Skip to content

Conversation

xiaolei373
Copy link
Collaborator

1、基于Uvicorn原生能力,增加Server关闭时的优雅退出时长参数timeout_graceful_shutdown,交给用户赋值,该字段的作用是:当触发终止信号时,服务还会运行指定时间然后终止服务,保证一些请求执行完毕

Copy link

paddle-bot bot commented Sep 1, 2025

Thanks for your contribution!

@paddle-bot paddle-bot bot added the contributor External developers label Sep 1, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds graceful shutdown capabilities to the FastDeploy API server by introducing a configurable timeout parameter for Uvicorn's graceful shutdown feature. The change allows users to specify how long the server should wait for in-flight requests to complete before terminating.

Key changes:

  • Added --timeout-graceful-shutdown command line argument with default value of 0 seconds
  • Integrated the timeout parameter with Uvicorn's native graceful shutdown functionality
  • Added comprehensive documentation for graceful shutdown best practices

Reviewed Changes

Copilot reviewed 3 out of 5 changed files in this pull request and generated 2 comments.

File Description
fastdeploy/entrypoints/openai/api_server.py Added CLI argument and passed timeout parameter to Uvicorn configuration
docs/zh/best_practices/graceful_shutdown_service.md Added Chinese documentation for graceful shutdown solution architecture
docs/best_practices/graceful_shutdown_service.md Added English documentation for graceful shutdown solution architecture

Comment on lines +84 to +86
parser.add_argument(
"--timeout-graceful-shutdown",
default=0,
Copy link

Copilot AI Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The default value of 0 seconds effectively disables graceful shutdown. Consider using a more reasonable default like 30 seconds to provide better out-of-the-box behavior for production deployments.

Suggested change
parser.add_argument(
"--timeout-graceful-shutdown",
default=0,
default=30,

Copilot uses AI. Check for mistakes.

Comment on lines +84 to +89
parser.add_argument(
"--timeout-graceful-shutdown",
default=0,
type=int,
help="timeout for graceful shutdown in seconds (used by uvicorn)",
)
Copy link

Copilot AI Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add input validation to ensure the timeout value is non-negative. Negative values don't make sense for a timeout parameter and could cause unexpected behavior.

Copilot uses AI. Check for mistakes.

@Jiang-Jia-Jun Jiang-Jia-Jun merged commit ed97cf8 into PaddlePaddle:develop Sep 4, 2025
14 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants