-
Notifications
You must be signed in to change notification settings - Fork 637
Graceful shut down #3785
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Graceful shut down #3785
Conversation
Thanks for your contribution! |
966d63a
to
bf78165
Compare
bf78165
to
217948a
Compare
217948a
to
7ac0d02
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds graceful shutdown capabilities to the FastDeploy API server by introducing a configurable timeout parameter for Uvicorn's graceful shutdown feature. The change allows users to specify how long the server should wait for in-flight requests to complete before terminating.
Key changes:
- Added
--timeout-graceful-shutdown
command line argument with default value of 0 seconds - Integrated the timeout parameter with Uvicorn's native graceful shutdown functionality
- Added comprehensive documentation for graceful shutdown best practices
Reviewed Changes
Copilot reviewed 3 out of 5 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
fastdeploy/entrypoints/openai/api_server.py | Added CLI argument and passed timeout parameter to Uvicorn configuration |
docs/zh/best_practices/graceful_shutdown_service.md | Added Chinese documentation for graceful shutdown solution architecture |
docs/best_practices/graceful_shutdown_service.md | Added English documentation for graceful shutdown solution architecture |
parser.add_argument( | ||
"--timeout-graceful-shutdown", | ||
default=0, |
Copilot
AI
Sep 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The default value of 0 seconds effectively disables graceful shutdown. Consider using a more reasonable default like 30 seconds to provide better out-of-the-box behavior for production deployments.
parser.add_argument( | |
"--timeout-graceful-shutdown", | |
default=0, | |
default=30, |
Copilot uses AI. Check for mistakes.
parser.add_argument( | ||
"--timeout-graceful-shutdown", | ||
default=0, | ||
type=int, | ||
help="timeout for graceful shutdown in seconds (used by uvicorn)", | ||
) |
Copilot
AI
Sep 4, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add input validation to ensure the timeout value is non-negative. Negative values don't make sense for a timeout parameter and could cause unexpected behavior.
Copilot uses AI. Check for mistakes.
1、基于Uvicorn原生能力,增加Server关闭时的优雅退出时长参数timeout_graceful_shutdown,交给用户赋值,该字段的作用是:当触发终止信号时,服务还会运行指定时间然后终止服务,保证一些请求执行完毕