tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-05 11:07:29 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: suggest Gunicorn instead of uvicorn when using max_requests_before_restart (#17788)

Browse Source
This commit is contained in:
Alexsander Hamir
2025-12-10 13:09:00 -08:00
committed by GitHub
parent 7ad2a58dcd
commit 439bb5bfe3
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/docs/proxy/prod.md
+7
View File
@@ -81,6 +81,13 @@ CMD ["--port", "4000", "--config", "./proxy_server_config.yaml", "--num_workers"
export MAX_REQUESTS_BEFORE_RESTART=10000
```
> **Tip:** When using `--max_requests_before_restart`, the `--run_gunicorn` flag is more stable and mature as it uses Gunicorn's battle-tested worker recycling mechanism instead of Uvicorn's implementation.
```shell
# Use Gunicorn for more stable worker recycling
CMD ["--port", "4000", "--config", "./proxy_server_config.yaml", "--num_workers", "$(nproc)", "--run_gunicorn", "--max_requests_before_restart", "10000"]
```
## 4. Use Redis 'port','host', 'password'. NOT 'redis_url'