* [Fix] Containers API - Allow routing to regional endpoints (#19118)
* fix get_complete_url
* fix url resolution containers API
* TestContainerRegionalApiBase
* feat(proxy): add keepalive_timeout support for Gunicorn server
Add configurable keepalive timeout parameter for Gunicorn workers to
match existing Uvicorn functionality. This allows users to tune the
keep-alive connection timeout based on their deployment requirements.
Changes:
- Add keepalive_timeout parameter to _run_gunicorn_server method
- Configure Gunicorn's keepalive setting (defaults to 90s if not specified)
- Update --keepalive_timeout CLI help text to document both Uvicorn and Gunicorn behavior
- Pass keepalive_timeout from run_server to _run_gunicorn_server
Tests:
- Add test to verify keepalive_timeout flag is properly passed to Gunicorn
- Add test to verify default 90s timeout when flag is not specified
Co-Authored-By: lizhen921 <294474470@qq.com>
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
---------
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: lizhen921 <294474470@qq.com>
* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params
Fixes issue where 'user' was not being sent in request to litellm proxy via sdk
* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default
* docs: cleanup docs
* fix(proxy_cli.py): remove --use_prisma_migrate flag
* refactor: remove references to use_prisma_migrate env var
This is now the default flow for db migrations
* Enhance proxy CLI with Rich formatting and improved user experience
- Integrated Rich library for better console output in `proxy_cli.py`, including version display, health check results, and test completion responses.
- Updated health check and test completion methods to provide progress indicators and formatted tables.
- Refactored feedback display in `proxy_server.py` to use Rich for a more visually appealing user interface.
- Adjusted tests in `test_proxy_cli.py` to mock console output instead of using print statements, ensuring compatibility with Rich formatting.
* fix linting error
* refactor(proxy_cli.py): simplify DB setup logging
- Removed progress indicators for IAM token generation and environment variable decryption to simplify the code.
- Consolidated the logic for generating the database URL and setting environment variables.
- Enhanced error handling for configuration loading and database setup, ensuring clearer feedback
* Update test-linting workflow to include proxy-dev dependencies in Poetry installation
* Enhance proxy server initialization with Rich console for improved model display. Added support for loading model parameters from environment variables and refined provider identification logic. Fallback to original print formatting if Rich is not available.
* Refactor feedback handling: Moved feedback message generation and custom warning display to utils.py. Enhanced feedback box with rich formatting and fallback to ASCII for environments without rich. Cleaned up proxy_server.py by removing obsolete code.
* fix linting error
* Refactor model initialization display: Moved model initialization logic to a new utility function `display_model_initialization` for improved readability and maintainability. Enhanced model provider extraction with a dedicated function. Fallback to basic logging if Rich console is unavailable.
* Refactor model provider extraction: Replace the `_extract_provider_from_model` function with a more robust approach using `get_llm_provider`. Implement fallback logic for provider identification and improve error handling. Ensure compatibility with Rich console for model initialization display.
Ishaan Jaffer