* Fix vercel_ai_gateway entry for glm-4.6
* Fix: add unit test for vercel_ai_gateway/zai/glm-4.6
* Fix:
Add fix for unit test on vercel_ai_gateway/zai/glm-4.6
* Remove print statements in test_vercel_ai_gateway_glm46_cost_math
* Add gemini/gemini-embedding-001 pricing entry for Google GenAI API
Add pricing and configuration entry for "gemini/gemini-embedding-001"
model when called via Google GenAI API (google-genai provider). This
mirrors the existing Vertex AI entry ("gemini-embedding-001") but uses
the 'gemini' provider instead.
Co-Authored-By: Claude <noreply@anthropic.com>
* Update tpm and rpm for gemini/gemini-embedding-001
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix(mcp): respect X-Forwarded-Proto header in OAuth endpoints
When LiteLLM proxy is deployed behind a reverse proxy (like nginx or a load balancer) that terminates SSL/TLS, the proxy receives HTTP requests internally but should expose HTTPS URLs externally. This change detects the X-Forwarded-Proto header and uses it to construct correct redirect URIs and endpoint URLs.
Changes:
- Added X-Forwarded-Proto detection to authorize, token, oauth_protected_resource_mcp, oauth_authorization_server_mcp, and register_client endpoints
- Added comprehensive tests for X-Forwarded-Proto header support across all affected endpoints
- Fixed existing tests to properly mock request.headers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix formatting
* feat(mcp): support X-Forwarded-Host for proxy base URL reconstruction
Extended X-Forwarded-Proto support to also handle X-Forwarded-Host and X-Forwarded-Port headers. This allows LiteLLM to correctly construct redirect URIs and endpoint URLs when deployed behind a reverse proxy that changes the host/port.
Example scenario:
- Internal URL: http://localhost:8888/github/mcp
- External URL: https://proxy.abc.com/github/mcp
- Proxy sets: X-Forwarded-Proto: https, X-Forwarded-Host: proxy.abc.com
Changes:
- Added get_request_base_url() helper function to centralize X-Forwarded-* header handling
- Replaced all inline X-Forwarded-Proto checks with calls to the helper function
- Helper handles X-Forwarded-Proto, X-Forwarded-Host, and X-Forwarded-Port
- Added tests for X-Forwarded-Host scenarios in authorize and token endpoints
Fixes issue where protected resource URL mismatch occurred:
Error: Protected resource http://proxy.abc.com:8888/github/mcp
does not match expected https://proxy.abc.com/github/mcp🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: replace Yelp-specific hostnames with generic examples
Changed all references from chatproxy.yelpcorp.com to proxy.example.com in:
- test_proxy_forwarding.py (default host parameter)
- TEST_PROXY_FORWARDING.md (documentation examples)
- discoverable_endpoints.py (docstring example)
- test_discoverable_endpoints.py (test mock data)
This makes the code more generic and suitable for open source.
All 13 tests still passing.
* remove accidentally added files
* fix formatting
* add new test for get_base_url
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: prevent httpx DeprecationWarning memory leak in AsyncHTTPHandler
Route bytes/str to content= parameter instead of data= to avoid deprecation warning that causes memory leak
* refactor: extract data/content preparation into helper function
Create _prepare_request_data_and_content() helper to DRY up the logic
for routing data/content parameters correctly in httpx requests.
This helper prevents httpx DeprecationWarnings (which cause memory leaks)
by moving bytes/str from data= to content= parameter while keeping
dict/Mapping in data= parameter.
Applied the helper consistently across all HTTP methods in both
AsyncHTTPHandler and HTTPHandler classes:
- post(), put(), patch(), delete()
- single_connection_post_request()
Related to: b850ed1188aced4e8c3cd7ef8b1a065fbc43edbb
* fix: Python 3.8 compatibility - use Tuple instead of tuple in type hints
Replace lowercase tuple[...] with typing.Tuple[...] in http_handler.py
to fix 'TypeError: type object is not subscriptable' on Python 3.8
* fix(apscheduler): prevent memory leaks from jitter and frequent job intervals
Fixes critical memory leak in APScheduler that causes 35GB+ memory allocations
during proxy startup and operation. The leak was identified through Memray
analysis showing massive allocations in normalize() and _apply_jitter()
functions.
Key changes:
1. Remove jitter parameters from all scheduled jobs - jitter was causing
expensive normalize() calculations leading to memory explosion
2. Configure AsyncIOScheduler with optimized job_defaults:
- misfire_grace_time: 3600s (increased from 120s) to prevent backlog
calculations that trigger memory leaks
- coalesce: true to collapse missed runs
- max_instances: 1 to prevent concurrent job execution
- replace_existing: true to avoid duplicate jobs on restart
3. Increase minimum job intervals:
- PROXY_BATCH_WRITE_AT: 30s (was 10s)
- add_deployment/get_credentials jobs: 30s (was 10s)
4. Use fixed intervals with small random offsets instead of jitter for
job distribution across workers
5. Explicitly configure jobstores and executors to minimize overhead
6. Disable timezone awareness to reduce computation
Memory impact:
- Before: 35GB with 483M allocations during startup
- After: <1GB with normal allocation patterns
Performance notes:
- Minimum job intervals increased from 10s to 30s (configurable via env vars)
- Jobs can still be distributed across workers using random start offsets
- No functional changes to job behavior, only timing and memory optimization
Testing:
- Added comprehensive test suite for scheduler configuration
- Verified no job execution backlog on startup
- Tested duplicate job prevention with replace_existing
Related issue: Memory leak in production proxy servers with APScheduler
\ud83e\udd16 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* docs: update PROXY_BATCH_WRITE_AT default value from 10s to 30s
Update documentation to reflect the new default value for PROXY_BATCH_WRITE_AT
changed in PR #15846. The default was increased from 10 seconds to 30 seconds
to prevent memory leaks in APScheduler.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: Move APScheduler config to constants.py
Address code review feedback from ishaan-jaff:
- Move scheduler configuration variables (coalesce, misfire_grace_time,
max_instances, replace_existing) to litellm/constants.py
- Update all references in proxy_server.py to use the constants
- Improves maintainability and makes configuration values centralized
Requested-by: @ishaan-jaff
Related: #15846🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
Fixes#15995
When using the Responses API with providers that don't have a native
ResponsesAPIConfig implementation (like Gemini, Anthropic, Cohere, etc.),
the text.format parameter was being ignored. This happened because these
providers fall back to using the LiteLLMCompletionTransformationHandler,
which converts Responses API requests to Chat Completion API requests.
Changes:
- Added 'text' to the list of supported parameters in
LiteLLMCompletionResponsesConfig.get_supported_openai_params()
- Added transformation logic to convert text.format (Responses API format)
to response_format (Chat Completion API format) in
transform_responses_api_request_to_chat_completion_request()
- Created _transform_text_format_to_response_format() method to handle
the conversion between the two format structures
The transformation supports:
- json_schema: Converts to Chat Completion's json_schema format with
proper nested structure
- json_object: Converts to Chat Completion's json_object mode
- text: Returns None (default text format)
This fix benefits all providers that use the fallback transformation
handler, ensuring consistent behavior with OpenAI's native Responses API
implementation.
