Alexsander Hamir and GitHub
439bb5bfe3
fix: suggest Gunicorn instead of uvicorn when using max_requests_before_restart ( #17788 )
2025-12-10 13:09:00 -08:00
Ishaan Jaff and GitHub
7ad2a58dcd
[Feat] A2a Gateway - allow using bedrock agentcore, langgraph agents with A2a Gateway ( #17786 )
...
* init LANGGRAPH
* init LangGraphConfig
* init LangGraphConfig types
* init langgraph
* init getting api base and key
* init transform langgraph
* fix SSE issues
* test_langgraph_acompletion_non_streaming
* add LangGraph to docs
* docs: Setting Up a Local LangGraph Server
* fix langgraph SSE
* fix import uuid
* init A2A to LiteLLM Completion Bridge
* add send message for bridge
* test_a2a_completion_bridge_non_streaming
* add A2ACompletionBridgeTransformation
* add a2a send message support
* init a2a bridge
2025-12-10 13:08:43 -08:00
Ishaan Jaff and GitHub
49b91c4a35
[Feat] A2a gateway - Add cost per token pricing ( #17780 )
...
* fix calculate_a2a_cost
* add cost_per_query
* add test_asend_message_uses_cost_per_query
* fix: _initialize_slack_alerting_jobs
* feat: add token tracking for agents invoke
* add A2ARequestUtils
* add _set_usage_on_logging_obj
* test_asend_message_token_tracking
* add _handle_a2a_response_logging
* test_asend_message_streaming_token_tracking
* add A2AStreamingIterator
* add cost calculator for agents
* test_asend_message_uses_input_output_cost_per_token
* docs gix
2025-12-10 13:08:15 -08:00
Alexsander Hamir and GitHub
ccb6cd880a
[Perf] - Cut memory leak in half ( #17784 )
...
Stop passing references to large objects to fire-and-forget tasks.
2025-12-10 12:31:58 -08:00
Ishaan Jaff and GitHub
5ee32167c0
[Feat] New Provider - add langgraph ( #17783 )
...
* init LANGGRAPH
* init LangGraphConfig
* init LangGraphConfig types
* init langgraph
* init getting api base and key
* init transform langgraph
* fix SSE issues
* test_langgraph_acompletion_non_streaming
* add LangGraph to docs
* docs: Setting Up a Local LangGraph Server
* fix langgraph SSE
* fix import uuid
2025-12-10 12:30:35 -08:00
Ishaan Jaff and GitHub
115a14891d
[Feat] Agent Gateway - Add token counting non streaming + streaming ( #17779 )
...
* fix calculate_a2a_cost
* add cost_per_query
* add test_asend_message_uses_cost_per_query
* fix: _initialize_slack_alerting_jobs
* feat: add token tracking for agents invoke
* add A2ARequestUtils
* add _set_usage_on_logging_obj
* test_asend_message_token_tracking
* add _handle_a2a_response_logging
* test_asend_message_streaming_token_tracking
* add A2AStreamingIterator
2025-12-10 11:45:28 -08:00
8bc5e2ca7f
Add /v1/messages/count_tokens endpoint documentation ( #17772 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2025-12-10 11:34:11 -08:00
7a773b738b
Add docstring to add_new_model endpoint ( #17778 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2025-12-10 11:25:07 -08:00
Krrish Dholakia
4ea758767a
build: update poetry.lock
2025-12-10 11:09:28 -08:00
Ishaan Jaff and GitHub
7a33579af6
[Feat] Agent Gateway - Add cost per query for agent invocations ( #17774 )
...
* fix calculate_a2a_cost
* add cost_per_query
* add test_asend_message_uses_cost_per_query
* fix: _initialize_slack_alerting_jobs
2025-12-10 10:43:52 -08:00
saisurya237 and GitHub
f3ead2c153
add a new feature fix to expose the team alias when authenticating th… ( #17725 )
...
* CLI SSO: show team aliases in team selection
* temp poetry lock fix
* add poetry.lock to gitignore
* Revert proxy_server background job refactor
* Revert proxy_server background job refactor
* rever gitignore and poetry lock file
2025-12-10 10:10:28 -08:00
yuneng-jiang and GitHub
ba554a86b9
Merge pull request #16843 from BerriAI/litellm_allow_custom_mount_paths
...
[Feature] Allow Root Path to Redirect when Docs not on Root Path
2025-12-10 09:52:30 -08:00
Sameer Kankute and GitHub
dea62e3f60
Merge pull request #17769 from BerriAI/litellm_test_fix
...
Fix nvdia and geminin tests
2025-12-10 22:29:04 +05:30
Sameer Kankute and GitHub
ad45fc7cce
Merge pull request #17770 from BerriAI/litellm_deepseek_v3.2
...
Add DeepSeek-V3.2 model support
2025-12-10 22:28:50 +05:30
Sameer Kankute and GitHub
4c78c1afc8
Merge pull request #17756 from BerriAI/litellm_add_gemini_computer_use
...
Add support for computer use for gemini
2025-12-10 22:28:37 +05:30
Sameer Kankute
05a82ae570
Add support for deepseek v3.2
2025-12-10 22:21:04 +05:30
Sameer Kankute
ec30032bc3
Fix nvdia and geminin tests
2025-12-10 22:05:11 +05:30
Sameer Kankute and GitHub
0d2f8ce931
Merge pull request #17711 from BerriAI/litellm_add_additional_drop_params_support
...
feat: Add nested field removal support to additional_drop_params
2025-12-10 15:37:39 +05:30
Lucas Sugi and GitHub
c7fd8fabdb
fix: Avoid error when we have just the tool_calls in input ( #17753 )
...
* fix: Avoid error when we have just the tool_calls in input
* fix: Remove the tool call validation
* feat: Add unit test
2025-12-09 22:59:59 -08:00
Krish Dholakia and GitHub
b0a5a4b81d
Arize Phoenix OSS - Prompt Management Integration ( #17750 )
...
* docs(prompt_management.md): document how to onboard prompts to litellm
* feat(arize_phoenix_prompt_manager.py): support new prompt management integration
allows users to connect arize phoenix prompt manager to litellm
* fix(proxy/utils.py): remove prompt variables to avoid re-processing prompt
* docs(arize_phoenix_prompts.md): document new prompt management integration
2025-12-09 22:53:42 -08:00
Sameer Kankute
bcac9e41f6
Add support for computer use for gemini
2025-12-10 10:34:08 +05:30
Ishaan Jaff and GitHub
e546f0e021
[Fix] UI SSO - allow up-serting user role when SSO provider role changes ( #17754 )
...
* use _build_sso_user_update_data
* test_build_sso_user_update_data_with_valid_role
2025-12-09 19:16:10 -08:00
Peter Chanthamynavong and GitHub
539ce89d4e
fix(guardrails): mask all regex pattern matches, not just first ( #17727 )
...
Before: search() + replace() only replaced first match
After: sub() replaces all matches of each pattern
Closes #17687
2025-12-09 18:51:28 -08:00
Cesar Garcia and GitHub
b4e0dabb37
fix: use absolute URL for Supported Endpoints link to avoid Docusaurus slug conflict ( #17710 )
...
The relative link was causing Docusaurus to incorrectly associate the
/supported_endpoints page with SDK Functions category instead of the
actual Supported Endpoints generated-index.
2025-12-09 18:49:49 -08:00
82f0c3c887
Support model names with slashes on Gemini endpoints ( #17743 )
...
* Support model names with slashes on Gemini endpoints
* Fix test
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-12-09 18:40:51 -08:00
Krrish Dholakia
8b125ff74e
test: remove test_helicone - integration test made live calls to api and broke ci/cd
2025-12-09 18:36:51 -08:00
254c1155a2
Remove streaming_logging.md documentation ( #17739 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2025-12-09 18:26:08 -08:00
Cesar Garcia and GitHub
01dec55c2f
fix(anthropic): preserve server_tool_use and web_search_tool_result in multi-turn conversations ( #17746 )
...
- Extract web_search_tool_result blocks in extract_response_content()
- Store web_search_results in provider_specific_fields for round-trip
- Detect srvtoolu_ prefix to reconstruct as server_tool_use (not tool_use)
- Add corresponding web_search_tool_result after server_tool_use blocks
This ensures multi-turn conversations with Anthropic web search + custom
tools work correctly without Anthropic expecting tool_result for server-
side tool executions.
2025-12-09 18:25:23 -08:00
9fa6c51678
Fix: Add Gemini context window exception mapping ( #17751 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2025-12-09 18:24:13 -08:00
Krrish Dholakia
f23a78fddc
test: remove expensive test
2025-12-09 18:23:03 -08:00
Krrish Dholakia
c276a87ab0
fix(anthropic/chat/transformation.py): pass output_config + thinking to claude opus 4.5
2025-12-09 18:21:41 -08:00
Sameer Kankute and GitHub
8fafd81f9d
Merge pull request #17732 from BerriAI/litellm_videos_bugs_2
...
Fix : use litellm params for all videos apis
2025-12-10 07:49:55 +05:30
Krrish Dholakia
d3531be9a0
docs(community.md): add new integration partner doc
2025-12-09 18:17:14 -08:00
Ishaan Jaff and GitHub
42f5770cfa
[docs] add docs for containers files api + code interpreter on LiteLLM ( #17749 )
...
* add new container api on OpenAI
* add related
* docs fix
* docs code interpreter
* code interp
* docs code interptert
* docs code int
* docs code interp
* docs code interp
2025-12-09 18:11:28 -08:00
Shivam Rawat and GitHub
4ada6bee49
fixed flex tier pricing ( #17748 )
2025-12-09 18:10:51 -08:00
8d5e6cc62d
Add community doc link ( #17734 )
...
* Add community contribution guide for integration partners
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com >
* Update community docs to direct users to #integration-partners
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com >
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2025-12-09 18:10:00 -08:00
Cesar Garcia and GitHub
63a97db663
feat(voyage): add rerank API support ( #17744 )
...
* feat(voyage): add rerank API support
Add support for Voyage AI rerank models (rerank-2.5, rerank-2.5-lite,
rerank-2, rerank-2-lite) to the LiteLLM rerank API.
Changes:
- Add VoyageRerankConfig transformation class
- Register voyage provider in rerank_api/main.py
- Add voyage case in utils.py get_provider_rerank_config
- Add rerank-2.5 and rerank-2.5-lite models to pricing JSON
- Add unit tests for transformation logic
- Update documentation for voyage.md and rerank.md
Usage:
```python
from litellm import rerank
response = rerank(
model="voyage/rerank-2.5",
query="What is the capital of France?",
documents=["Paris is...", "London is..."],
top_n=3,
)
```
* refactor(voyage): simplify rerank transformation code
Remove verbose docstrings to align with other providers (jina_ai pattern).
No functional changes - 168 lines vs 169 for jina_ai.
* fix(voyage): remove incorrect input_cost_per_query from rerank models
Voyage AI charges per token, not per query. The input_cost_per_query
field was incorrectly set to the same value as input_cost_per_token
in the existing rerank-2 and rerank-2-lite models.
Removes input_cost_per_query from all Voyage rerank models:
- voyage/rerank-2
- voyage/rerank-2-lite
- voyage/rerank-2.5
- voyage/rerank-2.5-lite
Pricing source: https://docs.voyageai.com/docs/pricing
2025-12-09 17:34:09 -08:00
Ishaan Jaff and GitHub
3631e8fa1d
[Feat] Containers API - add new container API file management + UI Interface ( #17745 )
...
* test_router_acreate_container_without_model
* _init_containers_api_endpoints
* test_init_containers_api_endpoints
* init container files endpoints
* init files api
* init container files API
* add containers api file content
* add code interpreter output UI
* add code interpreter input ui
* refactor code interpreter ui
* fix: require model selection
* cleaner container provision
* fix ContainerFileObject
* def container_file_content_handler(
add
* add retrieve_container_file_content
* aretrieve_container_file_content
* UI fix model
* fix linting errors
2025-12-09 17:33:26 -08:00
Ishaan Jaff and GitHub
142567e143
[Fix] Containers API - Allow using LIST, Create Containers using custom-llm-provider ( #17740 )
...
* test_router_acreate_container_without_model
* _init_containers_api_endpoints
* test_init_containers_api_endpoints
2025-12-09 17:00:35 -08:00
yuneng-jiang
d99cf81386
Fixing test
2025-12-09 16:11:17 -08:00
yuneng-jiang and GitHub
9a7831dd33
Merge pull request #17697 from BerriAI/litellm_ui_settings_ui
...
[Feature] UI - UI Settings
2025-12-09 15:58:08 -08:00
ephrimstanley and GitHub
a91dda1194
Return 403 instead of 503 for unauthorized routes ( #17723 )
2025-12-09 15:16:11 -08:00
Krrish Dholakia
cc8a46b6b2
bump: version 1.80.9 → 1.80.10
2025-12-09 14:55:43 -08:00
yuneng-jiang and GitHub
bdfc3308b1
Merge pull request #17741 from BerriAI/litellm_ui_cred_fix_2
...
[Fix] Change credential encryption to only affect db credentials
2025-12-09 14:04:20 -08:00
Alexsander Hamir and GitHub
9a0432d013
[Fix] Perf - Reduce memory accumulation of spend_logs ( #17742 )
...
Replaces time-based spend log processing with a queue-size-based approach to improve performance and reduce memory usage. The new implementation:
- Adds background task that monitors spend_log_transactions queue size
- Triggers processing when queue reaches configurable threshold (default: 100)
- Implements exponential backoff when queue is idle to reduce CPU usage
- Increases batch processing limits (BATCH_SIZE: 100→1000, MAX_LOGS_PER_INTERVAL: 1000→10000)
- Adds configurable environment variables: SPEND_LOG_QUEUE_SIZE_THRESHOLD, SPEND_LOG_QUEUE_POLL_INTERVAL
This approach prevents unbounded queue growth while reducing unnecessary polling overhead.
2025-12-09 14:00:34 -08:00
yuneng-jiang
879ae45421
Change credential encryption to only affect db credentials
2025-12-09 13:36:40 -08:00
YutaSaito and GitHub
80a18f989a
feat: propagate Langfuse trace_id ( #17669 )
2025-12-09 12:25:52 -08:00
yuneng-jiang
39bf7a9f7c
Merge remote-tracking branch 'origin' into litellm_allow_custom_mount_paths
2025-12-09 11:58:05 -08:00
yuneng-jiang and GitHub
9253a9c365
Merge pull request #17689 from BerriAI/litellm_ui_settings_backend
...
[Feature] Get and Update Backend Routes for UI Settings
2025-12-09 11:55:28 -08:00
Shivam Rawat and GitHub
43a7bbeeaf
added note for using Azure Active Directory Tokens with all the other endpoints ( #17733 )
2025-12-09 11:51:28 -08:00
