tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-18 04:28:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
28,597 Commits 2,072 Branches 1,425 Tags
13df50830d1b9a2daa26dd511b1b0780db73d5ea
Commit Graph
28597 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
YutaSaitoandGitHub 13df50830d chore: prefer standard trace id for Langfuse logging (#17791) 2025-12-11 08:18:45 -08:00
CyrusTCandGitHub 5d326386fb feat(bedrock): add serviceTier support for Converse API (#17810) 
Add support for the Bedrock Converse API serviceTier parameter to allow
specifying processing tier (priority, default, or flex).

Changes:
- Add ServiceTierBlock type in litellm/types/llms/bedrock.py
- Add serviceTier to CommonRequestObject
- Add serviceTier to get_config_blocks() in AmazonConverseConfig
- Add comprehensive tests for serviceTier functionality
- Add documentation for serviceTier usage

This allows users to configure service tier via:
- litellm_params in proxy config
- optional_params in SDK calls
 2025-12-11 08:16:32 -08:00
Cesar GarciaandGitHub 97be0da0d2 fix(azure_ai): Remove unsupported params from Azure AI Anthropic requests (#17822) 
* fix(azure_ai): Remove unsupported params from Azure AI Anthropic requests

Azure AI Anthropic endpoint rejects max_retries and stream_options parameters
with "Extra inputs are not permitted" error. These are LiteLLM-internal
parameters that should not be sent to the API.

Fixes 400 Bad Request error when using azure_ai/claude-sonnet-4-5 and other
Azure AI Anthropic models.

* test(azure_ai): Add test for unsupported params removal in Azure AI Anthropic

Verifies that max_retries, stream_options, and extra_body are properly
removed from the request before sending to Azure AI Anthropic endpoint.
 2025-12-11 08:09:13 -08:00
Ashton SidhuandGitHub a514313540 Add Hiddenlayer Guardrail Hooks (#17728) 
* Core logic working, need to add tests

* Re add removed files

* Remove mistaken files

* one more file

* Add deployment params

* Add tests

* Remove unused imports

* Update docs from feedback

* Update guardrails
 2025-12-11 07:43:26 -08:00
Sameer KankuteandGitHub 3de46889bb Merge pull request #17805 from BerriAI/litellm_fix_azure_anthropic_model_map 
Fix llm provider for azure_ai in model map
 2025-12-11 16:04:11 +05:30
Sameer Kankute 51065295ba Fix llm provider for azure_ai in model map 2025-12-11 11:36:56 +05:30
Sameer KankuteandGitHub 8942053c8b Merge pull request #17700 from BerriAI/litellm_batches_passthrough_cost_tracking 
Add anthropic retrieve batches and retreive file content support
 2025-12-11 10:31:54 +05:30
Sameer KankuteandGitHub 1107febf05 Merge pull request #17803 from BerriAI/litellm_preserve_system_instructions 
fix: Preserve systemInstructions for vertex ai generate content request
 2025-12-11 10:31:08 +05:30
Sameer Kankute 9344d29a15 fix: Preserve systemInstructions for vertex ai generate content request 2025-12-11 09:52:32 +05:30
Sameer KankuteandGitHub 2ea855d225 Merge pull request #17707 from raghav-stripe/raghav-fix-responsesapi-rl 
fix: responses api not applying tpm rate limits on api keys
 2025-12-11 08:57:16 +05:30
Shivam RawatandGitHub 9d7a255d55 made litellm proxy and sdk difference cleaner in overview (#17790) 2025-12-10 19:14:49 -08:00
Cesar GarciaandGitHub b2e3f56f69 feat(models): add Mistral Codestral 2508, Devstral 2512, and Labs Devstral Small 2512 (#17801) 
Add newly released Mistral coding models:
- mistral/codestral-2508: 256K context, $0.30/$0.90 per M tokens
- mistral/devstral-2512: 256K context, $0.40/$2.00 per M tokens
- mistral/labs-devstral-small-2512: 256K context, $0.10/$0.30 per M tokens
 2025-12-10 19:14:02 -08:00
Ishaan JaffandGitHub 4a7437ba5f [Feat] Agent Gateway - allow adding langgraph, bedrock agent core agents (#17802) 
* fix: langgraph bridge streaming

* add public/agents/fields

* test_a2a_completion_bridge_non_streaming

* TestA2AStreamingTransformation

* AgentCredentialFieldMetadata

* add new logo

* refactor add agent

* fix add dynamic fields

* feat allow adding langgraph agent

* add langgraph provider

* stash

* add AgentCreateInfo

* agent_create_fields

* fix fields

* test_a2a_completion_bridge_bedrock_agentcore

* test_a2a_completion_bridge_bedrock_agentcore

* add public endpoints

* fix a2a endpoints

* fix dynamic fields
 2025-12-10 19:13:50 -08:00
Sameer KankuteandGitHub ad5d94a82f Merge pull request #17771 from BerriAI/litellm_db_videos_issue 
fix: respect videos content db creds
 2025-12-11 08:27:43 +05:30
Sameer Kankute 44d57b695a remove print statment 2025-12-11 08:22:22 +05:30
Ishaan JaffandGitHub 059fedbed5 [Feat] Agent Gateway - Track agent_id in SpendLogs (#17795) 
* add agent_id in metadata in spend logs

* add agent_id in SpendLogsPayload

* add agent_id in SpendLogsPayload

* add _set_agent_id_on_logging_obj

* add agent id tracking in SpendLogs

* add agent id in spend logs

* fix create_a2a_client

* test_asend_message_passes_agent_id_to_callback

* test_get_logging_payload_includes_agent_id_from_kwargs

* test_asend_message_streaming_triggers_callbacks

* fix asend_message_streaming

* asend_message_streaming

* A2AStreamingIterator

* _handle_stream_message

* test_asend_message_streaming_propagates_metadata
 2025-12-10 16:09:56 -08:00
Ishaan JaffandGitHub 5d456bcdc3 [Feat] UI SSO - allow fetching role from generic SSO provider (Keycloak) (#17787) 
* fix ui SSO

* TestGenericResponseConvertorUserRole

* Assigning User Roles via SSO
 2025-12-10 13:09:28 -08:00
Alexsander HamirandGitHub 439bb5bfe3 fix: suggest Gunicorn instead of uvicorn when using max_requests_before_restart (#17788) 2025-12-10 13:09:00 -08:00
Ishaan JaffandGitHub 7ad2a58dcd [Feat] A2a Gateway - allow using bedrock agentcore, langgraph agents with A2a Gateway (#17786) 
* init LANGGRAPH

* init LangGraphConfig

* init LangGraphConfig types

* init langgraph

* init getting api base and key

* init transform langgraph

* fix SSE issues

* test_langgraph_acompletion_non_streaming

* add LangGraph to docs

* docs: Setting Up a Local LangGraph Server

* fix langgraph SSE

* fix import uuid

* init A2A to LiteLLM Completion Bridge

* add send message for bridge

* test_a2a_completion_bridge_non_streaming

* add A2ACompletionBridgeTransformation

* add a2a send message support

* init a2a bridge
 2025-12-10 13:08:43 -08:00
Ishaan JaffandGitHub 49b91c4a35 [Feat] A2a gateway - Add cost per token pricing (#17780) 
* fix calculate_a2a_cost

* add cost_per_query

* add test_asend_message_uses_cost_per_query

* fix: _initialize_slack_alerting_jobs

* feat: add token tracking for agents invoke

* add A2ARequestUtils

* add _set_usage_on_logging_obj

* test_asend_message_token_tracking

* add _handle_a2a_response_logging

* test_asend_message_streaming_token_tracking

* add A2AStreamingIterator

* add cost calculator for agents

* test_asend_message_uses_input_output_cost_per_token

* docs gix
 2025-12-10 13:08:15 -08:00
Alexsander HamirandGitHub ccb6cd880a [Perf] - Cut memory leak in half (#17784) 
Stop passing references to large objects to fire-and-forget tasks.
 2025-12-10 12:31:58 -08:00
Ishaan JaffandGitHub 5ee32167c0 [Feat] New Provider - add langgraph (#17783) 
* init LANGGRAPH

* init LangGraphConfig

* init LangGraphConfig types

* init langgraph

* init getting api base and key

* init transform langgraph

* fix SSE issues

* test_langgraph_acompletion_non_streaming

* add LangGraph to docs

* docs: Setting Up a Local LangGraph Server

* fix langgraph SSE

* fix import uuid
 2025-12-10 12:30:35 -08:00
Ishaan JaffandGitHub 115a14891d [Feat] Agent Gateway - Add token counting non streaming + streaming (#17779) 
* fix calculate_a2a_cost

* add cost_per_query

* add test_asend_message_uses_cost_per_query

* fix: _initialize_slack_alerting_jobs

* feat: add token tracking for agents invoke

* add A2ARequestUtils

* add _set_usage_on_logging_obj

* test_asend_message_token_tracking

* add _handle_a2a_response_logging

* test_asend_message_streaming_token_tracking

* add A2AStreamingIterator
 2025-12-10 11:45:28 -08:00
Krish DholakiaGitHubCursor Agent
 		8bc5e2ca7f Add /v1/messages/count_tokens endpoint documentation (#17772) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-12-10 11:34:11 -08:00
Krish DholakiaGitHubCursor Agent
 		7a773b738b Add docstring to add_new_model endpoint (#17778) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-12-10 11:25:07 -08:00
Krrish Dholakia 4ea758767a build: update poetry.lock 2025-12-10 11:09:28 -08:00
Ishaan JaffandGitHub 7a33579af6 [Feat] Agent Gateway - Add cost per query for agent invocations (#17774) 
* fix calculate_a2a_cost

* add cost_per_query

* add test_asend_message_uses_cost_per_query

* fix: _initialize_slack_alerting_jobs
 2025-12-10 10:43:52 -08:00
saisurya237andGitHub f3ead2c153 add a new feature fix to expose the team alias when authenticating th… (#17725) 
* CLI SSO: show team aliases in team selection

* temp poetry lock fix

* add poetry.lock to gitignore

* Revert proxy_server background job refactor

* Revert proxy_server background job refactor

* rever gitignore and poetry lock file
 2025-12-10 10:10:28 -08:00
yuneng-jiangandGitHub ba554a86b9 Merge pull request #16843 from BerriAI/litellm_allow_custom_mount_paths 
[Feature] Allow Root Path to Redirect when Docs not on Root Path
 2025-12-10 09:52:30 -08:00
Sameer Kankute 15e5a8251e fix: respect videos content db creds 2025-12-10 23:00:01 +05:30
Sameer KankuteandGitHub dea62e3f60 Merge pull request #17769 from BerriAI/litellm_test_fix 
Fix nvdia and geminin tests
 2025-12-10 22:29:04 +05:30
Sameer KankuteandGitHub ad45fc7cce Merge pull request #17770 from BerriAI/litellm_deepseek_v3.2 
Add DeepSeek-V3.2 model support
 2025-12-10 22:28:50 +05:30
Sameer KankuteandGitHub 4c78c1afc8 Merge pull request #17756 from BerriAI/litellm_add_gemini_computer_use 
Add support for computer use for gemini
 2025-12-10 22:28:37 +05:30
Sameer Kankute 05a82ae570 Add support for deepseek v3.2 2025-12-10 22:21:04 +05:30
Sameer Kankute ec30032bc3 Fix nvdia and geminin tests 2025-12-10 22:05:11 +05:30
Sameer Kankute 0f99517170 Fix lint error 2025-12-10 18:28:29 +05:30
Sameer Kankute 9e3a04a725 Add batch passthrough endpoint cost tracking for anthropic 2025-12-10 18:24:31 +05:30
Sameer Kankute ad87aa1926 fix code quality qa 2025-12-10 17:24:16 +05:30
Sameer Kankute a69384598c Fix:Argument llm_provider 2025-12-10 17:21:15 +05:30
Sameer Kankute b9d3d7c059 Add tests for file and batch feat for anthropic 2025-12-10 17:14:47 +05:30
Sameer Kankute ec3c9191f3 Transform anthropic file content to openai file content 2025-12-10 17:07:00 +05:30
Sameer Kankute 854183e3b9 Revert batch utils with original logic 2025-12-10 17:01:21 +05:30
Sameer KankuteandGitHub 0d2f8ce931 Merge pull request #17711 from BerriAI/litellm_add_additional_drop_params_support 
feat: Add nested field removal support to additional_drop_params
 2025-12-10 15:37:39 +05:30
Lucas SugiandGitHub c7fd8fabdb fix: Avoid error when we have just the tool_calls in input (#17753) 
* fix: Avoid error when we have just the tool_calls in input

* fix: Remove the tool call validation

* feat: Add unit test
 2025-12-09 22:59:59 -08:00
Krish DholakiaandGitHub b0a5a4b81d Arize Phoenix OSS - Prompt Management Integration (#17750) 
* docs(prompt_management.md): document how to onboard prompts to litellm

* feat(arize_phoenix_prompt_manager.py): support new prompt management integration

allows users to connect arize phoenix prompt manager to litellm

* fix(proxy/utils.py): remove prompt variables to avoid re-processing prompt

* docs(arize_phoenix_prompts.md): document new prompt management integration
 2025-12-09 22:53:42 -08:00
Sameer Kankute bcac9e41f6 Add support for computer use for gemini 2025-12-10 10:34:08 +05:30
Ishaan JaffandGitHub e546f0e021 [Fix] UI SSO - allow up-serting user role when SSO provider role changes (#17754) 
* use _build_sso_user_update_data

* test_build_sso_user_update_data_with_valid_role
 2025-12-09 19:16:10 -08:00
Peter ChanthamynavongandGitHub 539ce89d4e fix(guardrails): mask all regex pattern matches, not just first (#17727) 
Before: search() + replace() only replaced first match
After: sub() replaces all matches of each pattern

Closes #17687
 2025-12-09 18:51:28 -08:00
Cesar GarciaandGitHub b4e0dabb37 fix: use absolute URL for Supported Endpoints link to avoid Docusaurus slug conflict (#17710) 
The relative link was causing Docusaurus to incorrectly associate the
/supported_endpoints page with SDK Functions category instead of the
actual Supported Endpoints generated-index.
 2025-12-09 18:49:49 -08:00
Hunter WittenbornGitHubCopilot
 		82f0c3c887 Support model names with slashes on Gemini endpoints (#17743) 
* Support model names with slashes on Gemini endpoints

* Fix test

* Update tests/proxy_unit_tests/test_google_endpoint_routing.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/proxy_unit_tests/test_google_endpoint_routing.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/proxy_unit_tests/test_google_endpoint_routing.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/proxy_unit_tests/test_google_endpoint_routing.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/proxy_unit_tests/test_google_endpoint_routing.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/proxy_unit_tests/test_google_endpoint_routing.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-12-09 18:40:51 -08:00