Commit Graph
28219 Commits
Author SHA1 Message Date
Alexsander HamirandGitHub 0c017f376c fix: code quality issues from ruff linter (#17536)
* fix: resolve code quality issues from ruff linter

- Fix duplicate imports in anthropic guardrail handler
  - Remove duplicate AllAnthropicToolsValues import
  - Remove duplicate ChatCompletionToolParam import

- Remove unused variable 'tools' in guardrail handler

- Replace print statement with proper logging in json_loader
  - Use verbose_logger.warning() instead of print()

- Remove unused imports
  - Remove _update_metadata_field from team_endpoints
  - Remove unused ChatCompletionToolCallChunk imports from transformation

- Refactor update_team function to reduce complexity (PLR0915)
  - Extract budget_duration handling into _set_budget_reset_at() helper
  - Minimal refactoring to reduce function from 51 to 50 statements

All ruff linter errors resolved. Fixes F811, F841, T201, F401, and PLR0915 errors.

* docs: add missing environment variables to documentation

Add 8 missing environment variables to the environment variables reference section:
- AIOHTTP_CONNECTOR_LIMIT_PER_HOST: Connection limit per host for aiohttp connector
- AUDIO_SPEECH_CHUNK_SIZE: Chunk size for audio speech processing
- CYBERARK_SSL_VERIFY: Flag to enable/disable SSL certificate verification for CyberArk
- LITELLM_DD_AGENT_HOST: Hostname or IP of DataDog agent for LiteLLM-specific logging
- LITELLM_DD_AGENT_PORT: Port of DataDog agent for LiteLLM-specific log intake
- WANDB_API_KEY: API key for Weights & Biases (W&B) logging integration
- WANDB_HOST: Host URL for Weights & Biases (W&B) service
- WANDB_PROJECT_ID: Project ID for Weights & Biases (W&B) logging integration

Fixes test_env_keys.py test that was failing due to undocumented environment variables.
2025-12-05 08:40:49 -08:00
yuneng-jiangandGitHub b9b5d638c8 Merge pull request #17524 from BerriAI/litellm_team_user_settings_fix
[Fix] Select in Edit Membership Modal
2025-12-04 23:13:13 -08:00
yuneng-jiang 37bfe65bdd Adding screenshot to debug 2025-12-04 23:05:00 -08:00
yuneng-jiang 50283a00a3 e2e fix 2025-12-04 22:51:52 -08:00
Sameer KankuteandGitHub acc0b5fe27 Merge pull request #17362 from BerriAI/litellm_vertex-bge-cherrypick
[Feat] VertexAI - Add BGE Embeddings support
2025-12-05 11:53:42 +05:30
Krish DholakiaandGitHub b3a3081e8e Guardrails API - new structured_messages param (#17518)
* fix(generic_guardrail_api.py): add 'structured_messages' support

allows guardrail provider to know if text is from system or user

* fix(generic_guardrail_api.md): document 'structured_messages' parameter

give api provider a way to distinguish between user and system messages

* feat(anthropic/): return openai chat completion format structured messages when calls made via `/v1/messages` on Anthropic

* feat(responses/guardrail_translation): support 'structured_messages' param for guardrails

structured openai chat completion spec messages, for guardrail checks when using /v1/responses api

allows guardrail checks to work consistently across APIs
2025-12-04 22:08:00 -08:00
Krish DholakiaandGitHub 51cc102c30 fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514)
* fix(unified_guardrail.py): support during_call event type for unified guardrails

allows guardrails overriding apply_guardrails to work 'during_call'

* feat(generic_guardrail_api.py): support new 'tool_calls' field for generic guardrail api

returns the tool calls emitted by the LLM API to the user

* fix(generic_guardrail_api.py): working anthropic /v1/messages tool call response

send llm tool calls to guardrail api when called via `/v1/messages` API

* fix(responses/): run generic_guardrail_api on responses api tool call responses

* fix: fix tests

* test: fix tests

* fix: fix tests
2025-12-04 22:06:13 -08:00
Cesar GarciaandGitHub 316f7671a9 fix(gemini): handle partial JSON chunks after first valid chunk (#17496)
* fix(gemini): allow JSON accumulation on any chunk, not just first

* test(gemini): add tests for partial JSON chunk handling
2025-12-04 22:01:59 -08:00
yuneng-jiang 8da265b901 Fix select in edit membership 2025-12-04 21:59:43 -08:00
Kristian BrünnandGitHub 63fae79493 fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage (#17504)
* fix: optimize SpendLogs queries to use timestamp filtering (#17487)

* use timestamptz & enhance test
2025-12-04 21:52:57 -08:00
8776336c3c Enable detailed debugging for reference (#17508)
* Deprecate set_verbose in favor of LITELLM_LOG

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Update debugging documentation links

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-04 21:51:56 -08:00
Devaj ModyandGitHub 48b5100c18 fix(guardrails): mask all matching keywords in content filter (#17521)
Fixes #17517

  - Fixed bug where only the first matching blocked keyword was masked
  - Now iterates through ALL blocked keywords and masks each one
  - Added 3 regression tests for multiple keyword masking
2025-12-04 21:50:15 -08:00
Dominic FelitonandGitHub 6af693de36 (fix): empty response + vllm streaming (#17516)
* Fix empty response + vllm streaming

* Add unit test
2025-12-04 19:24:28 -08:00
YutaSaitoandGitHub 7106509ef0 chore: add stack trace to mcp error message (#17269) 2025-12-04 19:19:00 -08:00
Ishaan Jaffer 4f3b843efe docs openai 2025-12-04 18:32:23 -08:00
b2e8d3fd42 [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider (#17448)
* feat: Add JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Add simple JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Implement JSON-based provider config and migrate PublicAI

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* docs fix

* undo change

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-04 17:59:25 -08:00
Ishaan JaffandGitHub fadfbb13d3 [Docs] A2a - Permission management (#17515)
* docs add a2a gateway + mcp gateway

* docs a2a permissions

* docs a2a permission

* docs

* docs a2a

* docs a2a

* add new img

* docs agent permissions
2025-12-04 17:29:47 -08:00
Ishaan Jaffer dfe3124335 UI new build 2025-12-04 16:34:59 -08:00
Ishaan Jaffer 5cc1a5031b fix invoke_agent_a2a 2025-12-04 16:32:15 -08:00
Ishaan JaffandGitHub 575e769bff [Feat] UI - Agent Gateway - set allowed agents by key, team (#17511)
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route

* add AgentPermissionsProps

* add agents on team/key create

* add agent selector on team/key

* add agent selector on key edit /info

* add AgentPermissions

* docs list + invoke agents
2025-12-04 16:31:17 -08:00
Ishaan JaffandGitHub 8f592088ed [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend (#17502)
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route
2025-12-04 16:31:00 -08:00
e3116da653 feat: Add /global/spend/tags to admin viewer routes (#17501)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-04 14:43:33 -08:00
Ishaan Jaffer c6e26a20b1 refactor invoke 2025-12-04 14:21:01 -08:00
Raghav JhavarandGitHub 72eb4c3a1c 🆕 feat: support routing to only websearch supported deployments (#17500)
* support routing to only websearch supported deployments

* add docs
2025-12-04 14:18:20 -08:00
Anas AbdelRandGitHub 2abcc77944 fix: resolve ruff lint errors (#17490)
Fixed 20 of 22 lint errors:

- batches/batch_utils.py: Added missing model_name parameter to functions
- integrations/custom_guardrail.py: Removed unused Tuple import
- llms/custom_httpx/http_handler.py: Removed unused AIOHTTP_NEEDS_CLEANUP_CLOSED import
- llms/anthropic/chat/guardrail_translation/handler.py: Prefixed unused variable with underscore
- llms/openai/responses/guardrail_translation/handler.py: Removed duplicate BaseModel import, prefixed unused variable
- llms/pass_through/guardrail_translation/handler.py: Prefixed unused variables with underscore
- proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py: Removed unused List and Tuple imports
- proxy/hooks/parallel_request_limiter_v3.py: Removed unused import

Remaining 2 errors are PLR0915 (too many statements) which require refactoring.
2025-12-04 14:12:57 -08:00
Yannay HammerandGitHub 2627f0d519 Fix aim security guardrail tests (#17499) 2025-12-04 13:29:20 -08:00
Krrish Dholakia 5aeba81538 docs(multi_tenant_architecture.md): add new architecture doc 2025-12-04 11:13:50 -08:00
Ishaan JaffandGitHub 8ee8cad4f1 feat - new model amazon.titan-image-generator-v2:0 (#17488) 2025-12-04 09:26:55 -08:00
Sameer KankuteandGitHub 31f1bf0539 Merge pull request #17485 from BerriAI/litellm_ragflow_test_fix
fix: test_default_api_base for ragfow
2025-12-04 21:57:18 +05:30
Sameer KankuteandGitHub f2c0029939 Merge pull request #17470 from BerriAI/litellm_batches_bedrock_content
Add support for file content download for bedrock batches
2025-12-04 21:57:04 +05:30
Sameer KankuteandGitHub 5b4542304d Merge pull request #17461 from BerriAI/litellm_qwen2_imported_model_support
Add support for bedrock qwen 2 imported model
2025-12-04 21:56:22 +05:30
Sameer KankuteandGitHub 1bd4050921 Merge pull request #17458 from BerriAI/litellm_map_output_tokens_details
Map output_tokens_details of responses API to completion_tokens_details
2025-12-04 21:55:21 +05:30
Sameer Kankute 6751badf3a fix: test_default_api_base for ragfow 2025-12-04 21:49:33 +05:30
Sameer Kankute 46ebf425d5 Fix : test_vertexai_model_garden_model_completion 2025-12-04 21:41:02 +05:30
Sameer Kankute edd392b50d Add support for file content download for bedrock batches 2025-12-04 13:27:53 +05:30
yuneng-jiangandGitHub 94b944e8ec Merge pull request #17465 from BerriAI/litellm_ui_model_page_perf_2
[Infra] UI - Change Credentials to use React Query
2025-12-03 23:53:40 -08:00
yuneng-jiangandGitHub ad86dfbf7d Merge pull request #17451 from BerriAI/litellm_new_login_page_sso_changes
[Feature] Add Auto Redirect to SSO to New Login Page
2025-12-03 23:53:21 -08:00
yuneng-jiang 5496e622c1 Unit tests 2025-12-03 23:46:09 -08:00
yuneng-jiang 59b32c28a8 Change selector in test 2025-12-03 23:37:15 -08:00
yuneng-jiang 0483336c53 change test to follow other logins 2025-12-03 23:21:09 -08:00
yuneng-jiang 19dc5d750c Adding waits 2025-12-03 23:10:32 -08:00
yuneng-jiang 45a9723467 view_user_info debugging 2025-12-03 22:20:04 -08:00
yuneng-jiang b8190c6b6d resolving build issues 2025-12-03 22:15:19 -08:00
yuneng-jiang 7519a3e30b Change credentials to use react-query 2025-12-03 22:08:41 -08:00
dc7c2b9b05 Update docs to link agent hub (#17462)
* Docs: Add AI Hub agent registry documentation

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Fix: Update AI Hub link in A2A documentation

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-03 21:59:45 -08:00
yuneng-jiang c634c08818 Find inputs by placeholder 2025-12-03 21:51:45 -08:00
Sameer Kankute 4710e772be Add support for bedrock qwen 2 imported model 2025-12-04 11:08:57 +05:30
Daniel KleinandGitHub 3134fc825b fix: make bedrock converse messages respect modify_params as expected (#17427) 2025-12-03 21:34:50 -08:00
Cesar GarciaandGitHub 664b384dce fix(pricing): add missing Moonshot turbo models and fix incorrect pricing (#17432)
Fixes #17417

- Add moonshot/kimi-k2-turbo-preview ($1.15/$8.00 input/output)
- Add moonshot/kimi-k2-thinking-turbo ($1.15/$8.00 input/output)
- Add moonshot/kimi-k2-0905-preview ($0.60/$2.50 input/output, 262K context)
- Fix moonshot/kimi-thinking-preview pricing ($30/$30 -> $0.60/$2.50)
- Fix wandb/moonshotai/Kimi-K2-Instruct absurd pricing ($135K/$400K -> $0.60/$2.50)
2025-12-03 21:31:49 -08:00
Cesar GarciaandGitHub b05f6d3cfd fix(groq): check supports_response_schema before using json_tool_call workaround (#17438)
Fixes issue where LiteLLM used the json_tool_call workaround for all Groq
models, even those that support native json_schema (e.g., gpt-oss-120b,
llama-4, kimi-k2). This caused errors when users passed their own tools
alongside structured outputs.

Changes:
- Check `supports_response_schema()` before using the workaround
- Only use json_tool_call workaround for models without native support
- Add clear error message when using workaround with user-provided tools
- Update model config to set `supports_response_schema: false` for models
  that don't support native json_schema
- Add unit tests for structured output handling
2025-12-03 21:31:13 -08:00