Commit Graph
28030 Commits
Author SHA1 Message Date
Sameer KankuteandGitHub 18a9af3488 Merge pull request #17291 from BerriAI/litellm_fix_correct_attribute_error_code_raise
Fix 500 error for malformed request
2025-12-02 22:17:48 +05:30
Ishaan JaffandGitHub 1bb9e1bde8 [Feat] Add vllm batch+files API support (#15823)
* add OPENAI_COMPATIBLE_BATCH_AND_FILES_PROVIDERS

* fix use OPENAI_COMPATIBLE_BATCH_AND_FILES_PROVIDERS

* add _get_batch_job_total_usage_from_file_content

* fixes for vLLM + 12 labs async invoke

* fix: vLLM Batch APIs

* afile_retrieve

* test_hosted_vllm_full_workflow

* fix SERVER_URL for test
2025-12-02 08:41:50 -08:00
Alexsander HamirandGitHub 9ff2ecc16d Fix: update default proxy_batch_write_at number (#17355)
The default is 10 seconds, not 30.
2025-12-02 05:52:00 -08:00
Alexsander HamirandGitHub 0acb7f418d Fix (Docs) - Update default database connection limit #17353
We recommend using 10–20 connections. Our actual default is 10, but the documentation incorrectly listed the default as 100.
2025-12-02 05:35:58 -08:00
AlexsanderHamir 7324905c95 fix: update default database connection number 2025-12-02 05:29:03 -08:00
Krish DholakiaandGitHub 4c7a988454 Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support (#17338)
* refactor(generic_guardrail_api.py): refactor to update to new guardrail api logic

* refactor: refactor llm api integrations to support passing in text as a list[str] instead of one at a time

* refactor: fix linting errors

* refactor: pass request type to guardrail api

allows request vs. response processing to occur

* feat: pass user api key dict information to the guardrail api

* fix: pass user api key dict information to the guardrail api

* feat: pass litellm call id + trace id, if present

* docs: update docs
2025-12-01 20:11:58 -08:00
YutaSaitoandGitHub fe41e14951 fix: remove URL format validation for MCP server endpoints (#17270) 2025-12-01 20:08:54 -08:00
Korbinian KochandGitHub 6e8e3b30f9 Update Databricks model pricing and add new models (including databricks pricing test). (#17277)
* update databricks pricing and add DBU<>USD test

* Refactor test_databricks_pricing.py

Removed unnecessary sys.path modification and cleaned up comments.
2025-12-01 20:06:47 -08:00
codgicianandGitHub e09e309371 feat(github-copilot): Add Embedding API support (#17278) 2025-12-01 20:05:28 -08:00
Boxuan LiandGitHub 89458573a2 Add context window exception mapping for Together AI (#17284) 2025-12-01 20:02:59 -08:00
YutaSaitoandGitHub da5b81c1ff feat: add experimental latest-user filtering for Bedrock (#17282)
* feat: add experimental latest-user filtering for Bedrock

* doc: add experimental bedrock latest-message flag
2025-12-01 20:02:28 -08:00
Saar wintrovandGitHub 860270a792 SSO: Clear sso integration for all users (#17287) 2025-12-01 20:01:36 -08:00
Cesar GarciaandGitHub 01dfc3561a Fix AttributeError when metadata is null in request body (#17263) (#17306)
Handle the case where metadata is explicitly set to null/None in the
request body. This was causing a 401 error with "'NoneType' object
has no attribute 'get'" when calling /v1/batches with metadata: null.

The fix uses `or {}` instead of a default dict value since the key
exists but has a None value.
2025-12-01 19:58:27 -08:00
Cesar GarciaandGitHub 965406c643 feat(provider): add Z.AI (Zhipu AI) as built-in provider (#17307)
* feat(provider): add Z.AI (Zhipu AI) as built-in provider

Add support for Z.AI GLM models as a native OpenAI-compatible provider.

- Add "zai" to openai_compatible_providers list
- Add ZAI enum to LlmProviders
- Add provider URL resolution for https://api.z.ai/api/paas/v4
- Add 8 GLM models with pricing to model cost maps:
  - glm-4.6 (200K context, $0.6/$2.2 per 1M tokens)
  - glm-4.5, glm-4.5v, glm-4.5-x, glm-4.5-air, glm-4.5-airx
  - glm-4-32b-0414-128k
  - glm-4.5-flash (free tier)
- Add unit tests for provider integration

Closes #17289

* docs: add Z.AI provider documentation

- Add zai.md with usage examples, model list, and pricing
- Add to sidebars.js navigation
2025-12-01 19:56:47 -08:00
idola9andGitHub 71efcb7115 Refactor Noma guardrail to use shared Responses transformation and include system instructions (#17315)
* Support system prompts in noma guardrails

* Use litellm util to covert chat completions to responses api
2025-12-01 19:56:14 -08:00
98a244450e Fix sso users not added to entra synced team (#17331)
* test: add failing tests for SSO user not added to Entra-synced teams bug

Adds tests reproducing the bug where new SSO users with teams=None
(from NewUserResponse) are not added to Entra ID synced teams because
add_missing_team_member() returns early when teams is None.

Tests demonstrate:
- NewUserResponse with teams=None fails to add user to teams (bug)
- LiteLLM_UserTable with teams=[] correctly adds user to teams (control)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: treat None as empty list in add_missing_team_member for new SSO users

Fixed bug where new SSO users logging in via Microsoft SSO were not added
to their Entra-synced teams. The issue was an early return when
user_info.teams is None (default for NewUserResponse). Now treats None
as an empty list so new users are properly added to all their SSO teams.

Location: litellm/proxy/management_endpoints/ui_sso.py:438-440

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-01 19:53:30 -08:00
rioiartandGitHub 70126d9130 Fix/new org team validate against org (#17333)
* fix: skip user budget/model validation for org-scoped teams

When creating a team with organization_id, budget and model constraints
should be validated against the organization's limits, not the user's
personal limits. This allows org admins with restrictive personal
budgets to create teams within their organization's more generous limits.

Adds 4 unit tests to verify:
- Org-scoped teams bypass user budget validation
- Org-scoped teams bypass user model validation
- Standalone teams still validate against user limits

* fix: enforce user budget/model limits for standalone teams in update_team

- Add user-level budget and model validation to update_team endpoint for standalone teams,
  matching the existing pattern in new_team
- Org-scoped teams correctly bypass user validation and use organization limits instead
- Add 5 new comprehensive tests covering standalone/org team budget/model validation

* fix: Add direct TPM/RPM org limit validation and consolidate user team limit checks

- Add direct TPM/RPM comparison against org limits in _check_org_team_limits()
- Consolidate budget/models/TPM/RPM user validation into _check_user_team_limits() helper
- Ensure user limits only apply to standalone teams (organization_id=None)
- Org-scoped teams now validate TPM/RPM against org limits (not user limits)
- Add 8 tests for TPM/RPM validation scenarios (org and user limits)
- Reduce code duplication between new_team() and update_team()
2025-12-01 19:51:42 -08:00
Ishaan JaffandGitHub 1cdfb3da8f [Bug Fix] - Fix litellm_enterprise ensure imported routes exist (#17337)
* test_enterprise_routes.py

* test_enterprise_routes_all_imports_exist
2025-12-01 19:14:12 -08:00
Sameer KankuteandGitHub 289c13ca5d Merge pull request #17260 from abi-jey/main
fix: GA path for azure openai realtime models
2025-12-02 08:34:10 +05:30
Sameer KankuteandGitHub dbf1cd591d Merge pull request #17271 from colinlin-stripe/cherry-pick-invoke-headers
[fix] extra_headers in messages api bedrock invoke
2025-12-02 08:26:54 +05:30
Ishaan JaffandGitHub 860cdc81d3 [Fix] Fix Watsonx Audio Transcription API (#17326)
* """
add

* fix transform_audio_transcription_request

* fix tests

* test_watsonx_transcription_request_body
2025-12-01 18:26:56 -08:00
EliasandGitHub 37ecb03d4f Add support of audio transcription for OVHcloud (#17305) 2025-12-01 18:26:39 -08:00
Danny KoppingandGitHub be920d75d3 Add claude-opus-4-5 alias (#17313)
Similar to `claude-sonnet-4-5`.
2025-12-01 18:25:26 -08:00
Krish DholakiaandGitHub 1eb06f8031 Revert "fix: respect guardrail mock_response during during_call to return blo…" (#17332)
This reverts commit 6de6107673.
2025-12-01 15:40:28 -08:00
b6d6f834e0 (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo (#17175)
* feat(generic_guardrail_api.py): new generic api for guardrails

Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM

* docs(generic_guardrail_api.md): document new generic guardrail api

* Fix: Improve PII detection and guardrail API integration

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* feat: correctly extract raw request from guardrail api

* docs(generic_guardrail_api.md): document this is a beta feature

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-01 14:29:52 -08:00
Ishaan JaffandGitHub f434ca61ec add kimi-k2-instruct-0905 (#17328) 2025-12-01 14:14:41 -08:00
Krrish Dholakia c9afb86994 docs(azure_ai.md): document anthropic model usage on azure ai 2025-12-01 14:06:31 -08:00
Krrish Dholakia 7a46f3a083 docs: document azure ai provider for anthropic 2025-12-01 14:05:54 -08:00
Ishaan JaffandGitHub 24f847b84c [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints (#17324)
* feat: allow fetching OIDC user info

* test: use test_auth_builder_with_oidc_userinfo_enabled gets user info when enabled

* fix tool permission doc

* docs fix diagram
2025-12-01 13:59:00 -08:00
Ishaan JaffandGitHub ce0dc0c8b9 [Feat] WatsonX - allow passing zen_api_key dynamically (#16655)
* test_watsonx_zen_api_key_from_client

* zen api key

* docs using zen api key
2025-12-01 12:55:47 -08:00
YutaSaitoandGitHub a73bd751fc doc: add images for tool permission guardrail (#17322) 2025-12-01 12:38:49 -08:00
Colin Lin 661bccbc39 fixed flaky test by sorting list 2025-12-01 14:26:14 -05:00
Colin Lin e420b633a1 add tests 2025-12-01 14:25:37 -05:00
Colin Lin 2a5082e6cf remove logs 2025-12-01 14:25:37 -05:00
Colin Lin c588e7854d use kwargs 2025-12-01 14:25:37 -05:00
orgersh92andGitHub 7808a610f8 Fix session consistency, move Lasso API version away from source code (#17316)
* store and fetch lasso-conversation id from cache

* include gateway/v# in the baseUrl to allow simpler version migrations in the future

* add tests for cached conversation ID
2025-12-01 10:03:51 -08:00
YutaSaitoandGitHub 6de6107673 fix: respect guardrail mock_response during during_call to return blocked output (#17247) 2025-12-01 09:59:01 -08:00
Sameer KankuteandGitHub 3347e4de11 Merge pull request #17292 from BerriAI/litellm_gemini_image_gen_handling
Add better handling in image generation for gemini models
2025-12-01 21:42:36 +05:30
Sameer KankuteandGitHub 353c779c34 Merge pull request #17301 from BerriAI/litellm_claude_code_beta_fix
Remove not compatible beta header from Bedrock
2025-12-01 21:39:37 +05:30
Sameer KankuteandGitHub 8d1113eac1 Merge pull request #17296 from BerriAI/litellm_video_passthrough_cost_tracking
(Feat)Add passthrough cost tracking for veo
2025-12-01 21:33:03 +05:30
Sameer Kankute 983ba7aa0f Remove not compatible beta header from claude code 2025-12-01 17:22:04 +05:30
Sameer Kankute ae132abff4 Revert auth change 2025-12-01 14:36:30 +05:30
Sameer Kankute 7dac498efb Add passthrough cost tracking for veo 2025-12-01 14:33:03 +05:30
Sameer Kankute 02510a908f Add better handling image generation for gemini models 2025-12-01 10:42:38 +05:30
Sameer Kankute 9edc50efbd Fix 500 error for malformed request 2025-12-01 10:21:44 +05:30
Sameer KankuteandGitHub 07434092f8 Merge pull request #17290 from BerriAI/litellm_fix_create_batch_Header
Respect custom llm provider in header
2025-12-01 08:37:50 +05:30
Sameer Kankute f7380a51de Respect custom llm provider in header 2025-12-01 08:36:29 +05:30
Alexsander HamirandGitHub b949ec90db Merge pull request #16739 from BerriAI/litellm_/audio/speech
[Perf] Fix `/audio/speech` performance by using `shared_sessions`
2025-11-30 17:27:44 -08:00
Alexsander HamirandGitHub 4fffd33ccf Merge pull request #17167 from BerriAI/litellm_spend_logs_api
[Refactor] Deprecate `spend/logs` & add `spend/logs/v2`
2025-11-29 06:19:53 -08:00
Sameer KankuteandGitHub 7f42b9b987 Merge pull request #17193 from BerriAI/litellm_twelvelabs_int
Added support for twelvelabs pegasus
2025-11-28 22:09:00 +05:30