* fix: enforce team MCP permissions when using JWT authentication
Root cause: When JWT auth was used with teams in groups (via team_ids_jwt_field),
the team's MCP permissions were not being enforced because:
1. The default team_allowed_routes did not include mcp_routes
2. allowed_routes_check() failed for MCP endpoints like /mcp/tools/list
3. find_team_with_model_access() skipped the team due to failed route check
4. team_id was None in UserAPIKeyAuth
5. MCPRequestHandler._get_allowed_mcp_servers_for_team() returned empty list
Fix: Add 'mcp_routes' to the default team_allowed_routes in LiteLLM_JWTAuth.
This ensures that teams can access MCP endpoints by default, allowing the
team's MCP server permissions to be properly enforced.
Added tests:
- test_reproduce_jwt_mcp_enforcement_issue: Reproduces the exact bug scenario
- test_verify_mcp_routes_in_default_team_allowed_routes: Verifies fix
- test_mcp_route_check_passes_for_team: Verifies route check works
Co-authored-by: ishaan <ishaan@berri.ai>
* test: add comprehensive E2E tests for JWT + team MCP permission enforcement
Added tests:
- test_e2e_jwt_team_mcp_permissions_enforced: Full E2E test verifying JWT auth
with teams in groups properly sets team_id and MCPRequestHandler returns
the team's MCP servers
- test_e2e_jwt_without_team_no_mcp_servers: Verifies no MCP servers returned
when JWT has no teams
- test_e2e_jwt_team_mcp_key_intersection: Verifies intersection logic when
both key and team have MCP permissions (result = intersection)
These tests verify the complete flow:
1. JWT token with team in groups field
2. JWT auth properly sets team_id on UserAPIKeyAuth
3. MCPRequestHandler.get_allowed_mcp_servers() returns team's MCP servers
4. Key/team permission intersection works correctly
Co-authored-by: ishaan <ishaan@berri.ai>
* test: add simple tests for JWT + MCP permission enforcement
Simple, focused tests that validate:
1. test_simple_jwt_mcp_permissions_enforced: JWT user with team gets team's MCP servers
2. test_simple_jwt_no_team_no_mcp_servers: JWT user without team gets no MCP servers
3. test_simple_jwt_team_id_required_for_mcp_permissions: Verifies team_id is required
4. test_jwt_auth_sets_team_id_for_mcp_route: JWT auth sets team_id for MCP routes
These tests directly verify the core MCP permission enforcement logic works
when using JWT authentication with teams.
Co-authored-by: ishaan <ishaan@berri.ai>
* Add test: MCP route without model still returns team_id
Co-authored-by: ishaan <ishaan@berri.ai>
* Add 2 debug logs for JWT+MCP troubleshooting
- handle_jwt.py: Log team route check result (team_id, route, is_allowed)
- user_api_key_auth_mcp.py: Log team_id when looking up MCP permissions
Co-authored-by: ishaan <ishaan@berri.ai>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
AsyncHTTPHandler.__del__ was closing httpx clients still in use by
AsyncOpenAI/AsyncAzureOpenAI due to independent cache lifecycles.
Restores standalone httpx client creation for OpenAI/Azure providers.
* fix fail-open for grayswan; pass metadata to cygnal api endpoint; update docs
* pass litellm_metadata to cygnal in payload
* switch error msg to const, and clean exception handling.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 4eece154d056ba33689a5584c86c8fc352bb7cdd.
* fix: check for model_response_choices before guardrail input
* test: add tests for responses api translation
* fix: protect other guardrail translations
* refactor: remove type ignores
* anthropic request body got mutated fix
* add warning when extra_body is provided but user is non premium
* fix: resolve mypy union-attr errors in anthropic guardrail handler
Cast choices[0] to Choices type before accessing .message attribute
to satisfy mypy's union type checking for Choices | StreamingChoices.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* add logger when model response has no choices for streaming /response and /messages
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 541a2b075a91b1b2d9efaf0407572f35bf5d4324.
* update pyproject.toml as requested
* Revert "update pyproject.toml as requested"
This reverts commit 716ea0caa1fee5e5f028d3f86479fedea2fac68b.
---------
Co-authored-by: Xiaohan Fu <xiaohan@grayswan.ai>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Adds litellm.proxy_auth to automatically obtain and refresh OAuth2/JWT
tokens when connecting to LiteLLM Proxy or any OAuth2-protected endpoint.
- Add ProxyAuthHandler for token lifecycle (obtain, cache, refresh)
- Add AzureADCredential wrapper for azure-identity credentials
- Add GenericOAuth2Credential for any OAuth2 provider (Okta, Auth0, etc)
- Auto-inject Authorization headers in completion() and embedding()
Closes#19834
Add the following SCIM v2 discovery endpoints per RFC 7643/7644:
- GET /scim/v2 - Base resource discovery (ListResponse of ResourceTypes)
- GET /scim/v2/ResourceTypes - List all supported resource types
- GET /scim/v2/ResourceTypes/{id} - Get a specific resource type (User/Group)
- GET /scim/v2/Schemas - List all supported schemas
- GET /scim/v2/Schemas/{uri} - Get a specific schema by URI
These endpoints are required by identity providers (Okta, Azure AD, etc.)
for SCIM resource discovery. Previously, GET /scim/v2 returned 404.
Also adds SCIMResourceType, SCIMSchema, and SCIMSchemaAttribute Pydantic
models to the SCIM types module.
Fixes#20295
* fix: strip stale mcp-session-id header to prevent 'Session not found' error loop
When VSCode reconnects to LiteLLM's MCP endpoint after a reload, it sends
a stale mcp-session-id header. The session was already cleaned up, causing
a 404 'Session not found' error. VSCode retries with the same stale ID,
creating an infinite error loop.
Before forwarding requests to the StreamableHTTP session manager, check if
the mcp-session-id header references a valid session. If the session doesn't
exist, strip the header so a new session is created automatically.
Fixes#20292
* refactor: extract stale session handling into _strip_stale_mcp_session_header helper
* perf: cache _get_relevant_args_to_use_for_logging() as module-level frozenset
The set of valid LLM API parameter names for logging was being rebuilt
on every request from 8 OpenAI SDK type annotations + set operations.
Since these are static TypedDict annotations that never change at
runtime, compute once at import time and store as a class-level
frozenset.
Line profiler: get_standard_logging_model_parameters() dropped from
774ms to 77ms across 12K calls (90% reduction, ~25µs/req saved).
* test: add tests for cached ModelParamHelper logging args
Verify cached frozenset matches dynamic computation and that
prompt content keys (messages, prompt, input) are excluded from
logged model parameters.
Pass-through endpoints (like vLLM classify) were not setting
standard_logging_object because _get_assembled_streaming_response
returns None for non-ModelResponse results.
This caused model_max_budget_limiter.async_log_success_event to raise
ValueError('standard_logging_payload is required').
The fix adds an elif branch in async_success_handler that mirrors the
non-pass-through code path.
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Add x-api-key header to CountTokens handler to match chat completion
authentication. Azure AI Anthropic requires this header per Microsoft's
native API format.
Bedrock rejects requests when toolResult or toolUse blocks within a
single message contain duplicate IDs. The Converse message transformer
merges consecutive tool/assistant messages without checking for
duplicate toolUseId values, causing BedrockException errors.
Add _deduplicate_bedrock_content_blocks() — a generalized helper that
removes duplicate blocks by ID, logs a warning for each dropped
duplicate via verbose_logger, and preserves non-tool blocks (e.g.
cachePoint). Apply it at all four merge sites (sync/async × toolResult/
toolUse).
The Anthropic /messages path was fixed in PR #19324; this applies the
equivalent fix to the Bedrock Converse path.
Fixes#20048
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Previously, when Model Armor guardrail blocked a request/response,
the `applied_guardrails` field was not populated in the logs because
`add_guardrail_to_applied_guardrails_header()` was called after the
HTTPException was raised.
This fix moves the `add_guardrail_to_applied_guardrails_header()` call
to before the blocking check in all hooks:
- async_pre_call_hook (pre_call mode)
- async_moderation_hook (during_call mode)
- async_post_call_success_hook (post_call mode)
- async_post_call_streaming_iterator_hook (streaming)
This ensures that even when a guardrail blocks content, the guardrail
name is properly recorded in the logs for observability.
Added regression tests to verify applied_guardrails is populated when
content is blocked.
Co-authored-by: Cursor <cursoragent@cursor.com>
When using LiteLLM's Anthropic /v1/messages endpoint to route requests to
OpenAI models, requests fail if any tool name exceeds OpenAI's 64-character
limit. Anthropic API has no such limit, causing compatibility issues.
Changes:
- Add truncate_tool_name() function using {55-char-prefix}_{8-char-hash} format
- Modify translate_anthropic_tools_to_openai() to truncate and return mapping
- Modify translate_anthropic_tool_choice_to_openai() to truncate tool name
- Restore original tool names in responses using the mapping
- Support tool name restoration in streaming responses
- Add backwards-compatible API (existing methods still work)
The fix only applies when routing Anthropic requests to OpenAI models.
Native Anthropic/Claude requests pass through unchanged.
* fix: models loadbalancing billing issue by filter (#18891)
* fix: models loadbalancing billing issue by filter
* fix: separate key and team access groups in metadata
* fix: lint issues
Fixes#19788
- Add `supported_regions: ["global"]` to Qwen MaaS models in model_prices_and_context_window.json
- Update `get_supported_regions()` to read directly from `model_cost` dict
- Update `get_complete_vertex_url()` to use `get_vertex_region()` for global-only models
- Update `create_vertex_url()` to generate correct URL for global location (without region prefix)
- Add tests for Qwen global endpoint support
Per review feedback, thought_signature should not be a root-level
param on ImageObject as it's not OpenAI compatible. Moved to
provider_specific_fields dict to match the pattern used in chat
completions (Message, Delta, Choices, etc).
Fixes#17184 - Gemini 3 Pro image preview model returns a thoughtSignature
field required for interactive image editing. This change:
- Adds thought_signature field to ImageObject class
- Updates Gemini and Vertex AI transformations to extract thoughtSignature
- Adds test for thought_signature in response transformation
