Empty system messages were skipped for Anthropic's system param but
not removed from the messages list, causing BadRequestError when
anthropic_messages_pt encountered the unsupported "system" role.
Fixes#21622
The admin bulk-delete path (prisma_client.delete_data) was never comparing
the DB return value against the requested tokens, so failed_tokens was
always empty for admins even when the DB silently skipped some tokens.
Adds the same mismatch check as the non-admin path, and a new test that
exercises admin bulk-delete returning fewer tokens than requested.
Fixes gap identified by Greptile review on #21609.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of raising an exception when some requested tokens are not found
in the DB, populate `failed_tokens` in the return dict so callers can
inspect which keys were not deleted without catching a generic Exception.
Supersedes #12577.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix aviation safety topic filter: remove overly broad exceptions, add cockpit access block words
* fix airline brand protection filter: add identifier words, competitor/ops block words, tighten exceptions
* add example_sentences to all policy templates + topic-filtering and prompt-injection templates
* add policy_endpoints package with AI policy suggester
* update test patch targets for policy_endpoints package move
* add unit tests for AI policy suggester
* add suggestPolicyTemplates networking function
* add AI suggestion modal component
* add Use AI button and template loading callback to PolicyTemplates
* wire up AI suggestion modal in policies page
* fix policy_templates_backup.json path after package move
* add estimated_latency field to all policy templates
* use llm_router and accept model parameter in ai_policy_suggester
* add model param to suggest templates endpoint
* pass model param in suggestPolicyTemplates
* polish ai suggestion modal: model selector, auto-growing textareas, latency badges
* add template queue for processing multiple AI-suggested templates
* show template progress badge in guardrail selection modal
AgentCore MCP server endpoints require the Accept header to contain
both application/json and text/event-stream per the MCP specification
(Streamable HTTP transport). Without this header, requests are rejected
with a 406 Not Acceptable error (JSON-RPC code -32011).
Sets the Accept header at the top of sign_request() so both JWT/Bearer
and SigV4 authentication paths include it.
* fix(scim): handle deprovisioning operations without path field
When SCIM providers send deprovisioning requests without a path field
(e.g., {"op": "replace", "value": {"active": false}}), the code was
storing the value under an empty string key in metadata.
This fix:
- Detects operations with no path where value is a dict
- Extracts and handles known fields like 'active' correctly
- Sets metadata["scim_active"] = false instead of metadata[""] = {"active": false}
Fixes: SCIM deprovisioning creating empty string keys in user metadata
* fix(scim): handle all known fields in operations without path
Extended the fix to handle all SCIM fields (not just active) when
operations have no path field:
- active -> scim_active
- displayName -> user_alias
- externalId -> user_id
- name.givenName/familyName -> scim_metadata
Added comprehensive test for multiple fields without path.
Addresses Greptile review feedback on RFC 7644 compliance.
* trigger PR update
For multi-turn conversations, convert thinking_blocks on assistant
messages into content blocks prepended before the rest of the content,
so reasoning context is passed back to the hosted_vllm API.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Virtual keys only track created_at and updated_at, which don't indicate
when a key was last used. This adds a last_active field that gets updated
during the async batch spend update, giving admins visibility into which
keys are actively being used.
Changes:
- Add last_active DateTime? to VerificationToken and
DeletedVerificationToken in all 3 schema files and Python types
- Set last_active in the batch key spend update alongside spend increment
- Add Last Active column to virtual keys UI table with info popover
and hover tooltip showing full date/time with timezone
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix aviation safety topic filter: remove overly broad exceptions, add cockpit access block words
* fix airline brand protection filter: add identifier words, competitor/ops block words, tighten exceptions
* update policy templates with competitor pre/post guardrails and streaming enrichment
* sync policy_templates_backup.json with policy_templates.json
* add streaming enrichment endpoint, competitor variations, and model selection
* add streaming enrichPolicyTemplate networking function
* rewrite template parameter modal with streaming tags, AI/manual toggle, model selector
* update policies panel to pass enrichment options and show discovered competitors
* show discovered competitors as tags in guardrail selection modal
* use llm_router instead of litellm.acompletion, extract helpers, move constants
* validate competitors list size, cap variation prompt input
* add refinement instruction support for competitor discovery
* add instruction and existingCompetitors params to streaming enrichment
* add refinement input for iterating on competitor list with AI
* emit status events during variation generation so UI shows progress
* add onStatus callback to streaming enrichment
* show status spinner during variation generation, widen modal to 700px
* add tests for competitor enrichment helper functions
Add a new team member permission `/team/daily/activity` that allows
non-admin team members to see all team usage data, not just their own.
## Changes
- Add `TEAM_DAILY_ACTIVITY` to `KeyManagementRoutes` enum and available
team member permissions
- Add `_team_member_has_permission` helper in `common_utils.py`
- Modify `/team/daily/activity` endpoint to skip API key filtering when
the member has this permission
- Add permission description and method detection in the UI
- Add backend and frontend tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Four tests were broken by commit e00c181f0c (Mcp user permissions #21462):
1. test_list_tools_single_server_unprefixed_names: The commit changed
_get_tools_from_mcp_servers to always add server prefixes (add_prefix=True),
removing the conditional that skipped prefixing for single servers.
Updated assertion from "toolA" → "zapier-toolA".
2. test_mcp_get_prompt_success: mcp_get_prompt now extracts the server name
from a prefixed prompt name via split_server_prefix_from_name(). Passing
unprefixed "hello" returns server_name="" which matches no server → 403.
Updated call to use "server_a-hello" so the server lookup succeeds.
3. test_e2e_jwt_team_mcp_permissions_enforced &
4. test_e2e_jwt_team_mcp_key_intersection:
The commit replaced `from typing import List` with
`from litellm.proxy.proxy_server import general_settings` in
MCPRequestHandler.get_allowed_mcp_servers(). Both tests mock
litellm.proxy.proxy_server with a types.ModuleType that lacked
general_settings, causing ImportError. Added general_settings={} to
both mock modules.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_complete_reload_flow and test_distributed_reload_check_function both
trigger code paths that assign a minimal stub dict to litellm.model_cost
(via the /reload/model_cost_map endpoint and _check_and_reload_model_cost_map).
Without restoring, subsequent tests in the same worker can't find gpt-4o
pricing and calculate spend=0.0 instead of the expected value.
Added try/finally save-and-restore of litellm.model_cost in both tests,
matching the pattern used in test_reload_model_cost_map_admin_access.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(schema.prisma): add object permissions for end users
allows controlling if end user can call specific mcp servers
* feat: cleanup for customer_endpoints support of object permission id
* fix: cleanup str
* feat(customers/): enforce end user can only call allowed mcps - if configured
* docs: document customer/end user object permission usage
* feat: enforce end user permissions on MCP tool calls
This commit implements end user permission enforcement for MCP servers:
1. Always add server prefixes to MCP tool names
- Removed conditional logic that only added prefixes when multiple servers existed
- Now always adds server prefix for consistent tool naming across all scenarios
- Updated 5 locations in server.py (list_tools, get_prompts, get_resources,
get_resource_templates, get_prompt)
2. Created MCP End User Permission Guardrail Hook
- New guardrail hook: litellm/proxy/guardrails/guardrail_hooks/mcp_end_user_permission.py
- Runs on post_call to validate tool calls in LLM responses
- Extracts MCP server name from tool names (splits on first '-')
- Checks if end_user_id has permissions for the MCP server
- Raises GuardrailRaisedException if end user lacks permission
- Supports both streaming and non-streaming responses
3. Added comprehensive tests
- Test file: tests/test_litellm/proxy/guardrails/guardrail_hooks/test_mcp_end_user_permission.py
- Tests cover: authorized/unauthorized tools, non-MCP tools, no end_user scenarios
- Tests permission checking logic and exception raising
The hook integrates with the existing MCPRequestHandler._get_allowed_mcp_servers_for_end_user
to fetch end user permissions and enforce access control at the response level.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* refactor: remove redundant add_prefix variable assignments
Simplified the code by removing intermediate `add_prefix` variable
assignments and passing `True` directly to function calls since
we now always add server prefixes.
Changes:
- Removed `add_prefix = True` variable assignments in 5 locations
- Changed `add_prefix=add_prefix` to `add_prefix=True` in function calls
- Added inline comments to clarify the behavior
This makes the code more concise and clearer in intent.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(auth_utils.py): support safety_identifier as a valid way of passing the end user id for responses api
* feat(llms): ensure 'tools' is correctly updated for responses api
* fix: fix greptile feedback
* feat: transformation.py
proper responses api tool handling for guardrail translation layer
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(ui/): initial commit adding a compliance testing playground
allow proxy admins to test policies and guardrails against datasets
* feat(ui/): make score more friendly
* feat(policy_endpoints.py): new helper function for testing policies
* feat(policy_endpoints.py): expose new endpoint for testing policies and guardrails
enables compliance playground to work as expected
* feat(complianceui.tsx): show returned text
* docs: add DATABRICKS_API_KEY to environment settings reference
* fix: streaming test usage check on Pydantic model
* fix: mock litellm.proxy.proxy_server in test_skip_server_startup
Users had to set store_model_in_db in the config YAML and restart the proxy,
causing service downtime. This change allows the value to be written to the
LiteLLM_Config table and read from the database at runtime, with DB values
overriding config file values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Chesars