* fix(guardrails): apply team-level guardrails alongside global policy guardrails
Two bugs prevented team-direct guardrails from being automatically applied
when using a team-scoped API key:
1. Auth caching: `valid_token.team_metadata` was never refreshed from the
freshly-fetched team object at the "Check 6" step in
`_user_api_key_auth_builder`. Guardrails added to a team after the key
was first cached were therefore invisible to `move_guardrails_to_metadata`.
Fix: propagate `_team_obj.metadata` → `valid_token.team_metadata` after
every "Check 6" team fetch (user_api_key_auth.py).
2. Guardrail execution: `get_guardrail_from_metadata` checked
`data["litellm_metadata"]` before `data["metadata"]`. When a request
carried a non-empty `litellm_metadata` without a "guardrails" key, the
merged guardrail list written to `data["metadata"]` by
`move_guardrails_to_metadata` was shadowed and the guardrail received an
empty requested-guardrails list (custom_guardrail.py).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix merge conflict
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep anthropic_messages as the logged call_type for non-Anthropic /v1/messages adapter paths and add a regression test to prevent fallback to completion/acompletion.
Made-with: Cursor
* refactor: new agentic loop event hook
simplifies how to create logic for tool based multi llm calls
* fix: compress - make it work on anthropic input as well
* fix(compress.py): working prompt compression for claude code
ensures claude code messages can run through proxy easily
* docs: add agentic loop hook guide
* docs: add agentic_loop_hook to sidebar
* fix: fix multiple arguments error
* fix: fix tool call loop for compression on streaming /v1/messages
* fix: fix linting errors
* fix: fix ci/cd errors
* feat(litellm_pre_call_utils.py): use claude code session for litellm session id
allows claude code logs to be stitched together, making it easy to know they were all part of the same conversation
* fix: suppress incorrect mypy warning rE: module
* revert: drop PR's changes to litellm/proxy/_experimental/out/
Restores the 34 HTML files under _experimental/out/ to their pre-PR
paths (X/index.html -> X.html). All renames are R100 (content
unchanged); no other files are touched.
* fix: address greptile review comments on PR #25729
- Skip ``kwargs["tools"] = []`` injection when compression is a no-op —
Anthropic Messages rejects empty tool arrays on requests that did not
originally declare tools.
- Move agentic-loop safety guards (fingerprint cycle / max depth) out of
the per-callback try/except so they propagate instead of being swallowed
by the generic exception handler. Extracted _check_agentic_loop_safety.
- Gate generic ``x-<vendor>-session-id`` capture behind the
LITELLM_CAPTURE_VENDOR_SESSION_HEADERS env var (off by default) to
preserve backwards compatibility; explicit x-litellm-* headers are
unaffected.
- Fix monkeypatch target in pre-call-hook test to patch the actual
module-level binding
(litellm.integrations.compression_interception.handler.compress).
- Add regression tests for empty-tools skip and opt-in session capture.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: drop LITELLM_CAPTURE_VENDOR_SESSION_HEADERS flag
Generic x-<vendor>-session-id header capture is a new feature and only
runs *after* the explicit x-litellm-trace-id / x-litellm-session-id
checks, so it does not change behavior for any existing caller that was
already using the LiteLLM headers — no backwards-incompatibility to gate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(compress): replace input_type with CallTypes call_type
Drop the bespoke ``CompressionInputType`` literal and use the existing
``litellm.types.utils.CallTypes`` enum instead. ``litellm.compress()``
now takes ``call_type: Union[CallTypes, str]`` (default
``CallTypes.completion``) — no new concept to learn, and the enum is
already the way the rest of the codebase talks about request shapes.
Supported values: ``completion`` / ``acompletion`` (OpenAI chat-completions
shape) and ``anthropic_messages`` (Anthropic structured content blocks).
Updated: compress(), the compression_interception handler, tests, docs,
and the two eval scripts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Greptile P2. The admin-inject gate only removed tags from data['metadata']
and data['litellm_metadata']; and the
policy engine read directly, so a caller without
allow_client_tags could still drive tag-based policy decisions by moving
tags to the body root. Also strip the root key in the same branch.
Post-merge audit found 6 adjacent variants of the VERIA-28 class. All
fixed here with regression tests:
1. Strip widened from 3 named keys to the full user_api_key_* prefix.
The proxy writes a dozen user_api_key_* fields (user_id, alias,
spend, team_id, request_route, end_user_id, …) into
data[_metadata_variable_name]; the 3-key strip left the rest
exploitable for identity/spend forgery in audit logs and guardrails.
2. proxy_server_request['body'] snapshot moved to AFTER the strip.
Was captured at line ~990 before the strip ran, so
standard_logging_object, lago, and spend_tracking readers saw the
attacker-forged payload even though the live data dict was clean.
3. get_tags_from_request_body (auth-time) now coerces JSON-string
metadata via safe_json_loads. Previously crashed with
AttributeError on string metadata (DoS; potential RBAC bypass if
a caller swallowed the exception).
4. get_end_user_id_from_request_body coerces JSON-string
metadata/litellm_metadata. Previously isinstance(dict) guard
caused end-user budget attribution to be silently skipped when
the caller sent metadata as a JSON string.
5. Four hand-rolled 'if data.get("metadata") is None: data["metadata"] = {}'
blocks in proxy_server.py (7160, 7341, 7590, 11375) now guard on
isinstance(dict). They crashed with TypeError when metadata was a
JSON string (DoS).
6. _get_admin_metadata defensively guards with isinstance(dict);
previously AttributeError'd on any leaked string metadata.
Also hoists the inline safe_json_loads import in _guardrail_modification_check
to module level per CLAUDE.md style.
Close three variant bypasses adjacent to VERIA-28 found during post-fix
variant audit:
1. _guardrail_modification_check had the same isinstance(dict) bypass
Veria-AI just flagged on the pre-call strip. A caller sending
`{"metadata": "{…}"}` as a JSON-encoded string (multipart/form-data
or extra_body) skipped the guard, got parsed to dict downstream, and
reached guardrail logic with bypass flags intact. Coerce strings via
safe_json_loads before evaluating.
2. The allow_client_tags strip only covered body metadata.tags and
litellm_metadata.tags — caller-supplied tags arriving via the
x-litellm-tags header or root-level data["tags"] bypassed it. Gate
add_request_tag_to_metadata's result on the same flag.
3. requester_metadata was deepcopied BEFORE the strip, so attacker
injections (user_api_key_metadata shadows, disallowed tags,
_pipeline_managed_guardrails) persisted in the snapshot. The PANW
guardrail (and any future consumer) trusting requester_metadata
would see forged values. Move the deepcopy to after the strip.
Regression tests added for each.
Veria AI caught a bypass: metadata can arrive as a JSON string via
multipart/form-data or extra_body, and the existing strip block ran
before the string-to-dict parse. The isinstance(_user_meta, dict)
guard returned False on the string, the strip was skipped, and then
the parse turned the string into a dict — leaving user_api_key_metadata
/ user_api_key_team_metadata / _pipeline_managed_guardrails / tags
intact in the parsed dict.
Move the strip to run AFTER the parse and BEFORE the merge of
litellm_metadata into data[_metadata_variable_name], closing the bypass
for both raw-dict and string-encoded payloads.
Regression test: test_add_litellm_data_to_request_strips_string_encoded_admin_injection.
VERIA-28 (High) follow-up: tag-based routing and tag budget enforcement
read metadata.tags directly from the request, letting an attacker reach
restricted tag-routed deployments or misattribute spend to a victim
team's tag.
Strip metadata.tags (and litellm_metadata.tags) at the pre-call boundary
unless the caller's key or team metadata opts in with
allow_client_tags=True. Default-deny: existing clients that need to pass
routing tags must have the flag set explicitly on their key or team.
Preserves the tag-routing feature for admins who trust their callers;
closes the injection path for everyone else.
Expand the pre-call metadata strip to also remove user_api_key_metadata
and user_api_key_team_metadata. The proxy writes these fields into
data[_metadata_variable_name] with admin-authoritative values, but only
into that one metadata key; the caller's value in the OTHER metadata
key (metadata vs litellm_metadata) would otherwise persist and be
picked up by _get_admin_metadata, letting a caller supply their own
'admin' config to disable guardrails, opt out of global policies, etc.
VERIA-28 (High): Security Policy and Guardrail Bypass via Unsanitized
Request Metadata.
Add regression test at the proxy boundary verifying the strip, and
extend the guardrail test to cover the post-strip admin-config path.
* style(ui/): distinguish agent calls from llm calls on ui
* feat: initial grouping working
* feat: set stable contextid for a2a calls - allows for easily passing to downstream llm/mcp calls
* feat(a2a_endpoints.py): fix tracing to avoid recreating logging objects for the same call
allows stable trace id usage
* fix(guardrail_endpoints): handle string ui_type values in _build_field_dict
_build_field_dict unconditionally called .value on ui_type, which crashes
for guardrail configs that use plain strings (e.g. BlockCodeExecutionGuardrailConfigModel
uses "multiselect" and "percentage"). Now checks with hasattr before calling .value.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: propagate trace/session id from headers in MCP server calls
Cherry-picked mcp_server/server.py fixes from 6feb9bab: adds
get_chain_id_from_headers to extract x-litellm-trace-id /
x-litellm-session-id from raw headers, and uses it in call_tool
and list_tools to keep spend logs and tracing consistent with A2A.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Fix plaintext JWTs leaking in debug logs
Wrap raw request headers in RedactedDict (dict subclass with redacted
str/repr) at the single entry point where they enter the system. This
prevents any downstream logging path from exposing Bearer tokens.
Also remove a redundant log that re-read request.headers directly,
bypassing the already-cleaned _headers variable.
* Add e2e test for JWT redaction in debug logs
* Preserve RedactedDict type through copy()
- test_litellm_pre_call_utils.py: wrap test body in try/finally so
litellm.callbacks is always restored even when an assertion fails,
addressing greptile review comment
- test_langfuse_otel.py: resolve trivial merge conflict in comment
("unpatched" vs "unpatch-ed"), keeping correct spelling
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three test isolation issues fixed:
1. test_mcp_debug.py: Replace deprecated asyncio.get_event_loop().run_until_complete()
with asyncio.run() in TestWrapSendWithDebugHeaders. In Python 3.10+,
get_event_loop() raises RuntimeError when no event loop is set in the
current thread, causing test_injects_headers and test_body_messages_unchanged
to fail in isolation.
2. test_mcp_server_manager.py: After _reload_mcp_manager_module() creates a new
global_mcp_server_manager instance, server.py still holds a stale reference
to the old instance. Tests in test_mcp_server.py that populate the new
manager's registry and then call server.py functions (e.g. _get_tools_from_mcp_servers)
get empty results because server.py reads from the old manager. Fix: update
server.py's module-level reference after each reload.
3. test_litellm_pre_call_utils.py: test_add_litellm_metadata_from_request_headers
sets litellm.callbacks without restoring it afterward. Add cleanup to restore
original callbacks after the test to prevent state leaking to subsequent tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Reload litellm_pre_call_utils module inside test to get fresh litellm reference
- Use string-based patch("litellm.model_group_settings") instead of patch.object
- These changes ensure the patch targets the correct module after conftest reloads litellm
Fix several tests that fail in CI due to parallel test execution and
module reloading in conftest.py.
1. test_empty_assistant_message_handling:
- Use patch.object on factory_module.litellm instead of direct assignment
- Ensures the correct litellm reference is modified after conftest reloads
2. test_embedding_header_forwarding_with_model_group:
- Use patch.object on pre_call_utils_module.litellm instead of direct assignment
- Same fix for module reloading issue
3. test_embedding_input_array_of_tokens:
- Move mock inside test function (after fixture initializes router)
- Add skip condition if llm_router is None
- Fixes "AttributeError: None does not have 'aembedding'" in parallel execution
Root cause: conftest.py reloads litellm at module scope, which can cause:
- Different litellm references between test code and library code
- Global state (like llm_router) being None at decorator execution time
- isinstance checks failing due to class identity mismatches
Cache request.url.path once instead of accessing it 6 times (2 in
assistants check + 4 in LITELLM_METADATA_ROUTES loop). Reduces
per-call time from 47µs to 20µs (-58%).
Also inline the assistants API check to avoid function call overhead.
Adds 8 tests for _get_metadata_variable_name covering all return paths.
* Draft commit.
* user header mapping feature with backward compatibility with user_header_name field.
* user header mapping feature with backward compatibility with user_header_name field optimizations.
* Added unit tests.
* feat(litellm_pre_call_utils.py): add num_retries to litellm data for backend call
allow user to pass in num retries via request headers
* test(test_litellm_pre_call_utils.py): add unit test
* docs(request_headers.md): document new request header
* fix(common_daily_activity.py): show spend breakdown by model group
Partial fix for https://github.com/BerriAI/litellm/issues/12887
* feat(new_usage.tsx): new tab switcher for viewing usage by model group vs. received model
Closes https://github.com/BerriAI/litellm/issues/12887
* fix(litellm_pre_call_utils.py): add user agent tags to spend logs in standard logging payload logic
avoid clash when tag based routing is enabled
* test: remove redundant test
* test: rename oidc test to run earlier
quicker debuging
* fix(azure.py): return more detailed error message
* fix(azure/common_utils.py): use default scope, if scope is none
fixes oidc test
* fix: always default to cognitiveservices.azure.com
* test: update test
* feat(ui_sso.py): allow admin to specify additional headers for sso provider
some sso providers require special headers to return a json response
* test(test_ui_sso.py): add unit tests to ensure custom headers are respect3ed
* docs(config_settings.md): document new header param
* fix(litellm_pre_call_utils.py): add spend tag tracking by user agent
allows checking spend for cli tools like claude code
* feat(litellm_pre_call_utils.py): track spend by user agent part if user agent contains "/"
allows tracking spend across user agent versions
Better cost tracking for claude cod
* test(test_litellm_pre_call_utils.py): add testing for pre call utils, user agent parsing
* fix: fix linting check
