Per-user OAuth MCP requests now only skip pre-emptive 401 when a stored token is available, preserving token-reuse behavior while restoring fast PKCE kickoff for first-time or missing-token users.
`should_create_missing_views()` had `and result[0]["reltuples"]` which is
falsy when reltuples=0. On a fresh empty PostgreSQL table, CREATE INDEX sets
reltuples=0, causing the guard to return False and skip view creation entirely.
Views like MonthlyGlobalSpendPerKey are never created, and the
/global/spend/logs endpoint returns 500.
Fix: change to `and result[0]["reltuples"] is not None` so reltuples=0
(empty table) and reltuples=-1 (unanalyzed table) both correctly return True.
Also harden test_vertex_ai.py to return None instead of crashing with
JSONDecodeError when the spend-logs endpoint returns a non-JSON 500 response,
and add unit tests covering all three reltuples branches (0, -1, positive).
test_virtual_key_max_budget_alert_check_per_key_overrides_global asserted
override semantics but the implementation does additive merge. Renamed test
and updated assertion to match: per-key and global thresholds are unioned,
not replaced.
Project-level model rpm/tpm limits stored in project_metadata were never
checked during rate limit enforcement — only model-level limits applied.
Adds _add_project_model_rate_limit_descriptor_from_metadata() to the v3
limiter (mirrors the existing team metadata path) and calls it in
async_pre_call_hook, creating a model_per_project descriptor keyed as
"{project_id}:{model}" with the project's configured limits.
Also extends get_model_rate_limit_from_metadata's Literal to accept
"project_metadata" and adds get_project_model_rpm/tpm_limit helpers.
Fixes: LIT-2317
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Guard empty recipients in _handle_multi_threshold_max_budget_alert:
log warning and skip instead of falling through to old path error loop
- Widen max_budget_alert_emails type to Dict[str, Union[str, List[str]]]
to match _parse_email_list runtime behavior (accepts comma-separated strings)
- Pre-filter asyncio.create_task with min threshold check to avoid
unnecessary task allocation on every request when spend is below
all configured thresholds
When litellm.max_end_user_budget_id is configured, implicitly-created end users
(via /chat/completions) have budget_id=NULL in the DB since the default budget
is only applied in-memory. The budget reset job filtered by budget_id, so these
users were never reset and eventually permanently blocked.
Fix: when the default budget is in the reset list, also query for and reset
end users with budget_id=NULL and spend > 0. This keeps the hot auth path
unchanged (no DB writes on every request).
Fixes#22019
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add `default_key_max_budget_alert_emails` litellm_settings config as
global fallback for all virtual keys (per-key metadata takes priority)
- Fix crash when key has no user_id/user_email by passing recipient email
to _get_email_params (same pattern as team soft budget path)
- Use owner email for greeting, falling back to key_alias or token
- Rename setting from default_max_budget_alert_emails to
default_key_max_budget_alert_emails for clarity
Users can set metadata.max_budget_alert_emails as a JSON map of threshold
percentages to email recipients on virtual keys. When configured, the email
handler loops over each threshold, checks per-threshold dedup cache, and
sends to the configured recipients (auto-including the key owner's email).
When no map is set, the existing single 80% threshold behavior is preserved
unchanged. Teams support is out of scope for this v0.
- url_utils.py: narrow sockaddr[0] from str|int to str via a helper with a
fail-closed isinstance check. Fixes the two mypy errors introduced by
the SSRF hardening without masking unexpected stdlib behavior.
- key_management_endpoints.py: restore the documented team member_permissions
path for /key/update. The cross-key admin check added to close the
cross-org rewrite attack was over-broad: it rejected non-admin team
members even when can_team_member_execute_key_management_endpoint had
already validated their team membership and /key/update grant. Now skip
the admin check when the key has a team_id and the change is non-budget
(membership + permission already enforced above). Budget/spend changes
still require team/org admin. The cross-org attack remains blocked:
an outside org admin fails the earlier team membership check.
- test_logging_redaction_e2e_test.py: rename and rewrite two parametrized
tests to assert that request-body turn_off_message_logging has no effect.
Reflects the intentional removal of turn_off_message_logging from
_supported_callback_params so the caller cannot override admin logging
policy via the request body.
- test_key_management_endpoints.py: add two tests covering the restored
team member permission path — one positive (non-budget update succeeds
for a team member with /key/update grant), one negative (max_budget
change still rejected without admin role).
Compute supports_reasoning once per non-wildcard health-check resolution path and update the stale default-max-tokens test docstring.
Made-with: Cursor
Apply reasoning-first precedence for background health-check max tokens, parse reasoning env as optional, and raise non-wildcard fallback max_tokens from 1 to 5 for better reliability.
Made-with: Cursor
Audit-B #2. _check_key_admin_access was gated on max_budget/spend
changes only, which meant a non-admin caller could blanket-rewrite
any OTHER field on any key (key_alias, models, tpm_limit, rpm_limit,
metadata, tags, allowed_routes, guardrails, blocked, duration,
permissions, auto_rotate, access_group_ids, object_permission, …)
as long as they avoided budget/spend. Example attack:
POST /key/update {
key: sk-victim-in-org-B,
models: [],
blocked: true,
organization_id: org-A,
}
The caller is org-admin of org-A, which satisfies the route gate;
the handler then wipes models and blocks the victim's key.
Policy after this fix:
- PROXY_ADMIN: always allowed.
- Key OWNER (matching user_id): allowed for non-budget fields;
budget/spend changes still require team/org admin.
- Everyone else: must pass _check_key_admin_access (PROXY_ADMIN /
key-owner / team-admin / org-admin of the key).
Regression test confirms a non-owner INTERNAL_USER cannot rewrite
key_alias/blocked on someone else's key; existing test covers the
owner-can-update-alias case; existing test covers
internal-user-cannot-modify-max-budget.
Continuation of Veria E3NpkuAd / Audit-B hardening. All three are the
same anti-pattern PR #25904 already addressed for _user_is_org_admin
and /user/delete: route-level gate trusts a caller-supplied scope
field, handler operates on a different scope.
1. /user/update no longer silently creates a user when the target
email doesn't exist. Pre-fix, an org admin could supply a fresh
email + caller-chosen budget/models/metadata; the INSERT path
created the user with no org attachment, bypassing /user/new's
org/team authorization. Now require PROXY_ADMIN for the create
branch; return 404 otherwise. Also fixes /user/bulk_update because
it dispatches through the same _update_single_user_helper.
2. /team/bulk_member_add with all_users=true restricted to PROXY_ADMIN.
The flag pulls every user in the database into the target team,
ignoring org scope — any team admin could use it to capture every
user across every org into their team.
3. /team/update now verifies destination-org admin rights. When the
request carries an organization_id that differs from the team's
current org, an org admin of the caller's current org could
previously relocate the team into any other org (draining their
resources, or capturing a team they once administered). Require
PROXY_ADMIN or org-admin of the DESTINATION org for the relocation.
Regression tests for #1 and #3; #2 covered by the existing bulk_add
suite after the gate addition.
Greptile P2. The admin-inject gate only removed tags from data['metadata']
and data['litellm_metadata']; and the
policy engine read directly, so a caller without
allow_client_tags could still drive tag-based policy decisions by moving
tags to the body root. Also strip the root key in the same branch.
Greptile P1 on the /user/delete fix. Per CLAUDE.md 'No N+1 queries',
move the find_many inside the per-user loop to a single batched
fetch with {'user_id': {'in': data.user_ids}} before the loop, then
distribute to a per-user set in memory.
Veria admin-queue finding E3NpkuAd, Audit-B #1. The route-level gate
accepts this call when the caller is PROXY_ADMIN or ORG_ADMIN of any
org named in request_data["organization_id"]/["organizations"]. The
handler processes data.user_ids without cross-checking whether those
users belong to the caller's administered orgs, so an org-admin of
org-A could delete users in org-B via:
{"user_ids": ["victim_in_org_B"], "organization_id": "org-A"}
Add per-target authorization: org-admins may only delete users whose
entire org membership is within their admin scope; targets with any
org outside scope (or no org at all) require PROXY_ADMIN.
Regression test confirms an ORG_ADMIN call fails with 403 and no
cascade delete_many runs.
Post-merge audit found 6 adjacent variants of the VERIA-28 class. All
fixed here with regression tests:
1. Strip widened from 3 named keys to the full user_api_key_* prefix.
The proxy writes a dozen user_api_key_* fields (user_id, alias,
spend, team_id, request_route, end_user_id, …) into
data[_metadata_variable_name]; the 3-key strip left the rest
exploitable for identity/spend forgery in audit logs and guardrails.
2. proxy_server_request['body'] snapshot moved to AFTER the strip.
Was captured at line ~990 before the strip ran, so
standard_logging_object, lago, and spend_tracking readers saw the
attacker-forged payload even though the live data dict was clean.
3. get_tags_from_request_body (auth-time) now coerces JSON-string
metadata via safe_json_loads. Previously crashed with
AttributeError on string metadata (DoS; potential RBAC bypass if
a caller swallowed the exception).
4. get_end_user_id_from_request_body coerces JSON-string
metadata/litellm_metadata. Previously isinstance(dict) guard
caused end-user budget attribution to be silently skipped when
the caller sent metadata as a JSON string.
5. Four hand-rolled 'if data.get("metadata") is None: data["metadata"] = {}'
blocks in proxy_server.py (7160, 7341, 7590, 11375) now guard on
isinstance(dict). They crashed with TypeError when metadata was a
JSON string (DoS).
6. _get_admin_metadata defensively guards with isinstance(dict);
previously AttributeError'd on any leaked string metadata.
Also hoists the inline safe_json_loads import in _guardrail_modification_check
to module level per CLAUDE.md style.
Close three variant bypasses adjacent to VERIA-28 found during post-fix
variant audit:
1. _guardrail_modification_check had the same isinstance(dict) bypass
Veria-AI just flagged on the pre-call strip. A caller sending
`{"metadata": "{…}"}` as a JSON-encoded string (multipart/form-data
or extra_body) skipped the guard, got parsed to dict downstream, and
reached guardrail logic with bypass flags intact. Coerce strings via
safe_json_loads before evaluating.
2. The allow_client_tags strip only covered body metadata.tags and
litellm_metadata.tags — caller-supplied tags arriving via the
x-litellm-tags header or root-level data["tags"] bypassed it. Gate
add_request_tag_to_metadata's result on the same flag.
3. requester_metadata was deepcopied BEFORE the strip, so attacker
injections (user_api_key_metadata shadows, disallowed tags,
_pipeline_managed_guardrails) persisted in the snapshot. The PANW
guardrail (and any future consumer) trusting requester_metadata
would see forged values. Move the deepcopy to after the strip.
Regression tests added for each.
Per VERIA-28's secondary recommendation. The existing check only gated
metadata.guardrails. User-supplied values for disable_global_guardrails
(plural and the original singular typo variant) and opted_out_global_guardrails
are already silently ignored by _get_admin_metadata at read time, but the
silent-ignore makes diagnosis confusing and relies on one specific read
site catching them.
Reject at auth time with a 403 when any of:
- guardrails list (existing)
- disable_global_guardrails (new)
- disable_global_guardrail (new — historical singular-key variant)
- opted_out_global_guardrails (new)
are present in metadata, litellm_metadata, or at the request root, and the
caller's team lacks can_modify_guardrails. Defense in depth: the strip at
the pre-call layer still runs; this check fails loudly one layer earlier
so operators see an explicit 403 rather than a silent-ignore.
Veria AI caught a bypass: metadata can arrive as a JSON string via
multipart/form-data or extra_body, and the existing strip block ran
before the string-to-dict parse. The isinstance(_user_meta, dict)
guard returned False on the string, the strip was skipped, and then
the parse turned the string into a dict — leaving user_api_key_metadata
/ user_api_key_team_metadata / _pipeline_managed_guardrails / tags
intact in the parsed dict.
Move the strip to run AFTER the parse and BEFORE the merge of
litellm_metadata into data[_metadata_variable_name], closing the bypass
for both raw-dict and string-encoded payloads.
Regression test: test_add_litellm_data_to_request_strips_string_encoded_admin_injection.
- Move protected-headers set to module level as a frozenset
- Add x-api-key, x-goog-api-key to protected set (provider credential headers)
- Block x-amz- prefix to cover AWS SigV4 signing headers
- Normalize forwarded header names to lowercase on write
- Log at debug level when a protected header is skipped
- Add unit test covering protected-header drop and non-protected forwarding
Two pre-existing tests codified the pre-fix behavior where any caller-
supplied metadata.tags would flow through to spend logs and routing:
- test_add_key_or_team_level_spend_logs_metadata_to_request exercised
the request/key/team tag merge. Set allow_client_tags=True on the key
metadata so the merge path is still tested under the new regime.
- test_create_file_with_nested_litellm_metadata asserted that
litellm_metadata[tags] form-data propagated to the handler. Drop the
tag field; the test still proves nested form-parser correctness via
spend_logs_metadata and environment.
VERIA-28 (High) follow-up: tag-based routing and tag budget enforcement
read metadata.tags directly from the request, letting an attacker reach
restricted tag-routed deployments or misattribute spend to a victim
team's tag.
Strip metadata.tags (and litellm_metadata.tags) at the pre-call boundary
unless the caller's key or team metadata opts in with
allow_client_tags=True. Default-deny: existing clients that need to pass
routing tags must have the flag set explicitly on their key or team.
Preserves the tag-routing feature for admins who trust their callers;
closes the injection path for everyone else.
Expand the pre-call metadata strip to also remove user_api_key_metadata
and user_api_key_team_metadata. The proxy writes these fields into
data[_metadata_variable_name] with admin-authoritative values, but only
into that one metadata key; the caller's value in the OTHER metadata
key (metadata vs litellm_metadata) would otherwise persist and be
picked up by _get_admin_metadata, letting a caller supply their own
'admin' config to disable guardrails, opt out of global policies, etc.
VERIA-28 (High): Security Policy and Guardrail Bypass via Unsanitized
Request Metadata.
Add regression test at the proxy boundary verifying the strip, and
extend the guardrail test to cover the post-strip admin-config path.
Validate org admin role against all requested organizations instead
of returning on first match. Scope team list queries to the caller's
permitted organizations when filtering by user_id.
* feat(proxy): add NO_OPENAPI env var to disable /openapi.json endpoint (#25696)
* feat(proxy): add NO_OPENAPI env var to disable /openapi.json endpoint - Fixes#25538
* test(proxy): add tests for _get_openapi_url
---------
Co-authored-by: Progressive-engg <lov.kumari55@gmail.com>
* feat(prometheus): add api_provider label to spend metric (#25693)
* feat(prometheus): add api_provider label to spend metric
Add `api_provider` to `litellm_spend_metric` labels so users can
build Grafana dashboards that break down spend by cloud provider
(e.g. bedrock, anthropic, openai, azure, vertex_ai).
The `api_provider` label already exists in UserAPIKeyLabelValues and
is populated from `standard_logging_payload["custom_llm_provider"]`,
but was not included in the spend metric's label list.
* add api_provider to requests metric + add test
Address review feedback:
- Add api_provider to litellm_requests_metric too (same call-site as
spend metric, keeps label sets in sync)
- Add test_api_provider_in_spend_and_requests_metrics following the
existing pattern in test_prometheus_labels.py
* fix: ensure `litellm_metadata` is attached to `pre_call` guardrail to align with `post_call` guardrail (#25641)
* fix: ensure `litellm_metadata` is attached to pre_call to align with post_call
* refactor: remove unused BaseTranslation._ensure_litellm_metadata
* refactor: module level imports for ensure_litellm_metadata and CodeQL
* fix: update based off of Codex comment
* revert: undo usage of `_guardrail_litellm_metadata`
* feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite-preview (#25610)
* fix(bedrock): skip synthetic tool injection for json_object with no schema (#25740)
When response_format={"type": "json_object"} is sent without a JSON
schema, _create_json_tool_call_for_response_format builds a tool with an
empty schema (properties: {}). The model follows the empty schema and
returns {} instead of the actual JSON the caller asked for.
This patch:
- Skips synthetic json_tool_call injection when no schema is provided.
The model already returns JSON when the prompt asks for it.
- Fixes finish_reason: after _filter_json_mode_tools strips all
synthetic tool calls, finish_reason stays "tool_calls" instead of
"stop". Callers (like the OpenAI SDK) misinterpret this as a pending
tool invocation.
json_schema requests with an explicit schema are unchanged.
Co-authored-by: Claude <noreply@anthropic.com>
* fix(utils): allowed_openai_params must not forward unset params as None
`_apply_openai_param_overrides` iterated `allowed_openai_params` and
unconditionally wrote `optional_params[param] = non_default_params.pop(param, None)`
for each entry. If the caller listed a param name but did not actually
send that param in the request, the pop returned `None` and `None` was
still written to `optional_params`. The openai SDK then rejected it as
a top-level kwarg:
AsyncCompletions.create() got an unexpected keyword argument 'enable_thinking'
Reproducer (from #25697):
allowed_openai_params = ["chat_template_kwargs", "enable_thinking"]
body = {"chat_template_kwargs": {"enable_thinking": False}}
Here `enable_thinking` is only present nested inside
`chat_template_kwargs`, so the helper should forward
`chat_template_kwargs` and leave `enable_thinking` alone. Instead it
wrote `optional_params["enable_thinking"] = None`.
Fix: only forward a param if it was actually present in
`non_default_params`. Behavior is unchanged for the happy path (param
sent → still forwarded), and the explicit `None` leakage is gone.
Adds a regression test exercising the helper in isolation so the test
does not depend on any provider-specific `map_openai_params` plumbing.
Fixes#25697
---------
Co-authored-by: lovek629 <59618812+lovek629@users.noreply.github.com>
Co-authored-by: Progressive-engg <lov.kumari55@gmail.com>
Co-authored-by: Ori Kotek <ori.k@codium.ai>
Co-authored-by: Alexander Grattan <51346343+agrattan0820@users.noreply.github.com>
Co-authored-by: Mohana Siddhartha Chivukula <103447836+iamsiddhu3007@users.noreply.github.com>
Co-authored-by: Amiram Mizne <amiramm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Ryan Crabbe