Our `uv.lock` already resolves jinja2 to 3.1.6, so Docker / CI installs
get that version. The `pyproject.toml` floor was lagging at 3.1.0,
which means downstream consumers using `--resolution=lowest-direct` or
older constraint files can land on 3.1.0-3.1.5 instead of the version
we actually test against.
Aligns the declared floor with the resolved version so external
installers see the same baseline our test matrix exercises.
`uv lock` diff is metadata-only (no resolved-version drift).
* fix(mcp): forward extra_headers for OpenAPI MCP tools
OpenAPI-generated tools only applied static closure headers and BYOK
Authorization via ContextVar. Copy MCPServer.extra_headers from the
incoming MCP request into _request_extra_headers (set in server.py before
local tool dispatch), merge in openapi_to_mcp_generator via a small helper.
OAuth2 M2M: do not forward caller Authorization from raw_headers (same rule
as _prepare_mcp_server_headers for managed MCP).
Adds TestRequestExtraHeaders and clarifies mcp_server_manager registration
comment.
Fixes#26794
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(mcp): access has_client_credentials on MCPServer directly
Greptile: getattr default was redundant; property exists on MCPServer and
mcp_server is non-None inside the extra_headers forwarding block.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
The antd mock omits Typography, which caused all 15 tests in
create_key_button.test.tsx to fail with "No 'Typography' export is
defined on the 'antd' mock" after #27218 switched the key-type
dropdown labels to Typography.Text / Typography.Paragraph.
Add Typography (with .Text, .Paragraph, .Title subcomponents) to the
mock so the dropdown renders in the test environment.
The Create New Key dialog had a "Default" key-type option that was
misleading: it was not the actual default selection (AI APIs is), and it
grants access to all routes — broader than the description implied.
Rename the option to "Full Access" with an accurate description, and
reorder the dropdown to AI APIs → Management → Full Access. Switch the
inner labels to antd Typography for consistency with the rest of the UI.
UI-only change; the underlying enum value ("default") is unchanged so
the API contract is preserved.
The previous detection treated any model with input_cost_per_image
or output_cost_per_image as image generation. Several chat and
embedding models carry those fields to price multimodal vision input,
not generated images:
- gemini-3.1-pro-preview (mode=chat) has output_cost_per_image=0.00012
alongside input/output token pricing.
- azure/gpt-realtime-* (mode=chat) has input_cost_per_image=5e-6.
- amazon.titan-embed-image-v1 (mode=embedding) has
input_cost_per_image=6e-5.
For these models the image-gen branch fired first and reserved a
fraction of a cent per request, short-circuiting the token-priced
path entirely. Long Gemini chats reserved 1 × $0.00012 instead of
the true token cost.
Gate strictly on mode in {"image_generation", "image_edit"}. All 197
real image_generation entries and all 31 image_edit entries
(Flux Kontext, Stability inpaint/outpaint, etc.) carry the right mode,
so the field-presence fallback was unnecessary.
Adds regression tests for the chat-model-with-image-cost-field case
and for image_edit reservation.
Image-generation routes (dall-e-3, flux, etc.) have no per-token output
cost so they fell through to the no-reservation read-time-only path.
Concurrent image requests against a depleted budget could all pass
common_checks (counter exactly at max_budget passes the strict-`>`
gate) and reach the provider before reconciliation caught up.
Add per-image reservation in _estimate_request_max_cost_for_model:
when the model has a per-image cost field, reserve `n × cost_per_image`
upfront. The atomic counter increment serializes concurrent admissions,
so the second request sees the post-first-reservation counter and
raises BudgetExceededError instead of silently leaking through.
Both `output_cost_per_image` and `input_cost_per_image` are honored —
naming is inconsistent across providers (OpenAI dall-e-3 uses
input_cost_per_image, aiml/dall-e-3 uses output_cost_per_image for
the same per-generated-image price).
Per-pixel pricing (DALL-E 2 size variants) and TTS/STT routes still
fall through to read-time enforcement; those are follow-ups.
- Introduce RoutingPrismaWrapper that transparently routes read operations (find_*, count, group_by, query_raw, query_first) to a reader endpoint while writes remain on the writer, enabling Aurora-style reader/writer endpoint splits
- Add IAMEndpoint dataclass and parse_iam_endpoint_from_url() to capture static connection fields from a reader URL so only the IAM token needs to rotate, avoiding the need for separate DATABASE_HOST_READ_REPLICA/etc. env vars
- Enhance PrismaWrapper with per-instance knobs (db_url_env_var, iam_endpoint, recreate_uses_datasource, log_prefix) so writer and reader wrappers are independent: the reader writes its fresh URL to DATABASE_URL_READ_REPLICA and passes datasource override to Prisma since Prisma only auto-reads DATABASE_URL
- Fix deadlock in PrismaWrapper.__getattr__: when called from inside a running event loop, schedule the token refresh as a background task instead of blocking with run_coroutine_threadsafe + future.result(), which would deadlock the loop thread waiting for a coroutine that needs the loop to run
- Fix botocore crash when DATABASE_PORT is unset by defaulting to "5432" in both proxy_cli.py and PrismaWrapper.get_rds_iam_token(); passing None caused botocore to embed the literal string "None" in the presigned URL
- Implement graceful reader degradation: reader connect/recreate failures are non-fatal; wrapper sets _reader_unavailable=True and silently routes reads to the writer to keep the proxy serving traffic during transient reader outages
- Add PrismaClient.writer_db property so the reconnect smoke-test always validates the writer engine specifically; query_raw on the routing wrapper would route to the reader and not verify the newly-recreated writer
- Expose DATABASE_URL_READ_REPLICA in Helm chart (values.yaml + deployment.yaml) via both plain value and secret key reference, and document the field in docker-compose.yml
- Add 887-line test suite covering routing logic, IAM token refresh paths, reader degradation scenarios, datasource override behavior, and the deadlock regression
Co-authored-by: Yassin Kortam <yassinkortam@g.ucla.edu>
reserve_budget_for_request fell back to reserving the entire remaining
team/key/user headroom whenever a request omitted max_tokens, which
pinned the spend counter at max_budget for the duration of the
in-flight request and false-positive-blocked every concurrent or
back-to-back request until the success callback reconciled. Surfaced
as an integration-test team being budget-blocked at its $2000 cap
while DB spend was $0.144.
Switch the missing-max_tokens path to a fixed default of 16384 output
tokens (mirrors parallel_request_limiter_v3's DEFAULT_MAX_TOKENS_ESTIMATE
precedent), and clamp explicit max_tokens at the model's
max_output_tokens for reservation accounting only. The outbound request
body is unchanged, so providers see whatever the caller actually sent;
only the local integer used to compute reservation cost is bounded.
This also prevents a hostile max_tokens=999999999 from inflating one
request's reservation up to the entire team headroom.
For Opus 4.7 (output $25/M, max_output 128K) on a $2000 budget the
worst-case per-request reservation drops from "everything left" to
$3.20, raising admittable concurrency from 1 to ~625.
Operators upgrading past 35bbca60b0 (which made /metrics auth
default-on) see "Malformed API Key passed in. Ensure Key has 'Bearer '
prefix." with no hint that
litellm_settings.require_auth_for_metrics_endpoint: false restores the
previous unauthenticated behavior. Append that discovery hint to the
existing 401 body so a Prometheus scraper that breaks after upgrade
has a clear migration path. No behavior change.
* feat(sso): show full IdP claims in /sso/debug/callback
The debug callback only displayed the proxy-parsed OpenID summary, so
customers couldn't verify what custom claims (team_id, team_alias, roles,
etc.) the IdP was actually returning. Render two new sections — Raw
Claims (userinfo) and Access Token Claims (decoded JWT) — alongside the
existing parsed view. Strip bearer tokens defense-in-depth in case a
non-conforming IdP places them in its userinfo response.
Resolves LIT-2838
* Update litellm/proxy/management_endpoints/ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(sso): hoist json.dumps out of f-string for py3.10 ruff
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Adds a tri-state control (inherit / yes / no) when creating or editing
guardrails so admins can set litellm_params.skip_tool_message_in_guardrail
without YAML, mirroring the existing skip_system_message control.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the system-message skip in PR #25481 for tool-role messages.
Adds a global litellm.skip_tool_message_in_guardrail flag and a
per-guardrail litellm_params.skip_tool_message_in_guardrail override,
applied in the OpenAI and Anthropic chat translation handlers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore lineage between main and internal staging so the next
staging->main promotion (#27436) can merge without conflicts.
main was 2 commits ahead:
- 6ff668c7aa squash-merge of the previous staging->main promotion (#27245)
- 8c9830eef9 feat(xai): add grok-4.3 (#27396), already present on staging
The squash-merge has no shared lineage with the individual commits that
went into staging, which is why git surfaced 13 textual conflicts despite
both sides having the same logical content. Every conflicting file's
main-side change came from 6ff668c7aa only, and the matching staging-side
changes are the post-promotion evolution. Resolved all 13 with --ours
(staging's version is the latest evolution; main's snapshot is stale).
The grok-4.3 entries auto-merged in model_prices_and_context_window.json
and its backup, but were already on staging via an independent commit, so
the net diff vs HEAD is empty for those files.
Net new content from this merge: 12 lines added to
ui/litellm-dashboard/package-lock.json -- npm 11 libc array tags on four
existing entries, no functional impact.
Merge of cve-sweep-2026-05 into litellm_yj_may7 picked the older
npm@11.12.1 line, regressing the bump in f08b1b63fa that cleared
ip-address GHSA-v2v4-37r5-5v8g (npm@11.12.1 bundles ip-address@10.1.0;
11.14.0 bundles 10.1.1).
Google updated their Interactions OpenAPI spec
(https://ai.google.dev/static/api/interactions.openapi.json), removing
the readOnly 'outputs' property from CreateModelInteractionParams in
favor of 'steps' (a polymorphic transcript array). The compliance test
fetches the live spec, so it began failing on every PR once the spec
flipped over. Update the asserted output-field list to match.
Note: this only re-aligns the spec-shape assertion. Our SDK response
types (litellm/types/interactions/generated.py) still expose 'outputs'
and need to be regenerated separately to add 'steps'/Step variants and
decide on a back-compat path for callers reading .outputs.
The entry was added to cover the now-reverted black 24.10.0 -> 26.3.1
bump. With the bump dropped, upstream's existing liccheck setup is
unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Revert disk_cache.py JSONDisk swap + remove test_disk_cache.py. The
JSONDisk migration is backwards-incompatible (existing pickle caches
become unreadable; non-JSON values raise unguarded TypeError on set)
and warrants its own focused PR with a feature flag rather than riding
along with the CVE/dep-bump sweep.
CVE-2025-69872 remains unmitigated at the diskcache layer; users
concerned about pickle-RCE on cache-dir writers can avoid Cache(type="disk")
or pin a fork until upstream ships a fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Empirical grype scan of the built runtime image flagged
ip-address@10.1.0 (Medium) bundled inside /usr/local/lib/node_modules/npm.
npm@11.14.0 bundles ip-address@10.1.1 which carries the fix.
Verified by rebuilding the image and rescanning: ip-address finding gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pin the new digest published 2026-05-05:
sha256:31da6565... (from sha256:3258be47...).
Delta in baseline packages: zlib 1.3.2-r2 -> 1.3.2-r3. glibc stays at
2.43-r7 (still the latest available; whatever further glibc fixes for
CVE-2026-5450 / CVE-2026-5928 land in -r8+ from Chainguard, this PR
doesn't touch those).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR was blocked by .github/workflows/guard-fork-dependencies.yml: fork PRs
cannot modify uv.lock. Reverting:
- uv.lock + pyproject.toml black bump (24.10.0 -> 26.3.1) and the 295
files of mechanical Black 26 reformat coupled to it
- pyproject.toml diskcache extra change (kept the runtime mitigation in
litellm/caching/disk_cache.py via JSONDisk)
Kept:
- Dockerfile cache narrowing (drops ~660 MB of uv build cache that
surfaced cached setuptools as CVE findings)
- litellm/caching/disk_cache.py: dc.JSONDisk to neutralize CVE-2025-69872
- ui/litellm-dashboard/package-lock.json + litellm-js/spend-logs/package-lock.json:
next/postcss/hono/uuid CVE bumps (these are not blocked by the fork guard)
- tests/test_litellm/caching/test_disk_cache.py
- tests/code_coverage_tests/liccheck.ini: harmless black authorization
Black + gitpython + langchain dep upgrades will need a follow-up from a
maintainer pushing a branch in the canonical BerriAI/litellm repo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip VCR wiring from the batches test conftest. Drops:
- import of `_vcr_conftest_common` helpers
- the `vcr_config` fixture, `pytest_recording_configure`,
`_vcr_outcome_gate`, `pytest_runtest_makereport`
- the `apply_vcr_auto_marker_to_items` call in
`pytest_collection_modifyitems`
- `VerboseReporterState` / its `pytest_configure` /
`pytest_runtest_logreport` hooks (purely VCR-verdict plumbing)
Why: every test in this directory creates ephemeral OpenAI / Bedrock /
vLLM resources whose IDs change per run (file-XXX, batch-XXX,
ft-XXX, ...). VCR's path/query/body matchers don't match across runs,
so `record_mode="new_episodes"` was silently passing through to the
live API and recording many new cassette entries every run. Cassette
bloat without replay benefit.
Behaviour after this change is identical to running the directory
without `CASSETTE_REDIS_URL` set: tests that have keys hit live APIs,
tests that don't continue to skip via their existing skipif markers.
Conftest now keeps only path setup and the session-scoped `event_loop`
fixture.
OpenAI announced gpt-3.5-turbo-0125 (and fine-tuning of gpt-3.5-turbo
in general) for shutdown on 2026-10-23, with the announcement landing
2026-04-22. The hard-fail date is ~5 months out, but timing fits the
recent uptick in this test flaking and OpenAI may already be running
the deprecated model's pipeline with deprioritized infra.
Bump to gpt-4o-mini-2024-07-18 — currently supported for fine-tuning,
no announced shutdown. Updates the live test plus the mocked test for
consistency. Belt-and-suspenders with the existing propagation-retry
helper.
Previous fix polled `litellm.afile_retrieve` for `status == "processed"`
before calling the fine-tuning endpoint. That doesn't actually solve
the race:
- OpenAI's `FileObject.status` field is deprecated per the SDK type and
not authoritative — it can read "processed" before the file is usable.
- The retrieve and fine-tuning endpoints don't share a consistency
model, so retrieve succeeding tells you nothing about FT visibility.
Replace with a retry around the actual `acreate_fine_tuning_job` call
that catches the OpenAI 400 `'file-... does not exist'` and backs off
exponentially (1s → cap 8s, 12 attempts, ~70s total budget). The
operation succeeding is the only reliable signal that propagation
finished.
Yuneng Jiang