* feat(otel): expose http.response.status_code on failure spans
Set the OTel-standard http.response.status_code (integer) on failure
spans alongside the existing OpenInference error.code (kept for
back-compat). error.type is already emitted via ERROR_TYPE.
Crucially, also record structured error attributes on the proxy SERVER
span ('Received Proxy Server Request') from async_post_call_failure_hook
- the only place the SERVER span is in hand. _handle_failure records on
the litellm_request child span (the parent span is not propagated into
its kwargs), so prior to this change the SERVER span that dashboards
query carried only span status, never error.code/error.type. Reuses
_record_exception_on_span + StandardLoggingPayloadSetup.get_error_information
so values match the child span.
Tests: recorder unit coverage + a hook-driven test asserting the SERVER
span is stamped (the gap recorder-only tests missed). Full
test_opentelemetry.py suite: 197 passed.
* feat(otel): set http.route + url.path on the proxy SERVER span
Add the OTel-standard http.route (low-cardinality route template, e.g.
/v1/threads/{thread_id}/runs) and url.path (literal path) to the SERVER
span ('Received Proxy Server Request') so dashboards can group traffic
by endpoint instead of seeing every path param as a unique value.
Same architectural gap as the status-code commit: the success/failure
logging handlers write the litellm_request CHILD span, and
_handle_success explicitly refuses to copy to the SERVER span. Verified
with a console-exporter run that the SERVER span was bare on success.
Unlike error info, route/path are known at request time, so set them
directly on the freshly-created SERVER span in user_api_key_auth (one
edit point, works for success and failure, no hook-ordering risk):
- http.route from the matched FastAPI route (scope['route'].path),
empirically confirmed populated at auth-dependency time.
- url.path from the existing literal-path variable.
New get_request_route_template helper + set_proxy_request_route_attributes
(no-op on None span, so the Langfuse override stays safe).
Tests: route-attribute setter + route-template helper edges. Full
test_opentelemetry.py and test_auth_utils.py green.
* feat(otel): set litellm.preprocessing.duration_ms on the proxy SERVER span
Expose the total time LiteLLM spends before the upstream provider
request begins (auth + parsing + pre-call hooks) as a single number on
the SERVER span ('Received Proxy Server Request'). Window:
proxy-receive -> FIRST provider handoff.
Retry semantics: first attempt only (pure preprocessing, excludes
retry loops + backoff). api_call_start_time is overwritten on every
attempt, so a set-once first_api_call_start_time pins the first handoff.
Same architectural gap as the prior two commits: the success/failure
logging handlers write the litellm_request CHILD span, not the SERVER
span. Set it instead from the post-call hooks on
user_api_key_dict.parent_otel_span.
Failure-path subtlety: request_data.pop('litellm_logging_obj') runs
before the failure-hook loop, so the failure hook can't read the
logging object. litellm_received_at is propagated via the existing
request->metadata channel, and first_api_call_start_time is mirrored
onto litellm_params.metadata, so both anchors survive into request_data
and the OTel helper reads them uniformly for success and failure.
Edits: user_api_key_auth (stash receive instant), litellm_pre_call_utils
(propagate it), litellm_logging (set-once first handoff + metadata
mirror), opentelemetry (constant + set_preprocessing_duration_attribute,
called from both post-call hooks).
Tests: duration helper (both container shapes, missing/negative/None
edges) + set-once invariant (retry doesn't overwrite, metadata mirror).
test_opentelemetry.py + test_auth_utils.py + test_litellm_logging.py:
447 passed. Verified live: SERVER span carries the attribute on success
and failure, coexisting with the status-code and route attributes.
* fix(otel): MyPy type-narrowing for status-code + preprocessing-duration
No behavior change. MyPy (CI lint) flagged:
- error_information["error_code"] is str|None: narrow via a None-checked
local before int().
- _to_timestamp returns Optional[float]: resolve both anchors and return
early if either is None instead of subtracting possibly-None floats.
* fix(otel): stop polluting user request metadata with first_api_call_start_time
The PR3 set-once preprocessing anchor was mirrored into
litellm_params["metadata"] from core litellm_logging.py. That dict is
the caller's request metadata, mutated in place and shared across every
call path including pure SDK (litellm.acreate_batch). It got echoed into
LiteLLMBatch(metadata=...), which the OpenAI batch schema types as
Dict[str, str] -> pydantic ValidationError on a datetime value.
- litellm_logging.py: set first_api_call_start_time only on
model_call_details (success path reads it there directly).
- proxy/utils.py: post_call_failure_hook lifts it off the logging object
into request_data (internal top-level key, same convention as the
other proxy-internal request_data keys) right before the existing
litellm_logging_obj pop. Never touches user metadata.
- opentelemetry.py: read the anchor from the container top level
(model_call_details on success, request_data on failure).
- Tests updated; add TestPostCallFailureHookLiftsFirstApiCallStartTime.
Fixes the batches_testing regression introduced on this branch.
* chore(otel): trim verbose comments to concise rationale
Collapse multi-line why-blocks to one or two lines and drop process/plan references (PR-numbering, "the plan") from test comments. No behavior change.
* fix(bedrock-converse): drop blank-text fallback for empty thinking blocks
Claude Code with extended thinking replays prior assistant turns that
include an empty thinking block (`thinking=""`, `signature=""`) alongside
tool_use blocks. The unsigned-reasoning fallback in
`add_thinking_blocks_to_assistant_content` was emitting
`BedrockContentBlock(text="")`, which Bedrock Converse rejects with:
"The text field in the ContentBlock object at messages.X.content.0
is blank."
Guard the fallback with a strip() check, matching the existing
empty-text guards elsewhere in `_bedrock_converse_messages_pt`.
* style: remove unneeded comments
* chore: reject bare str at file-input sinks to prevent local-file read (#27667)
Squash-merged by litellm-agent from stuxf's PR.
* fix: use os.PathLike in ocr sink and check truthy reasoningSummary for bridge
- ocr/main.py: widen Path check to os.PathLike for consistency with other sinks
- main.py: bridge condition checks truthiness of reasoning_summary, not just None
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: remove unused pathlib.Path import in ocr/main.py
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(tests): drop deprecated OpenAI-Beta realtime header
OpenAI deprecated the 'OpenAI-Beta: realtime=v1' header; the live
service now returns code 4000 invalid_beta with
"Unknown beta requested: 'realtime'.". Two integration tests in
tests/llm_translation/realtime/test_realtime_guardrails_openai.py
hardcoded the header and started failing across all PRs.
Library code is unaffected: the OpenAI realtime handler only
forwards 'OpenAI-Beta: realtime=v1' upstream when the proxy *client*
sends it (litellm/llms/openai/realtime/handler.py). Default proxy
behavior uses the GA protocol.
Connect to OpenAI without the deprecated header, and accept the GA
event name 'response.output_audio_transcript.delta' alongside the
beta-protocol name 'response.audio_transcript.delta' for the
transcript-delta assertion.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(logging): post_call tolerates non-JSON-serializable values
post_call() did json.dumps(original_response) without default=str, so
any provider passing a dict containing datetime/Decimal/etc. would
raise TypeError. Bedrock batch retrieval hits this with
get_model_invocation_job() responses that include datetime fields
(submitTime, lastModifiedTime, endTime), failing
tests/batches_tests/test_bedrock_files_and_batches.py::test_async_file_and_batch
across all PRs.
Pass default=str so non-serializable values fall back to str().
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(tests): mock boto3 in bedrock retrieve batch test
The test patched AsyncHTTPHandler.get, but the bedrock retrieve
handler uses boto3.client('bedrock').get_model_invocation_job
directly, so the real AWS call was being made on every run, failing
with AccessDeniedException because the hardcoded test ARN belongs to
a different AWS account.
- Mock boto3.client and BedrockBatchesConfig.get_credentials so the
test never touches AWS.
- Use status=Completed in the mock response so output_file_id is
populated (the handler intentionally leaves it None for
non-completed jobs).
- Assert the predicted per-job output object URI (matches what the
handler actually returns) instead of the bare output prefix.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* docs(tests): include GA event name in guardrail-block test docstring
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* feat(realtime): OpenAI Realtime GA support and beta compatibility
- Normalize beta-style session.update to GA for upstream OpenAI; optional GA→beta
event translation when client sends OpenAI-Beta: realtime=v1
- Default upstream WebSocket without OpenAI-Beta; forward header when client opts in
- Extend OpenAI realtime types for GA event names and conversation item shapes
- Relax LiteLLMRealtimeStreamLoggingObject.results to List[Any] for GA events
- Update proxy client_secrets fallback to omit beta header; dashboard RealtimePlayground
- Add unit tests for remap, translation, and beta header helper
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix results
* fix greptile
* Fix mypy issues
* Remove unused class constants _GA_TEXT_DELTA_TYPES and _GA_AUDIO_DELTA_TYPES
These frozensets were defined as class-level constants in realtime_streaming.py
but never referenced anywhere in the codebase. Removing dead code.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
* fix(realtime): use GA-shaped session.update in guardrail injections
The guardrail VAD injection code sent a beta-style session.update with a
flat turn_detection field:
{"session": {"turn_detection": {"create_response": false}}}
When the upstream OpenAI backend operates in GA mode (no OpenAI-Beta
header forwarded), it requires the nested GA shape:
{"session": {"type": "realtime", "audio": {"input": {"turn_detection": {"create_response": false}}}}}
The _remap_beta_session_to_ga helper was only applied to client-
originated session.update messages in client_ack_messages. Internally-
generated session.updates (sent via _send_to_backend) in two paths:
- _handle_raw_backend_message (raw/no provider_config path, line 518)
- backend_to_client_send_messages provider_config path (line 481)
bypassed the remap, so GA upstreams ignored or rejected them, breaking
audio transcription guardrails for all non-beta clients.
Fix: add _make_disable_auto_response_message() helper that always emits
the correct GA-shaped session.update, and replace both injection sites
with it.
Update existing tests to assert the GA nested shape instead of the old
flat beta shape, and add a new unit test for the helper itself.
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
* Log realtime session type
* Fix beta realtime session payloads
* Fix realtime audio format remapping edge case
* Fix Azure realtime beta session shape
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Greptile follow-ups on the prior commit:
- (P1) ``is_request_body_safe`` recursed into ``litellm_embedding_config``
with no depth bound, so a request body 1000 levels deep could exhaust
Python's call stack and surface a 500 ``RecursionError``. Refactored
the check to be iterative (single-level descent into a fixed list of
nested-config keys) and extracted the per-dict banned-param scan into
a helper that's shared between the root and the nested call sites.
Also fixes the ``recursive_detector`` CI job that was triggered by
the recursive-by-name pattern.
- (P2) ``assert_same_origin`` error messages identified the mismatching
component but echoed the ``expected`` host and the candidate
hostname back to the caller. In the SSRF threat model the caller is
the attacker, so reflecting that information was a secondary leak of
operator infrastructure. Messages now identify only *which*
component mismatched (scheme / host / port) without naming names.
- (P2) ``_NESTED_CONFIG_KEYS`` was defined after the function that used
it. Hoisted the constant (and the new ``_BANNED_REQUEST_BODY_PARAMS``
tuple) above the function for readability.
Adds a 1000-level-deep nested config test that asserts no
``RecursionError`` and a hostname-leak test that asserts no operator
host appears in the rejection message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The async CustomStreamWrapper.__anext__ caught httpx.TimeoutException
and re-raised it raw, bypassing _handle_stream_fallback_error. The
Router's FallbackStreamWrapper only reacts to MidStreamFallbackError,
so stream_timeout firing mid-stream never triggered fallbacks the way
connection-phase timeout does.
Route timeouts through _handle_stream_fallback_error like every other
exception so they wrap into MidStreamFallbackError and the Router can
switch to a fallback model.
Two SSRF findings were OPEN with no in-flight fix; both are closed
now using narrow defenses that key off existing trust boundaries.
VERIA-6 (Milvus ``litellm_embedding_config``):
``is_request_body_safe`` already blocks ``api_base`` / ``api_key`` /
``langfuse_host`` / ``s3_endpoint_url`` / etc. at the *root* of the
request body, gated by an admin opt-in (``allow_client_side_credentials``
or per-deployment ``configurable_clientside_auth_params``). The bug is
that the Milvus vector-store transformer unpacks
``litellm_embedding_config`` into ``litellm.embedding(**embedding_config)``,
so a caller can smuggle the same banned params in via nesting and bypass
the check. Fix: ``is_request_body_safe`` now recurses into a known list
of nested-config dicts (``litellm_embedding_config`` for now) and applies
the same banned-param check with the same admin opt-in. Admin-side
vector-store config flows through ``litellm_params`` rather than the
request body, so it's unaffected.
VERIA-51 (polling URLs returned by upstream APIs):
Azure DALL-E 2, Azure Document Intelligence, and Black Forest Labs
all blindly fetched a polling URL returned by the upstream and
attached the operator's API key to the request. A compromised upstream
or a future API contract change could redirect credentials anywhere.
New ``url_utils.assert_same_origin(candidate, expected)`` helper checks
scheme, host (case-insensitive), and port (with default-port
normalization). Applied at all five polling sites: Azure DALL-E
sync+async, Azure DI sync+async, BFL image generation sync+async, BFL
image edit sync+async. Cross-origin polling URLs now raise rather than
forward credentials. The Azure DALL-E ``Expected 'status' in response``
exception no longer reflects the raw response body — that path turned
Blind SSRF into Full-Read SSRF for the limited window before the
origin check fully closed it.
Tests: 7 ``assert_same_origin`` unit tests, 6 ``is_request_body_safe``
nested-config tests, 5 polling-site rejection tests + 1 same-origin
sanity check.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test supplies a minimal PDF base64 payload but expected the wrong
constant (base64 for "test"). Assert against the same pdf_b64 value
and drop the unused import.
Made-with: Cursor
* fix(bedrock): handle document content blocks in Converse API message conversion
Document content blocks (used for PDF support) were silently dropped
during message conversion for Bedrock's Converse API. The content block
processing loop only handled text, image_url, and file types — document
blocks were skipped without warning, causing the model to respond as if
no document was provided.
Adds document block handling in three locations:
- Sync user message processing (_bedrock_converse_messages_pt)
- Async user message processing (_bedrock_converse_messages_pt_async)
- Tool result conversion (_convert_to_bedrock_tool_call_result)
Fixes#24641
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use _validate_format for proper MIME type to Bedrock format mapping
Address Greptile review: naive media_type.split("/")[1] produced invalid
Bedrock format names for complex MIME types (e.g. OOXML → docx, text/plain
→ txt, text/markdown → md). Now reuses BedrockImageProcessor._validate_format
which handles all MIME types correctly via mimetypes + fallback.
Also fixes test assertions to expect correct Bedrock format values and adds
text/plain and text/markdown test cases.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reject non-base64 document sources with a clear error
URL-type document sources (e.g. {"type": "url", "url": "..."}) would
crash with an opaque KeyError on missing 'media_type'. Guard at the top
of _process_document_message and raise a clear ValueError since Bedrock
Converse only supports base64-encoded document sources.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore(auth): validate clientside api_base against SSRF guard; clear admin secrets on base override
Two related issues with how the proxy handles client-supplied
``api_base`` / ``base_url`` overrides on chat-completion requests:
1. **SSRF gate bypass** — ``check_complete_credentials()`` returned
``True`` for any non-empty ``api_key``, allowing the
``is_request_body_safe`` ``banned_params`` loop to admit ``api_base``
/ ``base_url`` values that point at private (RFC 1918), loopback,
link-local, or cloud-metadata addresses. Now: when the gate sees a
client-supplied ``api_base`` / ``base_url``, it runs the URL through
``litellm_core_utils.url_utils.validate_url`` (DNS-resolves, blocks
internal/IMDS/LL networks, defends against rebinding). Rejection
raises with a clear message.
2. **Admin-config leak on base override** —
``get_dynamic_litellm_params`` only carried the three clientside keys
(``api_key``, ``api_base``, ``base_url``) from request to upstream
call. Other admin-configured fields on ``litellm_params`` —
``organization``, ``extra_body``, ``extra_headers``, ``api_version``,
``azure_ad_token``, AWS / Vertex creds, etc. — flowed through
unchanged. With base redirected to a client-controlled server, those
admin secrets were sent to the attacker. Now: when ``api_base`` /
``base_url`` is in ``request_kwargs``, drop those admin-config
fields from ``litellm_params`` unless the caller re-supplied them.
Tests cover the SSRF-target rejection per URL field, the admin-secret
clearing on base override, the don't-clear case when only ``api_key``
is overridden (BYOK pattern), and the don't-overwrite case when the
caller resupplies fields like ``organization`` themselves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(vertex-batches): wrap api_base GET in safe_get for defense-in-depth
The vertex batches status-poll fetches an attacker-influenceable
``api_base`` URL with a raw ``sync_handler.get()``. The proxy auth gate
already validates clientside ``api_base`` before reaching this sink, so
the proxy flow is covered. This adds the per-sink wrap so SDK callers
and any future code path that bypasses the proxy gate pick up the same
SSRF defense from ``url_utils.safe_get``.
Operators with a legitimate private Vertex base can either allowlist
the host via ``litellm.user_url_allowed_hosts`` or disable validation
with ``litellm.user_url_validation = False``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(auth): hoist url_utils import; derive admin-config field list from CredentialLiteLLMParams
/simplify pass:
- Move ``from litellm.litellm_core_utils.url_utils import SSRFError, validate_url``
to module top in ``proxy/auth/auth_utils.py``. CLAUDE.md prefers
module-level imports unless avoiding a circular dependency, and
there's no cycle here (``url_utils`` doesn't depend on ``proxy.auth``).
- Replace the hardcoded ``_ADMIN_CONFIG_FIELDS_TO_CLEAR_ON_BASE_OVERRIDE``
literal with ``_admin_config_fields_to_clear_on_base_override()`` that
derives the typed-field portion from
``CredentialLiteLLMParams.model_fields``. Adds three fields the
hardcoded list missed (``aws_bedrock_runtime_endpoint``,
``watsonx_region_name``, ``region_name``) and stays in sync as new
provider fields are declared on the model. The kwargs-only set
(``organization``, ``extra_body``, ``azure_ad_token``, ``aws_session_token``,
``aws_sts_endpoint``, ``aws_web_identity_token``, ``aws_role_name``, …)
remains explicit since those fields aren't on the typed model.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): close field-echo bypass; gate URL check on toggle; cover async batch path
Three issues from review:
1. ``get_dynamic_litellm_params`` used ``if field not in request_kwargs:
pop`` to clear admin-set provider config when the caller redirected
``api_base``. A caller could *echo* any clear-list field name (with any
value, including an empty string) to skip the pop, leaving the admin's
value in ``litellm_params`` to be forwarded to the redirected upstream.
Fix: always pop, then write the caller's value back if they resupplied
the field.
2. ``check_complete_credentials`` called ``validate_url`` directly. That
helper doesn't itself consult ``litellm.user_url_validation``; the
toggle is honoured by ``safe_get`` / ``async_safe_get``. Mirror that
here so admins who explicitly disabled URL validation aren't blocked
at the proxy boundary.
3. ``VertexAIBatchesHandler._async_retrieve_batch`` still used a bare
``await client.get(api_base, ...)`` while the sync sibling was wrapped
in ``safe_get``. Wrap the async call in ``async_safe_get`` so SDK
callers on the async path get the same DNS-rebind / private /
cloud-metadata defenses as the sync path.
Tests:
- ``TestCheckCompleteCredentialsBlocksSSRF`` is now mock-only; an autouse
fixture flips the toggle on, ``validate_url`` is patched in the
parametrized blocking tests, and the positive path no longer makes a
real DNS call to api.openai.com.
- ``test_skips_url_validation_when_toggle_is_off`` documents the new
toggle-off behaviour and asserts ``validate_url`` is not called.
- ``test_caller_resupplied_value_overrides_admin_value_on_base_override``
replaces the prior test that asserted the buggy
preserve-admin-value-on-echo behaviour.
- ``test_field_echo_does_not_preserve_admin_value`` is a focused
regression test for the empty-string echo vector.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): close provider-confusion credential exfil; expand banned-params; cover OCI
Three additions on top of the entry-point URL gate so the cluster is
fully closed against caller-supplied ``api_base`` redirection:
1. ``get_llm_provider_logic.py`` matched registered openai-compatible
endpoints against ``api_base`` with an unanchored substring search
(``if endpoint in api_base:``). A caller could pass an api_base like
``https://attacker.com/api.groq.com/openai/v1`` to coerce the proxy
into reading ``GROQ_API_KEY`` from the environment and forwarding it
as a Bearer credential to the attacker's host. Replaced with parsed-
URL semantics (hostname exact-match plus segment-bounded path-prefix)
in a new ``_endpoint_matches_api_base`` helper.
2. ``is_request_body_safe`` rejects ``api_base`` / ``base_url`` /
``user_config`` / a handful of AWS / vertex fields, but the list
omitted three other endpoint-targeting fields:
* ``aws_bedrock_runtime_endpoint`` — Bedrock endpoint redirect
* ``langsmith_base_url`` / ``langfuse_host`` — observability callback
hostnames; attacker-controlled values exfiltrate the entire request
payload (incl. message content) via the logging hook.
Added all three to the blocklist.
3. ``_admin_config_fields_to_clear_on_base_override`` derives its typed-
field list from ``CredentialLiteLLMParams.model_fields``, which does
not declare any of the OCI provider's auth fields. Added
``oci_signer``, ``oci_user``, ``oci_fingerprint``, ``oci_tenancy``,
``oci_key``, and ``oci_key_file`` to the kwargs-only fixed list so
they are cleared on caller-redirected ``api_base`` like the AWS /
Azure / Vertex equivalents.
Tests:
- ``TestEndpointMatchesApiBase`` — direct unit tests on the new
matcher: legitimate provider URLs (5 shapes) match; attacker
smuggling via path injection, suffix label, prefix label, userinfo
``@`` injection, and path-segment lookalikes (7 shapes) do not.
- ``TestGetLlmProviderRejectsAttackerSmuggledApiBase`` — end-to-end
invariant that ``GROQ_API_KEY`` is never read against an attacker-
controlled host while the legitimate ``api.groq.com`` path still
resolves the provider correctly.
- ``TestIsRequestBodySafeBlocksEndpointTargetingFields`` — parametrized
coverage that each of the three new banned-params raises a clear
rejection naming the offending field.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): remove implicit api-key bypass + add posthog/braintrust/slack to blocklist
The historical ``check_complete_credentials`` clause inside
``is_request_body_safe`` was a third, *implicit*, *caller-controlled*
BYOK path: any caller that supplied a non-empty ``api_key`` caused the
entire banned-params blocklist to be skipped. That turned every missing
entry on the blocklist into an exploitable SSRF / credential-exfil hole
and is the root cause of the chain of api_base advisories that have
been re-discovered with each new integration:
* GHSA-jh89-88fc-qrfp (critical, triage) — env-var exfil via api_base
* GHSA-3frq-6r6h-7j64 (high, triage) — admin org / extra_body leak
* veria-admin Dv_m860l, b_yRJeQ5, stN90yjP, LBlyOAc8, U2TD78kg —
variations on "list X is missing field Y"
Two explicit, admin-controlled BYOK paths already exist and remain:
``general_settings.allow_client_side_credentials = true`` (proxy-wide)
and ``configurable_clientside_auth_params: [...]`` per deployment.
Removing the implicit bypass converts the failure mode of a missing
blocklist entry from "live credential leak" to "predictable 400 with
a clear remediation message," which is the structural fix.
Also adds the three remaining endpoint-targeting fields the dynamic
callback layer reads from request body: ``posthog_host``,
``braintrust_host``, ``slack_webhook_url``. ``slack_webhook_url`` in
particular was a direct exfil channel (caller-set webhook → proxy
mirrors every request to attacker's Slack).
Tests:
- ``test_api_key_does_not_bypass_blocklist`` — parametrized regression
asserting api_key=anything no longer skips the gate for any of the
five highest-risk fields.
- ``test_admin_opt_in_proxy_wide_still_allows`` — confirms the
documented BYOK opt-in still works.
- Extends ``test_endpoint_targeting_field_in_request_body_is_rejected``
to cover posthog / braintrust / slack.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(auth): block sagemaker_base_url, s3_endpoint_url, deployment_url
Provider-specific endpoint overrides surfaced by a wider audit of
``optional_params`` consumers in ``litellm/llms/``. Same threat as
``api_base``: a caller-supplied value redirects the outbound request
to an attacker host.
* ``s3_endpoint_url`` — read in ``litellm/llms/bedrock/files/transformation.py``
to build the S3 upload URL for Bedrock files. Caller redirects file
uploads to attacker-controlled S3.
* ``sagemaker_base_url`` — read in ``litellm/llms/sagemaker/{chat,completion}/*``.
Caller redirects SageMaker traffic. This is the primary vector
described in veria-admin mNqEBBtG.
* ``deployment_url`` — popped in ``litellm/llms/sap/chat/transformation.py``.
Caller redirects SAP deployment requests.
Tests parametrize ``test_endpoint_targeting_field_in_request_body_is_rejected``
to cover the three new fields.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Unify cost calc in success_handler dict and typed branches
* Trim verbose comments and docstrings
---------
Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain>
Co-authored-by: Michael Riad Zaky <michaelr@Michaels-MacBook-Air.local>
AWS Bedrock has reached end-of-life for `claude-3-7-sonnet-20250219-v1:0`,
returning 404s with "This model version has reached the end of its life."
Update test references to `claude-sonnet-4-5-20250929-v1:0` (same capability
surface: thinking, tools, prompt caching, PDF input, vision, computer use).
The bedrock/invoke pass-through tests stay on Sonnet 3.5 since Sonnet 4.5
is converse-only on Bedrock.
- Tighten _is_anthropic_document_data_uri to match the mimes Anthropic
actually accepts as base64 `document` source ({application/pdf,
text/plain}). The previous application/* + text/* prefix match would
route e.g. data:application/json URIs through the document path,
producing blocks the Anthropic API rejects. Unsupported mimes now
stay on the existing image code path (same failure mode as before the
fix — no regression, just stops introducing a new one).
- On the Bedrock tool-result `type: "file"` branch, accept either
file_data or file_id and raise BadRequestError on both-None, mirroring
the user-message _process_file_message pattern. Previously a file
block with only file_id was silently dropped.
- Consolidate the six new PDF tool-result tests under tests/test_litellm/
only (the PR template's required location and where the unit-test CI
workflow runs with coverage). The duplicate copies under
tests/llm_translation/ added drift risk with no additional coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Duplicate the three Bedrock and three Anthropic tool-result tests into
tests/test_litellm/ so they're picked up by `make test-unit` (and its
coverage report). The originals in tests/llm_translation/ stay — they
run under integration and remain the canonical translation-suite
regression cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bedrock enforces non-increasing TTL ordering across cache_control blocks
(tools → system → messages). The tool cache_control TTL was being
unconditionally dropped to the default 5m, while system blocks preserved
the user-specified TTL for Claude 4.5+ models. This mismatch caused
"a ttl='1h' block must not come after a ttl='5m' block" errors when
users set ttl='1h' on both tools and system.
Converse path: add_cache_point_tool_block() now accepts a model param
and preserves TTL for Claude 4.5+, matching _get_cache_point_block().
Invoke path: _remove_ttl_from_cache_control() now also processes tools
(was only processing system and messages).
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants)
Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet
shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page
is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the
established precedent for azure/gpt-5.4* (which were in the cost map
before the Azure rollout) so cost tracking and capability flags work
the moment customers deploy.
Schema follows the existing azure/gpt-5.4* shape:
- Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat,
$60/$360 pro per 1M, with priority tier 2x base
- Azure variants drop the flex/batches keys (Azure has no flex tier)
but keep priority pricing, matching gpt-5.4* precedent
- mode=chat for the thinking model, mode=responses for pro
reasoning_effort capability flags mirror the OpenAI variants exactly
since Azure proxies the same API contract: minimal rejection on both
chat and pro, low/none rejection on pro. Once #26456 (which sets
supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*)
lands, OpenAI and Azure flag profiles align.
Tests pin entry presence + pricing for all four Azure variants and
verify the live-API-derived reasoning_effort flags.
* test: register supports_low_reasoning_effort in cost-map JSON schema
azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch
carry supports_low_reasoning_effort=false. The strict
'additionalProperties: false' schema in
test_aaamodel_prices_and_context_window_json_is_valid rejected the new
key. Register it alongside the other supports_*_reasoning_effort
entries.
Note: the runtime side of this flag (code that reads it) lands in
#26456. Until that PR merges the flag is inert for both Azure and
OpenAI pro entries, but having the schema accept it lets cost-map
tests pass on either merge order.
Mirrors the existing vertex_credentials handling for the newer
vertex_ai_credentials field, and extends the dynamic masker's sensitive
pattern set to recognize the plural form so other plural-named credential
fields are also covered.
* feat(proxy): add /v1/memory CRUD endpoints with user/team scoping
New LiteLLM_MemoryTable stores user/team-scoped key/value entries with
optional JSON metadata. Value is a String (LLM-readable text) and metadata
is an optional Json? envelope, matching the Letta + mem0 hybrid model so
future structured fields can be added without a schema migration.
Endpoints:
POST /v1/memory - create
GET /v1/memory - list (caller-scoped; admins see all)
GET /v1/memory/{key} - fetch one
PUT /v1/memory/{key} - upsert
DELETE /v1/memory/{key} - delete
Non-admin callers cannot set a user_id/team_id other than their own.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(proxy/memory): omit metadata field when None on create
Prisma's Python client rejects `metadata=None` on a `Json?` field with
"A value is required but not set" — the field must be omitted from the
`data` dict entirely to store SQL NULL. Build the create payload
conditionally in both `create_memory` and the PUT-create branch of
`upsert_memory`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(ui): add Memory page to view/manage /v1/memory entries
Adds a new "Memory" sidebar item under Tools so users can see what their
agents have stored. Lists all memories visible to the caller (scoped by
the backend), with a key-search filter, preview column, scope tags, and
view/edit/delete actions. Create modal accepts optional JSON metadata.
- networking.tsx: fetchMemoryList / createMemory / updateMemory / deleteMemory
wired to the /v1/memory CRUD endpoints.
- MemoryView + MemoryEditModal: new antd-based components (per CLAUDE.md:
use antd for new UI, not tremor).
- page.tsx + leftnav.tsx: wire the "memory" route + sidebar entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(memory): add key_prefix filter + promote Memory to AI GATEWAY nav
Backend:
- GET /v1/memory now accepts `key_prefix` for Redis-style namespace
scans (e.g. `?key_prefix=user:`). When both `key` and `key_prefix`
are passed, `key_prefix` wins.
- Prefix filter sits under the visibility filter in the Prisma where
clause, so it can never leak rows across user/team scopes.
- New tests: prefix match, and cross-scope isolation (another user's
`user:*` rows must not appear in the caller's results).
UI:
- Memory moved from a Tools submenu to a top-level AI GATEWAY item
(alongside Agents, MCP Servers, Skills) — it's an API primitive,
not a tool-management surface.
- Search box now drives prefix search, matching the Redis mental
model ("type the namespace, see everything under it").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(memory): enforce unique key per scope by using NULLS NOT DISTINCT
The unique constraint `(key, user_id, team_id)` on LiteLLM_MemoryTable
silently allowed duplicates when user_id or team_id was NULL, because
Postgres treats every NULL as distinct by default (ANSI semantics). A
caller with no team_id could POST the same key three times and get
three rows.
Migration:
1. Dedupe existing rows, keeping the most recent per (key, user_id,
team_id), using `IS NOT DISTINCT FROM` so NULL == NULL.
2. Drop the old unique index.
3. Recreate it with `NULLS NOT DISTINCT` (Postgres 15+).
No code change: POST already returns 409 on unique-violation error
messages — it just wasn't firing before because the constraint didn't
catch the NULL-team case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(memory): make key globally unique, 409 on any duplicate
Switches from the compound unique `(key, user_id, team_id)` to a simple
`key @unique`. The compound form silently allowed duplicates when
user_id or team_id was NULL (Postgres treats each NULL as distinct), so
callers could POST the same key repeatedly. Globally-unique key means
one row per key, period — any duplicate create → 409.
- schema.prisma (×3): `key String @unique`, drop `@@unique(...)`.
- initial add_memory_table migration: unique index on (key) only.
- Remove the now-unused follow-up NULLS NOT DISTINCT migration.
- Endpoint error message simplified ("already exists" — no "for this scope").
- Test fake's create() now enforces global key uniqueness.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ui/memory): full-width layout + user/teams-style columns
- Add `w-full` to the MemoryView outer div so the page fills the
flex-flex-1 container (was collapsing to intrinsic width).
- Replace the combined "Scope" column with separate User ID / Team ID
columns, matching the layout of the Users / Teams pages: ID, Name,
Preview, User ID, Team ID, Updated, Actions.
- IDs render with a truncated mono label + copy-to-clipboard button,
same pattern as view_users.
- Detail drawer now shows Memory ID / User ID / Team ID as separate
fields instead of stacked color tags.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ui/memory): use clean MCP-style ID pill, drop copy icons
The ID / User ID / Team ID columns showed a mono text blob with a
copy-to-clipboard icon next to each value — too busy compared to the
MCP Servers page. Swap the renderer for MCP's pill style:
- Truncated mono ID inside a blue Tailwind pill
(`font-mono text-blue-600 bg-blue-50 ... rounded-md border`).
- No copy icon. Full ID surfaces via tooltip.
- ID column is a button that opens the detail drawer on click;
user/team ID pills are static (not clickable).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(memory): address greptile review feedback
Addresses 5 greptile findings (3/5 → higher confidence target):
1. Identity-less orphan rows (P1): non-admin callers with no user_id AND
no team_id could create rows that the visibility filter would never
match again. Now rejected up front with 400 — caller must authenticate
with a scoped key or act as PROXY_ADMIN.
2. Upsert race returning 500 (P1): PUT's check-then-create isn't atomic;
a concurrent writer could slip a row in between the 404-check and the
create call. Now catch unique-violation on create, re-read, and fall
through to update — PUT stays idempotent. If the conflicting row
belongs to a different scope, surface a 409 instead of 500.
3. PUT-create scope inconsistency (P2): PUT's create branch always used
the caller's own user_id/team_id, so admins couldn't bootstrap rows
scoped elsewhere via PUT (only POST). Now PUT-create calls the shared
`_resolve_scope()` helper, matching POST semantics.
4. Stale schema comment (P2): schema said "Keyed by (key, user_id,
team_id)" but `key` is globally unique. Updated all three schema
copies to reflect the actual design.
5. UI silently truncated at 200 (P2): MemoryView fetched pageSize=200
with no load-more. Swapped to real server-side pagination driven by
`data.total`; page size is now 50 and the pager is a real AntD
control.
Also extracts a shared `_resolve_scope()` helper and `_is_unique_violation()`
from create_memory so POST and PUT don't drift on the scope/error logic.
Tests: +3 new (identity-less 400, PUT admin bootstrap, PUT race →
update), 18/18 pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(memory): typed Prisma error + explicit-null metadata on PUT
Two more greptile threads from the last review:
- Unique-violation detection was string-matching "Unique"/"UniqueViolation"
in the exception message, fragile across Prisma/driver versions. Now
check the typed error `code == "P2002"` first, with string fallback.
- PUT could not distinguish "metadata omitted" from "metadata: null" —
both parsed as `None`, so callers had no way to clear stored metadata.
Switch to Pydantic v2's `model_fields_set` to tell which fields the
caller actually sent; explicit null now clears the column.
New tests:
- explicit null clears metadata
- omitted metadata preserves existing value
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ui/memory): send explicit null when user clears metadata
Addresses the remaining P1 from the last greptile review:
When the edit modal's metadata textarea was cleared and saved,
`metadataParsed` stayed `undefined`, `JSON.stringify` dropped the key
entirely, and the backend's `model_fields_set` guard therefore left
the stored metadata untouched — UI showed success but nothing changed.
Now: empty textarea on edit → send explicit `null` so the backend
sees `metadata` in `model_fields_set` and clears the column.
Empty textarea on create still maps to `undefined` (field omitted)
to avoid Prisma's `Json? = None` quirk on insert.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ui/memory): preserve slashes in key path encoding
The backend route `/v1/memory/{key:path}` supports keys with slashes,
but `encodeURIComponent` encoded `/` as `%2F`. Some proxies (nginx
default, CloudFlare, AWS ALB) reject or re-decode `%2F` mid-flight,
so UI update/delete calls on slash-containing keys could fail or
silently misroute.
New helper `encodeMemoryKeyForPath` splits by `/`, URL-encodes each
segment, then rejoins with literal `/`. Every other unsafe char
(spaces, `?`, `#`, `%`) stays encoded per-segment; slashes stay as
path delimiters, matching what the `:path` converter expects.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ui/memory): drop misleading client-side column sorters
With server-side pagination, client sorters on `key` and `updated_at`
only reorder the current page while pretending to sort the full
dataset — users would see "sorted by name" but only the visible 50
rows would actually be sorted.
Remove the sorters. The backend already returns rows in
`updated_at DESC` order (sensible default for a memory view), and
users can narrow the result with the key-prefix filter.
Greptile also flagged missing `@@map` on the new model as a
"consistency" issue, but only 1 of 59 tables in this repo uses
`@@map` — the dominant pattern is to rely on Prisma's default
(model name == table name). Skipping that finding as a
false-positive on convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(memory): compose visibility + key filters via explicit AND
Greptile P1 (filter-fragility): `where.update(vis)` was semantically
correct today, but dict-merging by key meant any future visibility
filter that grew a new top-level "OR" would silently clobber the
existing key filter.
Compose explicitly instead:
where = {"AND": [key_filter, vis]}
Applied to both `list_memory` and `_find_memory_for_caller`. When
either side is empty (admin has no visibility filter; list has no
key filter), skip the wrapper and use the non-empty side directly
to keep the generated SQL clean.
Test fake's `_matches` now understands top-level `AND` too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(ui/memory): wrap write helpers with react-query useMutation
Previously the Memory view read via `useQuery` but called the raw
create/update/delete fetch helpers directly in handlers, tracking
loading state with a local `submitting` flag and invalidating state
via `refetch()`. That mixes two concerns:
- it skips react-query's mutation state (isPending / isError / isSuccess)
- `refetch()` only retouches the currently-mounted query instance, not
other cached pages, so navigating back to an older page could show
stale rows
Switch the three write paths to `useMutation`:
- `createMutation`, `updateMutation`, `deleteMutation` — each owns
the mutation fn, success toast, and error toast.
- Success handlers invalidate the whole `["memoryList", ...]` prefix
via `queryClient.invalidateQueries`, so every cached page refetches
(pagination + filter-aware).
- Refresh button now invalidates instead of `refetch()`, keeping all
behavior consistent.
- handleSave/handleDelete become thin adapters that call `.mutateAsync`;
their errors are swallowed locally since the mutation's onError has
already surfaced the toast.
Also tightened the edit modal's key-field tooltip to reflect the
actual global-unique semantics (was "Unique per user/team scope").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(memory): close cross-user write gap + sanitize 500 errors (Veria)
Addresses two Veria findings:
**High — cross-user memory tampering via team membership.** The
visibility filter uses an OR (`user_id == caller OR team_id == caller`)
so team members can SEE each other's team-scoped rows. That's
intentional for list/get. But because PUT/DELETE used the same filter
to find the target row, any team member could overwrite or delete a
teammate's *personal* row whenever both `user_id` and `team_id` were
stamped on it — broader visibility was being silently treated as
broader authority.
New `_assert_write_access(row, caller)` enforces ownership for
mutations. Non-admin rules:
- The row's `user_id` must match the caller (personal ownership), OR
- The row has no `user_id` and its `team_id` matches the caller's
team (a "pure team row" intended for shared writes).
Admins bypass the check. The same gate runs in PUT (both regular
and post-race-recovery branches) and DELETE.
**Medium — DB internals leaked through 500 detail.** Every `except`
block was raising `HTTPException(500, detail=str(e))`, which surfaces
Prisma error strings (table/column names, host:port, error class
names) to API callers. New `_internal_error()` helper logs the real
exception server-side and returns a generic, caller-safe `detail`.
Applied to create, list, upsert (general fallthrough), and delete.
Also tightened the race-recovery 409 message to drop the "in a
different scope" wording — the caller never needs to know whose
scope it lives in.
Tests (+5):
- teammate cannot overwrite personal row → 403
- teammate cannot delete personal row → 403
- teammate CAN modify pure team row (no user_id stamped) → 200
- admin bypasses write-auth → 200
- 500 response never echoes Prisma internals (table/host/class names)
25/25 unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(memory): require team admin to modify pure team rows
Tightens the write-authorization rule for "pure team rows" (rows with
no user_id stamped, only team_id) to match the pattern used by
team-management endpoints (`_is_user_team_admin` + `_is_user_org_admin_for_team`):
- Plain team members can READ team rows via the OR visibility filter
(intentional, unchanged).
- Only PROXY_ADMIN, team admins of the row's team_id, or org admins
for the team's organization may MODIFY them. Plain members get 403.
`_assert_write_access` is now async and takes the prisma_client so it
can fetch the team and run the existing `_is_user_team_admin` /
`_is_user_org_admin_for_team` helpers from
`litellm.proxy.management_endpoints.common_utils`. The org-admin path
is best-effort: it calls `get_user_object`, which depends on the
proxy_server module being initialized, so any exception there is
treated as "not an org admin" rather than crashing the request.
Tests:
- team admin can modify pure team row → 200
- plain team member cannot modify pure team row → 403
- plain team member cannot delete pure team row → 403
Updates the test fake to add a tiny `litellm_teamtable.find_unique`
implementation and a `_make_team(team_id, admin_user_ids=[...])`
helper.
27/27 unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: mypy + UI page-metadata sync for memory page
Two CI failures:
1. mypy: `_find_memory_for_caller` had `key_filter` inferred as
`dict[str, str]` (literal type) and the conditional `{"AND": [key_filter, vis]}`
returned `dict[str, list[...]]`, so the join site failed
`dict-item` typing. Annotate both intermediates as `dict` so mypy
widens the value type.
2. UI test (`page_utils.test.ts > should have descriptions for all
pages`): every leftnav entry must have a description in
`page_metadata.ts`, and `memory` was missing. Added a one-line
description, matching the style of neighboring entries.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [Feat] Day-0 support for GPT-5.5 and GPT-5.5 Pro (#26449)
* feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro
Add pricing + capability entries for the new GPT-5.5 family launched by
OpenAI on 2026-04-24:
- gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M
input/output/cached input
- gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6
per 1M input/output/cached input
Other fees (long-context >272k, flex, batches, priority, cache
discounts) follow the same ratios as GPT-5.4, with context window
retained at 1.05M input / 128K output.
No transformation / classifier code changes are required:
OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via
numeric version parsing, and model registration is driven from the
JSON. The existing responses-API bridge for tools + reasoning_effort
(litellm/main.py:970) already covers gpt-5.5-pro.
Tests:
- GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants
- New test_generic_cost_per_token_gpt55_pro cost-calc test
- Updated test_generic_cost_per_token_gpt55 for long-context fields
* fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants
gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the
supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and
supports_minimal_reasoning_effort flags that their non-dated
counterparts define. Reasoning-effort routing in OpenAIGPT5Config is
fully capability-driven from these JSON flags — since an absent flag
is treated as False for opt-in levels (xhigh), users pinning to a
dated snapshot would silently lose xhigh support and diverge from the
base alias on logprobs + flexible temperature handling.
Copy the flags onto both dated variants so every dated snapshot
inherits the base model's reasoning-effort capability profile.
Adds a parametrized regression test that asserts
supports_{none,minimal,xhigh}_reasoning_effort parity between each
dated variant and its non-dated counterpart, preventing future drift
when new snapshots are added.
* fix(schema): close LiteLLM_MemoryTable model brace dropped during merge
The rebase against `litellm_internal_staging` (which added
`LiteLLM_AdaptiveRouterState` / `LiteLLM_AdaptiveRouterSession`) left
the closing brace of `LiteLLM_MemoryTable` missing in all three
schema copies — the next model declaration ended up parsed as a field
of the memory table, surfacing as the CI prisma error:
error: This line is not a valid field or attribute definition.
--> schema.prisma:1250
|
1249 | // Per-(router, request_type, model) Beta posterior for the adaptive router.
1250 | model LiteLLM_AdaptiveRouterState {
Add the missing `}` (and the standard blank line) after the memory
table's `@@index([team_id])` in `schema.prisma`,
`litellm/proxy/schema.prisma`, and
`litellm-proxy-extras/litellm_proxy_extras/schema.prisma`.
`prisma generate --schema litellm/proxy/schema.prisma` now runs clean;
27/27 memory unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Verified against OpenAI's live Chat Completions API on 2026-04-24:
POST /v1/chat/completions
{"model": "gpt-5.5", "reasoning_effort": "minimal", ...}
-> 400 Unsupported value: 'reasoning_effort' does not support 'minimal'
with this model. Supported values are: 'none', 'low', 'medium',
'high', and 'xhigh'.
POST /v1/chat/completions
{"model": "gpt-5.5-pro", "reasoning_effort": "minimal", ...}
-> 400 Unsupported value: 'minimal' is not supported with the
'gpt-5.5-pro' model. Supported values are: 'medium', 'high', and
'xhigh'.
Set supports_minimal_reasoning_effort=false on all four entries
(gpt-5.5, gpt-5.5-2026-04-23, gpt-5.5-pro, gpt-5.5-pro-2026-04-23) so
OpenAIGPT5Config._is_reasoning_effort_level_explicitly_disabled fires
and LiteLLM either drops the param (drop_params=True) or raises a
local UnsupportedParamsError, instead of round-tripping to OpenAI for
a 400.
Adds a parametrized test_gpt55_reasoning_effort_flags_match_live_openai_api
test that pins supports_{none,minimal,xhigh}_reasoning_effort on each
entry to OpenAI's actual API contract.
Note: gpt-5.5-pro additionally rejects 'none' and 'low'. 'none' is
already handled (supports_none_reasoning_effort=false). 'low' is not
representable in the current JSON schema (no supports_low flag);
filing separately.
* feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro
Add pricing + capability entries for the new GPT-5.5 family launched by
OpenAI on 2026-04-24:
- gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M
input/output/cached input
- gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6
per 1M input/output/cached input
Other fees (long-context >272k, flex, batches, priority, cache
discounts) follow the same ratios as GPT-5.4, with context window
retained at 1.05M input / 128K output.
No transformation / classifier code changes are required:
OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via
numeric version parsing, and model registration is driven from the
JSON. The existing responses-API bridge for tools + reasoning_effort
(litellm/main.py:970) already covers gpt-5.5-pro.
Tests:
- GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants
- New test_generic_cost_per_token_gpt55_pro cost-calc test
- Updated test_generic_cost_per_token_gpt55 for long-context fields
* fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants
gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the
supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and
supports_minimal_reasoning_effort flags that their non-dated
counterparts define. Reasoning-effort routing in OpenAIGPT5Config is
fully capability-driven from these JSON flags — since an absent flag
is treated as False for opt-in levels (xhigh), users pinning to a
dated snapshot would silently lose xhigh support and diverge from the
base alias on logprobs + flexible temperature handling.
Copy the flags onto both dated variants so every dated snapshot
inherits the base model's reasoning-effort capability profile.
Adds a parametrized regression test that asserts
supports_{none,minimal,xhigh}_reasoning_effort parity between each
dated variant and its non-dated counterpart, preventing future drift
when new snapshots are added.
* feat: add gpt-5.5 to model cost map
Add gpt-5.5 entry with pricing from OpenAI flagship page:
input $5/1M, cached input $0.50/1M, output $30/1M, 272K context.
* test: add gpt-5.5 coverage for model cost map and gpt-5 routing
- Add gpt-5.5 to GPT5_MODELS parametrized list so both OpenAIGPT5Config
and AzureOpenAIGPT5Config routing tests cover the new model.
- Add test_generic_cost_per_token_gpt55 verifying the new entry's
cost-map values ($5/$0.50/$30 per 1M) and that generic_cost_per_token
returns the expected prompt/completion costs.
