The hooks gated on ``call_type == "completion"`` but the proxy ingress
passes ``route_type`` straight through as ``call_type`` —
``"acompletion"`` for /v1/chat/completions and ``"aresponses"`` for
/v1/responses. Tests passed because they used the literal sync
``"completion"`` value, masking the gap.
Switch both hooks to ``is_text_content_call_type`` (matches the
canonical runtime values: completion / acompletion / aresponses) and
update existing tests to assert against runtime values, plus parametrize
a regression test that pins the gate.
Hold the resolved config in a process-memory TTL cache so the
request-handling path doesn't run litellm_proxymodeltable.find_first
on every vector-store call.
The vector store create/update path previously called
``_resolve_embedding_config`` against the admin-configured router/DB
model and persisted the resolved ``litellm_embedding_config`` dict
(``api_key`` / ``api_base`` / ``api_version``) into the
``litellm_managedvectorstorestable.litellm_params`` column. Because the
resolver expanded ``os.environ/...`` references via ``get_secret``, the
DB row carried cleartext provider credentials, and the
``/vector_store/{new,info,update,list}`` responses returned them to any
authenticated caller who could supply a known admin model name.
Move the auto-resolve out of ``create_vector_store_in_db`` and out of
the update path. Persist only the user-supplied ``litellm_embedding_model``
reference. Resolve at request-handling time inside
``_update_request_data_with_litellm_managed_vector_store_registry`` so
the resolved config lives in the per-request ``data`` dict and is
garbage-collected after the response. Legacy rows that were created by
an earlier proxy version and already carry a resolved
``litellm_embedding_config`` skip the re-resolution and pass through
unchanged so embedding calls keep working.
The ``new_vector_store`` response now also runs the existing
``_redact_sensitive_litellm_params`` masker (already used by ``info``,
``update``, and ``list``), defending against caller-supplied cleartext
on the create path and against legacy rows whose persisted credentials
are still in the database.
Existing tests that asserted the old write-time-resolve behaviour are
updated to assert the new persistence shape (no embedding config
stored, just the model reference). Two new tests cover the use-time
path: one asserting fresh resolution happens when a row carries only
the model reference, the other asserting legacy rows with persisted
config skip re-resolution and continue to work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A pre-existing logic bug in ``_check_banned_params``: when the
deployment-level ``configurable_clientside_auth_params`` permitted one
banned field, the loop ``return``-ed on the first match instead of
``continue``-ing, so any other banned param later in the same body or
metadata dict was never checked. This PR's metadata walk multiplies the
surface where that bypass matters — a body pairing an allowed
``api_base`` with an observability credential like ``langfuse_host``
would silently pass.
Proxy-wide ``allow_client_side_credentials`` keeps ``return`` (it's a
global opt-in for every banned param). The per-param branch becomes
``continue`` so only the one explicitly-permitted field is skipped.
Adds a regression test that exercises the api_base + langfuse_host pair.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related gaps in the proxy's request bouncer:
1. ``is_request_body_safe`` (auth_utils.py) walked the request-body root
and the ``litellm_embedding_config`` nested dict, but not ``metadata``
or ``litellm_metadata``. The same fields it bans at root — Langfuse /
Langsmith / Arize / PostHog / Braintrust / Phoenix / W&B Weave / GCS /
Humanloop / Lunary credentials and routing — were silently accepted
when the caller put them inside metadata, retargeting observability
callbacks to a caller-controlled host with caller-supplied creds.
Walk both metadata containers (and parse the JSON-string form sent via
multipart / ``extra_body``) through the same banned-params helper, so
the existing ``allow_client_side_credentials`` opt-in covers both
paths consistently.
2. The banned-params list was hand-maintained and lagged the canonical
``_supported_callback_params`` allow-list in
``initialize_dynamic_callback_params``. Derive the observability bans
from that allow-list (minus a small ``_SAFE_CLIENT_CALLBACK_PARAMS``
set for informational fields like ``langfuse_prompt_version`` and
``langsmith_sampling_rate``) so future integrations are covered
automatically; ``_EXTRA_BANNED_OBSERVABILITY_PARAMS`` carries the
handful of fields integrations read but the allow-list hasn't caught
up to. A guard test fails CI if a new entry is added to
``_supported_callback_params`` without an explicit safe-list decision.
Separately in ``litellm_pre_call_utils.py``: add ``callbacks``,
``service_callback``, ``logger_fn``, and ``litellm_disabled_callbacks``
to ``_UNTRUSTED_ROOT_CONTROL_FIELDS``. The first three are appended to
worker-wide ``litellm.{input,success,failure,_async_*,service}_callback``
lists / ``litellm.user_logger_fn`` from inside ``function_setup`` — one
request poisons every subsequent caller in that worker. The last is the
inverse primitive: the legitimate path reads it from key/team metadata,
the request-body version silently disables admin-configured audit /
observability for the call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The strip ran before the proxy parses ``litellm_metadata`` from a JSON
string into a dict (a path used by multipart/form-data and ``extra_body``
callers), so ``isinstance(metadata, dict)`` was False and ``model_info``
survived the strip. Move the call to the same post-parse position the
``user_api_key_*`` strip already uses for the same reason. Adds a
regression test exercising the JSON-string ``litellm_metadata`` path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operators upgrading would otherwise see client-supplied pricing overrides
silently stop applying with no diagnostic. Emit a debug-level line listing
the dropped fields and pointing at the opt-in flag when any are stripped;
stay silent on the no-op path so the log isn't filled with noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The proxy currently forwards request-body pricing parameters (the fields
on `CustomPricingLiteLLMParams`, plus `metadata.model_info`) into the
core call path. Those fields belong to deployment configuration, not to
per-request input — sending them from a client mutates the request's
recorded cost and, via `litellm.completion` → `register_model`, the
process-wide `litellm.model_cost` map for every later caller in the
worker. Strip them at the boundary.
The strip set is built from `CustomPricingLiteLLMParams.model_fields` so
pricing fields added later are covered automatically. Operators who do
want clients to supply per-request pricing can opt back in per key or
team via `metadata.allow_client_pricing_override = true`, mirroring the
existing `allow_client_mock_response` and
`allow_client_message_redaction_opt_out` flags.
Tests cover the strip set's coverage, root and metadata strips, the
opt-in skip on both key and team metadata, and a regression check that
the global `litellm.model_cost` map is unmutated after a stripped
request.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous per-provider guards in HuggingFace, Oobabooga, and Gemini
files lived in the SDK layer, breaking SDK callers who legitimately pass
URL-valued model identifiers. Move the check to the proxy boundary in
add_litellm_data_to_request so SDK users keep working while proxy users
default-deny URL-valued model and file_id, with admin opt-in via
litellm.provider_url_destination_allowed_hosts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflict resolution for #26968 dropped the `Iterator` typing import
(NameError at module load), left a dead `fallback_models = cast(...)`
block, and the new tests called `_enforce_key_and_fallback_model_access`
without the now-required `request` kwarg.
Extend LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS to the failure callback so the
per-row Metadata pane in the UI no longer shows the stack trace when the
opt-in env var is set, matching the existing console-side suppression.
https://claude.ai/code/session_014dztoRbRnRvq54HL9EyHx6
