* fix(security): bump tar to 7.5.11 and tornado to 6.5.5
- tar >=7.5.11: fixes CVE-2026-31802 (HIGH) in node-pkg
- tornado >=6.5.5: fixes CVE-2026-31958 (HIGH) and GHSA-78cv-mqj4-43f7 (MEDIUM) in python-pkg
Addresses vulnerabilities found in ghcr.io/berriai/litellm:main-v1.82.0-stable Trivy scan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: document tar override is enforced via Dockerfile, not npm
* fix: revert invalid JSON comment in package.json tar override
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Handle response.failed, response.incomplete, and response.cancelled terminal events in background streaming
Previously the background streaming task only handled response.completed and
hardcoded the final status to "completed". This missed three other terminal
event types from the OpenAI streaming spec, causing failed/incomplete/cancelled
responses to be incorrectly marked as completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove unused terminal_response_data variable
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Address code review: derive fallback status from event type, rewrite tests as integration tests
1. Replace hardcoded "completed" fallback in response_data.get("status")
with _event_to_status lookup so that response.incomplete and
response.cancelled events get the correct fallback if the response
body ever omits the status field.
2. Replace duplicated-logic unit tests with integration tests that
exercise background_streaming_task directly using mocked streaming
responses and assert on the final update_state call arguments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove dead mock_processor and unused mock_response parameter from test helper
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Remove FastAPI and UserAPIKeyAuth imports from test file
These types were only used as Mock(spec=...) arguments. Drop the spec
constraints and remove the top-level imports to avoid pulling FastAPI
into test files outside litellm/proxy/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
* Log warning when streaming response has no body_iterator
If base_process_llm_request returns a non-streaming response (no
body_iterator), log a warning since this likely indicates a
misconfiguration or provider error rather than a successful completion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Committed-By-Agent: claude
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."
* PR #22867 added _remove_scope_from_cache_control for Bedrock and Azure AI but omitted Vertex AI. This applies the same pattern to VertexAIPartnerModelsAnthropicMessagesConfig."
* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
inherit it, and removed the now-redundant copy from the Azure AI subclass.
* PR #22867 added _remove_scope_from_cache_control to AzureAnthropicMessagesConfig
but missed VertexAIPartnerModelsAnthropicMessagesConfi Rather than duplicating the method again, moved it up to the base AnthropicMessagesConfig so all providers
inherit it, and removed the now-redundant copy from the Azure AI subclass.
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
The model dropdown menus in the Add Fallbacks modal were rendering behind
the modal overlay because Ant Design portals Select dropdowns to document.body
by default. By setting getPopupContainer to attach the dropdown to its parent
element, the dropdown inherits the modal's stacking context and renders above
the modal.
Fixes#17895
Fixes#23502
The huggingface_embed.embedding() call was not receiving the headers
parameter, causing extra_headers (e.g., X-HF-Bill-To) to be silently
dropped. Other providers (openrouter, vercel_ai_gateway, bedrock) already
pass headers correctly. This fix adds headers=headers to match the
behavior of other providers.
Co-authored-by: Jah-yee <sparklab@outlook.com>
Extract permission checks and constraint validation from update_key_fn
into _validate_update_key_data helper to reduce statement count below
the 50-statement limit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tests were outdated after #23472 added pagination (take/order) to find_many
and stale-row cleanup via update_many. Updated assertions to match new call
signatures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The self-exclusion filter compared raw key strings against SHA-256
hashed tokens from the DB, so keys were never excluded and
double-counting persisted. Now hash data.key before comparison.
Also add tpm_limit_type/rpm_limit_type to _throughput_fields_changed
guard, fall back to existing_key_row.team_id for team limit checks
(matching the org pattern), and add team self-exclusion test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only run org validation (get_org_object + _check_org_key_limits) when
the update actually touches throughput-related fields (tpm_limit,
rpm_limit, or organization_id). Previously, any update to a key
belonging to an org would trigger the check, which would fail with a
400 if the org had been deleted — blocking unrelated field changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When updating a key, _check_org_key_limits and _check_team_key_limits
would include the key being updated in the find_many results, causing
its current limits to be counted twice (once from the DB query, once
from the new requested limits). This caused false 400 errors on valid
limit adjustments.
Fix: exclude the key being updated (by matching token) from the
allocated totals before checking limits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of hardcoding SPEND_PER_REQUEST (which broke when the model
changed from gpt-3.5-turbo-0301 to gpt-3.5-turbo), make a single
calibration request first, poll for its spend, and use that as the
per-request cost. Fails fast with pytest.fail() after 5 retries if
calibration cannot determine the cost.
Also fixes a bug in test_basic_spend_accuracy where the user spend
assertion error message referenced user_info['info'] instead of
user_info['user_info'].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Same deprecated model fix as proxy_server_config.yaml — these two CI
configs also referenced gpt-3.5-turbo-0301 which has no pricing data.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase wait timeout to 90s and pytest.fail() instead of silently
continuing, so the failure message points at the real cause.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gpt-3.5-turbo-0301 was removed from the model cost map, so every call
had response_cost=0 and team member spend never increased. The wait
helper also returned True after 3s regardless of whether spend updated.
- Switch fake-openai-endpoint to gpt-3.5-turbo (has pricing in cost map)
- Remove premature early-return in wait_for_team_member_spend_update
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase max_retries from 6 to 9 and retry_delay from 10s to 20s
(180s total wait, up from 60s) to give batch cost tracking more time
to finish before cleanup attempts file deletion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The terminal-state DB shortcut in retrieve_batch returned a LiteLLMBatch
with empty _hidden_params, causing the managed_files hook to skip encoding
output_file_id into a unified ID. This adds the same model_id extraction
from unified_batch_id that the non-terminal path already has.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- realtime_api/main.py: Revert param types to Dict, explicitly construct
RealtimeSessionConfig/RealtimeExpiresAfter before passing to
RealtimeClientSecretRequest
- presidio.py: Move type:ignore[override] to def line where mypy reports it
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_e2e_managed_batch test intermittently fails during cleanup
when deleting the input file — the batch cost tracking hasn't finished
processing yet (batch_processed=true not set), causing a 400 error.
This is a timing race condition unrelated to batch retrieval logic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Union[RealtimeClientSecretResponse, Response] annotation breaks
FastAPI's response model generation. Revert to the original annotation
and suppress mypy on the error-path return instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The terminal-state DB shortcut in retrieve_batch returned a LiteLLMBatch
with empty _hidden_params, causing the managed_files hook to skip encoding
output_file_id into a unified ID. This adds the same model_id extraction
from unified_batch_id that the non-terminal path already has.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- realtime_api/main.py: Widen param types to accept both Dict and Pydantic models
- proxy/realtime_endpoints/endpoints.py: Widen return type to Union[..., Response]
- proxy/guardrails/guardrail_hooks/presidio.py: Add type:ignore[override] for bytes in streaming return
- proxy/_experimental/mcp_server/rest_endpoints.py: Annotate _oauth2_flow with Literal type
- proxy/management_endpoints/ui_sso.py: Add httpx import under TYPE_CHECKING, remove invalid timeout kwarg from AsyncHTTPHandler.get()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(proxy): cap managed-object poll size + expire stale rows + kill-switch flag to prevent OOM/Prisma connection loss
* fix(constants): simplify PROXY_BATCH_POLLING_ENABLED readability
* docs+test: document new polling env vars, add pagination+stale-cleanup tests
* fix: exclude stale_expired from batch poll queries; fix update_many assertions in tests
* fix: scope stale cleanup to file_purpose, fix file_object mocks, add CheckBatchCost tests
* fix: avoid duplicate cost logging in fallback path; guard integer constants against zero/negative values
* fix: cache _has_batch_processed_column; guard cleanup from aborting poll; narrow fallback except
* fix: add complete/completed to primary query not_in; fix vacuous test assertion
- Primary find_many was missing "complete" and "completed" in its not_in
filter, creating asymmetry with the fallback query. A job whose status
was set to "complete" but whose batch_processed flag update failed would
be silently re-fetched and re-processed every cycle, emitting duplicate
cost logs.
- test_fallback_completion_update_omits_batch_processed patched
_is_base64_encoded_unified_file_id to return None, causing an immediate
continue — so update() was never called and the assertion looped over an
empty list (vacuously true). Rewrote the test to mock the full
completion pipeline, verify update() is called exactly once, and assert
batch_processed is absent from the update data.
- Added symmetric test (primary path) proving batch_processed IS included
when the column exists.
Made-with: Cursor
