Brings the date-range branch in line with the non-date-range branch which
already hashes sk- prefixed tokens before querying. Adds coverage for
filter-combination behavior in view_spend_logs.
The raw SQL queries in spend_management_endpoints.py, spend_tracking_utils.py,
and analytics_endpoints.py cast date params to ::timestamptz while comparing
against the plain-timestamp "startTime" column. Postgres resolves the type
mismatch by promoting the column using the DB session timezone, which drifts
the filter window and date_trunc buckets whenever session TZ is not UTC —
silently dropping rows at UTC day boundaries and, for narrow windows, losing
rows entirely.
Wraps every such comparison with `AT TIME ZONE 'UTC'` so the param side
resolves to a plain timestamp matching the column type. Both sides end up
as plain timestamp, Postgres does no implicit conversion, and session TZ
plays no role in the query. The fix is constant-foldable so the existing
startTime index (PR #17504) remains usable.
Also marks strptime-produced datetimes as tz-aware UTC at the call sites
for intent clarity and consistency with parse_date.
Fixes#22529 (Logs page missing recent rows under non-UTC session TZ) and
the Global Usage single-day-returns-two-days symptom reported internally.
Non-admins previously skipped RBAC when prisma_client was None but could
still read payloads from custom loggers. Return 403 unless admin view.
Add test_ui_view_request_response_forbids_non_admin_without_db.
Made-with: Cursor
- Drop module-level common_utils import; import team helpers inside callers.
- Inline admin-view role check in _is_admin_view_safe to break import cycle.
- Require non-null row.user before treating spend log as owned by the key
(fixes None==None bypass for service keys).
- Document deferred proxy_server imports in _get_permitted_team_ids_for_spend_logs.
- Update tests (common_utils patches, regression test, ruff cleanups).
Made-with: Cursor
Keep both sets of tests: upstream's OAuth2 token injection test and
our case-insensitive tool matching tests. Use upstream's version of
the bedrock output_config test (more comprehensive).
When a request arrives without an API key, auth fails and the failure
hook triggers spend tracking. _is_master_key was called with api_key=None,
causing secrets.compare_digest to raise TypeError. Add a None guard for
api_key matching the existing guard for _master_key.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the messages or response JSON fields in spend logs are truncated
before being written to the database, the truncation marker now includes
a note explaining:
- This is a DB storage safeguard
- Full, untruncated data is still sent to logging callbacks (OTEL, Datadog, etc.)
- The MAX_STRING_LENGTH_PROMPT_IN_DB env var can be used to increase the limit
Also emits a verbose_proxy_logger.info message when truncation occurs in
the request body or response spend log paths.
Adds 3 new tests:
- test_truncation_includes_db_safeguard_note
- test_response_truncation_logs_info_message
- test_request_body_truncation_logs_info_message
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* fix(lint): suppress PLR0915 for 3 complex methods that exceed 50-statement limit
- streaming_iterator.py: _process_event (84 statements)
- transformation.py: translate_messages_to_responses_input (51 statements)
- transformation.py: transform_realtime_response (54 statements)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(mypy): resolve type errors in public_endpoints, user_api_key_auth, common_utils, transformation
- public_endpoints.py: fix _cached_endpoints type annotation
- user_api_key_auth.py: accept Optional[str] for end_user_id parameter
- common_utils.py: add NewProjectRequest/UpdateProjectRequest to Union type
- transformation.py: add ChatCompletionRedactedThinkingBlock and list[Any] to content type
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(proxy-extras): bump version to 0.4.50 and sync schema
- Bump litellm-proxy-extras from 0.4.49 to 0.4.50
- Sync schema.prisma with main proxy schema
- Includes new LiteLLM_ClaudeCodePluginTable model
- Includes new @@index([startTime, request_id]) on SpendLogs
- Update version references in requirements.txt and pyproject.toml
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(router): use string id in test_add_deployment and add defensive str() in register_model
- Change test to use string '100' instead of int 100 for model_info.id
- Add str() conversion in register_model to prevent AttributeError on non-string keys
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(security): update minimatch to 10.2.4 to fix CVE-2026-27903 and CVE-2026-27904
- Run npm audit fix in docs/my-website
- Updates minimatch from 10.2.1 to 10.2.4 (fixes HIGH severity ReDoS vulnerabilities)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(test): update realtime guardrail test assertions to match actual guardrail behavior
- test_text_message_blocked_by_guardrail_no_ai_response: allow guardrail's own block
message text in response.done (previously expected empty content)
- test_voice_transcript_blocked_by_guardrail: allow guardrail to send response.cancel
+ block message + response.create flow (previously expected no response.create)
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: revert proxy-extras version in requirements.txt and pyproject.toml
The litellm-proxy-extras 0.4.50 is not published to PyPI yet, so consumer
references must stay at 0.4.49. Only the source package pyproject.toml
should be bumped to 0.4.50 for the publish_proxy_extras CI job.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: make transcript delta check optional in voice guardrail test
The guardrail sends an error event (guardrail_violation) when blocking
voice transcripts; it does not always produce transcript deltas. Remove
the assertion requiring response.audio_transcript.delta since the error
event is the primary signal that blocked content was handled.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Add missing env keys to documentation: LITELLM_MAX_STREAMING_DURATION_SECONDS and LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES
These two environment variables were used in code but not documented in the
environment variables reference section of config_settings.md, causing the
test_env_keys.py CI test to fail.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Fix 13 mypy type errors across 6 files
- in_flight_requests_middleware.py: Fix type: ignore error codes from
[union-attr] to [attr-defined], add [arg-type] for Gauge **kwargs
- transformation.py: Add [assignment] ignore for output_format reassignment,
add fallback empty string for tool use id to fix arg-type
- responses/main.py: Remove redundant type annotation on second
secret_fields assignment to fix no-redef
- streaming_iterator.py: Add [assignment] ignores for intermediate
cache token assignments
- handler.py: Add [typeddict-item] ignore for AnthropicMessagesRequest
construction from dict
- public_endpoints.py: Add [arg-type] ignore for _load_endpoints()
return type mismatch with SupportedEndpoint model
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: add auth overrides to spend tracking tests, fix realtime guardrail assertion, update UI minimatch
- Add app.dependency_overrides for user_api_key_auth in 4 spend tracking tests
that were returning 401 Unauthorized (error_code, error_message,
error_code_and_key_alias, key_hash)
- Fix realtime guardrail test to check ANY error event for guardrail_violation
instead of just the first (OpenAI may send its own errors first)
- Update ui/litellm-dashboard/package-lock.json to fix minimatch vulnerability
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Fix failing MCP e2e and create_mcp_server UI tests
Test 1 (test_independent_clients_no_shared_session):
- Add allow_all_keys: true to MCP servers in test config. With master_key
and no DB, get_allowed_mcp_servers returned empty, causing 0 tools and
403 on tool calls. allow_all_keys bypasses per-key restrictions.
- Add asyncio.sleep(0.5) between client connections to allow MCP SDK
TaskGroup cleanup and avoid ExceptionGroup on connection close (MCP #915).
Test 2 (create_mcp_server 'auth value is provided'):
- Use userEvent.setup({ delay: null }) for instant keystrokes to avoid
timeout from default typing delay on CI.
- Increase per-test timeout to 15000ms for CI environments.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: stabilize proxy unit tests for parallel execution
- test_response_polling_handler: add xdist_group to prevent heavy import OOM
- test_db_schema_migration: use temp dir for worker isolation, sync schema.prisma index
- test_custom_tokenizer_bug: use lighter tokenizer to prevent OOM in parallel
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: add auth overrides to more spend tracking and model info tests
- Fix test_ui_view_spend_logs_pagination missing auth override (401)
- Fix test_view_spend_tags missing auth override (401)
- Fix test_view_spend_tags_no_database missing auth override (401)
- Fix test_empty_model_list.py to use app.dependency_overrides instead of patch()
for FastAPI dependency injection auth
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(test): use patch.object for aiohttp transport test to work in parallel execution
The @patch decorator was not intercepting the static method call in parallel
xdist workers. Using patch.object on the directly-imported class is more reliable.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(security): update minimatch from 10.2.1 to 10.2.4 in Dockerfile
The Docker image was explicitly pinning minimatch@10.2.1 which has HIGH
severity ReDoS vulnerabilities (GHSA-7r86-cg39-jmmj, GHSA-23c5-xmqv-rm74).
Update to 10.2.4 which includes fixes for both CVEs.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(ui): prevent MCP and TeamInfo test timeouts on CI
- Add userEvent.setup({ delay: null }) to all tests using userEvent in both files
- Add timeout: 15000 to tests with significant user interaction (typing, multiple clicks)
- Fixes: create_mcp_server Bearer Token test, TeamInfo cancel button test
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: stabilize parallel test execution and aiohttp transport test
- test_aiohttp_handler: rewrite transport test to not rely on static method mock
(consistently fails in parallel xdist workers)
- test_proxy_cli: add xdist_group to prevent timeout during heavy imports
- test_swagger_chat_completions: add xdist_group to prevent timeout
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(security): add serialize-javascript override to fix GHSA-5c6j-r48x-rmvq
Add npm override for serialize-javascript>=7.0.3 in docs/my-website
to fix HIGH severity RCE vulnerability via RegExp.flags.
Also bump minimatch override to >=10.2.4.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Fix flaky tests: remove broken Vertex model, add retries for Anthropic
- Remove vertex_ai/meta/llama-4-scout-17b-16e-instruct-maas from
test_partner_models_httpx_streaming - consistently returns 400 BadRequest
- Add @pytest.mark.flaky(retries=6, delay=10) to test_function_call_parsing
for transient Anthropic API overload errors
- Add @pytest.mark.flaky(retries=6, delay=10) to test_openai_stream_options_call
for transient Anthropic InternalServerError
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(ci): add xdist_group(proxy_heavy) to prevent OOM in parallel proxy tests
- Add pytestmark = pytest.mark.xdist_group('proxy_heavy') to test_proxy_utils.py
- Change test_db_schema_migration.py from schema_migration to proxy_heavy group
- Add @pytest.mark.xdist_group('proxy_heavy') to test_proxy_server.py::test_health
Groups heavy proxy tests to run on same worker, avoiding worker OOM crashes.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* Fix vertex AI qwen global endpoint test to mock vertexai module import
The test_vertex_ai_qwen_global_endpoint_url test was failing because the
VertexAIPartnerModels.completion() method tries to 'import vertexai' before
any of the mocked code runs. In environments without google-cloud-aiplatform
installed, this import fails with a VertexAIError(status_code=400).
Fix by:
- Adding patch.dict('sys.modules', {'vertexai': MagicMock()}) to mock the
vertexai module import
- Adding vertex_ai_location parameter to the acompletion call for completeness
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(ci): add xdist_group to health endpoint and watsonx tests for parallel stability
- test_health_liveliness_endpoint: add xdist_group('proxy_health') to prevent timeout
- test_watsonx_gpt_oss tests: add xdist_group('watsonx_heavy') to prevent mock interference
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(test): pre-populate WatsonX IAM token cache to prevent parallel test interference
The watsonx prompt transformation test was failing in parallel execution because
litellm.module_level_client.post mock was being interfered with by other tests.
Pre-populating the IAM token cache avoids the HTTP call entirely.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(test): add spend data polling with retries for e2e pass-through tests
- test_vertex_with_spend.test.js: Replace 15s fixed wait with polling loop
(up to 6 attempts, 10s apart) for spend data to appear in DB
- Increase test timeout from 25s to 90s to accommodate polling
- base_anthropic_messages_tool_search_test.py: Add flaky(retries=3) for
streaming test that depends on live Anthropic API
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(ci): reduce parallel workers from 8 to 4 for proxy tests to prevent OOM
- litellm_proxy_unit_testing_part2: -n 8 -> -n 4
- litellm_mapped_tests_proxy_part2: -n 8 -> -n 4, timeout 60 -> 120
- Worker crashes consistently caused by too many parallel proxy tests
each loading the full FastAPI app and heavy dependency tree
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(db): add migration for SpendLogs composite index (startTime, request_id)
The @@index([startTime, request_id]) was added to schema.prisma but had no
corresponding migration. This caused test_aaaasschema_migration_check to fail
because prisma migrate diff detected the missing index.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(db): add migration for MCP available_on_public_internet default change to true
The schema.prisma changed the default for available_on_public_internet from
false to true, but no migration was created. This caused the schema migration
test to detect drift.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(test): increase server wait time and add retry to flaky external API tests
- test_basic_python_version.py: increase server startup wait from 60s to 90s
for slower CI environments (fixes installing_litellm_on_python_3_13)
- test_a2a_agent.py: add flaky(retries=3, delay=5) for non-streaming test
that depends on live A2A agent endpoint
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(test): add flaky retries to all intermittent external API tests for 0-fail CI
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(test): add auth overrides to file endpoint tests that return 500
The test_target_storage tests were getting 500 because the FastAPI auth
dependency wasn't overridden. Added app.dependency_overrides for proper
auth bypass in test environment.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Backend - Spend Log Storage for Realtime Calls:
- Collect user voice transcripts and text input during WebSocket sessions
- Store collected messages in spend logs when store_prompts_in_spend_logs enabled
- Capture tool definitions from session.update and tool calls from response.done
- Enrich proxy_server_request with tools and response with tool_calls for UI
Backend - WebSocket Auth:
- Support browser-based auth via Sec-WebSocket-Protocol subprotocol
- Echo back subprotocol on WebSocket accept
UI - Realtime Playground:
- New RealtimePlayground component with WebSocket voice+text chat
- Mic recording (PCM16 24kHz), server VAD, audio playback, text input
- Handle binary WebSocket frames (Blob/ArrayBuffer decoding)
- Add /v1/realtime endpoint option to playground endpoint selector
UI - Tools Section for Realtime Logs:
- Extract tool calls from realtime response format (response.tool_calls
and response.results[].response.output[].type=function_call)
Tests:
- 15 new backend tests for realtime streaming and spend log storage
- 4 new UI tests for realtime tool call extraction
Fixes pre-existing build errors:
- ToolPolicies.tsx: duplicate import, antd styles type
- create_key_button.tsx: missing message import
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Add a `request_duration_ms` column to `LiteLLM_SpendLogs` to track request
duration. New rows are computed at write time. Legacy rows use a COALESCE
fallback in the `/spend/logs/ui` query to compute duration on the fly from
`endTime - startTime`. The field is also sortable in the UI endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
test_vertex_passthrough_with_default_credentials and
test_view_spend_logs_with_date_range_summarized fail intermittently when a
prior xdist worker sets master_key — auth then rejects the unauthenticated
test requests before the code under test is reached.
- mock user_api_key_auth in test_vertex_passthrough_with_default_credentials
(same pattern used for test_vertex_passthrough_with_no_default_credentials
in #21810)
- wrap test_view_spend_logs_with_date_range_summarized in
app.dependency_overrides[ps.user_api_key_auth] with try/finally cleanup
(same pattern used for the other spend log tests in #21810)
- TestSpendLogsPayload: save/restore litellm.cache in setup_method/teardown_method
so tests that run after a cache-setting test don't see a non-None cache and get
a hash instead of "Cache OFF" in the cache_key field
- test_use_prisma_db_push_flag_behavior: apply clean_env pattern (strip DATABASE_URL/DIRECT_URL,
then set DATABASE_URL to test value) inside the with block instead of using @patch.dict
decorator, matching the pattern from test_skip_server_startup to avoid Click 8.3.x
StreamMixer stream lifecycle issues in CI
Tests hardcode expected api_base as https://api.anthropic.com/v1/messages but
if ANTHROPIC_BASE_URL is set in the environment the recorded api_base changes,
causing a mismatch. Clear both env vars via monkeypatch at the start of each test.
* fix(tests): add app.dependency_overrides for auth in spend logs tests
test_ui_view_spend_logs_with_status, test_ui_view_spend_logs_with_model,
test_ui_view_spend_logs_with_model_id, and test_view_spend_logs_summarize_parameter
all send Bearer sk-test without mocking user_api_key_auth. When a prior test
in the same xdist worker sets master_key, the auth check fails for sk-test
and the test fails intermittently.
Fix: use app.dependency_overrides[ps.user_api_key_auth] to bypass auth,
same pattern as other tests in the same file.
* fix(tests): mock user_api_key_auth in test_vertex_passthrough_with_no_default_credentials
vertex_proxy_route calls user_api_key_auth internally. When a prior test in the
same xdist worker sets master_key, the auth check fails for the test request
and create_pass_through_route is never called, causing assert_called_once_with to fail.
Fix: patch user_api_key_auth as an AsyncMock in the with mock.patch() block.
* fix(tests): isolate flaky files endpoint tests from global proxy state
* test(secret_managers): add mocked unit test for write/read JSON secret cycle
* fix(tests): restore litellm.callbacks in TestSpendLogsPayload setup/teardown
* fix(tests): clear app.openapi_schema in TestSwaggerChatCompletions setup/teardown
* fix(tests): add flaky marker to test_async_increment_tokens_with_ttl_preservation
* fix: reset db_health_cache in source module to prevent stale cache hits
The test was reassigning db_health_cache via `global` in the test module,
which doesn't affect the _health_endpoints module's variable. When a prior
test set the cache to "connected" within 2 minutes, _db_health_readiness_check
returned early without calling health_check(), causing assert_called_once to fail.
Also use PrismaError with a connection message so it's properly recognized
as a connection error by PrismaDBExceptionHandler.is_database_connection_error.
* fix: replace asyncio.sleep with polling loop in spend logs tests
The GLOBAL_LOGGING_WORKER processes callbacks via an async queue, so
asyncio.sleep(1) is a race condition - under CI load the worker may not
have processed the queued task within 1 second. Replace with a polling
helper that waits up to 10 seconds for the mock to be called.
Also add metadata.attempted_retries and metadata.max_retries to
ignored_keys since these are new fields.
* fix: isolate test_skip_server_startup from CI environment
Remove mix_stderr=False (unsupported in some Click versions). Strip
DATABASE_URL/DIRECT_URL from environment during the test to prevent
real prisma operations when these are set in CI.
- Add new SpendLogsMetadata keys to ignored_keys in spend logs tests
(regression from ccecc10c82 which intentionally includes all keys)
- Mock PrismaManager.setup_database and should_update_prisma_schema in
proxy CLI tests to prevent real DB migrations from running in CI
- Use CliRunner(mix_stderr=False) to fix Click stream lifecycle issues
- Use unique UUID suffix for Redis TTL test keys to avoid stale state
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add attempted_retries and max_retries fields to SpendLogsMetadata so the
Logs page can display how many retries occurred for each request. The
router now injects retry tracking metadata before each make_call, which
flows through the logging pipeline into the spend logs metadata JSON.
The UI shows "Not Retried" when the first attempt succeeded, and
"N / M" (attempted / max) when retries occurred. The field is hidden
for requests that did not go through the router.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ishaan Jaffer