Commit Graph
33178 Commits
Author SHA1 Message Date
0fe3145e83 Fix/presidio controls (#21798)
* check should_run_guardrail in sync logging hook path

* Add tests for CustomGuardrail logging behavior

Added tests to ensure CustomGuardrail logging behavior based on the guardrail execution state.

---------

Co-authored-by: Miguel Armenta <ma826r@att.com>
2026-02-21 12:39:17 -08:00
ryan-crabbeandGitHub f5139716a1 Merge pull request #20882 from ryan-crabbe/perf/optimize-model-dump-preserved-fields
perf: optimize model_dump_with_preserved_fields
2026-02-21 12:36:02 -08:00
Ishaan JaffandGitHub daa682e125 fix(tests): add missing start_db_health_watchdog_task mock (#21804)
* fix(tests): add missing start_db_health_watchdog_task mock in test_proxy_server_prisma_setup

* fix(tests): add missing start_db_health_watchdog_task mock in test_health_check_not_called_when_disabled
2026-02-21 12:31:52 -08:00
Ishaan Jaffer 0b69c21eca Revert "fix(http_handler): bypass cache when shared_session is provided for aiohttp tracing (#20630)"
This reverts commit 7ee36c2a3a.
2026-02-21 12:27:46 -08:00
ryan-crabbeandGitHub d7b3b2eb9e Merge pull request #20374 from ryan-crabbe/perf/cache-access-groups
perf: cache get_model_access_groups() no-args result on Router
2026-02-21 12:25:45 -08:00
ryan-crabbeandGitHub 469951466e Merge pull request #20541 from ryan-crabbe/perf/cost-calculator-optimizations
Perf/cost calculator optimizations
2026-02-21 12:18:38 -08:00
ryan-crabbeandGitHub a8dbcb1a30 Merge pull request #20526 from ryan-crabbe/perf/add-litellm-data-to-request-optimizations
Perf: add_litellm_data_to_request optimizations
2026-02-21 12:16:05 -08:00
ryan-crabbeandGitHub d04e5c6a3e Merge pull request #20448 from ryan-crabbe/perf/optimize-completion-cost
perf: optimize completion_cost()
2026-02-21 12:15:48 -08:00
Ishaan JaffandGitHub 08ae43ace1 fix(migrations): add ensure_project_id migration + bump litellm-proxy-extras to 0.4.46 (#21800)
* fix(migrations): add ensure_project_id_verification_token migration

Ensures project_id column exists on LiteLLM_VerificationToken. The original
migration (20251113000000_add_project_table) adds this column, but may have
been skipped if LiteLLM_ProjectTable already existed and the migration was
resolved as idempotent. Uses IF NOT EXISTS for safety.

* bump: litellm-proxy-extras 0.4.45 → 0.4.46
2026-02-21 12:15:21 -08:00
Ryan Crabbe dc2c835e7b perf: optimize completion_cost() — eliminate enum overhead, reduce function call indirection
Pre-resolve CallTypes enum values into module-level frozensets to avoid
repeated .value attribute access in the elif chain. Inline the hot-path
_store_cost_breakdown_in_logging_obj as a direct dict literal. Remove
unnecessary cast(CallTypesLiteral, call_type) call. Guard
_get_additional_costs() with azure_ai-only check since no other provider
implements additional costs.

Line profile shows 20.5% reduction in completion_cost() total time
(7.14s → 5.68s across 6,006 calls). The four targeted bottlenecks
dropped from 2.82s to 0.28s combined.
2026-02-21 12:14:55 -08:00
ryan-crabbeandGitHub d8dda93bba Merge pull request #20593 from ryan-crabbe/perf/reuse-litellm-params
perf: reuse LiteLLM_Params
2026-02-21 12:11:02 -08:00
ryan-crabbeandGitHub be016b683b Merge pull request #20440 from ryan-crabbe/perf/skip-duplicate-logging-payload
perf: skip duplicate get_standard_logging_object_payload for non-streaming req's
2026-02-21 12:07:31 -08:00
ryan-crabbeandGitHub 1ab11396f0 Merge pull request #20434 from ryan-crabbe/perf/prometheus-asgi-middleware
Perf/prometheus asgi middleware
2026-02-21 12:07:07 -08:00
Ryan Crabbe 498dad0af1 test: add backwards compatibility tests for PrometheusAuthMiddleware
Add tests verifying non-metrics requests pass through unaffected and
don't trigger auth even when auth is enabled.
2026-02-21 12:06:31 -08:00
Ryan Crabbe 6ae2d6e73b perf: convert PrometheusAuthMiddleware from BaseHTTPMiddleware to pure ASGI
BaseHTTPMiddleware creates a new asyncio task, wraps the response in a
StreamingResponse, and coordinates via events on every request — even for
non-/metrics paths where this middleware is a pure passthrough. This added
~3.8s of overhead in profiled benchmarks (15.3s total attributed time).

The pure ASGI implementation checks scope["path"] directly and passes
through to the inner app with zero object construction for non-/metrics
requests. A Request object is only created when auth is actually needed.
2026-02-21 12:06:31 -08:00
Ishaan JaffandGitHub 87feca0b4a fix: 2 failing CI tests in litellm_mapped_tests_proxy_part2 (#21797)
* fix(test): remove deprecated Click mix_stderr param in test_use_prisma_db_push_flag_behavior

Click 8.2+ removed the mix_stderr parameter from CliRunner. Use CliRunner() without it.

* fix(test): use app.dependency_overrides for auth mock in test_role_mappings_stored_and_retrieved

monkeypatch.setattr doesn't affect FastAPI's Depends() resolution in parallel
test execution. Use app.dependency_overrides which is the proper FastAPI pattern.
2026-02-21 12:04:58 -08:00
Ishaan JaffandGitHub a939fa37ae fix(proxy): restore broad is_database_connection_error; add is_database_transport_error for reconnect (#21796)
Any PrismaError should be treated as a DB connection error for the
allow_requests_on_db_unavailable feature and 503 responses. The narrow
keyword-based check is now in is_database_transport_error, which is
what the reconnect logic in auth_checks.py should use.

Fixes test_delete_access_group_503_on_db_connection_error and
test_handle_authentication_error_db_unavailable failures caused by
PR #21706 narrowing is_database_connection_error.
2026-02-21 12:04:00 -08:00
Ryan Crabbe 461a01311b perf: skip duplicate get_standard_logging_object_payload for non-streaming requests
- Add early return in _get_assembled_streaming_response for non-streaming
  requests, preventing duplicate standard_logging_object computation
- Move emit_standard_logging_payload into _process_hidden_params_and_response_cost
  so non-streaming requests still emit the debug payload
- Add emit_standard_logging_payload for dict/list result edge cases
2026-02-21 12:01:21 -08:00
Ishaan JaffandGitHub 74ef034110 fix(tests): add flaky retries to flaky CI tests (#21795)
* fix(tests): add flaky retries and error handling to test_create_eval

* fix(tests): add flaky retries to test_cohere_v2_conversation_history

* fix(tests): add flaky retries to test_gemini_url_context
2026-02-21 11:56:29 -08:00
Ishaan JaffandGitHub a0e76f4f25 fix(tests): mock httpx in RPM limit pass-through tests (#21793)
* fix(tests): isolate flaky files endpoint tests from global proxy state

* test(secret_managers): add mocked unit test for write/read JSON secret cycle

* fix(tests): mock httpx in rpm limit pass-through tests to avoid real Cohere API calls
2026-02-21 11:55:48 -08:00
Ryan Crabbe f976a01fce fix: use precise tuple[str, str] type annotation for _PRESERVED_NONE_FIELDS 2026-02-21 11:34:30 -08:00
Ishaan JaffandGitHub 349516280f fix(ui): fix failing ui_unit_tests (#21792)
- TeamMemberTab: pass canEditTeam=true so Add Member button renders
- UsagePageView: remove stale banner assertions (org/customer/agent banners were removed from source)
- LogDetailContent: scope '-' query to Provider description item using within() to avoid multiple-match error
2026-02-21 11:33:28 -08:00
Ishaan JaffandGitHub c6a8034184 fix(tests): isolate flaky tests - restore global state in setup/teardown (#21791)
* fix(tests): isolate flaky files endpoint tests from global proxy state

* test(secret_managers): add mocked unit test for write/read JSON secret cycle

* fix(tests): restore litellm.callbacks in TestSpendLogsPayload setup/teardown

* fix(tests): clear app.openapi_schema in TestSwaggerChatCompletions setup/teardown

* fix(tests): add flaky marker to test_async_increment_tokens_with_ttl_preservation
2026-02-21 11:30:35 -08:00
Ishaan JaffandGitHub 21a549d78d fix(tests): isolate flaky files endpoint tests from global proxy state (#21788)
* fix(tests): isolate flaky files endpoint tests from global proxy state

* test(secret_managers): add mocked unit test for write/read JSON secret cycle
2026-02-21 11:20:32 -08:00
Ishaan JaffandGitHub 2acc5cc457 fix(security): fix CVE-2025-69873, CVE-2026-26996 in docs deps; allowlist nodejs_wheel CVEs in Grype scan (#21787)
* fix(security): fix CVE-2025-69873 and CVE-2026-26996 in docs dependencies

Use npm overrides to pin patched versions:
- ajv@6.12.6 → 6.14.0 (fixes ReDoS CVE-2025-69873)
- ajv@8.17.1 → 8.18.0 (fixes ReDoS CVE-2025-69873)
- minimatch@3.1.2 → 10.2.1 (fixes DoS CVE-2026-26996)

serve-handler only calls minimatch(path, pattern) so the 3.x→10.x
upgrade is safe.

* fix(ruff): add missing Set and Dict imports to fix F821 errors

* fix(security): scope ajv overrides to avoid top-level version conflict

Replacing global 'ajv: 8.18.0' override with scoped 'schema-utils@4'
override. The global override conflicted with the nested file-loader/
null-loader/url-loader overrides, causing npm to install ajv@6 at the
top level where ajv-keywords@5.x requires ajv@8 (ajv/dist/compile/codegen).

Now:
- schema-utils@3 + loaders → ajv@6.14.0 (safe minor bump)
- schema-utils@4 → ajv@8.18.0 (safe minor bump)
- top-level ajv unmodified (stays at 8.x for ajv-keywords@5)

* fix(security): allowlist minimatch and tar CVEs from nodejs_wheel, bump tar override to >=7.5.8
2026-02-21 11:18:52 -08:00
Ishaan JaffandGitHub d95c3e9cd4 fix(ruff): add missing Set and Dict imports (F821) (#21785)
* fix(ruff): add missing Set and Dict imports to fix F821 errors

* fix(ci): add semantic_router dep to guardrails_testing job

The test_build_routes_combined_templates test calls build_routes() which
imports from semantic_router, but the guardrails_testing CI job didn't
install it.
2026-02-21 10:56:27 -08:00
Ishaan JaffandGitHub 8a145da793 fix(security): fix CVE-2025-69873 and CVE-2026-26996 in docs dependencies (#21782)
* fix(security): fix CVE-2025-69873 and CVE-2026-26996 in docs dependencies

Use npm overrides to pin patched versions:
- ajv@6.12.6 → 6.14.0 (fixes ReDoS CVE-2025-69873)
- ajv@8.17.1 → 8.18.0 (fixes ReDoS CVE-2025-69873)
- minimatch@3.1.2 → 10.2.1 (fixes DoS CVE-2026-26996)

serve-handler only calls minimatch(path, pattern) so the 3.x→10.x
upgrade is safe.

* fix(ruff): add missing Set and Dict imports to fix F821 errors

* fix(security): scope ajv overrides to avoid top-level version conflict

Replacing global 'ajv: 8.18.0' override with scoped 'schema-utils@4'
override. The global override conflicted with the nested file-loader/
null-loader/url-loader overrides, causing npm to install ajv@6 at the
top level where ajv-keywords@5.x requires ajv@8 (ajv/dist/compile/codegen).

Now:
- schema-utils@3 + loaders → ajv@6.14.0 (safe minor bump)
- schema-utils@4 → ajv@8.18.0 (safe minor bump)
- top-level ajv unmodified (stays at 8.x for ajv-keywords@5)
2026-02-21 10:56:11 -08:00
Ryan Crabbe c071e01fb6 fix: address PR review comments for model_dump_with_preserved_fields
- Restore preserve_fields param for backward compatibility (deprecated)
- Use zip() instead of index-based iteration to prevent IndexError
- Add backward compatibility test
2026-02-21 10:48:10 -08:00
Ishaan JaffandGitHub a5e886de79 fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var instead of hardcoding model (#21781)
* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in bedrock KB tests

* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in test_router

* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in test_router_retries

* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in test_router_timeout
2026-02-21 10:46:49 -08:00
Ishaan JaffandGitHub 0726bdb67c fix(tests): update gcs pubsub v1 fixture with new SpendLogsMetadata fields (#21779)
SpendLogsMetadata added new fields (user_api_key, status, error_information,
etc.) that weren't in the expected spend_logs_payload.json fixture, causing
test_async_gcs_pub_sub_v1 to fail.
2026-02-21 10:40:26 -08:00
32b09b29fc feat(ui): add forward_client_headers_to_llm_api toggle to general settings (#21776)
* feat(ui): add forward_client_headers_to_llm_api toggle to general settings

* feat(ui): add forward_client_headers_to_llm_api toggle to UI Settings tab

- Add toggle to UISettings.tsx frontend (switch + label + description)
- Add field to UISettings model and ALLOWED_UI_SETTINGS_FIELDS
- Sync setting to general_settings on get/update so proxy picks it up at runtime

---------

Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
2026-02-21 10:38:40 -08:00
Ishaan JaffandGitHub c5b695e71c fix(test): skip 'projects' field in team update assertion (#21777) 2026-02-21 10:24:53 -08:00
Ishaan JaffandGitHub afdf70be73 fix(test): update claude model name in test_get_valid_models_from_dynamic_api_key (#21771) 2026-02-21 10:15:23 -08:00
Ishaan JaffGitHubgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
3840497109 fix: resolve 7 mypy linting errors on main (#21768)
* fix(types): add session_id to LitellmMetadataFromRequestHeaders TypedDict

* fix(mypy): resolve no-redef and typeddict-item errors in competitor intent base

* fix(mypy): add type annotation for words_in_match in airline competitor check

* fix(mypy): suppress typeddict-item error for tracing_kw expansion

* fix(mypy): add type annotations in test_eval.py

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-21 10:05:59 -08:00
Ishaan JaffandGitHub 3e67cb5287 fix: resolve flaky test failures in health, spend logs, and CLI tests (#21769)
* fix: reset db_health_cache in source module to prevent stale cache hits

The test was reassigning db_health_cache via `global` in the test module,
which doesn't affect the _health_endpoints module's variable. When a prior
test set the cache to "connected" within 2 minutes, _db_health_readiness_check
returned early without calling health_check(), causing assert_called_once to fail.

Also use PrismaError with a connection message so it's properly recognized
as a connection error by PrismaDBExceptionHandler.is_database_connection_error.

* fix: replace asyncio.sleep with polling loop in spend logs tests

The GLOBAL_LOGGING_WORKER processes callbacks via an async queue, so
asyncio.sleep(1) is a race condition - under CI load the worker may not
have processed the queued task within 1 second. Replace with a polling
helper that waits up to 10 seconds for the mock to be called.

Also add metadata.attempted_retries and metadata.max_retries to
ignored_keys since these are new fields.

* fix: isolate test_skip_server_startup from CI environment

Remove mix_stderr=False (unsupported in some Click versions). Strip
DATABASE_URL/DIRECT_URL from environment during the test to prevent
real prisma operations when these are set in CI.
2026-02-21 10:02:24 -08:00
Ishaan JaffandGitHub 061a3cdc3e Fix ruff PLR0915 lint errors (#21766)
* fix(content_filter): extract helpers to reduce __init__ statement count

Fixes PLR0915 ruff lint error (too many statements).

* fix(test_eval): extract print/save helpers to reduce statement count

Fixes PLR0915 ruff lint error (too many statements).

* fix(proxy/utils): extract lock-timeout logic to reduce statement count

Fixes PLR0915 ruff lint error (too many statements).
2026-02-21 09:48:31 -08:00
Henrique CavarsanandGitHub 977ad015ca fix(proxy): recover from prisma-query-engine zombie process (#21707) 2026-02-21 09:31:44 -08:00
yuneng-jiangandGitHub 1e452338b9 Merge pull request #21765 from BerriAI/yj_build_fix
[Infra] Building UI for Release
2026-02-21 09:15:09 -08:00
yuneng-jiang b35869a3ba chore: update Next.js build artifacts (2026-02-21 17:14 UTC, node v22.16.0) 2026-02-21 09:14:05 -08:00
yuneng-jiang 8ebaeb7229 fixing build 2026-02-21 09:13:33 -08:00
yuneng-jiangandGitHub 425f4e53f1 Merge pull request #21502 from milan-berri/fix/ui-model-credentials-batch-files
fix: resolve credentials for UI-created models in batch file uploads
2026-02-21 09:05:00 -08:00
Harshit JainandGitHub e463fc22d9 Merge pull request #21763 from Harshit28j/litellm_feat_sticky_sessions
feat: add session_id to have better routing
2026-02-21 21:21:52 +05:30
Harshit JainandGitHub f3ff9bf54f Merge pull request #21009 from BerriAI/litellm_docker-count-no-req
add tests for hotpath & docker container
2026-02-21 20:57:28 +05:30
Harshit Jain 456d8f5524 feat: add session_id to have better routing 2026-02-21 18:45:50 +05:30
github-actions[bot]GitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
e487d711d6 chore: regenerate poetry.lock to match pyproject.toml (#21758)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-21 11:26:55 +00:00
Harshit JainandGitHub 1df9bd96c7 Merge pull request #21743 from Harshit28j/litellm_rollback_safety_checks
doc: add rollback safety check
2026-02-21 13:13:43 +05:30
kang hee yongandGitHub 03ecdf9284 feat(openrouter): add openrouter/minimax/minimax-m2.5 pricing (#21664) 2026-02-20 22:24:58 -08:00
yuneng-jiangandGitHub 1fb7320b6d Merge pull request #21745 from BerriAI/litellm_org_member_email_ui
[Feature] UI - Organization Info: Show member email, AntD tabs, reusable MemberTable
2026-02-20 21:49:27 -08:00
yuneng-jiangandClaude Sonnet 4.6 ffc673da09 fix: use include user:True instead of nested select for Prisma Python compat
Prisma Python client does not support nested select within include.
Use include user:True to fetch the full user object; model_validator
extracts user_email from it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 21:41:53 -08:00
yuneng-jiangandClaude Sonnet 4.6 4597d34344 feat: org info page - AntD tabs, MemberTable with user_email
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 21:41:53 -08:00