Commit Graph

6748 Commits

Author SHA1 Message Date
Julio Quinteros Pro fb8b11cc0a fix(tests): use counter-based mock for time.time in prisma self-heal test
The test used a fixed side_effect list for time.time(), but the number
of calls varies by Python version, causing StopIteration on 3.12 and
AssertionError on 3.14. Replace with an infinite counter-based callable
and assert the timestamp was updated rather than checking for an exact
value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 13:25:02 -03:00
Sameer Kankute f97ee62fb0 Merge pull request #21909 from BerriAI/litellm_cost_tracking_gemini
Add  Priority PayGo cost tracking gemini/vertex ai
2026-02-23 18:58:57 +05:30
Sameer Kankute 61e63b6553 Merge pull request #21904 from BerriAI/litellm_fix_model_cost_map
fix model cost map for anthropic fast and inference_geo
2026-02-23 18:57:15 +05:30
Sameer Kankute 2f8d36be1b Fix test_aaamodel_prices_and_context_window_json_is_valid 2026-02-23 18:56:12 +05:30
Sameer Kankute 9b5bbee906 Merge pull request #21786 from BerriAI/litellm_oss_staging_02_21_2026
Litellm oss staging 02 21 2026
2026-02-23 18:51:55 +05:30
Sameer Kankute 8decf04d8a Merge pull request #21877 from BerriAI/litellm_oss_staging_02_22_2026
Litellm oss staging 02 22 2026
2026-02-23 18:50:47 +05:30
Sameer Kankute 37d45139f2 Merge pull request #21917 from BerriAI/litellm_fix_model_cost_map_wildcard
Fix: Anthropic model wildcard access issue
2026-02-23 18:45:49 +05:30
TomAlon 99184c48d9 Add Noma guardrails v2 based on custom guardrails (#21400) 2026-02-23 05:05:27 -08:00
Sameer Kankute c7aafdf794 Merge pull request #21926 from BerriAI/main
merge main in oss 21 02
2026-02-23 18:17:30 +05:30
Sameer Kankute 57af8e6a93 Merge pull request #21924 from BerriAI/main
merge main in oss 22 02
2026-02-23 18:11:36 +05:30
Sameer Kankute 4ff1651699 Fix: Anthropic model wildcard access issue 2026-02-23 17:12:55 +05:30
Harshit Jain 9fc3c77c42 fix: ensure arrival_time is set before calculating queue time 2026-02-23 17:04:47 +05:30
Sameer Kankute 3561bfb96c Merge pull request #21640 from ta-stripe/feat/regional-sts-endpoint-for-auth_with_role_name
feat(bedrock): support optional regional STS endpoint in role assumption
2026-02-23 13:28:54 +05:30
Sameer Kankute 55ee8cdd56 Merge pull request #21701 from ta-stripe/fix/bedrock-openai-imported-model-name-encoding
fix(bedrock): encode model arns for OpenAI compatible bedrock imported models
2026-02-23 13:25:02 +05:30
Sameer Kankute f54fb9aeb1 Add tests for fast and us 2026-02-23 11:25:47 +05:30
Sameer Kankute 22bccc4f61 Fix entries with fast and us/ 2026-02-23 11:23:24 +05:30
Harshit Jain 304862175e Merge branch 'main' into litellm_fix_langfuse_otel_trace_v2 2026-02-22 19:54:05 +05:30
Harshit28j 1cc185eb16 fix: update opentelemetry with greptile review 2026-02-22 19:53:18 +05:30
Krish Dholakia 76ccc9e844 Guardrail Policy Versioning (#21862)
* feat: initial commit, adding support for policy versioning on litellm

* fix(policy_registry): support policy versioning

* fix: multiple QA fixes for policy flow builder with guardrail versioning on litellm

* feat: ui improvements

* feat: add prisma migration

* fix: address greptile fixes
2026-02-21 20:14:31 -08:00
Krish Dholakia 52585eb2d7 Revert "fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)" (#21876)
This reverts commit bce078a796.
2026-02-21 20:12:01 -08:00
Edwin Isac bce078a796 fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)
* server root path regression doc

* fixing syntax

* fix: replace Zapier webhook with Google Form for survey submission (#21621)

* Replace Zapier webhook with Google Form for survey submission

* Add back error logging for survey submission debugging

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth"

This reverts commit 0e1db3f7e4, reversing
changes made to 7e2d6f2355.

* test_vertex_ai_gemini_2_5_pro_streaming

* UI new build

* fix rendering

* ui new build

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* release note docs

* docs

* adding image

* fix(vertex_ai): enable context-1m-2025-08-07 beta header

The `context-1m-2025-08-07` Anthropic beta header was set to `null` for vertex_ai,
causing it to be filtered out when users set `extra_headers: {anthropic-beta: context-1m-2025-08-07}`.

This prevented using Claude's 1M context window feature via Vertex AI, resulting in
`prompt is too long: 460500 tokens > 200000 maximum` errors.

Fixes #21861

---------

Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: milan-berri <milan@berri.ai>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2026-02-21 20:11:13 -08:00
LeeJuOh 50f36d9ca6 fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo (#21754)
* fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo

* fix(budget): update stale docstring on get_budget_reset_time
2026-02-21 19:35:06 -08:00
Ishaan Jaffer 52294029a0 test_vertex_ai_gemini_2_5_pro_streaming 2026-02-21 16:59:22 -08:00
Ishaan Jaffer 2270a3aaf3 Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth"
This reverts commit 0e1db3f7e4, reversing
changes made to 7e2d6f2355.
2026-02-21 16:57:42 -08:00
Ryan Crabbe 643c9b6c04 Merge remote-tracking branch 'origin/main' into litellm_perf_user_api_key_auth 2026-02-21 16:03:31 -08:00
Ishaan Jaffer d31d5b8486 fix failing tests 2026-02-21 15:48:26 -08:00
Ishaan Jaffer 26ea29afd3 test_get_usage_as_dict 2026-02-21 15:39:06 -08:00
Krish Dholakia 1f7eeb274c Agent Builder - improve rejected response detection based on agent response (#21850)
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent

* feat: new response rejection detector

* fix: multiple fixes

* feat: add mcp tools support to agent builder

create an agent with access to llm's + mcp servers
2026-02-21 15:34:42 -08:00
Krish Dholakia 9fc6fd647c Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817)
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent
2026-02-21 15:32:47 -08:00
Ishaan Jaff bab4127cae fix(tests): fix flaky test_use_prisma_db_push_flag_behavior (#21849)
Replace Click CliRunner with standalone_mode=False to avoid
"I/O operation on closed file" errors caused by Click's stream
isolation in CI environments.
2026-02-21 15:23:55 -08:00
Ishaan Jaff f74a1c94df test(router): add coverage tests for _is_complexity_router_deployment and init_complexity_router_deployment (#21848) 2026-02-21 15:21:10 -08:00
Darien Kindlund ca5c109a92 feat: add optional digest mode for Slack alert types (#21683)
Adds per-alert-type digest mode that aggregates duplicate alerts
within a configurable time window and emits a single summary message
with count, start/end timestamps.

Configuration via general_settings.alert_type_config:
  alert_type_config:
    llm_requests_hanging:
      digest: true
      digest_interval: 86400

Digest key: (alert_type, request_model, api_base)
Default interval: 24 hours
Window type: fixed interval

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-02-21 15:19:17 -08:00
Ryan Crabbe c7ad8053b1 Merge origin/main into litellm_perf_user_api_key_auth
Resolve conflicts:
- pass_through_endpoints.py: take main's version, re-apply
  MAPPED_PASS_THROUGH_PREFIXES startswith(tuple) optimization
- test_user_api_key_auth.py: keep both auth optimization regression
  tests and JWT admin identity field tests
2026-02-21 15:14:20 -08:00
Ishaan Jaff 9f459c5c57 fix(logging): preserve pass-through endpoint response_cost (#21844)
* fix(logging): preserve pass-through endpoint response_cost in async_success_handler

Two places in the logging pipeline were overwriting response_cost that
pass-through handlers (Gemini/Vertex) had already calculated:

1. _process_hidden_params_and_response_cost fell through to
   _response_cost_calculator which returns None for pass-through calls
2. async_success_handler pass-through branch unconditionally set
   response_cost = None (introduced in PR #19887)

Now both places check if response_cost is already set before overwriting.

* test: add regression test for pass-through endpoint response_cost preservation
2026-02-21 15:09:45 -08:00
Ryan Crabbe bb24ebebd9 Merge origin/main into litellm_perf_convert_model_response_frozensets
Resolve conflict: keep main's provider_specific_fields passthrough
preservation while using frozenset set-difference optimization.
2026-02-21 14:52:43 -08:00
Ishaan Jaff 8afeaf8da4 fix(tests): fix flaky test_create_vertex_fine_tune_jobs_mocked - handle background Datadog flush (#21838) 2026-02-21 14:44:01 -08:00
Ishaan Jaff d7b22d340b fix(tests): move test_router_azure_acompletion to llm_translation testing (#21837) 2026-02-21 14:41:53 -08:00
Ishaan Jaff 59e5b7e8c6 fix(tests): use monkeypatch.setenv for Redis pool max_connections tests (#21834)
Replace patch('litellm._redis._get_redis_client_logic') with monkeypatch.setenv
in test_max_connections_url_config and test_max_connections_url_config_string_value.

The mock was unreliable in CI (REDIS_URL is set to the real Redis Cloud server),
causing the pool to silently use the real config instead of the test config.
Using monkeypatch.setenv tests the full env-var→pool chain more robustly and
matches the actual production code path.
2026-02-21 14:38:28 -08:00
Ishaan Jaff 235a47c576 fix(tests): mock test_claude_tool_use_with_gemini to fix flaky CI (#21832)
* ui fixes

* fix(tests): mock test_claude_tool_use_with_gemini to avoid MALFORMED_FUNCTION_CALL flakiness
2026-02-21 14:34:54 -08:00
Ishaan Jaff fb4249005e fix(tests): add atexit.register mock to prevent Click isolation stream closure in test_use_prisma_db_push_flag_behavior (#21829) 2026-02-21 14:28:02 -08:00
yuneng-jiang 8c5be4cb62 Revert "fix(proxy): recover from prisma-query-engine zombie process (#21707)"
This reverts commit 977ad015ca.
2026-02-21 14:20:06 -08:00
Ishaan Jaff 56a095a079 fix(tests): update deprecated Anthropic model in test_user_model_access (#21826)
* fix(tests): clear _async_success_callback in vertex fine-tune mocked tests to prevent Datadog interference

* fix(tests): update deprecated claude-3-5-haiku-20241022 to claude-haiku-4-5-20251001
2026-02-21 14:18:24 -08:00
Ishaan Jaff a1ead765ed fix(tests): clear _async_success_callback in vertex fine-tune mocked tests to prevent Datadog interference (#21825) 2026-02-21 14:16:18 -08:00
Ishaan Jaff 494aad4a68 fix(tests): isolate auth in vertex passthrough and spend logs date range tests (#21824)
test_vertex_passthrough_with_default_credentials and
test_view_spend_logs_with_date_range_summarized fail intermittently when a
prior xdist worker sets master_key — auth then rejects the unauthenticated
test requests before the code under test is reached.

- mock user_api_key_auth in test_vertex_passthrough_with_default_credentials
  (same pattern used for test_vertex_passthrough_with_no_default_credentials
  in #21810)
- wrap test_view_spend_logs_with_date_range_summarized in
  app.dependency_overrides[ps.user_api_key_auth] with try/finally cleanup
  (same pattern used for the other spend log tests in #21810)
2026-02-21 14:14:50 -08:00
Ishaan Jaff dd6a74da63 fix(tests): isolate litellm.cache and CLI env vars in flaky tests (#21821)
- TestSpendLogsPayload: save/restore litellm.cache in setup_method/teardown_method
  so tests that run after a cache-setting test don't see a non-None cache and get
  a hash instead of "Cache OFF" in the cache_key field
- test_use_prisma_db_push_flag_behavior: apply clean_env pattern (strip DATABASE_URL/DIRECT_URL,
  then set DATABASE_URL to test value) inside the with block instead of using @patch.dict
  decorator, matching the pattern from test_skip_server_startup to avoid Click 8.3.x
  StreamMixer stream lifecycle issues in CI
2026-02-21 14:11:48 -08:00
Ishaan Jaff 6acfa1c71d fix(tests): clear ANTHROPIC_BASE_URL/ANTHROPIC_API_BASE in spend log api_base tests (#21820)
Tests hardcode expected api_base as https://api.anthropic.com/v1/messages but
if ANTHROPIC_BASE_URL is set in the environment the recorded api_base changes,
causing a mismatch. Clear both env vars via monkeypatch at the start of each test.
2026-02-21 14:11:18 -08:00
Ishaan Jaff 6ec16d583d fix(test): add timeout to flush() call to prevent 300s hang in CI (#21819)
GLOBAL_LOGGING_WORKER.flush() calls queue.join() which blocks until all
items are task_done(). In CI with pytest-asyncio, each test gets a fresh
event loop so the worker reinitializes its queue - items from a previous
test never get task_done(), causing an infinite hang.

Fix: wrap flush() with asyncio.wait_for(..., timeout=10.0).
2026-02-21 14:10:57 -08:00
ryan-crabbe b17d37eceb Merge pull request #21815 from BerriAI/litellm_fix_openai_init_params_immutable
fix: make cached OpenAI init params immutable and fix import ordering
2026-02-21 13:33:03 -08:00
Ishaan Jaff c810f5cd63 fix(tests): replace fake France Azure endpoint in test_router_azure_acompletion (#21818) 2026-02-21 13:26:59 -08:00
Ryan Crabbe 9e1d83e3de fix: make LITELLM_CLIENT_SPECIFIC_PARAMS a tuple to prevent TypeError
tuple + list raises TypeError in get_openai_client_cache_key. Also add
test coverage for get_openai_client_cache_key to catch type mismatches.
2026-02-21 13:25:23 -08:00