mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-24 04:20:27 +00:00
fe63650ebd0945933f080d1f79fd73db45b341dd
1121
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Yuneng Jiang
|
b5db7ed37d |
test(fireworks): mock remaining live smoke tests
test_completion_fireworks_ai and test_completion_cost_fireworks_ai made real Fireworks calls and broke whenever Fireworks rotated its serverless catalog (no externally-verifiable model list exists). They also asserted nothing — just printed. Mock the HTTP post and assert real behavior instead: the request is built with the right model/messages and the OpenAI-compatible response parses back; the cost path yields a non-zero cost against the local cost map. No network, no model dependency, stronger than the old smoke checks. |
||
|
Yuneng Jiang
|
39a1d438f2 |
test(fireworks): replace deprecated llama-v3p3-70b-instruct model
Fireworks removed llama-v3p3-70b-instruct from serverless, so every
live test using it now fails with NotFoundError ("Model not found,
inaccessible, and/or not deployed").
Swap the 6 references (3 files) to the currently-served
accounts/fireworks/models/deepseek-v3p1 — the canonical model in
Fireworks' current docs examples and present in LiteLLM's cost map.
test_get_model_params_fireworks_ai is a pure pricing-heuristic test
(no network) asserting the >16b branch, so it uses llama-v3p1-70b-
instruct instead to keep the "fireworks-ai-above-16b" assertion and
branch coverage intact.
|
||
|
Mateo WangandGitHub
|
2c733c00f5 |
chore(ci): modernize model references in tests and configs (#27856)
* test: modernize models used in CircleCI e2e test suites
Replaces obsolete models (gpt-4o, gpt-4o-mini, gpt-3.5-turbo,
claude-3-5-sonnet-20240620, claude-sonnet-4-20250514) with current
equivalents across the e2e_openai_endpoints and
proxy_e2e_anthropic_messages_tests CircleCI jobs.
- gpt-4o -> gpt-5.5 (responses API e2e tests)
- gpt-4o-mini -> gpt-5-mini (websocket responses, oai_misc_config)
- gpt-4o-mini-2024-07-18 -> gpt-4.1-mini-2025-04-14 (fine-tuning,
still actively fine-tunable)
- gpt-4 / gpt-3.5-turbo target_model_names example -> gpt-5.5 /
gpt-5-mini
- bedrock claude-3-5-sonnet-20240620 batch entry -> haiku-4-5-20251001
(also aligning oai_misc_config model_name with what
test_bedrock_batches_api.py actually requests)
- bedrock claude-sonnet-4-20250514 (deprecated, retires 2026-06-15)
-> claude-sonnet-4-5-20250929
* test: point bedrock-claude-sonnet-4 alias at Sonnet 4.6, not 4.5
Greptile/Cursor flagged that after the previous commit, the
bedrock-claude-sonnet-4 alias collided with bedrock-claude-sonnet-4.5
(both pointed to claude-sonnet-4-5-20250929). Rename to
bedrock-claude-sonnet-4.6 and point it at the Sonnet 4.6 Bedrock ID
(us.anthropic.claude-sonnet-4-6, already in the litellm model
registry) so the alias name matches the underlying model version.
* test: modernize models across remaining CI-mounted configs & tests
Expands the modernization sweep to all CircleCI-mounted proxy configs
and to test directories where the model literal is a fixture/route key
(not the test's subject).
Config changes:
- proxy_server_config.yaml: bump gpt-3.5-turbo / gpt-3.5-turbo-1106 /
gpt-4o / gemini-1.5-flash / dall-e-3 underlying models; rename
gpt-3.5-turbo-end-user-test alias to gpt-5-mini-end-user-test; bump
text-embedding-ada-002 underlying to text-embedding-3-small. User-
facing aliases (gpt-3.5-turbo, gpt-4, text-embedding-ada-002, etc.)
preserved for backward compatibility with tests.
- simple_config.yaml, otel_test_config.yaml, spend_tracking_config.yaml:
bump gpt-3.5-turbo underlying to gpt-5-mini.
- pass_through_config.yaml: claude-3-5-sonnet / claude-3-7-sonnet /
claude-3-haiku entries replaced with claude-sonnet-4-5 / claude-
haiku-4-5 / claude-opus-4-7.
- oai_misc_config.yaml: align alias name with the gpt-5-mini rename.
Test changes (proactive: claude-sonnet-4-20250514 / claude-opus-4-
20250514 retire 2026-06-15):
- tests/llm_translation/test_anthropic_completion.py: bump 3 references
+ paired Vertex AI ID to claude-sonnet-4-5.
- tests/llm_translation/test_optional_params.py: bump 2 references.
- tests/pass_through_unit_tests/test_anthropic_messages_passthrough.py
and test_bedrock_anthropic_messages_test.py: bump router fixtures
using the deprecated model IDs.
- tests/pass_through_unit_tests/base_anthropic_messages_tool_search_test.py:
modernize docstring examples.
- tests/test_end_users.py: update references to renamed alias.
* test: modernize placeholder model literals in router_unit_tests
Mass replace_all on fixture/placeholder model literals across the
router_unit_tests/ suite (model name is a routing key / label, not the
test subject). Sub-agent sweep so far — additional commits will follow
for logging_callback_tests/, enterprise/, top-level tests/test_*.py,
and other CI-mounted dirs.
Mappings applied:
- gpt-3.5-turbo -> gpt-5-mini
- gpt-4 (bare) -> gpt-5.5
- gpt-4o (bare) -> gpt-5
- text-embedding-ada-002 -> text-embedding-3-small
- claude-3-sonnet-20240229 / claude-3-opus-20240229 /
claude-3-haiku-20240307 / claude-3-5-sonnet-20240620 ->
claude-sonnet-4-5-20250929 / claude-opus-4-7 /
claude-haiku-4-5-20251001 as appropriate
Explicitly preserved:
- gpt-4o-mini-* variants (transcribe, tts, etc.) where they're current
- gpt-4-turbo / gpt-4-vision-preview / gpt-4-0613 (subject literals)
- JSONL batch body literals
- Mock LLM response model fields (must match upstream)
- Fake/mock identifiers
* test: modernize placeholder model literals across remaining CI suites
Sub-agent sweep across logging_callback_tests/, guardrails_tests/,
enterprise/, pass_through_unit_tests/, otel_tests/,
llm_responses_api_testing/, batches_tests/, spend_tracking_tests/,
litellm_utils_tests/, unified_google_tests/, and a few top-level
tests/test_*.py files where the model literal is a fixture or
placeholder (router model_list, mock standard logging payload, mock
callback data) rather than the test's subject.
Mappings applied (see scope notes below):
- gpt-3.5-turbo -> gpt-5-mini
- gpt-4 (bare) -> gpt-5.5
- gpt-4o (bare) -> gpt-5.5 (corrected from initial gpt-5 — bare gpt-5
is not a valid OpenAI alias; only gpt-5.5 / gpt-5.4 / gpt-5.2-codex
/ gpt-5-mini exist)
- gpt-4o-mini (bare) -> gpt-5-mini
- text-embedding-ada-002 -> text-embedding-3-small
- claude-3-sonnet-20240229 -> claude-sonnet-4-5-20250929
- claude-3-opus-20240229 -> claude-opus-4-7
- claude-3-haiku-20240307 -> claude-haiku-4-5-20251001
- claude-3-5-sonnet-20240620/20241022 -> claude-sonnet-4-5-20250929
- claude-3-7-sonnet-20250219 -> claude-sonnet-4-6
- gemini-1.5-flash -> gemini-2.5-flash
- gemini-1.5-pro -> gemini-2.5-pro
Explicitly preserved (not modernized):
- llm_translation/ tests where model is the SUBJECT (provider-specific
translation/transformation logic). Only the deprecated 20250514
references were already bumped in a prior commit.
- Cost-calc / tokenizer subject tests in test_utils.py (skip-ranges
documented by the sub-agent).
- Bedrock model IDs in test_health_check.py path-stripping tests.
- JSONL batch request bodies and mock LLM response bodies (must match
upstream literal).
- Langfuse expected-request-body JSON fixtures (cost values are exact-
match-asserted; changing the model would shift response_cost).
- gpt-3.5-turbo-instruct (text-completion endpoint; no modern OpenAI
equivalent).
- Top-level tests calling the proxy through user-facing aliases
(gpt-3.5-turbo, gpt-4, text-embedding-ada-002, dall-e-3) — aliases
in proxy_server_config.yaml stay; only the underlying model was
bumped.
- tests/test_gpt5_azure_temperature_support.py (the test's whole point
is model-name handling).
- Fake / mock / openai/fake identifiers.
Notable side fixes:
- test_spend_accuracy_tests.py: UPSTREAM_MODEL now matches what
spend_tracking_config.yaml's proxy actually routes to (gpt-5-mini),
resolving a latent inconsistency.
- proxy_server_config.yaml: bare `gpt-5` alias renamed to `gpt-5.5`
(bare gpt-5 is not a valid OpenAI alias).
- test_batches_logging_unit_tests.py: explicit_models list entries
kept distinct (gpt-5-mini + gpt-5.5) after bulk rename.
* test: fix CI failures from model modernization sweep
CI surfaced 4 categories of regression from the bulk modernization:
1. Azure deployment names are customer-specific. Reverted:
- tests/litellm_utils_tests/test_health_check.py: azure/text-
embedding-3-small -> azure/text-embedding-ada-002 (the CI Azure
account does not have a text-embedding-3-small deployment).
- tests/logging_callback_tests/test_custom_callback_router.py:
same revert for two router fixtures driving aembedding.
2. gpt-5 family does not accept temperature != 1. Tests that pass a
custom temperature swapped from gpt-5-mini to gpt-4.1-mini (modern
non-reasoning OpenAI mini that still accepts temperature/logprobs):
- tests/logging_callback_tests/test_datadog.py
- tests/logging_callback_tests/test_langsmith_unit_test.py
- tests/logging_callback_tests/test_otel_logging.py
3. proxy_server_config.yaml's gpt-3.5-turbo-large alias was routing to
gpt-5.5 (a reasoning model that rejects logprobs). The proxy test
tests/test_openai_endpoints.py::test_chat_completion_streaming
exercises logprobs/top_logprobs through that alias. Bumped the
underlying model to gpt-4.1 (non-reasoning, still modern).
4. tests/logging_callback_tests/test_gcs_pub_sub.py asserts against a
pinned JSON fixture (gcs_pub_sub_body/spend_logs_payload.json) with
hardcoded model="gpt-4o" and a model-specific spend value. Reverted
the litellm.acompletion calls in the test to model="gpt-4o" so the
fixture's exact-match assertions still hold.
5. tests/pass_through_unit_tests/test_anthropic_messages_passthrough.py:
anthropic.messages.create routing to openai/gpt-5-mini returned an
empty content[0] with max_tokens=100 (reasoning-token consumption).
Swapped to openai/gpt-4.1-mini.
* test: fix Assistants API model + 2 cursor[bot] review nits
1. pass_through_unit_tests/test_custom_logger_passthrough.py: gpt-5.5
isn't accepted by the /v1/assistants endpoint
("unsupported_model"). Switch to gpt-4.1-mini (modern, Assistants-
API-supported, non-reasoning).
2. example_config_yaml/pass_through_config.yaml: the previous sweep
bumped the claude-3-7-sonnet alias to claude-opus-4-7, which is a
tier change (Sonnet -> Opus). Map to claude-sonnet-4-6 to keep the
Sonnet tier intact. (Cursor bugbot review.)
3. example_config_yaml/simple_config.yaml: model_name was left as
gpt-3.5-turbo while the underlying was bumped to gpt-5-mini, which
muddles the "simple" example. Make both sides gpt-5-mini so the
most basic example is a straight 1:1 mapping again. (Cursor bugbot
review.)
* fix: revert gpt-4/gpt-3.5-turbo alias underlying to non-reasoning models
tests/test_openai_endpoints.py::test_completion calls the proxy alias
"gpt-4" with temperature=0, and other tests call gpt-3.5-turbo with
custom temperature / logprobs / the legacy /v1/completions endpoint.
The earlier modernization mapped both aliases to gpt-5.5 / gpt-5-mini,
which are reasoning models that reject temperature != 1 and don't
expose /v1/completions. Map the aliases to gpt-4.1 / gpt-4.1-mini
(modern non-reasoning OpenAI models) instead — keeps user-facing
aliases preserved while picking a current underlying that still
supports the parameters/endpoints the tests exercise.
|
||
|
Cursor AgentandMateo Wang
|
93300019bd |
test(vcr): drop dead 'from respx import MockRouter' imports
These seven test files were on _RESPX_CONFLICTING_FILES, which made the
auto-marker skip them entirely. Inspecting the source shows the only
respx artifact is a top-level 'from respx import MockRouter' that no
test ever uses - no @pytest.mark.respx, no respx_mock fixture, no
respx.mock context manager. The import is dead code left over from a
previous mocking pattern.
Now that apply_vcr_auto_marker_to_items detects respx per-item via the
marker / fixture chain (
|
||
|
Cursor AgentandMateo Wang
|
b637d9f64a |
test(vcr): classify cache verdicts, detect live calls, surface cost leaks
Convert the per-test VCR verdict line from a single 'NOOP / HIT / MISS /
PARTIAL' tag into a classified outcome that distinguishes the cases that
silently bill the live API on every CI run from the ones that don't:
HIT pure replay
PARTIAL mixed replay + new recordings
MISS:RECORDED new cassette saved to Redis (cached next run)
MISS:OVERFLOW cassette > MAX_EPISODES_PER_CASSETTE; persister
refused to save; re-bills every run
MISS:NOT_PERSISTED test failed; save_cassette skipped; re-bills
NOOP VCR-marked but no HTTP traffic (mocked elsewhere)
UNMARKED:LIVE_CALL test bypassed VCR AND opened a TCP connection
to a known LLM provider host -> wasted spend
UNMARKED:NO_TRAFFIC test bypassed VCR but didn't call out
The UNMARKED:LIVE_CALL signal is what converts 'this test probably hits
live' into 'this test connected to api.openai.com'. We install a
socket.connect / socket.create_connection wrapper for the duration of
each non-VCR-marked test and record any outbound TCP to a known LLM
provider hostname. The probe sits below the httpx layer so vcrpy and
respx (which both patch above the socket) are unaffected.
Replace the file-level _RESPX_CONFLICTING_FILES blacklists in the
llm_translation and local_testing conftests with per-item respx
detection in apply_vcr_auto_marker_to_items. A test now skips VCR when
it actually carries @pytest.mark.respx or has respx_mock in its fixture
chain - not just because some other test in the same file imports
MockRouter. Items skipped by skip_files are split into respx_conflict
(real conflict, the module wires up respx) vs file_opt_out (dead skip-
list entry whose module never touches respx) so the session summary
makes pruning obvious.
Stabilize the AWS SigV4 fingerprint: the Authorization header on
Bedrock requests rotates its Credential date and Signature on every
call, which previously pushed every Bedrock test past the 50-episode
overflow threshold. Extract the access-key id only
('aws-sigv4:AKIA...') so two requests with the same identity match.
Always emit verdict logging when VCR is active (set
LITELLM_VCR_VERBOSE=0 to opt back into the legacy quiet mode). Add a
session-end classification summary that lists overflow tests, unmarked
live-call tests, and the skip-reason breakdown.
Wire the live-call probe + summary hook into every test directory that
already uses the Redis-backed VCR cache (audio_tests, guardrails_tests,
image_gen_tests, litellm_utils_tests, llm_responses_api_testing,
llm_translation, local_testing, logging_callback_tests, ocr_tests,
pass_through_unit_tests, router_unit_tests, search_tests,
unified_google_tests).
Add tests/llm_translation/test_vcr_classification.py covering the
verdict classifier, skip-reason tagging, AWS SigV4 fingerprint stability,
live-host classification, and session summary rendering.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
|
||
|
Yuneng Jiang
|
945b10ded4 |
fix(tests): drop dall-e-only test classes; route live image tests via gpt-image-1
Second wave of failures from the 2026-05-12 DALL-E shutdown: - tests/image_gen_tests/test_image_edits.py::TestOpenAIImageEditDallE2 and tests/image_gen_tests/test_image_generation.py::TestOpenAIDalle3 are explicitly named for the deprecated models and can't pass; remove. gpt-image-1 coverage already exists in sibling classes. - tests/local_testing/test_router.py image gen tests use dall-e-3 only as a routing example; swap to gpt-image-1. - tests/local_testing/test_custom_callback_input.py image_generation success/failure paths swapped to gpt-image-1. |
||
|
|
4c1d91d96f |
fix(anthropic): inject dummy tool without modify_params (#27620)
Anthropic rejects tool_use/tool_result when tools is omitted. Always map and attach the dummy tool in transform_request so CLIs work without litellm.modify_params. - Add unit test for transform_request dummy tool with modify_params off - Adjust parallel function calling integration expectations: Bedrock Converse still requires modify_params for this path Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
yuneng-jiangandGitHub
|
9a338e1b6b |
[Test] Tests: Stop parametrizing API keys into pytest test IDs (#27249)
Several tests parametrized over (model, api_key, ...) tuples or raw token strings, causing pytest to embed those values in the test ID and print them in CI logs. Refactored each affected test to keep the same coverage without putting key material into parametrize. - audio_tests/test_audio_speech.py: split env-var keys into separate azure/openai test functions sharing a helper; sync_mode parametrize preserved. - audio_tests/test_whisper.py: split into openai_whisper / azure_whisper functions sharing a helper; response_format parametrize preserved. - local_testing/test_embedding.py: single-case parametrize inlined. - proxy_unit_tests/test_user_api_key_auth.py: 5 header parametrize cases split into 5 named tests sharing an _assert helper. - proxy_unit_tests/test_proxy_utils.py: 4 api_key_value cases split into 4 named tests. - test_litellm/proxy/auth/test_user_api_key_auth.py: 5 key-prefix cases (Bearer / Basic / lowercase bearer / raw / AWS SigV4) split into 5 named tests. Verified: black clean; 14 refactored unit tests pass; pytest collects audio/embedding tests with safe IDs (no key material in test IDs). |
||
|
|
7e13256fee |
test: add 24hr Redis-backed VCR cache to additional test suites (#27159)
* test: add 24hr Redis-backed VCR cache to additional test suites Extracts the existing llm_translation VCR plumbing into a reusable helper (tests/_vcr_conftest_common.py) and wires it into the conftest.py files of the test directories listed in LIT-2787: audio_tests, batches_tests, guardrails_tests, image_gen_tests, litellm_utils_tests, local_testing, logging_callback_tests, pass_through_unit_tests, router_unit_tests, unified_google_tests The same helper is also adopted by the pre-existing llm_translation and llm_responses_api_testing conftests to remove the copy-pasted VCR setup. Each consuming conftest: - registers the Redis persister via pytest_recording_configure - auto-marks collected tests with pytest.mark.vcr (skipping respx-using files where applicable, since respx and vcrpy both patch httpx) - gates cassette writes on test success via _vcr_outcome_gate The cache is opt-in via CASSETTE_REDIS_URL; when unset, VCR is disabled and tests hit live providers as before. LITELLM_VCR_DISABLE=1 still forces a bypass for ad-hoc local runs. Test directories that run LiteLLM proxy in Docker (build_and_test, proxy_logging_guardrails_model_info_tests, proxy_store_model_in_db_tests) are intentionally not included: VCR.py patches the in-process httpx transport and cannot intercept calls made from inside a Docker container. The installing_litellm_on_python* jobs make no LLM calls and don't benefit from caching. https://linear.app/litellm-ai/issue/LIT-2787/add-24hr-caching-to-additional-test-suites * test(vcr): add safe-body matcher to handle JSONL and binary request bodies vcrpy's stock body matcher inspects Content-Type and unconditionally runs json.loads on application/json bodies. JSON Lines payloads (used by the Bedrock batch S3 PUT and other upload paths) crash that with json.JSONDecodeError: Extra data, before the matcher can return 'not a match'. This was the root cause of the batches_testing CI job failing on test_async_create_file once VCR auto-marking was applied to the batches_tests directory. Add a conservative byte-equality body matcher and use it in place of 'body' in the shared match_on tuple. The matcher is strictly more conservative than vcrpy's default — the only thing it gives up is 'different JSON key order is treated as the same body', which doesn't apply to deterministic litellm-built request payloads. It can never produce a false positive that the default would have rejected, so there is no cross-contamination risk. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): exclude tests that VCR replay actively breaks A few tests are incompatible with cassette replay and were failing on the latest CI run after VCR auto-marking was extended to local_testing and logging_callback_tests: - test_amazing_s3_logs.py (logging_callback_tests): the test asserts on a per-run response_id that should round-trip through a real S3 PUT/LIST. vcrpy's boto3 stub intercepts the PUT and the LIST replays stale keys, so the freshly-generated id is never found. - test_async_embedding_azure (logging_callback_tests) and test_amazing_sync_embedding (local_testing): the failure branches deliberately pass api_key='my-bad-key' to assert that the failure callback fires. We scrub auth headers from cassettes (so the bad-key request matches the prior good-key request), and vcrpy replays the recorded 200 — the failure callback never fires. - test_assistants.py (local_testing): the OpenAI Assistants polling APIs mint fresh thread/run IDs every recording session and then poll until status=='completed'. Replays of those polled GETs can never match a freshly-generated run id, so every CI run effectively re-records and the suite blows past the 15m no_output_timeout. Skip these from VCR auto-marking so they continue to hit live providers as they did before this change. The remaining tests in each directory still get cached. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): expand skip lists for second batch of incompatible tests Followup to the previous commit. After re-running CI on the rebuilt branch, three more tests surfaced as VCR-replay-incompatible: - litellm_utils_testing :: test_get_valid_models_from_dynamic_api_key Calls GET /v1/models with api_key='123' to assert the result is empty. We scrub auth headers, so the bad-key request matches the prior good-key cassette and replays the recorded model list. - litellm_utils_testing :: test_litellm_overhead.py Measures litellm_overhead_time_ms as a percentage of total wall-clock time. With cached responses the upstream 'network' time collapses to microseconds, blowing past the 40%% threshold the test asserts on. Skip the whole file (every parametrization is at risk). - local_testing_part1 :: test_async_custom_handler_completion and test_async_custom_handler_embedding Same bad-key failure-callback pattern as the already-skipped test_amazing_sync_embedding. - litellm_router_testing :: test_router_caching.py Asserts on litellm's own router-level response cache by comparing response1.id to response2.id across repeat upstream calls (test bypasses litellm cache via ttl=0 and expects upstream to return a *new* id). With VCR replay both upstream calls return the same cassette body, so the ids are identical. Skip the whole file. - logging_callback_tests :: test_async_chat_azure (preemptive) Same shape as already-skipped test_async_embedding_azure; was masked by upstream OpenAI rate-limit failures on baseline. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): use item.path and tighten matcher docstring - Replace pytest's deprecated item.fspath with item.path in apply_vcr_auto_marker_to_items so we don't emit deprecation warnings under pytest 8. - Clarify _safe_body_matcher docstring to reflect actual behavior (direct == first, then UTF-8 bytes comparison, no repr fallback). Addresses Greptile review feedback on PR #27159. * test(vcr): swallow all RedisError on cassette save/load Cassette persistence is strictly best-effort: any Redis-side failure (connection blip, timeout, OutOfMemoryError when the maxmemory cap is hit, READONLY replicas, etc.) should degrade to 'test passed but cassette not cached' rather than fail the test on teardown. Previously the persister only caught ConnectionError and TimeoutError, so OutOfMemoryError — which Redis Cloud raises when the cassette cache hits its memory cap and there are no evictable keys — propagated out of vcrpy's autouse fixture and ERRORed otherwise-passing tests on teardown. This caused the litellm_utils_testing CircleCI job to fail on the latest commit's run, even though the underlying test was a unit test that used mock_response and produced no real upstream traffic (the cassette was dirtied by a background langfuse callback). The rerun only succeeded because Redis evictions happened to free enough room before the SET — i.e. it was timing-dependent flakiness. Catch redis.exceptions.RedisError (the common base of all server- and client-side Redis exceptions) on both save and load, and parametrize the regression tests across ConnectionError, TimeoutError, and OutOfMemoryError to pin the new behavior. * test(vcr): surface cassette-cache failures with warnings + session banner When the persister silently swallows a Redis OOM (or any RedisError) on save/load there is otherwise no visible signal that the cache is degraded — tests pass, the cassette just isn't persisted, and the next session still hits the same Redis at the same near-cap memory. Add three layers of observability so that failure mode is loud: 1. Per-process health counters ("save_failures", "load_failures", and the last error string for each), exposed via cassette_cache_health() and reset via reset_cassette_cache_health(). The persister increments these in addition to logging. 2. VCRCassetteCacheWarning (UserWarning subclass) emitted via warnings.warn() inside the persister's except block. Pytest's built-in warnings summary at session end automatically lists every such warning, so the failure is visible in CI logs without any conftest-level wiring. 3. Session-end banner via emit_cassette_cache_session_banner() and a stderr-fallback atexit handler registered from register_persister_if_enabled(). Two states: - red "VCR CASSETTE CACHE DEGRADED" when save_failures or load_failures > 0 - yellow "VCR CASSETTE CACHE NEAR CAPACITY" (no failures, but used_memory >= 85% of maxmemory) so the next session knows the Redis is approaching OOM before any SET actually fails Capacity comes from a best-effort INFO memory probe (cassette_cache_capacity_snapshot) that returns None on any failure or when maxmemory is uncapped. The atexit handler skips xdist workers so only the controller emits. Tests: parametrize the existing save/load swallow-error tests across ConnectionError/TimeoutError/OutOfMemoryError, add direct tests for the health counters and warning emission, and a new test_vcr_conftest_common_banner.py covering banner output for every state (silent/red/yellow/disabled/xdist-worker). * test(vcr): bucket cassettes by API key fingerprint, drop bad-key skips Tests that deliberately call an LLM API with a bad key (e.g. to assert that the failure callback fires, or that check_valid_key returns False) were being silently served the prior good-key cassette: we scrub the real Authorization / x-api-key header from the cassette before storing it, so a follow-up bad-key call is byte-identical to the good-key call under the existing match_on tuple. Add a 'key_fingerprint' custom matcher that distinguishes requests by the SHA-256 of their API-key headers. The fingerprint is stamped into a synthetic 'x-litellm-key-fp' header by a new before_record_request hook, which then strips the real auth headers (we have to do the scrubbing here instead of via vcrpy's filter_headers knob, because filter_headers runs *first* and would erase the value we want to hash). Bad-key requests now get a different cassette bucket than good-key requests, so vcrpy will not replay a recorded 200 in place of the expected 401. The fingerprint is a one-way hash of the secret, so cassettes never contain the key. This permanently removes the 'bad-key' category of skips: - tests/local_testing: dropped ::test_amazing_sync_embedding, ::test_async_custom_handler_completion, ::test_async_custom_handler_embedding - tests/logging_callback_tests: dropped ::test_async_chat_azure, ::test_async_embedding_azure - tests/litellm_utils_tests: dropped ::test_get_valid_models_from_dynamic_api_key Coverage: 7 new unit tests in tests/test_litellm/test_vcr_safe_body_matcher.py covering header stripping, fingerprint determinism, no-auth bucketing, good-vs-bad key discrimination, x-api-key (Anthropic/Azure) discrimination, and idempotence under replay. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): drop redundant comments and docstrings Trim narration of code that is already self-evident from function and variable names. Keep the two genuinely non-obvious bits: - ordering constraint between filter_headers and before_record_request, which would invite a maintainer to re-introduce the bug if removed - the per-directory _VCR_INCOMPATIBLE_FILES rationale, since 'why exactly is this skipped' is not knowable from the test name alone Also drop the 40-line commented-out drop-in conftest snippet at the bottom of _vcr_conftest_common.py — the consuming conftests are the canonical reference. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): make _before_record_request idempotent vcrpy invokes before_record_request more than once per request: can_play_response_for calls it, then __contains__ / _responses (reached via play_response) call it again on the result. The second invocation sees a request whose auth headers we already stripped, so a naive recompute yields "no-key" and overwrites the real fingerprint stored in the header. This makes can_play_response_for and play_response disagree on matchability — the former says "yes, we have a stored response for this" (matching no-key to no-key) and the latter throws UnhandledHTTPRequestError because it computes a fresh real fingerprint that doesn't match the stored no-key. In CI this manifested as ~30 failing tests across guardrails_testing, audio_testing, batches_testing, image_gen_testing, llm_responses_api, litellm_router_unit_testing, etc. Skip the recompute when the header is already set, so re-applying the hook is a no-op. Adds a regression test that fires the hook twice on the same dict and asserts the fingerprint stays put. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * test(vcr): drop more redundant docstrings and headers * test(vcr): enable 24hr cache for ocr_tests and search_tests These two directories were the only non-dockerized test suites in the build_and_test workflow that make live LLM/provider API calls but were not VCR-enabled by this PR. Together they account for 96 tests: - tests/ocr_tests/ (31): Mistral OCR, Azure AI OCR, Azure Document Intelligence, Vertex AI OCR. Pure-unit tests inside the same files (e.g. TestAzureDocumentIntelligencePagesParam) make no HTTP calls and become benign VCR NOOPs. - tests/search_tests/ (65): Brave, DataForSEO, DuckDuckGo, Exa, Firecrawl, Google PSE, Linkup, Parallel.ai, Perplexity, SearchAPI, Searxng, Serper, Tavily. Both directories use the canonical minimal conftest pattern from tests/audio_tests/conftest.py with no skip lists. None of the test files use respx, none assert on per-call upstream non-determinism (no response1.id != response2.id, no overhead-as-fraction-of-total, no live polling), so the default match_on tuple should cache cleanly. If a flake surfaces during the first cassette-recording CI run, we can add a targeted skip the same way we did for the other dirs. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> |
||
|
Yuneng Jiang
|
7f550a5d67 | Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/peaceful-bell-ba8ca5 | ||
|
Yassin Kortam
|
a56256e5ee | feat: routing groups ui | ||
|
Yuneng Jiang
|
22782f3c3f |
[Fix] Tests: Replace deprecated openrouter/claude-3.7-sonnet with claude-sonnet-4.5
OpenRouter has dropped active endpoints for anthropic/claude-3.7-sonnet, causing test_reasoning_content_completion to fail with a 404 "No endpoints found" error. Switch to anthropic/claude-sonnet-4.5, which is current and supports reasoning streaming. |
||
|
Yuneng Jiang
|
e3917c9d08 |
[Test] Anthropic: Replace Legacy Claude-4-Sonnet Alias With Haiku 4.5
Three live-API tests pinned to claude-4-sonnet-20250514, which is a non-canonical alias of claude-sonnet-4-20250514. Anthropic's main API no longer resolves the legacy form under freshly issued keys, so the tests fail with not_found_error. The token counter test pinned to claude-sonnet-4-20250514 itself (deprecation_date 2026-05-14, two weeks out) was on borrowed time too. Bump all four to claude-haiku-4-5-20251001 — capability superset for what these tests exercise (streaming, parallel tool calling, extended thinking, token counting), no upcoming deprecation, cheaper per-token. |
||
|
Mateo WangandGitHub
|
0b87bdff3d |
Merge pull request #26976 from BerriAI/litellm_default_embedding_encoding_format
feat(embedding): default OpenAI-path encoding_format to float |
||
|
yuneng-jiangandGitHub
|
02582466c4 |
Merge pull request #24340 from BerriAI/litellm_staging_03_21_2026
Litellm staging 03 21 2026 |
||
|
Sameer Kankute
|
e0398cade7 |
fix(caching): defer streaming cache-hit callbacks for all stream=True
Success handlers already run when CustomStreamWrapper or CachedResponsesAPIStreamingIterator finishes replay. Logging at cache-hit time for acompletion/completion streaming duplicated spend and callbacks. Align tests with deferred behavior. Made-with: Cursor |
||
|
Sameer Kankute
|
a1f0823393 |
test(embedding): align local_testing OpenAI encoding_format default
Made-with: Cursor |
||
|
NoahandSameer Kankute
|
8947a74e13 |
fix(cache): persist and replay streamed Responses API requests (#24580)
* fix(cache): persist and replay streamed Responses API requests * Add focused coverage for streamed responses cache * Cover streamed responses cache helper branches * Exercise streamed responses cache edge branches |
||
|
Cursor AgentandMateo Wang
|
3215874e40 |
fix(test): scope ERROR log assertion to LiteLLM logger in test_model_alias_map
The test was flaking on unrelated asyncio ERROR records (e.g. "Unclosed client session" from background tasks in other tests). Restrict the assertion to records emitted by LiteLLM loggers so the test only fails on errors actually produced by the code under test. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> |
||
|
Ryan Crabbe
|
dc46467235 |
fix(tests): replace deprecated Bedrock Claude 3.7 Sonnet model ID
AWS Bedrock has reached end-of-life for `claude-3-7-sonnet-20250219-v1:0`, returning 404s with "This model version has reached the end of its life." Update test references to `claude-sonnet-4-5-20250929-v1:0` (same capability surface: thinking, tools, prompt caching, PDF input, vision, computer use). The bedrock/invoke pass-through tests stay on Sonnet 3.5 since Sonnet 4.5 is converse-only on Bedrock. |
||
|
Yuneng JiangandClaude Opus 4.7
|
1af11d4371 |
fix(vertex): synthesize items for array types missing items entirely
Companion to the prior commit. process_items only converted empty
`items: {}` to `{"type": "object"}`. But anyOf branches like
`{"type": "array"}` (no items field at all) were untouched, so after
convert_anyof_null_to_nullable stripped the null branch and added
nullable, the array branch was sent to Vertex as
`{"type": "array", "nullable": true}` — which Vertex rejects with
INVALID_ARGUMENT (`any_of[0].items: missing field`).
Make process_items synthesize `items: {"type": "object"}` for any
`type == "array"` schema where items is missing or empty.
Also:
- Convert test_gemini_tool_calling_working_demo to a hermetic mock
test asserting items is present on the array branch in the sent
body. Was previously a real-network call to Vertex and was the
test the user reported still failing in CI.
- Add unit test test_build_vertex_schema_array_branch_missing_items_in_anyof
covering the missing-items shape directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Yuneng JiangandClaude Opus 4.7
|
3ca985451e |
fix(vertex): preserve items on array branches inside anyOf with null
convert_anyof_null_to_nullable was stripping the items field from array
branches inside anyOf when a sibling null branch was present, leaving
{"type": "array"} without items. Vertex requires items whenever
type == "array" (even inside anyOf) and rejects the call with
INVALID_ARGUMENT.
Leave the (possibly empty) items in place so the downstream process_items
step can convert {} to {"type": "object"}, which is what Vertex wants.
Also:
- Update test_build_vertex_schema expected output, which was codifying
the broken shape.
- Convert test_gemini_tool_calling_not_working to a hermetic mock test
that asserts the request body sent to Vertex includes items inside
the callbacks anyOf array branch. The previous form made a real
network call and was flaky in CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Chesars
|
c75f0c0566 |
test: drop duplicate openrouter prefix-strip test
The multi-segment case (openrouter/<provider>/<model> → <provider>/<model>) is already covered by tests/test_litellm/llms/openrouter/test_openrouter_provider_routing.py in internal_staging, with broader coverage including double-prefix native models, wildcard deployments, and the bridge double-call scenario. Keeping a duplicate in tests/local_testing/ adds maintenance load with no extra coverage. |
||
|
Chesars
|
232464e151 |
test: drop incorrect openrouter native-prefix test
The removed test asserted that get_llm_provider(model="openrouter/auto") should return model="openrouter/auto" with the prefix preserved. That contract is wrong: the LiteLLM convention strips one "openrouter/" prefix, so a native OpenRouter model is reached via "openrouter/openrouter/auto" (double-prefix) which the wire send as "openrouter/auto" — the ID the OpenRouter API expects for natives. E2E checks against api.openrouter.ai confirm this is the correct routing convention for all native models (auto, bodybuilder, free, pareto-code). |
||
|
Chesars
|
91e78eca3d |
Merge remote-tracking branch 'upstream/litellm_internal_staging' into upstream-litellm_staging_03_21_2026
# Conflicts: # .circleci/config.yml # .circleci/requirements.txt # .github/workflows/_test-unit-base.yml # .github/workflows/_test-unit-services-base.yml # .github/workflows/auto_update_price_and_context_window.yml # .github/workflows/create-release.yml # .github/workflows/llm-translation-testing.yml # .github/workflows/publish_to_pypi.yml # .github/workflows/scan_duplicate_issues.yml # .github/workflows/test-linting.yml # .github/workflows/test-litellm-matrix.yml # .github/workflows/test-litellm.yml # .github/workflows/test-mcp.yml # .github/workflows/test-model-map.yaml # .github/workflows/test-proxy-e2e-azure-batches.yml # .github/workflows/test-unit-core-utils.yml # .github/workflows/test-unit-documentation.yml # .github/workflows/test-unit-enterprise-routing.yml # .github/workflows/test-unit-integrations.yml # .github/workflows/test-unit-llm-providers.yml # .github/workflows/test-unit-misc.yml # .github/workflows/test-unit-proxy-auth.yml # .github/workflows/test-unit-proxy-db.yml # .github/workflows/test-unit-proxy-endpoints.yml # .github/workflows/test-unit-proxy-infra.yml # .github/workflows/test-unit-proxy-legacy.yml # .github/workflows/test-unit-responses-caching-types.yml # .github/workflows/test-unit-security.yml # .github/workflows/test_server_root_path.yml # docs/my-website/docs/embedding/supported_embedding.md # litellm/litellm_core_utils/get_llm_provider_logic.py # litellm/llms/vertex_ai/gemini_embeddings/batch_embed_content_transformation.py # litellm/proxy/_experimental/out/404/index.html # litellm/proxy/_experimental/out/__next.__PAGE__.txt # litellm/proxy/_experimental/out/__next._full.txt # litellm/proxy/_experimental/out/__next._head.txt # litellm/proxy/_experimental/out/__next._index.txt # litellm/proxy/_experimental/out/__next._tree.txt # litellm/proxy/_experimental/out/_next/static/3qyC5Vtvhd5fSC6sPp1iW/_buildManifest.js # litellm/proxy/_experimental/out/_next/static/3qyC5Vtvhd5fSC6sPp1iW/_clientMiddlewareManifest.json # litellm/proxy/_experimental/out/_next/static/3qyC5Vtvhd5fSC6sPp1iW/_ssgManifest.js # litellm/proxy/_experimental/out/_next/static/aKKihXXKRJWLQThZgi8Rq/_buildManifest.js # litellm/proxy/_experimental/out/_next/static/aKKihXXKRJWLQThZgi8Rq/_clientMiddlewareManifest.json # litellm/proxy/_experimental/out/_next/static/aKKihXXKRJWLQThZgi8Rq/_ssgManifest.js # litellm/proxy/_experimental/out/_next/static/bmMTxs1O5fQKYcsMNTRMT/_buildManifest.js # litellm/proxy/_experimental/out/_next/static/bmMTxs1O5fQKYcsMNTRMT/_clientMiddlewareManifest.json # litellm/proxy/_experimental/out/_next/static/bmMTxs1O5fQKYcsMNTRMT/_ssgManifest.js # litellm/proxy/_experimental/out/_next/static/chunks/11362340846735c3.js # litellm/proxy/_experimental/out/_next/static/chunks/1a04d31843c96649.js # litellm/proxy/_experimental/out/_next/static/chunks/342c7d7210247a5e.js # litellm/proxy/_experimental/out/_next/static/chunks/39768ec0eebd2554.js # litellm/proxy/_experimental/out/_next/static/chunks/3b3c0b070b14da06.js # litellm/proxy/_experimental/out/_next/static/chunks/3bddc72a3ecc2253.js # litellm/proxy/_experimental/out/_next/static/chunks/4472ece1be7379b3.js # litellm/proxy/_experimental/out/_next/static/chunks/54e29148cb2f2582.js # litellm/proxy/_experimental/out/_next/static/chunks/67ddb5107368a659.js # litellm/proxy/_experimental/out/_next/static/chunks/6a167cef4b09b496.js # litellm/proxy/_experimental/out/_next/static/chunks/7174130ddef406dd.js # litellm/proxy/_experimental/out/_next/static/chunks/7c36bfe1ba5e3ba8.js # litellm/proxy/_experimental/out/_next/static/chunks/7e5fe5584502da06.js # litellm/proxy/_experimental/out/_next/static/chunks/8dda507c226082ca.js # litellm/proxy/_experimental/out/_next/static/chunks/8dfde809dc4ad794.js # litellm/proxy/_experimental/out/_next/static/chunks/99109c78121231a0.js # litellm/proxy/_experimental/out/_next/static/chunks/9dd55e1f36a7225c.js # litellm/proxy/_experimental/out/_next/static/chunks/a230559fcabaea23.js # litellm/proxy/_experimental/out/_next/static/chunks/a6c7f80b3968f639.js # litellm/proxy/_experimental/out/_next/static/chunks/ac9e96d21c200b48.js # litellm/proxy/_experimental/out/_next/static/chunks/ae9cf43b8c0c76aa.js # litellm/proxy/_experimental/out/_next/static/chunks/cf06797ce4e438f9.js # litellm/proxy/_experimental/out/_next/static/chunks/d069df5baead6d90.js # litellm/proxy/_experimental/out/_next/static/chunks/d2e3b7dd6499c245.js # litellm/proxy/_experimental/out/_next/static/chunks/d44e73d8ebac5747.js # litellm/proxy/_experimental/out/_next/static/chunks/dc8a270fee94ced6.js # litellm/proxy/_experimental/out/_next/static/chunks/df6546cd8a44d3b3.js # litellm/proxy/_experimental/out/_next/static/chunks/ea0f22bd4b3393bd.js # litellm/proxy/_experimental/out/_next/static/chunks/eaa9f9b9bb3e054b.js # litellm/proxy/_experimental/out/_next/static/chunks/turbopack-901b35f89c1f6751.js # litellm/proxy/_experimental/out/_next/static/chunks/turbopack-d1b22f5e0bd58c57.js # litellm/proxy/_experimental/out/_next/static/chunks/turbopack-ddedb29a5eb0118f.js # litellm/proxy/_experimental/out/_not-found.txt # litellm/proxy/_experimental/out/_not-found/__next._full.txt # litellm/proxy/_experimental/out/_not-found/__next._head.txt # litellm/proxy/_experimental/out/_not-found/__next._index.txt # litellm/proxy/_experimental/out/_not-found/__next._not-found.__PAGE__.txt # litellm/proxy/_experimental/out/_not-found/__next._not-found.txt # litellm/proxy/_experimental/out/_not-found/__next._tree.txt # litellm/proxy/_experimental/out/_not-found/index.html # litellm/proxy/_experimental/out/api-reference.html # litellm/proxy/_experimental/out/api-reference.txt # litellm/proxy/_experimental/out/api-reference/__next.!KGRhc2hib2FyZCk.api-reference.__PAGE__.txt # litellm/proxy/_experimental/out/api-reference/__next.!KGRhc2hib2FyZCk.api-reference.txt # litellm/proxy/_experimental/out/api-reference/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/api-reference/__next._full.txt # litellm/proxy/_experimental/out/api-reference/__next._head.txt # litellm/proxy/_experimental/out/api-reference/__next._index.txt # litellm/proxy/_experimental/out/api-reference/__next._tree.txt # litellm/proxy/_experimental/out/chat.html # litellm/proxy/_experimental/out/chat.txt # litellm/proxy/_experimental/out/chat/__next._full.txt # litellm/proxy/_experimental/out/chat/__next._head.txt # litellm/proxy/_experimental/out/chat/__next._index.txt # litellm/proxy/_experimental/out/chat/__next._tree.txt # litellm/proxy/_experimental/out/chat/__next.chat.__PAGE__.txt # litellm/proxy/_experimental/out/chat/__next.chat.txt # litellm/proxy/_experimental/out/experimental/api-playground.html # litellm/proxy/_experimental/out/experimental/api-playground.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next.!KGRhc2hib2FyZCk.experimental.api-playground.__PAGE__.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next.!KGRhc2hib2FyZCk.experimental.api-playground.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next.!KGRhc2hib2FyZCk.experimental.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next._full.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next._head.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next._index.txt # litellm/proxy/_experimental/out/experimental/api-playground/__next._tree.txt # litellm/proxy/_experimental/out/experimental/budgets.html # litellm/proxy/_experimental/out/experimental/budgets.txt # litellm/proxy/_experimental/out/experimental/budgets/__next.!KGRhc2hib2FyZCk.experimental.budgets.__PAGE__.txt # litellm/proxy/_experimental/out/experimental/budgets/__next.!KGRhc2hib2FyZCk.experimental.budgets.txt # litellm/proxy/_experimental/out/experimental/budgets/__next.!KGRhc2hib2FyZCk.experimental.txt # litellm/proxy/_experimental/out/experimental/budgets/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/experimental/budgets/__next._full.txt # litellm/proxy/_experimental/out/experimental/budgets/__next._head.txt # litellm/proxy/_experimental/out/experimental/budgets/__next._index.txt # litellm/proxy/_experimental/out/experimental/budgets/__next._tree.txt # litellm/proxy/_experimental/out/experimental/caching.html # litellm/proxy/_experimental/out/experimental/caching.txt # litellm/proxy/_experimental/out/experimental/caching/__next.!KGRhc2hib2FyZCk.experimental.caching.__PAGE__.txt # litellm/proxy/_experimental/out/experimental/caching/__next.!KGRhc2hib2FyZCk.experimental.caching.txt # litellm/proxy/_experimental/out/experimental/caching/__next.!KGRhc2hib2FyZCk.experimental.txt # litellm/proxy/_experimental/out/experimental/caching/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/experimental/caching/__next._full.txt # litellm/proxy/_experimental/out/experimental/caching/__next._head.txt # litellm/proxy/_experimental/out/experimental/caching/__next._index.txt # litellm/proxy/_experimental/out/experimental/caching/__next._tree.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins.html # litellm/proxy/_experimental/out/experimental/claude-code-plugins.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next.!KGRhc2hib2FyZCk.experimental.claude-code-plugins.__PAGE__.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next.!KGRhc2hib2FyZCk.experimental.claude-code-plugins.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next.!KGRhc2hib2FyZCk.experimental.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next._full.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next._head.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next._index.txt # litellm/proxy/_experimental/out/experimental/claude-code-plugins/__next._tree.txt # litellm/proxy/_experimental/out/experimental/old-usage.html # litellm/proxy/_experimental/out/experimental/old-usage.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next.!KGRhc2hib2FyZCk.experimental.old-usage.__PAGE__.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next.!KGRhc2hib2FyZCk.experimental.old-usage.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next.!KGRhc2hib2FyZCk.experimental.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next._full.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next._head.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next._index.txt # litellm/proxy/_experimental/out/experimental/old-usage/__next._tree.txt # litellm/proxy/_experimental/out/experimental/prompts.html # litellm/proxy/_experimental/out/experimental/prompts.txt # litellm/proxy/_experimental/out/experimental/prompts/__next.!KGRhc2hib2FyZCk.experimental.prompts.__PAGE__.txt # litellm/proxy/_experimental/out/experimental/prompts/__next.!KGRhc2hib2FyZCk.experimental.prompts.txt # litellm/proxy/_experimental/out/experimental/prompts/__next.!KGRhc2hib2FyZCk.experimental.txt # litellm/proxy/_experimental/out/experimental/prompts/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/experimental/prompts/__next._full.txt # litellm/proxy/_experimental/out/experimental/prompts/__next._head.txt # litellm/proxy/_experimental/out/experimental/prompts/__next._index.txt # litellm/proxy/_experimental/out/experimental/prompts/__next._tree.txt # litellm/proxy/_experimental/out/experimental/tag-management.html # litellm/proxy/_experimental/out/experimental/tag-management.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next.!KGRhc2hib2FyZCk.experimental.tag-management.__PAGE__.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next.!KGRhc2hib2FyZCk.experimental.tag-management.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next.!KGRhc2hib2FyZCk.experimental.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next._full.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next._head.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next._index.txt # litellm/proxy/_experimental/out/experimental/tag-management/__next._tree.txt # litellm/proxy/_experimental/out/guardrails.html # litellm/proxy/_experimental/out/guardrails.txt # litellm/proxy/_experimental/out/guardrails/__next.!KGRhc2hib2FyZCk.guardrails.__PAGE__.txt # litellm/proxy/_experimental/out/guardrails/__next.!KGRhc2hib2FyZCk.guardrails.txt # litellm/proxy/_experimental/out/guardrails/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/guardrails/__next._full.txt # litellm/proxy/_experimental/out/guardrails/__next._head.txt # litellm/proxy/_experimental/out/guardrails/__next._index.txt # litellm/proxy/_experimental/out/guardrails/__next._tree.txt # litellm/proxy/_experimental/out/index.html # litellm/proxy/_experimental/out/index.txt # litellm/proxy/_experimental/out/login.html # litellm/proxy/_experimental/out/login.txt # litellm/proxy/_experimental/out/login/__next._full.txt # litellm/proxy/_experimental/out/login/__next._head.txt # litellm/proxy/_experimental/out/login/__next._index.txt # litellm/proxy/_experimental/out/login/__next._tree.txt # litellm/proxy/_experimental/out/login/__next.login.__PAGE__.txt # litellm/proxy/_experimental/out/login/__next.login.txt # litellm/proxy/_experimental/out/logs.html # litellm/proxy/_experimental/out/logs.txt # litellm/proxy/_experimental/out/logs/__next.!KGRhc2hib2FyZCk.logs.__PAGE__.txt # litellm/proxy/_experimental/out/logs/__next.!KGRhc2hib2FyZCk.logs.txt # litellm/proxy/_experimental/out/logs/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/logs/__next._full.txt # litellm/proxy/_experimental/out/logs/__next._head.txt # litellm/proxy/_experimental/out/logs/__next._index.txt # litellm/proxy/_experimental/out/logs/__next._tree.txt # litellm/proxy/_experimental/out/mcp/oauth/callback.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next._full.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next._head.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next._index.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next._tree.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next.mcp.oauth.callback.__PAGE__.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next.mcp.oauth.callback.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next.mcp.oauth.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/__next.mcp.txt # litellm/proxy/_experimental/out/mcp/oauth/callback/index.html # litellm/proxy/_experimental/out/model-hub.html # litellm/proxy/_experimental/out/model-hub.txt # litellm/proxy/_experimental/out/model-hub/__next.!KGRhc2hib2FyZCk.model-hub.__PAGE__.txt # litellm/proxy/_experimental/out/model-hub/__next.!KGRhc2hib2FyZCk.model-hub.txt # litellm/proxy/_experimental/out/model-hub/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/model-hub/__next._full.txt # litellm/proxy/_experimental/out/model-hub/__next._head.txt # litellm/proxy/_experimental/out/model-hub/__next._index.txt # litellm/proxy/_experimental/out/model-hub/__next._tree.txt # litellm/proxy/_experimental/out/model_hub.html # litellm/proxy/_experimental/out/model_hub.txt # litellm/proxy/_experimental/out/model_hub/__next._full.txt # litellm/proxy/_experimental/out/model_hub/__next._head.txt # litellm/proxy/_experimental/out/model_hub/__next._index.txt # litellm/proxy/_experimental/out/model_hub/__next._tree.txt # litellm/proxy/_experimental/out/model_hub/__next.model_hub.__PAGE__.txt # litellm/proxy/_experimental/out/model_hub/__next.model_hub.txt # litellm/proxy/_experimental/out/model_hub_table.html # litellm/proxy/_experimental/out/model_hub_table.txt # litellm/proxy/_experimental/out/model_hub_table/__next._full.txt # litellm/proxy/_experimental/out/model_hub_table/__next._head.txt # litellm/proxy/_experimental/out/model_hub_table/__next._index.txt # litellm/proxy/_experimental/out/model_hub_table/__next._tree.txt # litellm/proxy/_experimental/out/model_hub_table/__next.model_hub_table.__PAGE__.txt # litellm/proxy/_experimental/out/model_hub_table/__next.model_hub_table.txt # litellm/proxy/_experimental/out/models-and-endpoints.html # litellm/proxy/_experimental/out/models-and-endpoints.txt # litellm/proxy/_experimental/out/models-and-endpoints/__next.!KGRhc2hib2FyZCk.models-and-endpoints.__PAGE__.txt # litellm/proxy/_experimental/out/models-and-endpoints/__next.!KGRhc2hib2FyZCk.models-and-endpoints.txt # litellm/proxy/_experimental/out/models-and-endpoints/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/models-and-endpoints/__next._full.txt # litellm/proxy/_experimental/out/models-and-endpoints/__next._head.txt # litellm/proxy/_experimental/out/models-and-endpoints/__next._index.txt # litellm/proxy/_experimental/out/models-and-endpoints/__next._tree.txt # litellm/proxy/_experimental/out/onboarding.html # litellm/proxy/_experimental/out/onboarding.txt # litellm/proxy/_experimental/out/onboarding/__next._full.txt # litellm/proxy/_experimental/out/onboarding/__next._head.txt # litellm/proxy/_experimental/out/onboarding/__next._index.txt # litellm/proxy/_experimental/out/onboarding/__next._tree.txt # litellm/proxy/_experimental/out/onboarding/__next.onboarding.__PAGE__.txt # litellm/proxy/_experimental/out/onboarding/__next.onboarding.txt # litellm/proxy/_experimental/out/organizations.html # litellm/proxy/_experimental/out/organizations.txt # litellm/proxy/_experimental/out/organizations/__next.!KGRhc2hib2FyZCk.organizations.__PAGE__.txt # litellm/proxy/_experimental/out/organizations/__next.!KGRhc2hib2FyZCk.organizations.txt # litellm/proxy/_experimental/out/organizations/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/organizations/__next._full.txt # litellm/proxy/_experimental/out/organizations/__next._head.txt # litellm/proxy/_experimental/out/organizations/__next._index.txt # litellm/proxy/_experimental/out/organizations/__next._tree.txt # litellm/proxy/_experimental/out/playground.html # litellm/proxy/_experimental/out/playground.txt # litellm/proxy/_experimental/out/playground/__next.!KGRhc2hib2FyZCk.playground.__PAGE__.txt # litellm/proxy/_experimental/out/playground/__next.!KGRhc2hib2FyZCk.playground.txt # litellm/proxy/_experimental/out/playground/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/playground/__next._full.txt # litellm/proxy/_experimental/out/playground/__next._head.txt # litellm/proxy/_experimental/out/playground/__next._index.txt # litellm/proxy/_experimental/out/playground/__next._tree.txt # litellm/proxy/_experimental/out/policies.html # litellm/proxy/_experimental/out/policies.txt # litellm/proxy/_experimental/out/policies/__next.!KGRhc2hib2FyZCk.policies.__PAGE__.txt # litellm/proxy/_experimental/out/policies/__next.!KGRhc2hib2FyZCk.policies.txt # litellm/proxy/_experimental/out/policies/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/policies/__next._full.txt # litellm/proxy/_experimental/out/policies/__next._head.txt # litellm/proxy/_experimental/out/policies/__next._index.txt # litellm/proxy/_experimental/out/policies/__next._tree.txt # litellm/proxy/_experimental/out/settings/admin-settings.html # litellm/proxy/_experimental/out/settings/admin-settings.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next.!KGRhc2hib2FyZCk.settings.admin-settings.__PAGE__.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next.!KGRhc2hib2FyZCk.settings.admin-settings.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next.!KGRhc2hib2FyZCk.settings.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next._full.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next._head.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next._index.txt # litellm/proxy/_experimental/out/settings/admin-settings/__next._tree.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts.html # litellm/proxy/_experimental/out/settings/logging-and-alerts.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next.!KGRhc2hib2FyZCk.settings.logging-and-alerts.__PAGE__.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next.!KGRhc2hib2FyZCk.settings.logging-and-alerts.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next.!KGRhc2hib2FyZCk.settings.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next._full.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next._head.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next._index.txt # litellm/proxy/_experimental/out/settings/logging-and-alerts/__next._tree.txt # litellm/proxy/_experimental/out/settings/router-settings.html # litellm/proxy/_experimental/out/settings/router-settings.txt # litellm/proxy/_experimental/out/settings/router-settings/__next.!KGRhc2hib2FyZCk.settings.router-settings.__PAGE__.txt # litellm/proxy/_experimental/out/settings/router-settings/__next.!KGRhc2hib2FyZCk.settings.router-settings.txt # litellm/proxy/_experimental/out/settings/router-settings/__next.!KGRhc2hib2FyZCk.settings.txt # litellm/proxy/_experimental/out/settings/router-settings/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/settings/router-settings/__next._full.txt # litellm/proxy/_experimental/out/settings/router-settings/__next._head.txt # litellm/proxy/_experimental/out/settings/router-settings/__next._index.txt # litellm/proxy/_experimental/out/settings/router-settings/__next._tree.txt # litellm/proxy/_experimental/out/settings/ui-theme.html # litellm/proxy/_experimental/out/settings/ui-theme.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next.!KGRhc2hib2FyZCk.settings.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next.!KGRhc2hib2FyZCk.settings.ui-theme.__PAGE__.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next.!KGRhc2hib2FyZCk.settings.ui-theme.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next._full.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next._head.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next._index.txt # litellm/proxy/_experimental/out/settings/ui-theme/__next._tree.txt # litellm/proxy/_experimental/out/teams.html # litellm/proxy/_experimental/out/teams.txt # litellm/proxy/_experimental/out/teams/__next.!KGRhc2hib2FyZCk.teams.__PAGE__.txt # litellm/proxy/_experimental/out/teams/__next.!KGRhc2hib2FyZCk.teams.txt # litellm/proxy/_experimental/out/teams/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/teams/__next._full.txt # litellm/proxy/_experimental/out/teams/__next._head.txt # litellm/proxy/_experimental/out/teams/__next._index.txt # litellm/proxy/_experimental/out/teams/__next._tree.txt # litellm/proxy/_experimental/out/test-key.html # litellm/proxy/_experimental/out/test-key.txt # litellm/proxy/_experimental/out/test-key/__next.!KGRhc2hib2FyZCk.test-key.__PAGE__.txt # litellm/proxy/_experimental/out/test-key/__next.!KGRhc2hib2FyZCk.test-key.txt # litellm/proxy/_experimental/out/test-key/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/test-key/__next._full.txt # litellm/proxy/_experimental/out/test-key/__next._head.txt # litellm/proxy/_experimental/out/test-key/__next._index.txt # litellm/proxy/_experimental/out/test-key/__next._tree.txt # litellm/proxy/_experimental/out/tools/mcp-servers.html # litellm/proxy/_experimental/out/tools/mcp-servers.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next.!KGRhc2hib2FyZCk.tools.mcp-servers.__PAGE__.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next.!KGRhc2hib2FyZCk.tools.mcp-servers.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next.!KGRhc2hib2FyZCk.tools.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next._full.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next._head.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next._index.txt # litellm/proxy/_experimental/out/tools/mcp-servers/__next._tree.txt # litellm/proxy/_experimental/out/tools/vector-stores.html # litellm/proxy/_experimental/out/tools/vector-stores.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next.!KGRhc2hib2FyZCk.tools.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next.!KGRhc2hib2FyZCk.tools.vector-stores.__PAGE__.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next.!KGRhc2hib2FyZCk.tools.vector-stores.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next._full.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next._head.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next._index.txt # litellm/proxy/_experimental/out/tools/vector-stores/__next._tree.txt # litellm/proxy/_experimental/out/usage.html # litellm/proxy/_experimental/out/usage.txt # litellm/proxy/_experimental/out/usage/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/usage/__next.!KGRhc2hib2FyZCk.usage.__PAGE__.txt # litellm/proxy/_experimental/out/usage/__next.!KGRhc2hib2FyZCk.usage.txt # litellm/proxy/_experimental/out/usage/__next._full.txt # litellm/proxy/_experimental/out/usage/__next._head.txt # litellm/proxy/_experimental/out/usage/__next._index.txt # litellm/proxy/_experimental/out/usage/__next._tree.txt # litellm/proxy/_experimental/out/users.html # litellm/proxy/_experimental/out/users.txt # litellm/proxy/_experimental/out/users/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/users/__next.!KGRhc2hib2FyZCk.users.__PAGE__.txt # litellm/proxy/_experimental/out/users/__next.!KGRhc2hib2FyZCk.users.txt # litellm/proxy/_experimental/out/users/__next._full.txt # litellm/proxy/_experimental/out/users/__next._head.txt # litellm/proxy/_experimental/out/users/__next._index.txt # litellm/proxy/_experimental/out/users/__next._tree.txt # litellm/proxy/_experimental/out/virtual-keys.html # litellm/proxy/_experimental/out/virtual-keys.txt # litellm/proxy/_experimental/out/virtual-keys/__next.!KGRhc2hib2FyZCk.txt # litellm/proxy/_experimental/out/virtual-keys/__next.!KGRhc2hib2FyZCk.virtual-keys.__PAGE__.txt # litellm/proxy/_experimental/out/virtual-keys/__next.!KGRhc2hib2FyZCk.virtual-keys.txt # litellm/proxy/_experimental/out/virtual-keys/__next._full.txt # litellm/proxy/_experimental/out/virtual-keys/__next._head.txt # litellm/proxy/_experimental/out/virtual-keys/__next._index.txt # litellm/proxy/_experimental/out/virtual-keys/__next._tree.txt # scripts/install.sh # tests/local_testing/test_get_llm_provider.py |
||
|
Chesars
|
a2a506c4dc |
Revert "Merge pull request #16590 from Chesars/refactor/remove-backup-file-dry-principle"
This reverts commit |
||
|
Chesars
|
40eeffe089 |
test(vertex): use valid dimension (128) for multimodalembedding live test
dimensions=1 was silently dropped before #24415 wired the param through to Vertex. Now that it's forwarded, Vertex rejects it (valid values: 128, 256, 512, 1408). Switch the test to a valid dimension. |
||
|
Cesar GarciaandGitHub
|
8bd58fb82d | Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 | ||
|
Yuneng Jiang
|
c4ea0e93c8 |
fix: drain logging worker in test_router_caching_ttl to remove flake
The mocked async_increment_cache_pipeline is invoked from Router's deployment_callback_on_success, registered as an async success callback. Those callbacks are enqueued to GLOBAL_LOGGING_WORKER and run on a background task, so the mock may not have been called yet when the test asserts on it. Flush the worker before asserting. |
||
|
Cesar GarciaandGitHub
|
25c0aa8bfd |
Merge pull request #26283 from BerriAI/litellm_internal_staging
Sync litellm_staging_03_22_2026 with litellm_internal_staging |
||
|
Yuneng Jiang
|
ee550e1949 |
[Test] CI: add v2 migration resolver coverage with local Postgres
Adds end-to-end CI coverage for `--use_v2_migration_resolver` via a new job `installing_litellm_on_python_v2_migration_resolver`: - Clones the pytest smoke path from `installing_litellm_on_python` but uses a local Postgres sidecar instead of the shared DB to prevent collisions with the v1 variant. - Runs only the new `test_litellm_proxy_server_config_no_general_settings_v2_resolver` which spawns the proxy with `--use_v2_migration_resolver` and smoke-tests `/health/liveliness` and `/chat/completions`. Refactors `test_basic_python_version.py`: - Extracts the proxy spawn + smoke-test body into `_run_proxy_server_smoke_test` so the v1 and v2 tests share the same code path. - The existing `test_litellm_proxy_server_config_no_general_settings` is now a thin wrapper that passes no extra args (v1 default, unchanged). - Adds `..._v2_resolver` variant that passes `--use_v2_migration_resolver`. The existing `installing_litellm_on_python` / `installing_litellm_on_python_3_13` jobs filter out the v2 variant via `-k "not v2_resolver"` so they keep running only against their shared DB, unchanged behavior. |
||
|
Ryan Crabbe
|
eee51a99ad | replace retired claude-3-haiku-20240307 with claude-haiku-4-5-20251001 in local_testing part1 and router fallback tests | ||
|
Ryan Crabbe
|
74169b114a | replace retired claude-3-haiku-20240307 with claude-haiku-4-5-20251001 in streaming tests | ||
|
Sameer KankuteandGitHub
|
57eae8d01c | Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 | ||
|
Ishaan Jaffer
|
e8461b5b97 | style: run black formatter on files from main merge | ||
|
Yuneng Jiang
|
95e1babf67 |
[Fix] TogetherAIConfig.get_supported_openai_params recursion
TogetherAIConfig.get_supported_openai_params called get_model_info(), whose first line calls litellm.get_supported_openai_params() — which for together_ai routes straight back into this method. The recursion only terminated when Python's recursion limit was hit or when _get_model_info_helper raised "not mapped" at the deepest level. Either way the try/except caught it, so the bug stayed silent — but the cycle ran ~332 deep every time, emitting hundreds of DEBUG log lines per call. Surfaced as "infinite loop" in CI when the success_handler thread emitted that log spam against an already-closed stderr during test teardown. Replace the get_model_info() call with supports_function_calling(), which uses _get_model_info_helper directly and does not call get_supported_openai_params. Measured drop from 332 to 2 _get_model_info_helper calls per first uncached lookup. Also swap the test model from Qwen/Qwen3.5-9B (not in model_cost map) back to a mapped serverless model, Qwen/Qwen2.5-7B-Instruct-Turbo. The mapping gap is what made the recursion's tail end raise up into the success handler during teardown in the first place. |
||
|
Yuneng Jiang
|
dafa1bf97c |
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr15
# Conflicts: # litellm/litellm_core_utils/litellm_logging.py # uv.lock |
||
|
Sameer Kankute
|
83d0f28aa8 | Fix tests | ||
|
Yuneng Jiang
|
c8cfc5de21 |
fix(httpx): set response.request and strip content-encoding in MaskedHTTPStatusError
MaskedHTTPStatusError constructs a new httpx.Response from the original
error. Two bugs surfaced under real HTTP error responses:
1. The new Response was created without request=, so response.request
raised RuntimeError("The .request property has not been set.") for
any downstream caller (e.g. exception_mapping_utils) that inspected it.
2. The decoded response bytes were passed together with the original
Content-Encoding header. On construction httpx tried to decompress
the already-decoded bytes and raised httpx.DecodingError
("Error -3 while decompressing data: incorrect header check").
Set response.request to the masked Request and strip Content-Encoding
(and the now-stale Content-Length) before rebuilding the Response.
URL/message masking is unchanged; the new request carries the already
masked URL.
Also update test_logging_key_masking_gemini: the security commit
|
||
|
Ishaan Jaffer
|
def9c4ec47 | chore: merge litellm_internal_staging, resolve uv.lock conflict | ||
|
|
a588f76789 |
Litellm ishaan april15 2 (#25828)
* [Test] Add Azure async chat completion timeout test. WIP * Capture TTFT for /v1/messages streaming responses The pass-through streaming path for /v1/messages (Anthropic, Bedrock, Vertex AI, Azure AI, Minimax) logged completion_start_time only after the entire stream finished. async_success_handler then fell back to end_time, making TTFT equal to total duration or null in the UI and Prometheus. Record the timestamp of the first chunk in async_sse_wrapper and propagate it to model_call_details before the logging handler runs, so gen_ai.response.time_to_first_token reflects the real first-chunk latency. Fixes #25598 * [Refactor] Implement timeout resolution logic in completion function add fetch ``request_timeout`` from litellm_settings * remove stale test case * remove extra print statement * default request timeout value in constants to 600s to match timeout defaults handled in the proxy * fix request timeout if using default value from constants.py * update code structure, test cases * only override if the global timeout sets timeout to 6000s * update code structure, move hard coded values to const and make the reslve function readable by moving fallback logic to a seperate function * modify default timeout values, replacing hard coded ones with default values defined --------- Co-authored-by: harish876 <harishgokul01@gmail.com> Co-authored-by: Joaquin Hui Gomez <joaquinhuigomez@users.noreply.github.com> |
||
|
Ishaan Jaffer
|
9977e63e3c | Merge remote-tracking branch 'origin/main' into worktree-foamy-jumping-coral | ||
|
Ishaan Jaffer
|
c8a0fe193f | style: black format test_unit_test_caching.py | ||
|
Yuneng Jiang
|
a9c6156137 |
[Fix] Test - Together AI: replace deprecated Mixtral with serverless Qwen3.5-9B
Mixtral-8x7B-Instruct-v0.1 is no longer on Together AI's serverless tier and now requires a dedicated endpoint, causing multiple tests to fail in CI: - test_together_ai.py::TestTogetherAI::test_empty_tools - test_completion.py::test_completion_together_ai_stream - test_completion.py::test_customprompt_together_ai - test_completion.py::test_completion_custom_provider_model_name - test_text_completion.py::test_async_text_completion_together_ai Qwen/Qwen3.5-9B is currently serverless on Together AI and supports function calling, satisfying BaseLLMChatTest capability requirements. |
||
|
Sameer KankuteandGitHub
|
1a9a31e4a2 |
Merge pull request #25665 from BerriAI/litellm_oss_staging_04_13_2026_p1
litellm oss staging 04/13/2026 |
||
|
hatim-ezandSameer Kankute
|
17bfa420e4 |
fix(router): discard oldest entry when trimming latency list in lowest_latency strategy (#25548)
* fix(router): discard oldest entry when trimming latency list in lowest_latency strategy The lowest_latency routing strategy keeps a rolling window of the most recent latency and time-to-first-token measurements per deployment. When the window is full, the strategy was discarding the *newest* value instead of the oldest, because the trim used `[: max_latency_list_size - 1]` (keeping indices 0..N-2) rather than `[1:]` (dropping index 0 and keeping indices 1..N-1). Since new values are appended at the end, the bug meant the most recent measurement was always dropped once the list reached capacity. The routing decisions then relied on stale data (including any early-spike values that never aged out), and timeout penalties written via `async_log_failure_event` were silently discarded as well. Fix the slice in all five call sites (sync + async log_success_event for both latency and time_to_first_token, and async_log_failure_event for the timeout penalty) and add regression tests covering each path. * test(router): cover async TTFT trim path in lowest_latency regression tests Adds test_ttft_list_trimming_discards_oldest_entry_async, an async counterpart to test_ttft_list_trimming_discards_oldest_entry that drives async_log_success_event with a ModelResponse and completion_start_time so the async time_to_first_token trim branch is actually exercised. Previously no test touched that code path: the sync TTFT test used log_success_event, and the async latency test passed a plain dict response_obj without stream/completion_start_time, so TTFT was never computed and the async trim was unreached. Verified load-bearing by reverting only the async TTFT slice — the new test fails and all others pass. * format |
||
|
Sameer KankuteandGitHub
|
972e42c7fd | Merge branch 'main' into litellm_oss_staging_04_04_2026 | ||
|
michelligabriele
|
85497740ff | fix(caching): add Responses API params to cache key allow-list | ||
|
Yuneng Jiang
|
4b6eb02b66 |
[Fix] Pin uv/pip versions and fix bare prisma calls in CI
- Pin `pip==26.0.1` and `uv==0.10.9` in CCI jobs that used unpinned `pip install uv` (redis_caching_unit_tests, ui_e2e_tests) - Replace bare `prisma generate` with `uv run --no-sync prisma generate` in proxy_part1, proxy_part2, and enterprise test jobs - Remove duplicate `check=True` kwarg in test_basic_python_version.py that caused TypeError with `_run_uv()` helper |
||
|
joereynaandYuneng Jiang
|
148dd7096c | format vertex test file |