Azure audio models were charging audio output tokens at the text token
rate instead of the correct audio token rate. This resulted in costs
being ~6.65x lower than expected.
The fix replaces Azure's custom cost calculation logic with the generic
cost calculator that properly handles text, audio, cached, reasoning,
and image tokens.
Fixes#19764
When using the responses API with provider-specific params (aws_*, vertex_*)
without explicitly passing custom_llm_provider, the code crashed with:
AttributeError: 'NoneType' object has no attribute 'startswith'
Root cause: local_vars was captured via locals() before get_llm_provider()
detected the provider from the model string (e.g., "bedrock/..."), so
custom_llm_provider remained None when processing provider-specific params.
Fix: Update local_vars["custom_llm_provider"] after get_llm_provider() call
so the detected provider is available for param processing.
Affected provider-specific params:
- aws_* (aws_region_name, aws_access_key_id, etc.) for Bedrock/SageMaker
- vertex_* (vertex_project, vertex_location, etc.) for Vertex AI
Fixes#19754
The gpt-5.2-codex model only supports the responses API, not chat completions.
Updated azure/gpt-5.2-codex and openrouter/openai/gpt-5.2-codex entries to use
mode: "responses" and supported_endpoints: ["/v1/responses"].
* Fix broken mocks in 6 flaky tests to prevent real API calls
Added network-level HTTP blocking using respx to prevent tests from making real API calls when Python-level mocks fail. This makes tests more reliable and retryable in CI.
Changes:
- Azure OIDC test: Added Azure Identity SDK mock to prevent real Azure calls
- Vector store test: Added @respx.mock decorator to block HTTP requests
- Resend email tests (3): Added @respx.mock decorator for all 3 test functions
- SendGrid email test: Added @respx.mock decorator
All test assertions and verification logic remain unchanged - only added safety nets to catch leaked API calls.
* Fix failing OIDC secret manager tests
Fixed two test failures in test_secret_managers_main.py:
1. test_oidc_azure_ad_token_success: Corrected the patch path for get_bearer_token_provider from 'litellm.secret_managers.get_azure_ad_token_provider.get_bearer_token_provider' to 'azure.identity.get_bearer_token_provider' since the function is imported from azure.identity.
2. test_oidc_google_success: Added @patch('httpx.Client') decorator to prevent any real HTTP connections during test execution, resolving httpx.ConnectError issues.
Both tests now pass successfully.
* Fix PLR0915: Extract system message handling to reduce statement count
* fix mypy
* fix: add host_progress_callback parameter to mock_call_tool in test
The test_call_tool_without_broken_pipe_error was failing because the mock function did not accept the host_progress_callback keyword argument that the actual implementation passes to client.call_tool(). Updated the mock to accept this parameter to match the real implementation signature.
* fixing flaky tests around oidc and email
* Add documentation comment to test file
* add retry
* add dependency
* increase retry
---------
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
* cache control for user messages and system messages
* add cache createion tokens in reponse
* cache controls in tool calls and assistant turns
* refactor with _should_preserve_cache_control
* add cache control unit tests
* use simpler cache creation token count logic
* use helper function
* remove unused function
* fix unit tests
* fix: make HTTPHandler mockable in OIDC secret manager tests
- Add _get_oidc_http_handler() factory function to make HTTPHandler
easily mockable in tests
- Update test_oidc_github_success to patch factory function instead
of HTTPHandler directly
- Update Google OIDC tests for consistency
- Fixes test_oidc_github_success failure where mock was bypassed
This change allows tests to properly mock HTTPHandler instances used
for OIDC token requests, fixing the test failure where the mock was
not being used.
* fix: patch base_llm_http_handler method directly in container tests
- Use patch.object to patch container_create_handler method directly
on the base_llm_http_handler instance instead of patching the module
- Fixes test_provider_support[openai] failure where mock wasn't applied
- Also fixes test_error_handling_integration with same approach
The issue was that patching 'litellm.containers.main.base_llm_http_handler'
didn't work because the module imports it with 'from litellm.main import',
creating a local reference. Using patch.object patches the method on the
actual object instance, which works regardless of import style.
* fix: resolve flaky test_openai_env_base by clearing cache
- Add cache clearing at start of test_openai_env_base to prevent cache pollution
- Ensures no cached clients from previous tests interfere with respx mocks
- Fixes intermittent failures where aiohttp transport was used instead of httpx
- Test-only change with low risk, no production code modifications
Resolves flaky test marked with @pytest.mark.flaky(retries=3, delay=1)
Both parametrized versions (OPENAI_API_BASE and OPENAI_BASE_URL) now pass consistently
* test: add explicit mock verification in test_provider_support
- Capture mock handler with 'as mock_handler' for explicit validation
- Add assert_called_once() to verify mock was actually used
- Ensures test verifies no real API calls are made
- Follows same pattern as test_openai_env_base validation
Mock _create_mcp_client to avoid network calls in health checks.
This prevents asyncio.CancelledError when the test teardown closes
the event loop while health checks are still pending.
The test focuses on conversion logic (access_groups, description)
not health check functionality, so mocking the network call is appropriate.
- Add test_get_valid_args in test_router_helper_utils.py to cover get_valid_args
- Use encoding='utf-8' in router_code_coverage.py for cross-platform file reads
- Create a test user with auto_create_key=False to ensure known starting state
- Filter get_users by user_ids to target only the test user
- Verify initial key count is 0 before creating a key
- Clean up test user after test completes
- This ensures consistent behavior across CI and local environments
- Added a Pydantic validator to convert empty string inputs for max_budget to None, preventing float parsing errors from the frontend.
- Modified the internal user update logic to explicitly allow max_budget to be None, ensuring the value isn't filtered out and can be reset to unlimited in the database.
- Added unit tests for validation and logic.
Closes#19781
* fix(proxy): use return value from CustomLogger.async_post_call_success_hook
Previously the return value was ignored for CustomLogger callbacks,
preventing users from modifying responses. Now the return value is
captured and used to replace the response (if not None), consistent
with CustomGuardrail and streaming iterator hook behavior.
Fixes issue with custom_callbacks not being able to inject data into
LLM responses.
* fix(proxy): also fix async_post_call_streaming_hook to use return value
Previously the streaming hook only used return values that started with
"data: " (SSE format). Now any non-None return value is used, consistent
with async_post_call_success_hook and streaming iterator hook behavior.
Added tests for streaming hook transformation.
---------
Co-authored-by: Gabriele Michelli <michelligabriele0@gmail.com>
