Extend advanced-tool-use header translation to include Claude Sonnet 4.5
in addition to Opus 4.5 on Bedrock Invoke API.
When Claude Code sends the advanced-tool-use-2025-11-20 header, it now
gets correctly translated to Bedrock-specific headers for both:
- Claude Opus 4.5
- Claude Sonnet 4.5
Headers translated:
- tool-search-tool-2025-10-19
- tool-examples-2025-10-29
Fixes defer_loading validation error on Bedrock with Sonnet 4.5.
Ref: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
The error message for DISABLE_ADMIN_ENDPOINTS incorrectly said
"DISABLING LLM API ENDPOINTS is an Enterprise feature" instead of
"DISABLING ADMIN ENDPOINTS is an Enterprise feature".
This was a copy-paste bug from the is_llm_api_route_disabled() function.
Added regression tests to verify both error messages are correct.
* feat: add feature to make silent calls
* add test or silent feat
* add docs for silent feat
* fix lint issues and UI logs
* add docs of ab testing and deep copy
* Fix broken mocks in 6 flaky tests to prevent real API calls
Added network-level HTTP blocking using respx to prevent tests from making real API calls when Python-level mocks fail. This makes tests more reliable and retryable in CI.
Changes:
- Azure OIDC test: Added Azure Identity SDK mock to prevent real Azure calls
- Vector store test: Added @respx.mock decorator to block HTTP requests
- Resend email tests (3): Added @respx.mock decorator for all 3 test functions
- SendGrid email test: Added @respx.mock decorator
All test assertions and verification logic remain unchanged - only added safety nets to catch leaked API calls.
* Fix failing OIDC secret manager tests
Fixed two test failures in test_secret_managers_main.py:
1. test_oidc_azure_ad_token_success: Corrected the patch path for get_bearer_token_provider from 'litellm.secret_managers.get_azure_ad_token_provider.get_bearer_token_provider' to 'azure.identity.get_bearer_token_provider' since the function is imported from azure.identity.
2. test_oidc_google_success: Added @patch('httpx.Client') decorator to prevent any real HTTP connections during test execution, resolving httpx.ConnectError issues.
Both tests now pass successfully.
* Fix PLR0915: Extract system message handling to reduce statement count
* fix mypy
* fix: add host_progress_callback parameter to mock_call_tool in test
The test_call_tool_without_broken_pipe_error was failing because the mock function did not accept the host_progress_callback keyword argument that the actual implementation passes to client.call_tool(). Updated the mock to accept this parameter to match the real implementation signature.
* fixing flaky tests around oidc and email
* Add documentation comment to test file
* add retry
* add dependency
* increase retry
---------
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
* cache control for user messages and system messages
* add cache createion tokens in reponse
* cache controls in tool calls and assistant turns
* refactor with _should_preserve_cache_control
* add cache control unit tests
* use simpler cache creation token count logic
* use helper function
* remove unused function
* fix unit tests
* fix: make HTTPHandler mockable in OIDC secret manager tests
- Add _get_oidc_http_handler() factory function to make HTTPHandler
easily mockable in tests
- Update test_oidc_github_success to patch factory function instead
of HTTPHandler directly
- Update Google OIDC tests for consistency
- Fixes test_oidc_github_success failure where mock was bypassed
This change allows tests to properly mock HTTPHandler instances used
for OIDC token requests, fixing the test failure where the mock was
not being used.
* fix: patch base_llm_http_handler method directly in container tests
- Use patch.object to patch container_create_handler method directly
on the base_llm_http_handler instance instead of patching the module
- Fixes test_provider_support[openai] failure where mock wasn't applied
- Also fixes test_error_handling_integration with same approach
The issue was that patching 'litellm.containers.main.base_llm_http_handler'
didn't work because the module imports it with 'from litellm.main import',
creating a local reference. Using patch.object patches the method on the
actual object instance, which works regardless of import style.
* fix: resolve flaky test_openai_env_base by clearing cache
- Add cache clearing at start of test_openai_env_base to prevent cache pollution
- Ensures no cached clients from previous tests interfere with respx mocks
- Fixes intermittent failures where aiohttp transport was used instead of httpx
- Test-only change with low risk, no production code modifications
Resolves flaky test marked with @pytest.mark.flaky(retries=3, delay=1)
Both parametrized versions (OPENAI_API_BASE and OPENAI_BASE_URL) now pass consistently
* test: add explicit mock verification in test_provider_support
- Capture mock handler with 'as mock_handler' for explicit validation
- Add assert_called_once() to verify mock was actually used
- Ensures test verifies no real API calls are made
- Follows same pattern as test_openai_env_base validation
Mock _create_mcp_client to avoid network calls in health checks.
This prevents asyncio.CancelledError when the test teardown closes
the event loop while health checks are still pending.
The test focuses on conversion logic (access_groups, description)
not health check functionality, so mocking the network call is appropriate.
- Add test_get_valid_args in test_router_helper_utils.py to cover get_valid_args
- Use encoding='utf-8' in router_code_coverage.py for cross-platform file reads
- Create a test user with auto_create_key=False to ensure known starting state
- Filter get_users by user_ids to target only the test user
- Verify initial key count is 0 before creating a key
- Clean up test user after test completes
- This ensures consistent behavior across CI and local environments
- Added a Pydantic validator to convert empty string inputs for max_budget to None, preventing float parsing errors from the frontend.
- Modified the internal user update logic to explicitly allow max_budget to be None, ensuring the value isn't filtered out and can be reset to unlimited in the database.
- Added unit tests for validation and logic.
Closes#19781
Adds support for Anthropic-style 'thinking' parameter in hosted_vllm,
converting it to OpenAI-style 'reasoning_effort' since vLLM is
OpenAI-compatible.
This enables users to use Claude Code CLI with hosted vLLM models
like GLM-4.6/4.7 through the /v1/messages endpoint.
Mapping (same as Anthropic adapter):
- budget_tokens >= 10000 -> "high"
- budget_tokens >= 5000 -> "medium"
- budget_tokens >= 2000 -> "low"
- budget_tokens < 2000 -> "minimal"
Fixes#19761
* fix(proxy): use return value from CustomLogger.async_post_call_success_hook
Previously the return value was ignored for CustomLogger callbacks,
preventing users from modifying responses. Now the return value is
captured and used to replace the response (if not None), consistent
with CustomGuardrail and streaming iterator hook behavior.
Fixes issue with custom_callbacks not being able to inject data into
LLM responses.
* fix(proxy): also fix async_post_call_streaming_hook to use return value
Previously the streaming hook only used return values that started with
"data: " (SSE format). Now any non-None return value is used, consistent
with async_post_call_success_hook and streaming iterator hook behavior.
Added tests for streaming hook transformation.
---------
Co-authored-by: Gabriele Michelli <michelligabriele0@gmail.com>
* feat: add clientip and user agent in metrics
* fix: lint errors
* Add model id and other req labels
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)
OAuth discovery endpoints now support both URL patterns:
- Standard MCP pattern: /mcp/{server_name} (new)
- Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)
The standard pattern is required by MCP-compliant clients like
mcp-inspector and VSCode Copilot, which expect resource URLs
following the /mcp/{server_name} convention per RFC 9728.
Changes:
- Add _build_oauth_protected_resource_response() helper
- Add oauth_protected_resource_mcp_standard() endpoint
- Add oauth_authorization_server_mcp_standard() endpoint
- Keep legacy endpoints for backward compatibility
- Add tests for both URL patterns
Fixes#17272
* fix(mcp): Add standard MCP URL pattern support for OAuth discovery (#17272)
OAuth discovery endpoints now support both URL patterns:
- Standard MCP pattern: /mcp/{server_name} (new)
- Legacy LiteLLM pattern: /{server_name}/mcp (backward compatible)
The standard pattern is required by MCP-compliant clients like
mcp-inspector and VSCode Copilot, which expect resource URLs
following the /mcp/{server_name} convention per RFC 9728.
Changes:
- Add _build_oauth_protected_resource_response() helper
- Add oauth_protected_resource_mcp_standard() endpoint
- Add oauth_authorization_server_mcp_standard() endpoint
- Keep legacy endpoints for backward compatibility
- Add tests for both URL patterns
Fixes#17272
* Test was relocated
* refactor(mcp): Extract helper methods from run_with_session to fix PLR0915
Split the large run_with_session method (55 statements) into smaller
helper methods to satisfy ruff's PLR0915 rule (max 50 statements):
- _create_transport_context(): Creates transport based on type
- _execute_session_operation(): Handles session lifecycle
Also changed cleanup exception handling from Exception to BaseException
to properly catch asyncio.CancelledError (which is a BaseException subclass
in Python 3.8+).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(mcp): Fix flaky test by mocking health_check_server
The test_mcp_server_manager_config_integration_with_database test was
making real network calls to fake URLs which caused timeouts and
CancelledError exceptions.
Fixed by mocking health_check_server to return a proper
LiteLLM_MCPServerTable object instead of making network calls.
* test(mcp): Fix skip condition to properly detect claude model names
The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.
Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(mcp): Fix skip condition to properly detect claude model names
The skip condition for missing API keys was checking for "anthropic" in
the model name, but the test uses "claude-haiku-4-5" which doesn't match.
Updated to check for both "anthropic" and "claude" model patterns.
Also added skip condition for OpenAI models when OPENAI_API_KEY is not set.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(proxy): support slashes in google route params
* fix(proxy): extract google model ids with slashes
* test(proxy): cover google model ids with slashes
Sameer Kankute