12463809bd
Merge pull request #19638 from BerriAI/main
...
merge main in stagin 1 22 26
2026-01-23 14:54:17 +05:30
69c8698e62
fix: pass through endpoints update registry ( #19420 )
...
* fix: pass through endpoints update registry
* add test case, fix lint error and comment to avoid confusion
* fix pass through endpoints test case
2026-01-22 19:57:48 -08:00
ab606c9a73
[Feat] Add Structured output for /v1/messages with Anthropic API, Azure Anthropic API, Bedrock Converse ( #19545 )
...
* fix: add AnthropicMessagesRequestOptionalParams
* add _update_headers_with_anthropic_beta
* fix output format tests
* test_structured_output_e2e
* TestAnthropicAPIStructuredOutput
* test_structured_output_e2e
* fix BASE
* TestAzureAnthropicStructuredOutput
* fix: Bedrock Converse
* add nthropic Messages Pass-Through Architecture
* fix: bedrock invoke output_format
* fix: transform_anthropic_messages_request for vertex anthropic
* TestBedrockInvokeStructuredOutput
* docs anthropic vertex
* docs fix
* docs fix
2026-01-21 20:09:18 -08:00
1a9a7df437
use mock db for cluade code marketplace
2026-01-20 16:18:21 -08:00
a82467d679
[Feat] - Add self hosted Claude Code Plugin Marketplace ( #19378 )
...
* init schema
* init endpoints
* fix: claude_code_marketplace_router
* refactor
* fix: claude_code_marketplace_router
* claude_code_marketplace_router
2026-01-19 14:05:47 -08:00
e817aa713e
[Fix] Claude Code x Bedrock Invoke fails with advanced-tool-use-2025-11-20 ( #19373 )
...
* _filter_unsupported_beta_headers_for_bedrock
* test_bedrock_sonnet_4_5_with_advanced_tool_use_beta_header
2026-01-19 10:16:18 -08:00
1417b002a3
[Feat] Claude Code x LiteLLM WebSearch - QA Fixes to work with Claude Code ( #19294 )
...
* fix websearch_interception_converted_stream
* test_websearch_interception_no_tool_call_streaming
* FakeAnthropicMessagesStreamIterator
* LITELLM_WEB_SEARCH_TOOL_NAME
* fixes tools def for litellm web search
* fixes FakeAnthropicMessagesStreamIterator
* test_litellm_standard_websearch_tool
* use new hook for modfying before any transfroms from litellm
* init WebSearchInterceptionLogger + ARCHITECTURE
* fix config.yaml
* init doc for claude code web search
* docs fix
* doc fix
* fix mypy linting
2026-01-17 16:30:31 -08:00
104283ae8f
[Feat] Claude Code - Add Websearch support using LiteLLM /search (using web search interception hook) ( #19263 )
...
* init WebSearchInterceptionLogger
* test_websearch_interception_real_call
* init async_should_run_agentic_completion
* async_should_run_agentic_loop
* async_run_agentic_loop
* refactor folder
* fix organization
* WebSearchTransformation
* WebSearchInterceptionLogger
* _call_agentic_completion_hooks
* WebSearch Interception Architecture
* test_websearch_interception_real_call
* add streaming
* add transform_request for streaming
* get_llm_provider
* test fix
* fix info
* init from config.yaml
* fixes
* test handler
* fix _is_streaming_response
* async_run_agentic_loop
* mypy fix
2026-01-16 21:10:05 -08:00
362b1a1577
[Feat] Add support for Tool Search on /messages API - Azure, Bedrock, Anthropic API ( #19165 )
...
* fix _update_headers_with_anthropic_beta
* init ANTHROPIC_BETA_HEADER_VALUES
* fix ANTHROPIC_BETA_HEADER_VALUES
* fix: _update_headers_with_anthropic_beta - anthropic API
* init _update_headers_with_anthropic_beta - azure AI support
* init VertexAIPartnerModelsAnthropicMessagesConfig
* fix _get_total_tokens_from_usage
* working TestBedrockInvokeToolSearch
* fix get_extra_headers
* TestBedrockInvokeToolSearch
* _get_tool_search_beta_header_for_bedrock
* fix mypy linting
2026-01-15 16:35:00 -08:00
458f773861
[Feat] Claude Code - Add support for Prompt Caching with Bedrock Converse ( #19123 )
...
* init BaseAnthropicMessagesPromptCachingTest
* fix UsageDelta
* fix: _create_initial_usage_delta
* TestBedrockInvokePromptCaching
* translate_anthropic_messages_to_openai wiht cache control
* fix translate_anthropic_messages_to_openai
2026-01-14 18:05:10 -08:00
06ded8750e
[Fix] Claude Code (/messages) - Litellm fix claude code Bedrock Invoke usage, request signing ( #19111 )
...
* test_should_not_fail_with_forwarded_headers_bedrock_invoke_messages
* use common get_request_headers for BaseAWS
* fix get_request_headers
* test_should_not_fail_with_forwarded_headers_bedrock_invoke_messages
2026-01-14 14:51:50 -08:00
747829dadb
[Fix] Claude Code + Bedrock Converse Usage - ensure budget tokens are passed to converse api correctly ( #19107 )
...
* test_bedrock_converse_budget_tokens_preserved
* test_openai_model_with_thinking_converts_to_reasoning_effort
* fix translate_anthropic_thinking_to_reasoning_effort
* test_bedrock_converse_budget_tokens_preserved
* test_anthropic_messages_bedrock_converse_with_thinking
2026-01-14 12:02:27 -08:00
5534038e93
Fix CI: Revert security scan changes and add GitGuardian ignore rules ( #18358 )
2025-12-22 17:03:53 -08:00
6112160a16
Revert "[Fix] Security - Remove example API keys with high entropy ( #18255 )"
...
This reverts commit 24edbccf5c
2025-12-20 20:48:11 +05:30
24edbccf5c
[Fix] Security - Remove example API keys with high entropy ( #18255 )
2025-12-19 10:09:50 -08:00
8fd0c81e5b
Add cost tracking for streaming in vertex ai
2025-11-20 15:08:38 +05:30
159db27d5c
fix test claude-sonnet-4-5-20250929
2025-10-31 18:13:29 -07:00
c1369a07ba
Add Add per model group header forwarding for Bedrock Invoke API ( #16042 )
2025-10-30 20:10:17 -07:00
85d4142845
Fix litellm_param based costing
2025-10-08 21:14:23 +05:30
64083111d3
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
...
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
2025-09-30 21:14:16 -07:00
04b3ac89b8
test: QueryParams
2025-09-30 18:45:38 -07:00
ce0b815959
fix test
2025-09-27 02:08:09 +05:30
61a450f2e2
fix lint
2025-09-27 01:16:09 +05:30
67e7ad5aa9
Add vertex live api passthrough with cost tracking
2025-09-27 00:55:47 +05:30
706b9214c0
fix: test_init_kwargs_for_pass_through_endpoint_basic
2025-09-18 07:59:05 -07:00
433d1a4947
[Bug fix] - Fix /messages fallback from Anthropic API -> Bedrock API ( #13946 )
...
* use helper get_provider_specific_headers
* fix get_provider_specific_headers
* test_anthropic_messages_fallbacks
* bedrock/us.anthropic.claude-sonnet-4
* fix: get_provider_specific_headers
* TestProviderSpecificHeaderUtils
* test_anthropic_messages_fallbacks
2025-08-25 13:44:54 -07:00
b78495d398
[Fix] Ensure /messages works when using `bedrock/converse/<model> with LiteLLM ( #13627 )
...
* get_bedrock_provider_config_for_messages_api
* fixes for get_bedrock_provider_config_for_messages_api
* test_anthropic_messages_litellm_router_bedrock
* fix merge conflicts
* fix - refactor based on jugal's comment
2025-08-14 16:50:05 -07:00
039c8a922c
Azure api_version="preview" support + Bedrock cost tracking via Anthropic /v1/messages ( #13072 )
...
* fix(azure/chat/gpt_transformation.py): support api_version="preview"
Fixes https://github.com/BerriAI/litellm/issues/12945
* Fix anthropic passthrough logging handler model fallback for streaming requests (#13022 )
* fix: anthropic passthrough logging handler model fallback for streaming requests
- Add fallback logic to retrieve model from logging_obj.model_call_details when request_body.model is empty
- Fixes issue #12933 where streaming requests to anthropic passthrough endpoints would crash due to missing model field
- Ensures downstream logging and cost calculation work correctly for all streaming scenarios
- Maintains backwards compatibility with existing non-streaming requests
* test: add minimal tests for anthropic passthrough logging handler model fallback
- Add unit tests for the model fallback logic in _handle_logging_anthropic_collected_chunks
- Test existing behavior when request_body.model is present
- Test fallback logic when request_body.model is empty but logging_obj.model_call_details has model
- Test edge cases where both sources are empty or missing
- Ensure backwards compatibility and graceful degradation
* fix(anthropic_passthrough_logging_handler.py): add provider to model name (accurate cost tracking)
* fix(anthropic_passthrough_logging_handler.py): don't reset custom llm provider, if already set
* fix: fix check
---------
Co-authored-by: Haggai Shachar <haggai.shachar@backline.ai >
2025-07-29 08:13:55 -07:00
847c4514c4
test fix - test_anthropic_messages_passthrough.py
2025-06-30 21:56:31 -07:00
d65a9fdcc7
[Bug Fix] Using /messages with lowest latency routing ( #12180 )
...
* add validate_anthropic_api_metadata
* fixes for lowest latency deployment
* add _select_metadata_field
* test_anthropic_messages_litellm_router_latency_metadata_tracking
2025-06-30 15:57:19 -07:00
75298af605
[Bug Fix] Cost tracking and logging via the /v1/messages API are not working when using Claude Code ( #11928 )
...
* add test_anthropic_messages_litellm_router_streaming_with_logging to base tests
* move test
* fixes for base ant tests
* working bedrock ant logging
* use BaseAnthropicMessagesStreamingIterator
* use common iterator for messages streaming
* TestAnthropicDirectAPI
* test_anthropic_claude3_transformation.py
* fix code QA checks
* fix logging for anthropic messages in SLP
* fix TestAnthropicOpenAIAPI
* remove hard coded usage for adapter
* test_anthropic_messages_litellm_router_streaming_with_logging
2025-06-20 18:08:35 -07:00
931b2e4875
[Bug Fix] Fix model_group tracked for /v1/messages and /moderations ( #11933 )
...
* fixes _get_router_metadata_variable_name
* fixes _update_kwargs_before_fallbacks
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* test_moderations_api_logging
* fix _pass_through_moderation_endpoint_factory
2025-06-20 14:51:50 -07:00
9ec6df59e4
fixes for pass through tests
2025-06-18 21:47:37 -07:00
c92b6c175c
Prometheus - fix request increment + add route tracking for streaming requests ( #11731 )
...
* fix(prometheus.py): remove request increment from inside the log success event
it's only done on post-call success/failure
* fix(litellm_logging.py): add additional validation step for checking if 'stream' is true
prevent double counting on non-stream requests
* test: add unit testing to ensure stream is not incorrectly set to true
* feat(litellm_logging.py): emit request route in standard logging payload
used by prometheus streaming metrics for route
* fix: fix otel test
* fix: fix linting errors
* test: update test
* fix: fix linting error
2025-06-14 16:26:48 -07:00
362e358a77
[Feat] Allow using litellm.completion with /v1/messages API Spec (use gpt-4, gemini etc with claude code) ( #11502 )
...
* feat: add anthropic stream wrapper
* feat: add AnthropicExperimentalPassThroughConfig
* feat: working non streaming anthropic
* feat: working streaming anthropic-litellm bridge
* test - anthropic OpenAI bridge tests
* fix: add sync support for anthropic_messages
* fix: using is async check
* fix: ensure streams are SSE
* fix: imports
* fix code qa check
* fix: linting errors
* test_sync_openai_messages
* cleanup remove stash file
2025-06-06 20:35:53 -07:00
3a6802fef1
[Feat] - Add Support for Showing Passthrough endpoint Error Logs on LiteLLM UI ( #10990 )
...
* fix: add error logging for passthrough endpoints
* feat: add error logging for passthrough endpoints
* fix: post_call_failure_hook track errors on pt
* fix: use constant for MAXIMUM_TRACEBACK_LINES_TO_LOG
* docs MAXIMUM_TRACEBACK_LINES_TO_LOG
* test: ensure failure callback triggered
* fix: move _init_kwargs_for_pass_through_endpoint
2025-05-20 18:29:39 -07:00
eeb27d70c1
[Fix] Allow using dynamic aws_region with /messages on Bedrock ( #10779 )
...
* fix: fix get_complete_url
* test: test_anthropic_messages_bedrock_dynamic_region
2025-05-12 20:22:38 -07:00
51930c07c5
[Fix]: /messages - allow using dynamic AWS params ( #10769 )
...
* fix: dynamic AWS params added for messages routes
* Update tests/pass_through_unit_tests/test_anthropic_messages_passthrough.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-05-12 14:09:17 -07:00
e5a08a5ae1
[Feat] Add streaming support for using bedrock invoke models with /v1/messages ( #10710 )
...
* add basic bedrock transform
* test_anthropic_messages_streaming_bedrock_invoke
* fix: typing ant
* fix: get async response iterator
* fix: code quality check
2025-05-09 18:56:23 -07:00
a0ee31edf8
[Feat] Add support for using Bedrock Invoke models in /v1/messages format ( #10681 )
...
* fix: add transform_anthropic_messages_request
* fix: add get_requested_response_api_optional_param
* fix: use base llm http handler for anthropic messages
* fix: add anthropic transform response
* fix: transform_anthropic_messages_response
* fix: fixes for anthropic messages
* fix: code qa fixes
* fix: pass thinking to anthropic
* fix: linting
* fixes
* feat: add folder for bedrock invoke messages
* feat: init bedrock invoke messages for anthropic claude family
* test: add bedrock invoke test for us anthropic
* test: test_anthropic_messages_non_streaming_bedrock_invokec
* feat: update anthropic messages transforms
* feat: update anthropic messages transforms
* Update litellm/utils.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* fix: test_anthropic_messages_non_streaming
* fix: linting override
* fix: linting error
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-05-08 21:16:47 -07:00
9d8f570f14
[Refactor] Anthropic /v1/messages endpoint - Refactor to use base llm http handler and transformations ( #10677 )
...
* fix: add transform_anthropic_messages_request
* fix: add get_requested_response_api_optional_param
* fix: use base llm http handler for anthropic messages
* fix: add anthropic transform response
* fix: transform_anthropic_messages_response
* fix: fixes for anthropic messages
* fix: code qa fixes
* fix: pass thinking to anthropic
* fix: linting
* fixes
2025-05-08 17:56:50 -07:00
931c0c760c
fix test_anthropic_messages_litellm_router_streaming_with_logging
2025-05-03 09:34:32 -07:00
4eac0f64f3
[Feat] Pass through endpoints - ensure PassthroughStandardLoggingPayload is logged and contains method, url, request/response body ( #10194 )
...
* ensure passthrough_logging_payload is filled in kwargs
* test_assistants_passthrough_logging
* test_assistants_passthrough_logging
* test_assistants_passthrough_logging
* test_threads_passthrough_logging
* test _init_kwargs_for_pass_through_endpoint
* _init_kwargs_for_pass_through_endpoint
2025-04-21 19:46:22 -07:00
bd39a395f1
use new anthropic interface
2025-03-31 14:31:09 -07:00
9eb9a369bb
working anthropic API tests
2025-03-26 17:34:41 -07:00
6b2f385ddf
test: update tests
2025-03-22 12:56:42 -07:00
6f719d0461
test: fix test
2025-03-22 12:50:58 -07:00
3ce3689282
test: migrate testing
2025-03-22 12:48:53 -07:00
94d3413335
refactor(llm_passthrough_endpoints.py): refactor vertex passthrough to use common llm passthrough handler.py
2025-03-22 10:42:46 -07:00
f47987e673
(Refactor) /v1/messages to follow simpler logic for Anthropic API spec ( #9013 )
...
* anthropic_messages_handler v0
* fix /messages
* working messages with router methods
* test_anthropic_messages_handler_litellm_router_non_streaming
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* AnthropicMessagesConfig
* _handle_anthropic_messages_response_logging
* working with /v1/messages endpoint
* working /v1/messages endpoint
* refactor to use router factory function
* use aanthropic_messages
* use BaseConfig for Anthropic /v1/messages
* track api key, team on /v1/messages endpoint
* fix get_logging_payload
* BaseAnthropicMessagesTest
* align test config
* test_anthropic_messages_with_thinking
* test_anthropic_streaming_with_thinking
* fix - display anthropic url for debugging
* test_bad_request_error_handling
* test_anthropic_messages_router_streaming_with_bad_request
* fix ProxyException
* test_bad_request_error_handling_streaming
* use provider_specific_header
* test_anthropic_messages_with_extra_headers
* test_anthropic_messages_to_wildcard_model
* fix gcs pub sub test
* standard_logging_payload
* fix unit testing for anthopic /v1/messages support
* fix pass through anthropic messages api
* delete dead code
* fix anthropic pass through response
* revert change to spend tracking utils
* fix get_litellm_metadata_from_kwargs
* fix spend logs payload json
* proxy_pass_through_endpoint_tests
* TestAnthropicPassthroughBasic
* fix pass through tests
* test_async_vertex_proxy_route_api_key_auth
* _handle_anthropic_messages_response_logging
* vertex_credentials
* test_set_default_vertex_config
* test_anthropic_messages_litellm_router_non_streaming_with_logging
* test_ageneric_api_call_with_fallbacks_basic
* test__aadapter_completion
2025-03-06 00:43:08 -08:00
Sameer Kankute