* feat(azure/prompt_shield.py): initial commit adding prompt shield guardrail + auto discovery mechanism for guardrails
reduces amount of code needed outside of guardrail integration for instrumentation
* feat(azure/prompt_shield.py): working azure prompt shield guardrail integration
Addresses https://github.com/BerriAI/litellm/issues/12254
* test: unit tests for prompt_shield
* fix(prompt_shield.py): add event hook validation for prompt shield guardrail
ensures prompt shield guardrail raises error if asked to run post_call (only runs on user prompt)
* feat(azure/): working text_moderation integration
* fix(text_moderation.py): suppress linting error
* test(test_azure_text_moderation.py): add unit test
* test(test_azure_text_moderation.py): add unit test for responses
* fix(text_moderation.py): return streaming error correctly
ensures error returned to user
* fix: fix linting error
* fix: fix linting check
* test: change mistral model
service tier exceeded
* fix(exception_mapping_utils.py): cover mistral in exception mapping
* fixes for using bedrock guard
* fixes for output_content_bedrock guard
* test_convert_to_bedrock_format_input_source
* test_convert_to_bedrock_format_post_call_streaming_hook
* test bedrock guard
* Enhance Mistral API: Add support for parallel tool calls and refine name handling in tool messages. Plus, introduce a new test for parallel tool calls in the Mistral model.
* tests
* make mypy happy
* Refine name handling in Mistral chat transformation: clarify conditions for removing the 'name' field based on message role and content.
* refactor: streamline Mistral integration by removing deprecated references and adding a new handler
- Removed "mistral" from the list of compatible providers in constants.
- Updated the completion function in main.py to utilize the new Mistral handler.
- Deleted outdated Mistral chat and embedding files.
- Introduced a new handler for Mistral chat completions, implementing the llm_http_handler pattern.
- Added integration tests for the Mistral handler to ensure proper API base and key handling.
* lint
* fix: remove unneeded handler object
* add tests
* Addres PR comments
* fix(main.py): handle router custom azure model name for responses api bridge
* fix(responses/handler): ensure azure model name is stripped before sending to provider
Fixes model name error
* fix(google_genai/main.py): handle stream=true being set in kwargs
* docs: cleanup icons from sidebar
* fix(test-litellm.yml): add google-genai to test litellmyml
* fix(main.py): strip 'responses/' from bridge
* fix(main.py): fix linting errors
* fix(types/openai.py): allow item to be none
handle azure streaming response
* fix(base.py): allow extra fields + handle azure item = none value in response output item added event
* fix(main.py): correctly handle removing responses/
* test(test_main.py): add unit tests
When JSON_LOGS=True is set, error logs were not being formatted as JSON despite
the configuration. This was because the logging initialization code configured
individual loggers but failed to properly initialize all loggers with the JSON
formatter.
This fix ensures that when json_logs is enabled, the _initialize_loggers_with_handler()
function is called to:
- Configure all loggers (root, LiteLLM, Router, Proxy) with JSON formatter
- Disable logger propagation to prevent duplicate entries
- Set up exception handlers for JSON formatting
Fixes LIT-267
* Enhance Mistral API: Add support for parallel tool calls and refine name handling in tool messages. Plus, introduce a new test for parallel tool calls in the Mistral model.
* tests
* make mypy happy
* Refine name handling in Mistral chat transformation: clarify conditions for removing the 'name' field based on message role and content.
* handle mistral returning '' instead of None
* fix(batches_endpoints/endpoints.py): support passing target model names for batch list as a query param
Fixes issue where cloud run fails calls because GET can't contain request body
* test(test_openai_batches_endpoints.py): add unit test
* docs(managed_batches.md): update docs
* feat(spend_tracking_utils.py): support STORE_PROMPTS_IN_SPEND_LOGS env var
ensures prompt is stored in spend logs
* fix(streaming_iterator.py): fix anthropic - completion streaming iterator to yield content block stop
ensures claude code renders messages
* test: skip local test
* fix - tuple was never falsy so never triggered the exception
* test - add test suite for openmeter integration
* refactor - move tests for openmeter integration
* Move PANW Prisma AIRS test per feedback on PR #12116
- Move test to tests/test_litellm/proxy/guardrails/guardrail_hooks/
* Remove test file from old location
* Fix: Preserve full path structure for Gemini custom api_base (Fixes#11959)
This fix addresses an issue where custom api_base URLs (like Cloudflare AI Gateway)
were not working correctly with Google AI Studio (Gemini) models.
The problem was that the _check_custom_proxy method was simply appending the endpoint
to the custom base URL, resulting in malformed URLs like:
https://gateway.ai.cloudflare.com/v1/my-id/my-gateway/google-ai-studio:generateContent
Instead of the correct format:
https://gateway.ai.cloudflare.com/v1/my-id/my-gateway/google-ai-studio/v1beta/models/gemini-2.5-flash:generateContent
Changes:
- Modified _check_custom_proxy to preserve the full path structure from the original URL
- Extracts the path from the original Google AI URL and appends it to the custom base
- Maintains backward compatibility for Vertex AI models (unchanged behavior)
- Added comprehensive tests to verify the fix works correctly
Fixes#11959
* Fix: Update test to match actual Gemini URL format and fix double colon issue
- Fixed test expectation to include the full model path with 'gemini/' prefix
- Fixed double colon issue in Vertex AI URL construction when using custom api_base
- All tests now pass successfully
* fix(proxy_server.py): handle empty config yaml
Fixes https://github.com/BerriAI/litellm/issues/12163
* fix(gemini/common_utils.py): replace models/ as expected, instead of using 'strip'
Fixes https://github.com/BerriAI/litellm/issues/12160
* fix(anthropic/experimental_pass_through/messages/transformation.py): check for env var when selecting api key
* fix(anthropic/transformation.py): return tool_use content block start on anthropic bridge
Closes https://github.com/BerriAI/litellm/issues/12158
* fix(anthropic/streaming_iterator.py): fix setting index in block
ensure index is set just once and increments correctly when a new block is created
* fix(anthropic/adapters/handler.py): update logging obj with stream options value if set
* feat(anthropic/streaming_iterator.py): return usage from chat completion to messages bridge
enables usage tracking for non-anthropic models
Closes https://github.com/BerriAI/litellm/issues/12132
* fix(streaming_iterator.py): safely access usage chunk
* fix: suppress linting error
* test: update tests
* fix: fix streaming errors
The test_keys_delete_error_handling test was failing with:
- ConnectionError when the mock wasn't properly applied
- The test was checking str(result.exception) without first verifying exception exists
This fix adds an explicit check that result.exception is not None before
attempting to convert it to string, preventing potential AttributeError
and making the test more robust.
* Allow strings in calculate cost
Sometimes the cost per unit is a string (e.g.: If a value like "3e-7" was read from the config.yaml)
* Add comprehensive tests for string cost value handling
- Added test_string_cost_values() to test basic string cost conversion functionality
- Added test_calculate_cost_component_with_string_values() to test the calculate_cost_component function directly
- Added test_string_cost_values_edge_cases() to test mixed string/float costs and error handling
- Added test_string_cost_values_with_threshold() to test string costs with threshold pricing
- Enhanced _get_token_base_cost() to handle string-to-float conversion for base costs and threshold costs
- Enhanced generic_cost_per_token() to handle string-to-float conversion for audio and reasoning token costs
- All tests cover scientific notation (e.g., '3e-7'), decimal notation (e.g., '0.000001'), and error handling for invalid strings
- Maintains backward compatibility with existing float cost values
* Dry up code
* Fixed case where number was an integer
* Allowing None
---------
Co-authored-by: openhands <openhands@all-hands.dev>
* fix(rebuild-usage-object---ensure-cache_tokens-is-set): Ensures cache tokens is correctly set
Fixes https://github.com/BerriAI/litellm/issues/12149
* test(test_stream_chunk_builder_utils.py): add unit test to ensure cached tokens is part of stream chunk builder
Ensures standardized values are used
* fix(proxy_server.py): handle empty config yaml
Fixes https://github.com/BerriAI/litellm/issues/12163
* fix(gemini/common_utils.py): replace models/ as expected, instead of using 'strip'
Fixes https://github.com/BerriAI/litellm/issues/12160
* fix(anthropic/experimental_pass_through/messages/transformation.py): check for env var when selecting api key
* docs(config_settings.md): add api key to docs
* fix: support Cursor IDE tool_choice format {"type": "auto"}
- Update validate_chat_completion_tool_choice to normalize {"type": "auto"} to "auto"
- Handles Cursor IDE sending non-standard tool_choice format
- Add comprehensive tests for tool choice validation
Fixes#12098
* fix: return full tool_choice object for Cursor IDE format
Based on PR feedback, updated validate_chat_completion_tool_choice to return
the full tool_choice dictionary instead of just extracting the type string.
This maintains consistency with downstream code that expects the full object
structure.
- Changed behavior: {"type": "auto"} now returns {"type": "auto"} instead of "auto"
- Updated tests to reflect the new expected behavior
- Ensures compatibility with code that passes tool_choice to optional_params
Addresses feedback from PR #12168
* fix(team_endpoints.py): prevent overwriting current list of team models on new model add
* fix(networking.tsx): fix default proxy base url
* fix(proxy_server.py): include team only models when retrieving all deployments on `/v2/model/info` helper util
ensures team only models are shown to user
* fix(router.py): check model name by team public model name when team id given
Fixes issue where team member could not see team only models when clicking into that team on `Models + Endpoints`
* fix(team_member_view.tsx): fix rendering team member budget, when budget is set
* test: update tests
* test: update unit test
* fix(anthropic/experimental_pass_through): use given model name when returning streaming chunks
don't harcode model name on streaming
confusing for user
* fix(anthropic/streaming_iterator.py): remove scope of import
* feat(litellm_logging.py): allow admin to specify additional headers for using as spend tags
Closes https://github.com/BerriAI/litellm/issues/12129
* test(test_litellm_logging.py): add unit tests
* feat(openweb_ui.md): add custom tag tutorial to docs
* docs(cost_tracking.md): add tag based usage UI screenshot
* test: update test
* fix: fix import
* use common helper create_invitation_for_user
* use common util in proxy
* fix create_invitation_for_user
* refactor base email
* test_get_invitation_link_creates_new_when_none_exist
* fix code QA checks
