When extract_and_raise_litellm_exception tries to raise a LiteLLM exception
from an error string, it was always passing the response parameter. However,
some exceptions like APIConnectionError don't accept this parameter, causing
a TypeError.
This fix tries to raise the exception with the response parameter first,
and falls back to raising without it if a TypeError occurs.
This fixes the error:
TypeError: APIConnectionError.__init__() got an unexpected keyword argument 'response'
Which was occurring when Gemini returned UNEXPECTED_TOOL_CALL finish reason
and LiteLLM tried to convert the error to an APIConnectionError.
Fixes: cascading error when Gemini uses thinking feature (__thought__ tool calls)
* Update CLAUDE.md with qwen3 tool_calls bug fix instructions (#18922)
* fix(ollama): set finish_reason to "tool_calls" when tool_calls present
When qwen3 models return tool_calls through Ollama, the finish_reason
was incorrectly left as "stop" instead of being set to "tool_calls".
This caused clients to miss the tool_calls in the response.
Added _get_finish_reason helper method following OpenAI provider's
pattern, and fixed both streaming and non-streaming response paths.
Fixes: https://github.com/BerriAI/litellm/issues/18922
* fix(ollama): pass tools directly without model capability check
The previous code tried to check model capability via get_model_info()
which made network calls to localhost:11434. When Ollama is remote,
this fails and falls back to JSON format, breaking tool calling.
Ollama 0.4+ supports native tool calling - let Ollama handle
model capability detection instead of LiteLLM.
Fixes#18922
* fix(ollama): transform tool_calls response to OpenAI format
Ollama returns tool_calls with arguments as dict, but OpenAI format
requires arguments to be a JSON string. Also ensures 'type': 'function'
field is present.
Completes the fix for #18922
* fix(ollama): set finish_reason to "tool_calls" when tool_calls present
Fixes#18922
Two issues addressed:
1. Remove broken model capability check
- get_model_info() fails when Ollama runs on remote server
- Broken fallback triggered JSON prompt injection
- Now passes tools directly - Ollama 0.4+ handles detection
2. Set finish_reason correctly
- Was hardcoded to "stop" even with tool_calls present
- Clients use this to know how to process the response
- Now returns "tool_calls" when tool_calls are in response
Both streaming and non-streaming responses are fixed.
Tests:
- All 14 existing Ollama tests pass
- Added 3 focused tests for the fixes
Adds 'servers' field to OpenAPI schema when server_root_path is set, ensuring correct Swagger UI execute path for reverse proxies and subpath deployments. Includes tests to verify correct server URL handling for various root path formats.
* fix(text_completion): support token IDs (list of integers) as prompt
Add support for passing token IDs (list of integers) to the text_completion
endpoint for OpenAI-compatible providers (openai, azure, vllm, etc.).
Fixes#17118
* test(text_completion): replace live test with mock test for token IDs
Move token IDs test from local_testing to test_litellm with mocks
per PR review feedback.
* feat(bedrock): add OpenAI-compatible service_tier parameter translation
Translates OpenAI's service_tier parameter (string) to Bedrock's
serviceTier format (object with type field).
* docs(bedrock): add OpenAI-compatible service_tier parameter documentation
Document the automatic translation from OpenAI-style service_tier
parameter to Bedrock's native serviceTier format.
* feat(bedrock): add service_tier to response when present
According to OpenAI's API documentation, when service_tier is sent in the
request, it should be returned in the response. This commit implements
this behavior for Bedrock Converse API to maintain compatibility with
OpenAI's API.
Changes:
- Added serviceTier field to ConverseResponseBlock type definition
- Moved ServiceTierBlock definition before ConverseResponseBlock to fix
type reference order
- Added response transformation to map Bedrock serviceTier (object) to
OpenAI service_tier (string format)
- Added 4 new tests for response transformation with service_tier
The service_tier is only added to the response when present in Bedrock's
response, maintaining backward compatibility.
Fixes#18137
Similar to the fix for web_search_tool_result (#17746, #17798), this PR
preserves web_fetch_tool_result blocks in multi-turn conversations.
Changes:
- Add handling for web_fetch_tool_result in transformation.py (non-streaming)
- Add capture of web_fetch_tool_result in handler.py (streaming)
- Fix streaming tool arguments bug where empty input {} was prepended to
actual arguments by using empty string instead of str({})
- Add unit tests for web_fetch_tool_result handling
* docs: update message content types link and add content types table
- Update "See All Message Values" link to point to main branch (line 664)
instead of outdated commit 8600ec7 (line 392)
- Add Content Types table documenting all 6 multimodal content types:
text, image_url, input_audio, video_url, file, document
- Link to existing docs for vision, audio, and document understanding
* docs: add type definition links for text and video_url
* docs: fix text type definition link to line 598
* docs: remove provider labels from file/document types
* docs: add examples for all content types per review feedback
The OCI adapter now accepts both string and object formats for image_url:
- String: "image_url": "https://example.com/image.png"
- Object: "image_url": {"url": "https://example.com/image.png"}
This fixes compatibility with OpenAI Vision API format.
* fix: sync Helm chart versioning with production standards and Docker versions
- Update Chart.yaml version from 0.4.10 to 1.0.0 (SemVer 0.x is for development, 1.0+ for production)
- Update appVersion from v1.50.2 to v1.80.12 to match current Docker image version
- Update workflow defaults from 0.1.0 to 1.0.0 for new chart version scheme
- Maintain independent chart versioning per Helm best practices
This ensures:
- Helm chart follows SemVer production standards (1.x instead of 0.x)
- appVersion stays synchronized with Docker/application version
- Chart version remains independent for flexibility (can update chart without waiting for app releases)
* fix: sync Helm chart appVersion with Docker image tags in release workflow
Updates the GitHub workflow to ensure Helm chart appVersion matches the
Docker image tags that are actually published:
- For stable/rc releases: Uses the workflow input tag (e.g., v1.80.12)
- For latest/dev releases: Uses the release_type to match main-{type} tags
- Makes 'tag' input required to prevent accidental releases with wrong versions
- Simplifies fallback logic by removing git-describe dependency
This ensures the chart's appVersion correctly references Docker images
that exist, preventing deployment failures from missing image tags.
* Update ghcr_deploy.yml
* fix(gemini): prevent negative text_tokens with explicit caching (#18750)
## Problem
When using Gemini with explicit caching (especially with images),
text_tokens would become negative (e.g., -3327) due to incorrectly
subtracting total cached_tokens from modality-specific text_tokens.
## Root Cause
The old code did:
```python
text_tokens = text_tokens - cached_tokens # 737 - 4064 = -3327
```
This was wrong because:
- cached_tokens includes ALL modalities (text + image + audio + video)
- text_tokens only contains text
- Subtracting total from specific caused negative values
## Solution
Parse cacheTokensDetails to get per-modality cached token breakdown:
```python
if "cacheTokensDetails" in usage_metadata:
cached_text_tokens = parse from cacheTokensDetails["TEXT"]
text_tokens = text_tokens - cached_text_tokens # Correct!
```
Now we subtract cached tokens per modality, preventing negatives.
## Changes
- Parse cacheTokensDetails field from Gemini response
- Calculate non-cached tokens per modality (text, image, audio)
- Remove incorrect global cached_tokens subtraction
- Add tests for explicit caching and implicit/no caching scenarios
## Testing
- Added test_gemini_cache_tokens_details_no_negative_values
- Added test_gemini_without_cache_tokens_details
- All existing Gemini caching tests pass
Fixes#18750
* feat: add cache_read_input_tokens to Usage object
Addresses reviewer feedback to include cached tokens at the top level
of the Usage object. This aligns with how Anthropic provider handles
cached tokens and ensures they are visible in the final usage response.
* fix: add cacheTokensDetails field to UsageMetadata TypedDict
Fixes mypy error where cacheTokensDetails was being accessed but not defined
in the UsageMetadata TypedDict type definition.
