tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-13 14:23:04 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add Day 0 gemini-3-pro-preview support (#16719)

Browse Source 
* Add thinking signature support for gemini

* Add docs related to thinking signature

* remove double base64 import

* fix mypy errors

* fix litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py mypy

* Add new gemini 3 model and features

* Add docs related to gemini 3

* Update gemini 3 pricing

* fix llm translation tests

* fix mapped tests
This commit is contained in:
Sameer Kankute
2025-11-18 09:44:45 -08:00
committed by GitHub
parent 4dbbec233c
commit acf206bec6
12 changed files with 1362 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/docs/providers/gemini.md
+408 -1
View File
@@ -70,7 +70,11 @@ LiteLLM translates OpenAI's `reasoning_effort` to Gemini's `thinking` parameter.
Note: Reasoning cannot be turned off on Gemini 2.5 Pro models.
:::
**Mapping**
:::tip Gemini 3 Models
For **Gemini 3+ models** (e.g., `gemini-3-pro-preview`), LiteLLM automatically maps `reasoning_effort` to the new `thinking_level` parameter instead of `thinking_budget`. The `thinking_level` parameter uses `"low"` or `"high"` values for better control over reasoning depth.
:::
**Mapping for Gemini 2.5 and earlier models**
| reasoning_effort | thinking | Notes |
| ---------------- | -------- | ----- |
@@ -80,6 +84,17 @@ Note: Reasoning cannot be turned off on Gemini 2.5 Pro models.
| "medium" | "budget_tokens": 2048 | |
| "high" | "budget_tokens": 4096 | |
**Mapping for Gemini 3+ models**
| reasoning_effort | thinking_level | Notes |
| ---------------- | -------------- | ----- |
| "minimal" | "low" | Minimizes latency and cost |
| "low" | "low" | Best for simple instruction following or chat |
| "medium" | "high" | Maps to high (medium not yet available) |
| "high" | "high" | Maximizes reasoning depth |
| "disable" | "low" | Cannot fully disable thinking in Gemini 3 |
| "none" | "low" | Cannot fully disable thinking in Gemini 3 |
<Tabs>
<TabItem value="sdk" label="SDK">
@@ -137,6 +152,59 @@ curl http://0.0.0.0:4000/v1/chat/completions \
</TabItem>
</Tabs>
### Gemini 3+ Models - `thinking_level` Parameter
For Gemini 3+ models (e.g., `gemini-3-pro-preview`), you can use the new `thinking_level` parameter directly:
<Tabs>
<TabItem value="sdk" label="SDK">
```python
from litellm import completion
# Use thinking_level for Gemini 3 models
resp = completion(
model="gemini/gemini-3-pro-preview",
messages=[{"role": "user", "content": "Solve this complex math problem step by step."}],
reasoning_effort="high", # Options: "low" or "high"
)
# Low thinking level for faster, simpler tasks
resp = completion(
model="gemini/gemini-3-pro-preview",
messages=[{"role": "user", "content": "What is the weather today?"}],
reasoning_effort="low", # Minimizes latency and cost
)
```
</TabItem>
<TabItem value="proxy" label="PROXY">
```bash
curl http://0.0.0.0:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
-d '{
"model": "gemini-3-pro-preview",
"messages": [{"role": "user", "content": "Solve this complex problem."}],
"reasoning_effort": "high"
}'
```
</TabItem>
</Tabs>
:::warning
**Temperature Recommendation for Gemini 3 Models**
For Gemini 3 models, LiteLLM defaults `temperature` to `1.0` and strongly recommends keeping it at this default. Setting `temperature < 1.0` can cause:
- Infinite loops
- Degraded reasoning performance
- Failure on complex tasks
LiteLLM will automatically set `temperature=1.0` if not specified for Gemini 3+ models.
:::
**Expected Response**
@@ -951,6 +1019,295 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \
## Thought Signatures
Thought signatures are encrypted representations of the model's internal reasoning process for a given turn in a conversation. By passing thought signatures back to the model in subsequent requests, you provide it with the context of its previous thoughts, allowing it to build upon its reasoning and maintain a coherent line of inquiry.
Thought signatures are particularly important for multi-turn function calling scenarios where the model needs to maintain context across multiple tool invocations.
### How Thought Signatures Work
- **Function calls with signatures**: When Gemini returns a function call, it includes a `thought_signature` in the response
- **Preservation**: LiteLLM automatically extracts and stores thought signatures in `provider_specific_fields` of tool calls
- **Return in conversation history**: When you include the assistant's message with tool calls in subsequent requests, LiteLLM automatically preserves and returns the thought signatures to Gemini
- **Parallel function calls**: Only the first function call in a parallel set has a thought signature
- **Sequential function calls**: Each function call in a multi-step sequence has its own signature
### Enabling Thought Signatures
To enable thought signatures, you need to enable thinking/reasoning:
<Tabs>
<TabItem value="sdk" label="SDK">
```python
from litellm import completion
response = completion(
model="gemini/gemini-2.5-flash",
messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
tools=[...],
reasoning_effort="low", # Enable thinking to get thought signatures
)
```
</TabItem>
<TabItem value="proxy" label="PROXY">
```bash
curl http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-1234" \
-d '{
"model": "gemini-2.5-flash",
"messages": [{"role": "user", "content": "What'\''s the weather in Tokyo?"}],
"tools": [...],
"reasoning_effort": "low"
}'
```
</TabItem>
</Tabs>
### Multi-Turn Function Calling with Thought Signatures
When building conversation history for multi-turn function calling, you must include the thought signatures from previous responses. LiteLLM handles this automatically when you append the full assistant message to your conversation history.
<Tabs>
<TabItem value="sdk" label="OpenAI Client">
```python
from openai import OpenAI
import json
client = OpenAI(api_key="sk-1234", base_url="http://localhost:4000")
def get_current_temperature(location: str) -> dict:
"""Gets the current weather temperature for a given location."""
return {"temperature": 30, "unit": "celsius"}
def set_thermostat_temperature(temperature: int) -> dict:
"""Sets the thermostat to a desired temperature."""
return {"status": "success"}
get_weather_declaration = {
"name": "get_current_temperature",
"description": "Gets the current weather temperature for a given location.",
"parameters": {
"type": "object",
"properties": {"location": {"type": "string"}},
"required": ["location"],
},
}
set_thermostat_declaration = {
"name": "set_thermostat_temperature",
"description": "Sets the thermostat to a desired temperature.",
"parameters": {
"type": "object",
"properties": {"temperature": {"type": "integer"}},
"required": ["temperature"],
},
}
# Initial request
messages = [
{"role": "user", "content": "If it's too hot or too cold in London, set the thermostat to a comfortable level."}
]
response = client.chat.completions.create(
model="gemini-2.5-flash",
messages=messages,
tools=[get_weather_declaration, set_thermostat_declaration],
reasoning_effort="low"
)
# Append the assistant's message (includes thought signatures automatically)
messages.append(response.choices[0].message)
# Execute tool calls and append results
for tool_call in response.choices[0].message.tool_calls:
if tool_call.function.name == "get_current_temperature":
result = get_current_temperature(**json.loads(tool_call.function.arguments))
messages.append({
"role": "tool",
"content": json.dumps(result),
"tool_call_id": tool_call.id
})
# Second request - thought signatures are automatically preserved
response2 = client.chat.completions.create(
model="gemini-2.5-flash",
messages=messages,
tools=[get_weather_declaration, set_thermostat_declaration],
reasoning_effort="low"
)
print(response2.choices[0].message.content)
```
</TabItem>
<TabItem value="curl" label="cURL">
```bash
# Step 1: Initial request
curl --location 'http://localhost:4000/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-1234' \
--data '{
"model": "gemini-2.5-flash",
"messages": [
{
"role": "user",
"content": "If it'\''s too hot or too cold in London, set the thermostat to a comfortable level."
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_temperature",
"description": "Gets the current weather temperature for a given location.",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
}
},
{
"type": "function",
"function": {
"name": "set_thermostat_temperature",
"description": "Sets the thermostat to a desired temperature.",
"parameters": {
"type": "object",
"properties": {
"temperature": {"type": "integer"}
},
"required": ["temperature"]
}
}
}
],
"tool_choice": "auto",
"reasoning_effort": "low"
}'
```
The response will include tool calls with thought signatures in `provider_specific_fields`:
```json
{
"choices": [{
"message": {
"role": "assistant",
"tool_calls": [{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_current_temperature",
"arguments": "{\"location\": \"London\"}"
},
"index": 0,
"provider_specific_fields": {
"thought_signature": "CpcHAdHtim9+q4rstcbvQC0ic4x1/vqQlCJWgE+UZ6dTLYGHMMBkF/AxqL5UmP6SY46uYC8t4BTFiXG5zkw6EMJ...=="
}
}]
}
}]
}
```
```bash
# Step 2: Follow-up request with tool response
# Include the assistant message from Step 1 (with thought signatures in provider_specific_fields)
curl --location 'http://localhost:4000/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-1234' \
--data '{
"model": "gemini-2.5-flash",
"messages": [
{
"role": "user",
"content": "If it'\''s too hot or too cold in London, set the thermostat to a comfortable level."
},
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_c130b9f8c2c042e9b65e39a88245",
"type": "function",
"function": {
"name": "get_current_temperature",
"arguments": "{\"location\": \"London\"}"
},
"index": 0,
"provider_specific_fields": {
"thought_signature": "CpcHAdHtim9+q4rstcbvQC0ic4x1/vqQlCJWgE+UZ6dTLYGHMMBkF/AxqL5UmP6SY46uYC8t4BTFiXG5zkw6EMJ...=="
}
}
]
},
{
"role": "tool",
"content": "{\"temperature\": 30, \"unit\": \"celsius\"}",
"tool_call_id": "call_c130b9f8c2c042e9b65e39a88245"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_current_temperature",
"description": "Gets the current weather temperature for a given location.",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
}
},
{
"type": "function",
"function": {
"name": "set_thermostat_temperature",
"description": "Sets the thermostat to a desired temperature.",
"parameters": {
"type": "object",
"properties": {
"temperature": {"type": "integer"}
},
"required": ["temperature"]
}
}
}
],
"tool_choice": "auto",
"reasoning_effort": "low"
}'
```
</TabItem>
</Tabs>
### Important Notes
1. **Automatic Handling**: LiteLLM automatically extracts thought signatures from Gemini responses and preserves them when you include assistant messages in conversation history. You don't need to manually extract or manage them.
2. **Parallel Function Calls**: When the model makes parallel function calls, only the first function call will have a thought signature. Subsequent parallel calls won't have signatures.
3. **Sequential Function Calls**: In multi-step function calling scenarios, each step's first function call will have its own thought signature that must be preserved.
4. **Required for Context**: Thought signatures are essential for maintaining reasoning context across multi-turn conversations with function calling. Without them, the model may lose context of its previous reasoning.
5. **Format**: Thought signatures are stored in `provider_specific_fields.thought_signature` of tool calls in the response, and are automatically included when you append the assistant message to your conversation history.
## JSON Mode
<Tabs>
@@ -1022,6 +1379,56 @@ LiteLLM Supports the following image types passed in `url`
- Images with direct links - https://storage.googleapis.com/github-repo/img/gemini/intro/landmark3.jpg
- Image in local storage - ./localimage.jpeg
## Image Resolution Control (Gemini 3+)
For Gemini 3+ models, LiteLLM supports per-part media resolution control using OpenAI's `detail` parameter. This allows you to specify different resolution levels for individual images in your request.
**Supported `detail` values:**
- `"low"` - Maps to `media_resolution: "low"` (280 tokens for images, 70 tokens per frame for videos)
- `"high"` - Maps to `media_resolution: "high"` (1120 tokens for images)
- `"auto"` or `None` - Model decides optimal resolution (no `media_resolution` set)
**Usage Example:**
```python
from litellm import completion
messages = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": "https://example.com/chart.png",
"detail": "high" # High resolution for detailed chart analysis
}
},
{
"type": "text",
"text": "Analyze this chart"
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/icon.png",
"detail": "low" # Low resolution for simple icon
}
}
]
}
]
response = completion(
model="gemini/gemini-3-pro-preview",
messages=messages,
)
```
:::info
**Per-Part Resolution:** Each image in your request can have its own `detail` setting, allowing mixed-resolution requests (e.g., a high-res chart alongside a low-res icon). This feature is only available for Gemini 3+ models.
:::
## Sample Usage
```python
import os
litellm/litellm_core_utils/prompt_templates/factory.py
+31 -6
View File
@@ -1,3 +1,4 @@
import base64
import copy
import hashlib
import json
@@ -1161,6 +1162,14 @@ def _gemini_tool_call_invoke_helper(
return function_call
def _get_thought_signature_from_tool(tool: dict) -> Optional[str]:
"""Extract thought signature from tool call's provider_specific_fields"""
provider_fields = tool.get("provider_specific_fields") or {}
if isinstance(provider_fields, dict):
return provider_fields.get("thought_signature")
return None
def convert_to_gemini_tool_call_invoke(
message: ChatCompletionAssistantMessage,
) -> List[VertexPartType]:
@@ -1207,8 +1216,9 @@ def convert_to_gemini_tool_call_invoke(
_parts_list: List[VertexPartType] = []
tool_calls = message.get("tool_calls", None)
function_call = message.get("function_call", None)
if tool_calls is not None:
for tool in tool_calls:
for idx, tool in enumerate(tool_calls):
if "function" in tool:
gemini_function_call: Optional[VertexFunctionCall] = (
_gemini_tool_call_invoke_helper(
@@ -1216,9 +1226,14 @@ def convert_to_gemini_tool_call_invoke(
)
)
if gemini_function_call is not None:
_parts_list.append(
VertexPartType(function_call=gemini_function_call)
)
part_dict: VertexPartType = {
"function_call": gemini_function_call
}
thought_signature = _get_thought_signature_from_tool(dict(tool))
if thought_signature:
part_dict["thoughtSignature"] = thought_signature
_parts_list.append(part_dict)
else: # don't silently drop params. Make it clear to user what's happening.
raise Exception(
"function_call missing. Received tool call with 'type': 'function'. No function call in argument - {}".format(
@@ -1230,7 +1245,18 @@ def convert_to_gemini_tool_call_invoke(
function_call_params=function_call
)
if gemini_function_call is not None:
_parts_list.append(VertexPartType(function_call=gemini_function_call))
part_dict_function: VertexPartType = {
"function_call": gemini_function_call
}
# Extract thought signature from function_call's provider_specific_fields
provider_fields = function_call.get("provider_specific_fields") if isinstance(function_call, dict) else {}
if isinstance(provider_fields, dict):
thought_signature = provider_fields.get("thought_signature")
if thought_signature:
part_dict_function["thoughtSignature"] = thought_signature
_parts_list.append(part_dict_function)
else: # don't silently drop params. Make it clear to user what's happening.
raise Exception(
"function_call missing. Received tool call with 'type': 'function'. No function call in argument - {}".format(
@@ -2496,7 +2522,6 @@ def stringify_json_tool_call_content(messages: List) -> List:
###### AMAZON BEDROCK #######
import base64
from email.message import Message
import httpx
litellm/llms/gemini/chat/transformation.py
+1 -1
View File
@@ -99,7 +99,7 @@ class GoogleAIStudioGeminiConfig(VertexGeminiConfig):
return supported_params
def _transform_messages(
self, messages: List[AllMessageValues]
self, messages: List[AllMessageValues], model: Optional[str] = None
) -> List[ContentType]:
"""
Google AI Studio Gemini does not support HTTP/HTTPS URLs for files.
litellm/llms/vertex_ai/gemini/transformation.py
+42 -7
View File
@@ -64,7 +64,25 @@ else:
LiteLLMLoggingObj = Any
def _process_gemini_image(image_url: str, format: Optional[str] = None) -> PartType:
def _map_openai_detail_to_media_resolution(
detail: Optional[str],
) -> Optional[Literal["low", "medium", "high"]]:
"""
Map OpenAI's "detail" parameter to Gemini's "media_resolution" parameter.
"""
if detail == "low":
return "low"
elif detail == "high":
return "high"
# "auto" or None means let the model decide, so we don't set media_resolution
return None
def _process_gemini_image(
image_url: str,
format: Optional[str] = None,
media_resolution: Optional[Literal["low", "medium", "high"]] = None,
) -> PartType:
"""
Given an image URL, return the appropriate PartType for Gemini
"""
@@ -99,8 +117,19 @@ def _process_gemini_image(image_url: str, format: Optional[str] = None) -> PartT
elif "http://" in image_url or "https://" in image_url or "base64" in image_url:
# https links for unsupported mime types and base64 images
image = convert_to_anthropic_image_obj(image_url, format=format)
_blob = BlobType(data=image["data"], mime_type=image["media_type"])
return PartType(inline_data=_blob)
_blob: BlobType = {"data": image["data"], "mime_type": image["media_type"]}
if media_resolution is not None:
_blob["media_resolution"] = media_resolution
# Convert snake_case keys to camelCase for JSON serialization
# The TypedDict uses snake_case, but the API expects camelCase
_blob_dict = dict(_blob)
if "media_resolution" in _blob_dict:
_blob_dict["mediaResolution"] = _blob_dict.pop("media_resolution")
if "mime_type" in _blob_dict:
_blob_dict["mimeType"] = _blob_dict.pop("mime_type")
return PartType(inline_data=cast(BlobType, _blob_dict))
raise Exception("Invalid image received - {}".format(image_url))
except Exception as e:
raise e
@@ -205,13 +234,18 @@ def _gemini_convert_messages_with_history( # noqa: PLR0915
element = cast(ChatCompletionImageObject, element)
img_element = element
format: Optional[str] = None
media_resolution: Optional[Literal["low", "medium", "high"]] = None
if isinstance(img_element["image_url"], dict):
image_url = img_element["image_url"]["url"]
format = img_element["image_url"].get("format")
detail = img_element["image_url"].get("detail")
media_resolution = _map_openai_detail_to_media_resolution(detail)
else:
image_url = img_element["image_url"]
_part = _process_gemini_image(
image_url=image_url, format=format
image_url=image_url,
format=format,
media_resolution=media_resolution,
)
_parts.append(_part)
elif element["type"] == "input_audio":
@@ -250,7 +284,8 @@ def _gemini_convert_messages_with_history( # noqa: PLR0915
)
try:
_part = _process_gemini_image(
image_url=passed_file, format=format
image_url=passed_file,
format=format,
)
_parts.append(_part)
except Exception:
@@ -448,11 +483,11 @@ def _transform_request_body(
try:
if custom_llm_provider == "gemini":
content = litellm.GoogleAIStudioGeminiConfig()._transform_messages(
messages=messages
messages=messages, model=model
)
else:
content = litellm.VertexGeminiConfig()._transform_messages(
messages=messages
messages=messages, model=model
)
tools: Optional[Tools] = optional_params.pop("tools", None)
tool_choice: Optional[ToolConfig] = optional_params.pop("tool_choice", None)
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py
+148 -16
View File
@@ -218,6 +218,21 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
def get_config(cls):
return super().get_config()
@staticmethod
def _is_gemini_3_or_newer(model: str) -> bool:
"""
Check if the model is Gemini 3 Pro or newer.
Gemini 3 models include:
- gemini-3-pro-preview
- Any future Gemini 3.x models
"""
# Check for Gemini 3 models
if "gemini-3" in model:
return True
return False
def _supports_penalty_parameters(self, model: str) -> bool:
unsupported_models = ["gemini-2.5-pro-preview-06-05"]
if model in unsupported_models:
@@ -575,10 +590,80 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
else:
raise ValueError(f"Invalid reasoning effort: {reasoning_effort}")
@staticmethod
def _map_reasoning_effort_to_thinking_level(
reasoning_effort: str,
model: Optional[str] = None,
) -> GeminiThinkingConfig:
"""
Map reasoning_effort to thinking_level for Gemini 3+ models.
Args:
reasoning_effort: The reasoning effort value
model: The model name (for validation, currently unused but kept for consistency)
Returns:
GeminiThinkingConfig with thinkingLevel set
"""
if reasoning_effort == "minimal":
return {"thinkingLevel": "low"}
elif reasoning_effort == "low":
return {"thinkingLevel": "low"}
elif reasoning_effort == "medium":
return {"thinkingLevel": "high"} # medium is not out yet
elif reasoning_effort == "high":
return {"thinkingLevel": "high"}
elif reasoning_effort == "disable":
return {"thinkingLevel": "low"} # gemini 3 cannot fully disable thinking, so we use "low"
elif reasoning_effort == "none":
return {"thinkingLevel": "low"} # gemini 3 cannot fully disable thinking, so we use "low"
else:
raise ValueError(f"Invalid reasoning effort: {reasoning_effort}")
@staticmethod
def _is_thinking_budget_zero(thinking_budget: Optional[int]) -> bool:
return thinking_budget is not None and thinking_budget == 0
@staticmethod
def _validate_thinking_config_conflicts(
optional_params: Dict,
param_name: str,
param_description: str = "thinking_budget",
) -> None:
"""
Validate that thinking_level and thinking_budget are not both specified.
"""
if "thinkingConfig" in optional_params:
existing_config = optional_params["thinkingConfig"]
if "thinkingLevel" in existing_config:
raise litellm.utils.UnsupportedParamsError(
message=(
f"Cannot specify both `{param_name}` (which maps to `{param_description}`) "
"and `thinking_level` in the same request. "
"For Gemini 3 models, use `thinking_level` instead."
),
status_code=400,
)
@staticmethod
def _validate_thinking_level_conflicts(
optional_params: Dict,
) -> None:
"""
Validate that thinking_level and thinking_budget are not both specified.
Called when setting thinking_level.
"""
if "thinkingConfig" in optional_params:
existing_config = optional_params["thinkingConfig"]
if "thinkingBudget" in existing_config:
raise litellm.utils.UnsupportedParamsError(
message=(
"Cannot specify both `thinking_level` and `thinking_budget` in the same request. "
"For Gemini 3 models, use `thinking_level` instead of `thinking_budget`."
),
status_code=400,
)
@staticmethod
def _map_thinking_param(
thinking_param: AnthropicThinkingParam,
@@ -672,6 +757,13 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
) -> Dict:
for param, value in non_default_params.items():
if param == "temperature":
if VertexGeminiConfig._is_gemini_3_or_newer(model):
if value is not None and value < 1.0:
verbose_logger.info(
f"Warning: Setting temperature < 1.0 for Gemini 3 models ({model}) "
"can cause infinite loops, degraded reasoning performance, and failure on complex tasks. "
"Strongly recommended to use temperature = 1.0 (default)."
)
optional_params["temperature"] = value
elif param == "top_p":
optional_params["top_p"] = value
@@ -734,12 +826,31 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
elif param == "seed":
optional_params["seed"] = value
elif param == "reasoning_effort" and isinstance(value, str):
optional_params[
"thinkingConfig"
] = VertexGeminiConfig._map_reasoning_effort_to_thinking_budget(
value, model
# Validate no conflict with thinking_level
VertexGeminiConfig._validate_thinking_config_conflicts(
optional_params=optional_params,
param_name="reasoning_effort",
param_description="thinking_budget",
)
if VertexGeminiConfig._is_gemini_3_or_newer(model):
optional_params[
"thinkingConfig"
] = VertexGeminiConfig._map_reasoning_effort_to_thinking_level(
value, model
)
else:
optional_params[
"thinkingConfig"
] = VertexGeminiConfig._map_reasoning_effort_to_thinking_budget(
value, model
)
elif param == "thinking":
# Validate no conflict with thinking_level
VertexGeminiConfig._validate_thinking_config_conflicts(
optional_params=optional_params,
param_name="thinking",
param_description="thinking_budget",
)
optional_params[
"thinkingConfig"
] = VertexGeminiConfig._map_thinking_param(
@@ -764,6 +875,15 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
elif "AUDIO" not in optional_params["responseModalities"]:
optional_params["responseModalities"].append("AUDIO")
# Set default temperature to 1.0 for Gemini 3 models if not specified
if VertexGeminiConfig._is_gemini_3_or_newer(model):
if "temperature" not in optional_params:
optional_params["temperature"] = 1.0
if "thinkingConfig" not in optional_params or "thinkingLevel" not in optional_params.get("thinkingConfig", {}):
thinking_config = optional_params.get("thinkingConfig", {})
thinking_config["thinkingLevel"] = "low"
optional_params["thinkingConfig"] = thinking_config
return optional_params
def get_mapped_special_auth_params(self) -> dict:
@@ -1025,19 +1145,31 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
_tools: List[ChatCompletionToolCallChunk] = []
for part in parts:
if "functionCall" in part:
_function_chunk = ChatCompletionToolCallFunctionChunk(
name=part["functionCall"]["name"],
arguments=json.dumps(part["functionCall"]["args"], ensure_ascii=False),
)
_function_chunk: ChatCompletionToolCallFunctionChunk = {
"name": part["functionCall"]["name"],
"arguments": json.dumps(part["functionCall"]["args"], ensure_ascii=False),
}
# Extract thought signature if present
thought_signature = part.get("thoughtSignature")
if is_function_call is True:
function = _function_chunk
function_dict: Dict[str, Any] = dict(_function_chunk)
if thought_signature:
if "provider_specific_fields" not in function_dict:
function_dict["provider_specific_fields"] = {}
function_dict["provider_specific_fields"]["thought_signature"] = thought_signature
function = cast(ChatCompletionToolCallFunctionChunk, function_dict)
else:
_tool_response_chunk = ChatCompletionToolCallChunk(
id=f"call_{uuid.uuid4().hex[:28]}",
type="function",
function=_function_chunk,
index=cumulative_tool_call_idx,
)
_tool_response_chunk: ChatCompletionToolCallChunk = {
"id": f"call_{uuid.uuid4().hex[:28]}",
"type": "function",
"function": _function_chunk,
"index": cumulative_tool_call_idx,
}
if thought_signature:
_tool_response_chunk["provider_specific_fields"] = { # type: ignore
"thought_signature": thought_signature
}
_tools.append(_tool_response_chunk)
cumulative_tool_call_idx += 1
if len(_tools) == 0:
@@ -1718,7 +1850,7 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
return model_response
def _transform_messages(
self, messages: List[AllMessageValues]
self, messages: List[AllMessageValues], model: Optional[str] = None
) -> List[ContentType]:
return _gemini_convert_messages_with_history(messages=messages)
litellm/model_prices_and_context_window_backup.json
+89
View File
@@ -10981,6 +10981,50 @@
"supports_vision": true,
"supports_web_search": true
},
"gemini-3-pro-preview": {
"cache_read_input_token_cost": 1.25e-07,
"cache_creation_input_token_cost_above_200k_tokens": 2.5e-07,
"input_cost_per_token": 2e-06,
"input_cost_per_token_above_200k_tokens": 4e-06,
"litellm_provider": "vertex_ai-language-models",
"max_audio_length_hours": 8.4,
"max_audio_per_prompt": 1,
"max_images_per_prompt": 3000,
"max_input_tokens": 1048576,
"max_output_tokens": 65535,
"max_pdf_size_mb": 30,
"max_tokens": 65535,
"max_video_length": 1,
"max_videos_per_prompt": 10,
"mode": "chat",
"output_cost_per_token": 1.2e-05,
"output_cost_per_token_above_200k_tokens": 1.8e-05,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_audio_input": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_video_input": true,
"supports_vision": true,
"supports_web_search": true
},
"gemini-2.5-pro-exp-03-25": {
"cache_read_input_token_cost": 3.125e-07,
"input_cost_per_token": 1.25e-06,
@@ -12640,6 +12684,51 @@
"supports_web_search": true,
"tpm": 800000
},
"gemini/gemini-3-pro-preview": {
"cache_read_input_token_cost": 3.125e-07,
"input_cost_per_token": 2e-06,
"input_cost_per_token_above_200k_tokens": 4e-06,
"litellm_provider": "gemini",
"max_audio_length_hours": 8.4,
"max_audio_per_prompt": 1,
"max_images_per_prompt": 3000,
"max_input_tokens": 1048576,
"max_output_tokens": 65535,
"max_pdf_size_mb": 30,
"max_tokens": 65535,
"max_video_length": 1,
"max_videos_per_prompt": 10,
"mode": "chat",
"output_cost_per_token": 1.2e-05,
"output_cost_per_token_above_200k_tokens": 1.8e-05,
"rpm": 2000,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_audio_input": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_video_input": true,
"supports_vision": true,
"supports_web_search": true,
"tpm": 800000
},
"gemini/gemini-2.5-pro-exp-03-25": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 0.0,
litellm/types/llms/vertex_ai.py
+5 -2
View File
@@ -29,9 +29,10 @@ class FileDataType(TypedDict):
file_uri: str # the cloud storage uri of storing this file
class BlobType(TypedDict):
class BlobType(TypedDict, total=False):
mime_type: Required[str]
data: Required[str]
media_resolution: Literal["low", "medium", "high"]
class PartType(TypedDict, total=False):
@@ -59,9 +60,10 @@ class HttpxCodeExecutionResult(TypedDict):
output: str
class HttpxBlobType(TypedDict):
class HttpxBlobType(TypedDict, total=False):
mimeType: str
data: str
mediaResolution: Literal["low", "medium", "high"]
class HttpxPartType(TypedDict, total=False):
@@ -174,6 +176,7 @@ class SafetSettingsConfig(TypedDict, total=False):
class GeminiThinkingConfig(TypedDict, total=False):
includeThoughts: bool
thinkingBudget: int
thinkingLevel: Literal["low", "medium", "high"]
GeminiResponseModalities = Literal["TEXT", "IMAGE", "AUDIO", "VIDEO"]
model_prices_and_context_window.json
+89
View File
@@ -10993,6 +10993,50 @@
"supports_vision": true,
"supports_web_search": true
},
"gemini-3-pro-preview": {
"cache_read_input_token_cost": 1.25e-07,
"cache_creation_input_token_cost_above_200k_tokens": 2.5e-07,
"input_cost_per_token": 2e-06,
"input_cost_per_token_above_200k_tokens": 4e-06,
"litellm_provider": "vertex_ai-language-models",
"max_audio_length_hours": 8.4,
"max_audio_per_prompt": 1,
"max_images_per_prompt": 3000,
"max_input_tokens": 1048576,
"max_output_tokens": 65535,
"max_pdf_size_mb": 30,
"max_tokens": 65535,
"max_video_length": 1,
"max_videos_per_prompt": 10,
"mode": "chat",
"output_cost_per_token": 1.2e-05,
"output_cost_per_token_above_200k_tokens": 1.8e-05,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_audio_input": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_video_input": true,
"supports_vision": true,
"supports_web_search": true
},
"gemini-2.5-pro-exp-03-25": {
"cache_read_input_token_cost": 3.125e-07,
"input_cost_per_token": 1.25e-06,
@@ -12652,6 +12696,51 @@
"supports_web_search": true,
"tpm": 800000
},
"gemini/gemini-3-pro-preview": {
"cache_read_input_token_cost": 3.125e-07,
"input_cost_per_token": 2e-06,
"input_cost_per_token_above_200k_tokens": 4e-06,
"litellm_provider": "gemini",
"max_audio_length_hours": 8.4,
"max_audio_per_prompt": 1,
"max_images_per_prompt": 3000,
"max_input_tokens": 1048576,
"max_output_tokens": 65535,
"max_pdf_size_mb": 30,
"max_tokens": 65535,
"max_video_length": 1,
"max_videos_per_prompt": 10,
"mode": "chat",
"output_cost_per_token": 1.2e-05,
"output_cost_per_token_above_200k_tokens": 1.8e-05,
"rpm": 2000,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_audio_input": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_video_input": true,
"supports_vision": true,
"supports_web_search": true,
"tpm": 800000
},
"gemini/gemini-2.5-pro-exp-03-25": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 0.0,
tests/llm_translation/test_prompt_factory.py
+1 -1
View File
@@ -456,7 +456,7 @@ def test_vertex_only_image_user_message():
{
"inline_data": {
"data": "/9j/2wCEAAgGBgcGBQ",
"mime_type": "image/jpeg",
"mimeType": "image/jpeg",
}
},
{"text": " "},
tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py
+214
View File
@@ -176,3 +176,217 @@ def test_empty_content_handling():
assert len(contents[0]["parts"]) == 1
assert "text" in contents[0]["parts"][0]
assert contents[0]["parts"][0]["text"] == ""
def test_thought_signature_extraction_from_response():
"""Test that thought signatures are extracted from Gemini response parts and stored in provider_specific_fields"""
from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (
VertexGeminiConfig,
)
from litellm.types.llms.vertex_ai import HttpxPartType
# Test case: Single function call with thought signature
test_signature = "Co4CAdHtim/rWgXbz2Ghp4tShzLeMASrPw6JJyYIC3cbVyZnKzU3uv8/wVzyS2sKRPL2m8QQHHXbNQhEEz500G7n/4ZMmksdTtfQcJMoT76S1DGwhnAiLwTgWCNXs3lEb4M19EVYoWFxhrH5Lr9YMIquoU9U4paydGwvZyIyigamIg4B6WnxrRsf0KZV12gJed0DZuKczvOFtHz3zUnmZRlOiTzd5gBVyQM+5jv1VI8m4WUKd6cN/5a5ZvaA0ggiO6kdVhlpIVs7GczSEVJD8KH4u02X7VSnb7CvykqDntZzV0y8rZFBEFGKrChmeHlWXP4D1IB3F9KQyhuLgWImMzg4BajKVxxMU737JGnNISy5"
parts_with_signature = [
HttpxPartType(
functionCall={
"name": "get_current_temperature",
"args": {"location": "Paris"},
},
thoughtSignature=test_signature,
)
]
function, tools, _ = VertexGeminiConfig._transform_parts(
parts=parts_with_signature,
cumulative_tool_call_idx=0,
is_function_call=False,
)
# Verify thought signature is stored in provider_specific_fields
assert tools is not None
assert len(tools) == 1
assert "provider_specific_fields" in tools[0]
assert tools[0]["provider_specific_fields"]["thought_signature"] == test_signature
def test_thought_signature_parallel_function_calls():
"""Test that only the first function call in parallel calls has thought signature"""
from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (
VertexGeminiConfig,
)
from litellm.types.llms.vertex_ai import HttpxPartType
test_signature = "Co4CAdHtim/rWgXbz2Ghp4tShzLeMASrPw6JJyYIC3cbVyZnKzU3uv8/wVzyS2sKRPL2m8QQHHXbNQhEEz500G7n/4ZMmksdTtfQcJMoT76S1DGwhnAiLwTgWCNXs3lEb4M19EVYoWFxhrH5Lr9YMIquoU9U4paydGwvZyIyigamIg4B6WnxrRsf0KZV12gJed0DZuKczvOFtHz3zUnmZRlOiTzd5gBVyQM+5jv1VI8m4WUKd6cN/5a5ZvaA0ggiO6kdVhlpIVs7GczSEVJD8KH4u02X7VSnb7CvykqDntZzV0y8rZFBEFGKrChmeHlWXP4D1IB3F9KQyhuLgWImMzg4BajKVxxMU737JGnNISy5"
# Parallel function calls - only first has signature
parts_parallel = [
HttpxPartType(
functionCall={"name": "get_current_temperature", "args": {"location": "Paris"}},
thoughtSignature=test_signature, # First FC has signature
),
HttpxPartType(
functionCall={"name": "get_current_temperature", "args": {"location": "London"}},
# Second FC has no signature (parallel call)
),
]
function, tools, _ = VertexGeminiConfig._transform_parts(
parts=parts_parallel,
cumulative_tool_call_idx=0,
is_function_call=False,
)
# Verify only first tool call has thought signature
assert tools is not None
assert len(tools) == 2
assert "provider_specific_fields" in tools[0]
assert tools[0]["provider_specific_fields"]["thought_signature"] == test_signature
# Second tool call should not have thought signature
assert "provider_specific_fields" not in tools[1] or "thought_signature" not in tools[1].get("provider_specific_fields", {})
def test_thought_signature_preservation_in_conversion():
"""Test that thought signatures are preserved when converting assistant messages back to Gemini format"""
from litellm.litellm_core_utils.prompt_templates.factory import (
convert_to_gemini_tool_call_invoke,
)
test_signature = "Co4CAdHtim/rWgXbz2Ghp4tShzLeMASrPw6JJyYIC3cbVyZnKzU3uv8/wVzyS2sKRPL2m8QQHHXbNQhEEz500G7n/4ZMmksdTtfQcJMoT76S1DGwhnAiLwTgWCNXs3lEb4M19EVYoWFxhrH5Lr9YMIquoU9U4paydGwvZyIyigamIg4B6WnxrRsf0KZV12gJed0DZuKczvOFtHz3zUnmZRlOiTzd5gBVyQM+5jv1VI8m4WUKd6cN/5a5ZvaA0ggiO6kdVhlpIVs7GczSEVJD8KH4u02X7VSnb7CvykqDntZzV0y8rZFBEFGKrChmeHlWXP4D1IB3F9KQyhuLgWImMzg4BajKVxxMU737JGnNISy5"
# Assistant message with tool calls containing thought signatures
assistant_message = {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_current_temperature",
"arguments": '{"location": "Paris"}',
},
"index": 0,
"provider_specific_fields": {
"thought_signature": test_signature,
},
},
{
"id": "call_def456",
"type": "function",
"function": {
"name": "get_current_temperature",
"arguments": '{"location": "London"}',
},
"index": 1,
# No thought signature for parallel call
},
],
}
gemini_parts = convert_to_gemini_tool_call_invoke(assistant_message)
# Verify thought signature is preserved in first function call part
assert len(gemini_parts) == 2
assert "function_call" in gemini_parts[0]
assert "thoughtSignature" in gemini_parts[0]
assert gemini_parts[0]["thoughtSignature"] == test_signature
# Verify second function call part does not have thought signature
assert "function_call" in gemini_parts[1]
assert "thoughtSignature" not in gemini_parts[1]
def test_thought_signature_sequential_function_calls():
"""Test that each sequential function call preserves its own thought signature"""
from litellm.litellm_core_utils.prompt_templates.factory import (
convert_to_gemini_tool_call_invoke,
)
signature_1 = "Co4CAdHtim/rWgXbz2Ghp4tShzLeMASrPw6JJyYIC3cbVyZnKzU3uv8/wVzyS2sKRPL2m8QQHHXbNQhEEz500G7n/4ZMmksdTtfQcJMoT76S1DGwhnAiLwTgWCNXs3lEb4M19EVYoWFxhrH5Lr9YMIquoU9U4paydGwvZyIyigamIg4B6WnxrRsf0KZV12gJed0DZuKczvOFtHz3zUnmZRlOiTzd5gBVyQM+5jv1VI8m4WUKd6cN/5a5ZvaA0ggiO6kdVhlpIVs7GczSEVJD8KH4u02X7VSnb7CvykqDntZzV0y8rZFBEFGKrChmeHlWXP4D1IB3F9KQyhuLgWImMzg4BajKVxxMU737JGnNISy5"
signature_2 = "DifferentSignatureForSecondCall1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ"
# Sequential function calls - each has its own signature
# This simulates a multi-step conversation where each step has a signature
assistant_message_step1 = {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_step1",
"type": "function",
"function": {
"name": "check_flight",
"arguments": '{"flight": "AA100"}',
},
"index": 0,
"provider_specific_fields": {
"thought_signature": signature_1,
},
},
],
}
assistant_message_step2 = {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_step2",
"type": "function",
"function": {
"name": "book_taxi",
"arguments": '{"destination": "airport"}',
},
"index": 0,
"provider_specific_fields": {
"thought_signature": signature_2,
},
},
],
}
gemini_parts_step1 = convert_to_gemini_tool_call_invoke(assistant_message_step1)
gemini_parts_step2 = convert_to_gemini_tool_call_invoke(assistant_message_step2)
# Verify each step preserves its own signature
assert len(gemini_parts_step1) == 1
assert gemini_parts_step1[0]["thoughtSignature"] == signature_1
assert len(gemini_parts_step2) == 1
assert gemini_parts_step2[0]["thoughtSignature"] == signature_2
def test_thought_signature_with_function_call_mode():
"""Test thought signature extraction in function_call mode (is_function_call=True)"""
from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (
VertexGeminiConfig,
)
from litellm.types.llms.vertex_ai import HttpxPartType
test_signature = "Co4CAdHtim/rWgXbz2Ghp4tShzLeMASrPw6JJyYIC3cbVyZnKzU3uv8/wVzyS2sKRPL2m8QQHHXbNQhEEz500G7n/4ZMmksdTtfQcJMoT76S1DGwhnAiLwTgWCNXs3lEb4M19EVYoWFxhrH5Lr9YMIquoU9U4paydGwvZyIyigamIg4B6WnxrRsf0KZV12gJed0DZuKczvOFtHz3zUnmZRlOiTzd5gBVyQM+5jv1VI8m4WUKd6cN/5a5ZvaA0ggiO6kdVhlpIVs7GczSEVJD8KH4u02X7VSnb7CvykqDntZzV0y8rZFBEFGKrChmeHlWXP4D1IB3F9KQyhuLgWImMzg4BajKVxxMU737JGnNISy5"
parts_with_signature = [
HttpxPartType(
functionCall={
"name": "get_current_weather",
"args": {"location": "Tokyo"},
},
thoughtSignature=test_signature,
)
]
function, tools, _ = VertexGeminiConfig._transform_parts(
parts=parts_with_signature,
cumulative_tool_call_idx=0,
is_function_call=True,
)
# Verify thought signature is stored in function's provider_specific_fields
assert function is not None
# Function should be dict-like (TypedDict or dict)
assert hasattr(function, "__getitem__") or isinstance(function, dict)
assert "provider_specific_fields" in function
assert function["provider_specific_fields"]["thought_signature"] == test_signature
assert tools is None
tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py
+333 -2
View File
@@ -1048,7 +1048,7 @@ def test_vertex_ai_code_line_length():
# Find the line that generates the ID
id_line = None
for line in source_lines:
if 'id=f"call_{uuid.uuid4().hex' in line:
if '"id": f"call_' in line and 'uuid.uuid4().hex[:28]' in line:
id_line = line.strip() # Remove indentation for length check
break
@@ -1425,4 +1425,335 @@ def test_vertex_ai_annotation_empty_grounding_metadata():
annotations = VertexGeminiConfig._convert_grounding_metadata_to_annotations(
[metadata_empty_supports], "test content"
)
assert len(annotations) == 0
assert len(annotations) == 0
# ==================== Gemini 3 Pro Preview Tests ====================
def test_is_gemini_3_or_newer():
"""Test the _is_gemini_3_or_newer method for version detection"""
from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (
VertexGeminiConfig,
)
# Gemini 3 models
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-3-pro-preview") == True
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-3-flash") == True
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-3-pro") == True
assert VertexGeminiConfig._is_gemini_3_or_newer("vertex_ai/gemini-3-pro-preview") == True
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini/gemini-3-pro-preview") == True
# Gemini 2.5 and older models
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-2.5-pro") == False
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-2.5-flash") == False
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-2.0-flash") == False
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-1.5-pro") == False
assert VertexGeminiConfig._is_gemini_3_or_newer("gemini-pro") == False
# Edge cases
assert VertexGeminiConfig._is_gemini_3_or_newer("") == False
def test_reasoning_effort_maps_to_thinking_level_gemini_3():
"""Test that reasoning_effort maps to thinking_level for Gemini 3+ models"""
from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (
VertexGeminiConfig,
)
v = VertexGeminiConfig()
model = "gemini-3-pro-preview"
optional_params = {}
# Test minimal -> low
non_default_params = {"reasoning_effort": "minimal"}
result = v.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model=model,
drop_params=False,
)
assert result["thinkingConfig"]["thinkingLevel"] == "low"
# Test low -> low
optional_params = {}
non_default_params = {"reasoning_effort": "low"}
result = v.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model=model,
drop_params=False,
)
assert result["thinkingConfig"]["thinkingLevel"] == "low"
# Test medium -> high (medium not available yet)
optional_params = {}
non_default_params = {"reasoning_effort": "medium"}
result = v.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model=model,
drop_params=False,
)
assert result["thinkingConfig"]["thinkingLevel"] == "high"
# Test high -> high
optional_params = {}
non_default_params = {"reasoning_effort": "high"}
result = v.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model=model,
drop_params=False,
)
assert result["thinkingConfig"]["thinkingLevel"] == "high"
# Test disable -> low (cannot fully disable in Gemini 3)
optional_params = {}
non_default_params = {"reasoning_effort": "disable"}
result = v.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model=model,
drop_params=False,
)
assert result["thinkingConfig"]["thinkingLevel"] == "low"
# Test none -> low (cannot fully disable in Gemini 3)
optional_params = {}
non_default_params = {"reasoning_effort": "none"}
result = v.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model=model,
drop_params=False,
)
assert result["thinkingConfig"]["thinkingLevel"] == "low"
def test_temperature_default_for_gemini_3():
"""Test that temperature defaults to 1.0 for Gemini 3+ models when not specified"""
from litellm.llms.vertex_ai.gemini.vertex_and_google_ai_studio_gemini import (
VertexGeminiConfig,
)
v = VertexGeminiConfig()
model = "gemini-3-pro-preview"
optional_params = {}
# No temperature specified
non_default_params = {}
result = v.map_openai_params(
non_default_params=non_default_params,
optional_params=optional_params,
model=model,
drop_params=False,
)
# Should default to 1.0
assert "temperature" in result
assert result["temperature"] == 1.0
def test_media_resolution_from_detail_parameter():
"""Test that OpenAI's detail parameter is correctly mapped to media_resolution"""
from litellm.llms.vertex_ai.gemini.transformation import (
_gemini_convert_messages_with_history,
_map_openai_detail_to_media_resolution,
)
# Test detail -> media_resolution mapping
assert _map_openai_detail_to_media_resolution("low") == "low"
assert _map_openai_detail_to_media_resolution("high") == "high"
assert _map_openai_detail_to_media_resolution("auto") is None
assert _map_openai_detail_to_media_resolution(None) is None
# Test with actual message transformation using base64 image
# Using a minimal valid base64-encoded 1x1 PNG
base64_image = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
messages = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": base64_image,
"detail": "high"
}
}
]
}
]
contents = _gemini_convert_messages_with_history(messages=messages)
# Verify media_resolution is set in the inline_data
# Note: Gemini adds a blank text part when there's no text, so we expect 2 parts
assert len(contents) == 1
assert len(contents[0]["parts"]) >= 1
# Find the part with inline_data
image_part = None
for part in contents[0]["parts"]:
if "inline_data" in part:
image_part = part
break
assert image_part is not None
assert "inline_data" in image_part
# The TypedDict uses snake_case internally, but mediaResolution is camelCase in the dict
assert "mediaResolution" in image_part["inline_data"]
assert image_part["inline_data"]["mediaResolution"] == "high"
def test_media_resolution_low_detail():
"""Test that detail='low' maps to media_resolution='low'"""
from litellm.llms.vertex_ai.gemini.transformation import (
_gemini_convert_messages_with_history,
)
# Using a minimal valid base64-encoded 1x1 PNG
base64_image = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
messages = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": base64_image,
"detail": "low"
}
}
]
}
]
contents = _gemini_convert_messages_with_history(messages=messages)
# Find the part with inline_data
image_part = None
for part in contents[0]["parts"]:
if "inline_data" in part:
image_part = part
break
assert image_part is not None
assert "inline_data" in image_part
assert image_part["inline_data"]["mediaResolution"] == "low"
def test_media_resolution_auto_detail():
"""Test that detail='auto' or None doesn't set media_resolution"""
from litellm.llms.vertex_ai.gemini.transformation import (
_gemini_convert_messages_with_history,
)
# Using a minimal valid base64-encoded 1x1 PNG
base64_image = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
# Test with auto
messages_auto = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": base64_image,
"detail": "auto"
}
}
]
}
]
contents = _gemini_convert_messages_with_history(messages=messages_auto)
# Find the part with inline_data
image_part = None
for part in contents[0]["parts"]:
if "inline_data" in part:
image_part = part
break
assert image_part is not None
assert "inline_data" in image_part
# mediaResolution should not be set for auto
assert "mediaResolution" not in image_part["inline_data"] or image_part["inline_data"].get("mediaResolution") is None
# Test with None
messages_none = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": base64_image
}
}
]
}
]
contents = _gemini_convert_messages_with_history(messages=messages_none)
# Find the part with inline_data
image_part = None
for part in contents[0]["parts"]:
if "inline_data" in part:
image_part = part
break
assert image_part is not None
assert "inline_data" in image_part
# mediaResolution should not be set
assert "mediaResolution" not in image_part["inline_data"] or image_part["inline_data"].get("mediaResolution") is None
def test_media_resolution_per_part():
"""Test that different images can have different media_resolution values"""
from litellm.llms.vertex_ai.gemini.transformation import (
_gemini_convert_messages_with_history,
)
# Using minimal valid base64-encoded 1x1 PNGs
base64_image1 = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
base64_image2 = "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
messages = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {
"url": base64_image1,
"detail": "low"
}
},
{
"type": "text",
"text": "Compare these images"
},
{
"type": "image_url",
"image_url": {
"url": base64_image2,
"detail": "high"
}
}
]
}
]
contents = _gemini_convert_messages_with_history(messages=messages)
# Should have one content with multiple parts
assert len(contents) == 1
assert len(contents[0]["parts"]) == 3 # image1, text, image2
# First image should have low resolution (first part is the image)
image1_part = contents[0]["parts"][0]
assert "inline_data" in image1_part
assert image1_part["inline_data"]["mediaResolution"] == "low"
# Second image should have high resolution (third part is the second image)
image2_part = contents[0]["parts"][2]
assert "inline_data" in image2_part
assert image2_part["inline_data"]["mediaResolution"] == "high"
tests/test_litellm/llms/vertex_ai/test_vertex.py
+1 -1
View File
@@ -1241,7 +1241,7 @@ def test_process_gemini_image():
base64_image = "data:image/jpeg;base64,/9j/4AAQSkZJRg..."
base64_result = _process_gemini_image(base64_image)
print("base64_result", base64_result)
assert base64_result["inline_data"]["mime_type"] == "image/jpeg"
assert base64_result["inline_data"]["mimeType"] == "image/jpeg"
assert base64_result["inline_data"]["data"] == "/9j/4AAQSkZJRg..."