tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-18 04:28:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #25876 from BerriAI/litellm_hotfix_opus_4.7

Browse Source 
Litellm hotfix opus 4.7
This commit is contained in:
ishaan-berri
2026-04-16 12:19:00 -07:00
committed by GitHub
parent 72a461ba4a fe6fef97d1
commit c0fc4c4234
14 changed files with 1265 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/blog/claude_opus_4_7/index.md
+366
View File
@@ -0,0 +1,366 @@
---
slug: claude_opus_4_7
title: "Day 0 Support: Claude Opus 4.7"
date: 2026-04-16T10:00:00
authors:
- sameer
- ishaan-alt
- krrish
description: "Day 0 support for Claude Opus 4.7 on LiteLLM AI Gateway - use across Anthropic, Azure, Vertex AI, and Bedrock."
tags: [anthropic, claude, opus 4.7]
hide_table_of_contents: false
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
LiteLLM now supports [Claude Opus 4.7](https://www.anthropic.com/news/claude-opus-4-7) on Day 0. Use it across Anthropic, Azure, Vertex AI, and Bedrock through the LiteLLM AI Gateway.
{/* truncate */}
## Docker Image
```bash
docker pull ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7
```
## Usage - Anthropic
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: anthropic/claude-opus-4-7
api_key: os.environ/ANTHROPIC_API_KEY
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Azure
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: azure_ai/claude-opus-4-7
api_key: os.environ/AZURE_AI_API_KEY
api_base: os.environ/AZURE_AI_API_BASE # https://<resource>.services.ai.azure.com
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e AZURE_AI_API_KEY=$AZURE_AI_API_KEY \
-e AZURE_AI_API_BASE=$AZURE_AI_API_BASE \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Vertex AI
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: vertex_ai/claude-opus-4-7
vertex_project: os.environ/VERTEX_PROJECT
vertex_location: us-east5
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e VERTEX_PROJECT=$VERTEX_PROJECT \
-e GOOGLE_APPLICATION_CREDENTIALS=/app/credentials.json \
-v $(pwd)/config.yaml:/app/config.yaml \
-v $(pwd)/credentials.json:/app/credentials.json \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Usage - Bedrock
<Tabs>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Setup config.yaml**
```yaml
model_list:
- model_name: claude-opus-4-7
litellm_params:
model: bedrock/anthropic.claude-opus-4-7
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
aws_region_name: us-east-1
```
**2. Start the proxy**
```bash
docker run -d \
-p 4000:4000 \
-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-v $(pwd)/config.yaml:/app/config.yaml \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.83.3-stable.opus-4.7 \
--config /app/config.yaml
```
**3. Test it!**
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
```
</TabItem>
</Tabs>
## Advanced Features
### Adaptive Thinking
:::note
When using `reasoning_effort` with Claude Opus 4.7, all values (`low`, `medium`, `high`, `xhigh`) are mapped to `thinking: {type: "adaptive"}`. To use explicit thinking budgets with `type: "enabled"`, pass the native `thinking` parameter directly.
:::
<Tabs>
<TabItem value="completions" label="/chat/completions">
LiteLLM supports adaptive thinking through the `reasoning_effort` parameter:
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "Solve this complex problem: What is the optimal strategy for..."
}
],
"reasoning_effort": "high"
}'
```
</TabItem>
<TabItem value="messages" label="/v1/messages">
Use the `thinking` parameter with `type: "adaptive"` to enable adaptive thinking mode:
```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-7",
"max_tokens": 16000,
"thinking": {
"type": "adaptive"
},
"messages": [
{
"role": "user",
"content": "Explain why the sum of two even numbers is always even."
}
]
}'
```
</TabItem>
</Tabs>
### Effort Levels
Claude Opus 4.7 supports four effort levels: `low`, `medium`, `high` (default), and `xhigh`. These give you finer-grained control over how much reasoning the model applies to a task. Pass the effort level via the `output_config` parameter.
`xhigh` is a new effort level introduced with Opus 4.7 that sits above `high`. The `max` effort level is Claude Opus 4.6 only and is not available on 4.7.
<Tabs>
<TabItem value="completions" label="/chat/completions">
```bash
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_KEY' \
--data '{
"model": "claude-opus-4-7",
"messages": [
{
"role": "user",
"content": "Explain quantum computing"
}
],
"output_config": {
"effort": "xhigh"
}
}'
```
**Using OpenAI SDK:**
```python
import openai
client = openai.OpenAI(
api_key="your-litellm-key",
base_url="http://0.0.0.0:4000"
)
response = client.chat.completions.create(
model="claude-opus-4-7",
messages=[{"role": "user", "content": "Explain quantum computing"}],
extra_body={"output_config": {"effort": "xhigh"}}
)
```
**Using LiteLLM SDK:**
```python
from litellm import completion
response = completion(
model="anthropic/claude-opus-4-7",
messages=[{"role": "user", "content": "Explain quantum computing"}],
output_config={"effort": "xhigh"},
)
```
You can combine `reasoning_effort` with `output_config` for even more fine-grained control over the model's behavior.
</TabItem>
<TabItem value="messages" label="/v1/messages">
```bash
curl --location 'http://0.0.0.0:4000/v1/messages' \
--header 'x-api-key: sk-12345' \
--header 'content-type: application/json' \
--data '{
"model": "claude-opus-4-7",
"max_tokens": 4096,
"messages": [
{
"role": "user",
"content": "Explain quantum computing"
}
],
"output_config": {
"effort": "xhigh"
}
}'
```
</TabItem>
</Tabs>
**Effort level guide:**
| Effort | When to use |
|--------|-------------|
| `low` | Short, fast responses — simple lookups, formatting, classification |
| `medium` | Balanced tradeoff for everyday Q&A and light reasoning |
| `high` (default) | Complex reasoning, code generation, analysis |
| `xhigh` | Hardest problems — multi-step math, deep research, agentic planning |
litellm/anthropic_beta_headers_config.json
+4 -4
View File
@@ -71,12 +71,12 @@
"computer-use-2025-01-24": "computer-use-2025-01-24",
"computer-use-2025-11-24": "computer-use-2025-11-24",
"context-1m-2025-08-07": "context-1m-2025-08-07",
"context-management-2025-06-27": "context-management-2025-06-27",
"context-management-2025-06-27": null,
"effort-2025-11-24": null,
"fast-mode-2026-02-01": null,
"files-api-2025-04-14": null,
"fine-grained-tool-streaming-2025-05-14": null,
"interleaved-thinking-2025-05-14": "interleaved-thinking-2025-05-14",
"interleaved-thinking-2025-05-14": null,
"mcp-client-2025-11-20": null,
"mcp-client-2025-04-04": null,
"mcp-servers-2025-12-04": null,
@@ -102,12 +102,12 @@
"computer-use-2025-01-24": "computer-use-2025-01-24",
"computer-use-2025-11-24": "computer-use-2025-11-24",
"context-1m-2025-08-07": "context-1m-2025-08-07",
"context-management-2025-06-27": "context-management-2025-06-27",
"context-management-2025-06-27": null,
"effort-2025-11-24": null,
"fast-mode-2026-02-01": null,
"files-api-2025-04-14": null,
"fine-grained-tool-streaming-2025-05-14": null,
"interleaved-thinking-2025-05-14": "interleaved-thinking-2025-05-14",
"interleaved-thinking-2025-05-14": null,
"mcp-client-2025-11-20": null,
"mcp-client-2025-04-04": null,
"mcp-servers-2025-12-04": null,
litellm/constants.py
+1
View File
@@ -1113,6 +1113,7 @@ BEDROCK_CONVERSE_MODELS = [
"openai.gpt-oss-120b-1:0",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-opus-4-7",
"anthropic.claude-opus-4-6-v1:0",
"anthropic.claude-opus-4-6-v1",
"anthropic.claude-sonnet-4-6",
litellm/llms/anthropic/chat/transformation.py
+50 -6
View File
@@ -67,6 +67,7 @@ from litellm.types.utils import (
from litellm.utils import (
ModelResponse,
Usage,
_supports_factory,
add_dummy_tool,
any_assistant_message_has_thinking_blocks,
get_max_tokens,
@@ -189,6 +190,30 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
v in model_lower for v in ("opus-4-6", "opus_4_6", "opus-4.6", "opus_4.6")
)
@staticmethod
def _is_opus_4_7_model(model: str) -> bool:
"""Check if the model is specifically Claude Opus 4.7."""
model_lower = model.lower()
return any(
v in model_lower for v in ("opus-4-7", "opus_4_7", "opus-4.7", "opus_4.7")
)
@staticmethod
def _supports_effort_level(model: str, level: str) -> bool:
"""Check ``supports_{level}_reasoning_effort`` in the model map.
Mirrors the pattern used in ``openai/chat/gpt_5_transformation.py`` so
that adding support for a new effort level is a pure model-map change.
"""
try:
return _supports_factory(
model=model,
custom_llm_provider="anthropic",
key=f"supports_{level}_reasoning_effort",
)
except Exception:
return False
def get_supported_openai_params(self, model: str):
params = [
"stream",
@@ -212,6 +237,7 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
if (
"claude-3-7-sonnet" in model
or AnthropicConfig._is_claude_4_6_model(model)
or AnthropicConfig._is_claude_4_7_model(model)
or supports_reasoning(
model=model,
custom_llm_provider=self.custom_llm_provider,
@@ -771,7 +797,9 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
) -> Optional[AnthropicThinkingParam]:
if reasoning_effort is None or reasoning_effort == "none":
return None
if AnthropicConfig._is_claude_4_6_model(model):
if AnthropicConfig._is_claude_4_6_model(
model
) or AnthropicConfig._is_claude_4_7_model(model):
return AnthropicThinkingParam(
type="adaptive",
)
@@ -1020,6 +1048,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
"opus-4-5",
"opus-4.6",
"opus-4-6",
"opus-4.7",
"opus-4-7",
"sonnet-4.6",
"sonnet-4-6",
"sonnet_4.6",
@@ -1061,14 +1091,17 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
optional_params["thinking"] = AnthropicConfig._map_reasoning_effort(
reasoning_effort=value, model=model
)
# For Claude 4.6 models, effort is controlled via output_config,
# For Claude 4.6+ models, effort is controlled via output_config,
# not thinking budget_tokens. Map reasoning_effort to output_config.
if AnthropicConfig._is_claude_4_6_model(model):
if AnthropicConfig._is_claude_4_6_model(
model
) or AnthropicConfig._is_claude_4_7_model(model):
effort_map = {
"low": "low",
"minimal": "low",
"medium": "medium",
"high": "high",
"xhigh": "xhigh",
"max": "max",
}
mapped_effort = effort_map.get(value, value)
@@ -1495,13 +1528,24 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
if not output_config or not isinstance(output_config, dict):
return
effort = output_config.get("effort")
if effort and effort not in ["high", "medium", "low", "max"]:
valid_efforts = ["high", "medium", "low", "xhigh", "max"]
if effort and effort not in valid_efforts:
raise ValueError(
f"Invalid effort value: {effort}. Must be one of: 'high', 'medium', 'low', 'max'"
f"Invalid effort value: {effort}. Must be one of: "
f"'high', 'medium', 'low', 'xhigh', 'max'"
)
# ``max`` is Claude Opus 4.6 only (not Sonnet 4.6, not Opus 4.5/4.7).
# Keep this hardcoded so the error message is specific and stable.
if effort == "max" and not self._is_opus_4_6_model(model):
raise ValueError(
f"effort='max' is only supported by Claude Opus 4.6. Got model: {model}"
f"effort='max' is only supported by Claude Opus 4.6. "
f"Got model: {model}"
)
# ``xhigh`` is data-driven via ``supports_xhigh_reasoning_effort`` so
# enabling it for a new model is a pure model-map change.
if effort == "xhigh" and not self._supports_effort_level(model, "xhigh"):
raise ValueError(
f"effort='xhigh' is not supported by this model. Got model: {model}"
)
data["output_config"] = output_config
litellm/llms/anthropic/common_utils.py
+24 -3
View File
@@ -256,6 +256,27 @@ class AnthropicModelInfo(BaseLLMModelInfo):
)
)
@staticmethod
def _is_claude_4_7_model(model: str) -> bool:
"""Check if the model is a Claude 4.7 model (Opus 4.7)."""
model_lower = model.lower()
return any(
v in model_lower
for v in (
"opus-4-7",
"opus_4_7",
"opus-4.7",
"opus_4.7",
)
)
@staticmethod
def _is_adaptive_thinking_model(model: str) -> bool:
"""Claude 4.6+ models use adaptive thinking with output_config effort."""
return AnthropicModelInfo._is_claude_4_6_model(
model
) or AnthropicModelInfo._is_claude_4_7_model(model)
def is_effort_used(
self, optional_params: Optional[dict], model: Optional[str] = None
) -> bool:
@@ -263,14 +284,14 @@ class AnthropicModelInfo(BaseLLMModelInfo):
Check if effort parameter is being used and requires a beta header.
Returns True if effort-related parameters are present and
the model requires the effort beta header. Claude 4.6 models
the model requires the effort beta header. Claude 4.6+ models
use output_config as a stable API feature — no beta header needed.
"""
if not optional_params:
return False
# Claude 4.6 models use output_config as a stable API feature — no beta header needed
if model and self._is_claude_4_6_model(model):
# Claude 4.6+ models use output_config as a stable API feature — no beta header needed
if model and self._is_adaptive_thinking_model(model):
return False
# Check if reasoning_effort is provided for Claude Opus 4.5
litellm/llms/anthropic/experimental_pass_through/messages/transformation.py
+35
View File
@@ -166,6 +166,36 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
return headers, api_base
@staticmethod
def _translate_legacy_thinking_for_adaptive_model(
model: str, optional_params: Dict
) -> None:
"""Translate legacy ``thinking.type=enabled`` to adaptive for 4.6/4.7.
Caller-provided ``output_config.effort`` is never overridden.
"""
if not AnthropicModelInfo._is_adaptive_thinking_model(model):
return
thinking = optional_params.get("thinking")
if not isinstance(thinking, dict) or thinking.get("type") != "enabled":
return
budget = int(thinking.get("budget_tokens") or 0)
if budget >= 24000:
effort = "xhigh"
elif budget >= 10000:
effort = "high"
elif budget >= 5000:
effort = "medium"
else:
effort = "low"
optional_params["thinking"] = {"type": "adaptive"}
existing_output_config = optional_params.get("output_config")
if not isinstance(existing_output_config, dict):
existing_output_config = {}
existing_output_config.setdefault("effort", effort)
optional_params["output_config"] = existing_output_config
def transform_anthropic_messages_request(
self,
model: str,
@@ -187,6 +217,11 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
status_code=400,
)
self._translate_legacy_thinking_for_adaptive_model(
model=model,
optional_params=anthropic_messages_optional_request_params,
)
# Filter out x-anthropic-billing-header from system messages
system_param = anthropic_messages_optional_request_params.get("system")
if system_param is not None:
litellm/llms/bedrock/chat/converse_transformation.py
+6 -2
View File
@@ -1298,12 +1298,16 @@ class AmazonConverseConfig(BaseConfig):
# Add computer use tools and anthropic_beta if needed (only when computer use tools are present)
if computer_use_tools:
# Determine the correct computer-use beta header based on model
# "computer-use-2025-11-24" for Claude Opus 4.6, Claude Opus 4.5
# "computer-use-2025-11-24" for Claude Opus 4.7, Opus 4.6, and Opus 4.5
# "computer-use-2025-01-24" for Claude Sonnet 4.5, Haiku 4.5, Opus 4.1, Sonnet 4, Opus 4, and Sonnet 3.7
# "computer-use-2024-10-22" for older models
model_lower = model.lower()
if (
"opus-4.6" in model_lower
"opus-4.7" in model_lower
or "opus_4.7" in model_lower
or "opus-4-7" in model_lower
or "opus_4_7" in model_lower
or "opus-4.6" in model_lower
or "opus_4.6" in model_lower
or "opus-4-6" in model_lower
or "opus_4_6" in model_lower
litellm/llms/bedrock/common_utils.py
+4
View File
@@ -589,6 +589,10 @@ def is_claude_4_5_on_bedrock(model: str) -> bool:
"opus_4.6",
"opus-4-6",
"opus_4_6",
"opus-4.7",
"opus_4.7",
"opus-4-7",
"opus_4_7",
]
return any(pattern in model_lower for pattern in claude_4_5_patterns)
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py
+84
View File
@@ -13,6 +13,8 @@ from typing import (
import httpx
from litellm.anthropic_beta_headers_manager import filter_and_transform_beta_headers
from litellm.constants import BEDROCK_MIN_THINKING_BUDGET_TOKENS
from litellm.litellm_core_utils.litellm_logging import verbose_logger
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
from litellm.llms.anthropic.experimental_pass_through.messages.transformation import (
AnthropicMessagesConfig,
@@ -205,10 +207,78 @@ class AmazonAnthropicClaudeMessagesConfig(
"opus_4.6",
"opus-4-6",
"opus_4_6",
"opus-4.7",
"opus_4.7",
"opus-4-7",
"opus_4_7",
]
return any(pattern in model_lower for pattern in supported_patterns)
def _ensure_thinking_for_clear_thinking_context_management(
self,
anthropic_messages_request: Dict,
model: str,
) -> bool:
"""
Bedrock rejects ``clear_thinking_20251015`` context-management edits unless
extended thinking is ``enabled`` or ``adaptive``. Claude Code often sends
context management without a top-level ``thinking`` field.
When we detect that edit type on a model that supports extended thinking on
Bedrock, inject a minimal ``thinking`` config so the request succeeds.
Returns:
True if ``thinking`` was added or upgraded for this fix (caller may
need to add the interleaved-thinking beta header).
"""
cm = anthropic_messages_request.get("context_management")
if not isinstance(cm, dict):
return False
edits = cm.get("edits")
if not isinstance(edits, list):
return False
needs_thinking = any(
isinstance(e, dict) and e.get("type") == "clear_thinking_20251015"
for e in edits
)
if not needs_thinking:
return False
if not self._supports_extended_thinking_on_bedrock(model):
return False
thinking = anthropic_messages_request.get("thinking")
if isinstance(thinking, dict):
t = thinking.get("type")
if t in ("enabled", "adaptive"):
return False
# ``disabled`` or unknown — replace with enabled so clear_thinking is valid
verbose_logger.debug(
"Bedrock clear_thinking_20251015: replacing thinking=%s with minimal enabled thinking",
thinking,
)
max_tokens = anthropic_messages_request.get("max_tokens")
budget = BEDROCK_MIN_THINKING_BUDGET_TOKENS
if isinstance(max_tokens, int) and max_tokens <= budget:
verbose_logger.warning(
"Bedrock clear_thinking_20251015: max_tokens=%s is not greater than "
"minimum thinking budget (%s); cannot inject thinking safely",
max_tokens,
budget,
)
return False
anthropic_messages_request["thinking"] = {
"type": "enabled",
"budget_tokens": budget,
}
verbose_logger.debug(
"Bedrock clear_thinking_20251015: injected thinking with budget_tokens=%s",
budget,
)
return True
def _is_claude_opus_4_5(self, model: str) -> bool:
"""
Check if the model is Claude Opus 4.5.
@@ -281,6 +351,10 @@ class AmazonAnthropicClaudeMessagesConfig(
"sonnet_4.6",
"sonnet-4-6",
"sonnet_4_6",
# NOTE: Opus 4.7 on Bedrock does not support server-side tool search
# as of launch (2026-04-16). Bedrock rejects the tool type with:
# "tool type 'tool_search_tool_..._20251119' is not supported for this model".
# Re-add the opus-4.7 patterns here once AWS announces support.
]
return any(pattern in model_lower for pattern in supported_patterns)
@@ -406,6 +480,13 @@ class AmazonAnthropicClaudeMessagesConfig(
if "model" in anthropic_messages_request:
anthropic_messages_request.pop("model", None)
injected_thinking_for_clear_thinking = (
self._ensure_thinking_for_clear_thinking_context_management(
anthropic_messages_request=anthropic_messages_request,
model=model,
)
)
# 4. Remove `ttl` field from cache_control in messages (Bedrock doesn't support it for older models)
self._remove_ttl_from_cache_control(
anthropic_messages_request=anthropic_messages_request, model=model
@@ -455,6 +536,9 @@ class AmazonAnthropicClaudeMessagesConfig(
)
beta_set.update(auto_betas)
if injected_thinking_for_clear_thinking:
beta_set.add("interleaved-thinking-2025-05-14")
self._get_tool_search_beta_header_for_bedrock(
model=model,
tool_search_used=tool_search_used,
litellm/model_prices_and_context_window_backup.json
+317 -29
View File
@@ -1005,7 +1005,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"global.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.25e-06,
@@ -1032,7 +1033,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"us.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@@ -1059,7 +1061,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"eu.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@@ -1086,7 +1089,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"au.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@@ -1113,6 +1117,147 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"global.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"us.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token": 5.5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.75e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"eu.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token": 5.5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.75e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"au.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token": 5.5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.75e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"anthropic.claude-sonnet-4-6": {
@@ -1765,6 +1910,35 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_max_reasoning_effort": true
},
"azure_ai/claude-opus-4-7": {
"input_cost_per_token": 5e-06,
"output_cost_per_token": 2.5e-05,
"litellm_provider": "azure_ai",
"max_input_tokens": 200000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"cache_creation_input_token_cost": 6.25e-06,
"cache_creation_input_token_cost_above_1hr": 1e-05,
"cache_read_input_token_cost": 5e-07,
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 159
},
"azure_ai/claude-opus-4-1": {
@@ -8928,7 +9102,8 @@
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
}
},
"supports_max_reasoning_effort": true
},
"claude-opus-4-6-20260205": {
"cache_creation_input_token_cost": 6.25e-06,
@@ -8956,6 +9131,71 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
},
"supports_max_reasoning_effort": true
},
"claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_creation_input_token_cost_above_1hr": 1e-05,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "anthropic",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
}
},
"claude-opus-4-7-20260416": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_creation_input_token_cost_above_1hr": 1e-05,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "anthropic",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
@@ -18991,13 +19231,11 @@
"cache_read_input_token_cost_above_272k_tokens": 5e-07,
"cache_read_input_token_cost_flex": 1.3e-07,
"cache_read_input_token_cost_priority": 5e-07,
"cache_read_input_token_cost_above_272k_tokens_priority": 1e-06,
"input_cost_per_token": 2.5e-06,
"input_cost_per_token_above_272k_tokens": 5e-06,
"input_cost_per_token_flex": 1.25e-06,
"input_cost_per_token_batches": 1.25e-06,
"input_cost_per_token_priority": 5e-06,
"input_cost_per_token_above_272k_tokens_priority": 1e-05,
"litellm_provider": "openai",
"max_input_tokens": 1050000,
"max_output_tokens": 128000,
@@ -19007,8 +19245,7 @@
"output_cost_per_token_above_272k_tokens": 2.25e-05,
"output_cost_per_token_flex": 7.5e-06,
"output_cost_per_token_batches": 7.5e-06,
"output_cost_per_token_priority": 2.25e-05,
"output_cost_per_token_above_272k_tokens_priority": 3.375e-05,
"output_cost_per_token_priority": 3e-05,
"supported_endpoints": [
"/v1/chat/completions",
"/v1/batch",
@@ -19041,13 +19278,11 @@
"cache_read_input_token_cost_above_272k_tokens": 5e-07,
"cache_read_input_token_cost_flex": 1.3e-07,
"cache_read_input_token_cost_priority": 5e-07,
"cache_read_input_token_cost_above_272k_tokens_priority": 1e-06,
"input_cost_per_token": 2.5e-06,
"input_cost_per_token_above_272k_tokens": 5e-06,
"input_cost_per_token_flex": 1.25e-06,
"input_cost_per_token_batches": 1.25e-06,
"input_cost_per_token_priority": 5e-06,
"input_cost_per_token_above_272k_tokens_priority": 1e-05,
"litellm_provider": "openai",
"max_input_tokens": 1050000,
"max_output_tokens": 128000,
@@ -19057,8 +19292,7 @@
"output_cost_per_token_above_272k_tokens": 2.25e-05,
"output_cost_per_token_flex": 7.5e-06,
"output_cost_per_token_batches": 7.5e-06,
"output_cost_per_token_priority": 2.25e-05,
"output_cost_per_token_above_272k_tokens_priority": 3.375e-05,
"output_cost_per_token_priority": 3e-05,
"supported_endpoints": [
"/v1/chat/completions",
"/v1/batch",
@@ -19086,14 +19320,10 @@
"gpt-5.4-pro": {
"cache_read_input_token_cost": 3e-06,
"cache_read_input_token_cost_above_272k_tokens": 6e-06,
"cache_read_input_token_cost_priority": 6e-06,
"cache_read_input_token_cost_above_272k_tokens_priority": 1.2e-05,
"input_cost_per_token": 3e-05,
"input_cost_per_token_above_272k_tokens": 6e-05,
"input_cost_per_token_flex": 1.5e-05,
"input_cost_per_token_batches": 1.5e-05,
"input_cost_per_token_priority": 6e-05,
"input_cost_per_token_above_272k_tokens_priority": 0.00012,
"litellm_provider": "openai",
"max_input_tokens": 1050000,
"max_output_tokens": 128000,
@@ -19103,8 +19333,6 @@
"output_cost_per_token_above_272k_tokens": 0.00027,
"output_cost_per_token_flex": 9e-05,
"output_cost_per_token_batches": 9e-05,
"output_cost_per_token_priority": 0.00027,
"output_cost_per_token_above_272k_tokens_priority": 0.000405,
"supported_endpoints": [
"/v1/responses",
"/v1/batch"
@@ -19135,14 +19363,10 @@
"gpt-5.4-pro-2026-03-05": {
"cache_read_input_token_cost": 3e-06,
"cache_read_input_token_cost_above_272k_tokens": 6e-06,
"cache_read_input_token_cost_priority": 6e-06,
"cache_read_input_token_cost_above_272k_tokens_priority": 1.2e-05,
"input_cost_per_token": 3e-05,
"input_cost_per_token_above_272k_tokens": 6e-05,
"input_cost_per_token_flex": 1.5e-05,
"input_cost_per_token_batches": 1.5e-05,
"input_cost_per_token_priority": 6e-05,
"input_cost_per_token_above_272k_tokens_priority": 0.00012,
"litellm_provider": "openai",
"max_input_tokens": 1050000,
"max_output_tokens": 128000,
@@ -19152,8 +19376,6 @@
"output_cost_per_token_above_272k_tokens": 0.00027,
"output_cost_per_token_flex": 9e-05,
"output_cost_per_token_batches": 9e-05,
"output_cost_per_token_priority": 0.00027,
"output_cost_per_token_above_272k_tokens_priority": 0.000405,
"supported_endpoints": [
"/v1/responses",
"/v1/batch"
@@ -19183,11 +19405,13 @@
},
"gpt-5.4-mini": {
"cache_read_input_token_cost": 7.5e-08,
"cache_read_input_token_cost_flex": 1e-08,
"cache_read_input_token_cost_batches": 3.8e-08,
"cache_read_input_token_cost_flex": 3.75e-08,
"cache_read_input_token_cost_batches": 3.75e-08,
"cache_read_input_token_cost_priority": 1.5e-07,
"input_cost_per_token": 7.5e-07,
"input_cost_per_token_flex": 3.75e-07,
"input_cost_per_token_batches": 3.75e-07,
"input_cost_per_token_priority": 1.5e-06,
"litellm_provider": "openai",
"max_input_tokens": 272000,
"max_output_tokens": 128000,
@@ -19196,6 +19420,7 @@
"output_cost_per_token": 4.5e-06,
"output_cost_per_token_flex": 2.25e-06,
"output_cost_per_token_batches": 2.25e-06,
"output_cost_per_token_priority": 9e-06,
"supported_endpoints": [
"/v1/chat/completions",
"/v1/batch",
@@ -26698,6 +26923,13 @@
"supports_reasoning": false,
"supports_function_calling": true
},
"perplexity/anthropic/claude-opus-4-7": {
"litellm_provider": "perplexity",
"mode": "responses",
"supports_web_search": true,
"supports_reasoning": false,
"supports_function_calling": true
},
"perplexity/anthropic/claude-opus-4-5": {
"litellm_provider": "perplexity",
"mode": "responses",
@@ -31060,7 +31292,8 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346
"tool_use_system_prompt_tokens": 346,
"supports_max_reasoning_effort": true
},
"vertex_ai/claude-opus-4-6@default": {
"cache_creation_input_token_cost": 6.25e-06,
@@ -31086,6 +31319,61 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_max_reasoning_effort": true
},
"vertex_ai/claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "vertex_ai-anthropic_models",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346
},
"vertex_ai/claude-opus-4-7@default": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "vertex_ai-anthropic_models",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346
},
"vertex_ai/claude-sonnet-4-5": {
@@ -38251,4 +38539,4 @@
"supports_native_structured_output": true,
"supports_pdf_input": true
}
}
}
litellm/setup_wizard.py
+7 -2
View File
@@ -52,11 +52,16 @@ PROVIDERS: List[Dict] = [
{
"id": "anthropic",
"name": "Anthropic",
"description": "Claude Opus 4.6, Sonnet 4.6, Haiku 4.5",
"description": "Claude Opus 4.7, Opus 4.6, Sonnet 4.6, Haiku 4.5",
"env_key": "ANTHROPIC_API_KEY",
"key_hint": "sk-ant-...",
"test_model": "claude-haiku-4-5-20251001",
"models": ["claude-opus-4-6", "claude-sonnet-4-6", "claude-haiku-4-5-20251001"],
"models": [
"claude-opus-4-7",
"claude-opus-4-6",
"claude-sonnet-4-6",
"claude-haiku-4-5-20251001",
],
},
{
"id": "gemini",
model_prices_and_context_window.json
+310 -7
View File
@@ -1005,7 +1005,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"global.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.25e-06,
@@ -1032,7 +1033,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"us.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@@ -1059,7 +1061,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"eu.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@@ -1086,7 +1089,8 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"au.anthropic.claude-opus-4-6-v1": {
"cache_creation_input_token_cost": 6.875e-06,
@@ -1113,6 +1117,147 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true,
"supports_max_reasoning_effort": true
},
"anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"global.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"us.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token": 5.5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.75e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"eu.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token": 5.5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.75e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"au.anthropic.claude-opus-4-7": {
"cache_creation_input_token_cost": 6.875e-06,
"cache_read_input_token_cost": 5.5e-07,
"input_cost_per_token": 5.5e-06,
"litellm_provider": "bedrock_converse",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.75e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"supports_native_structured_output": true
},
"anthropic.claude-sonnet-4-6": {
@@ -1765,6 +1910,35 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 159,
"supports_max_reasoning_effort": true
},
"azure_ai/claude-opus-4-7": {
"input_cost_per_token": 5e-06,
"output_cost_per_token": 2.5e-05,
"litellm_provider": "azure_ai",
"max_input_tokens": 200000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"cache_creation_input_token_cost": 6.25e-06,
"cache_creation_input_token_cost_above_1hr": 1e-05,
"cache_read_input_token_cost": 5e-07,
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 159
},
"azure_ai/claude-opus-4-1": {
@@ -8928,7 +9102,8 @@
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
}
},
"supports_max_reasoning_effort": true
},
"claude-opus-4-6-20260205": {
"cache_creation_input_token_cost": 6.25e-06,
@@ -8956,6 +9131,71 @@
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
},
"supports_max_reasoning_effort": true
},
"claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_creation_input_token_cost_above_1hr": 1e-05,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "anthropic",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
}
},
"claude-opus-4-7-20260416": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_creation_input_token_cost_above_1hr": 1e-05,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "anthropic",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346,
"provider_specific_entry": {
"us": 1.1,
"fast": 6.0
@@ -26683,6 +26923,13 @@
"supports_reasoning": false,
"supports_function_calling": true
},
"perplexity/anthropic/claude-opus-4-7": {
"litellm_provider": "perplexity",
"mode": "responses",
"supports_web_search": true,
"supports_reasoning": false,
"supports_function_calling": true
},
"perplexity/anthropic/claude-opus-4-5": {
"litellm_provider": "perplexity",
"mode": "responses",
@@ -31045,7 +31292,8 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346
"tool_use_system_prompt_tokens": 346,
"supports_max_reasoning_effort": true
},
"vertex_ai/claude-opus-4-6@default": {
"cache_creation_input_token_cost": 6.25e-06,
@@ -31071,6 +31319,61 @@
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"tool_use_system_prompt_tokens": 346,
"supports_max_reasoning_effort": true
},
"vertex_ai/claude-opus-4-7": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "vertex_ai-anthropic_models",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346
},
"vertex_ai/claude-opus-4-7@default": {
"cache_creation_input_token_cost": 6.25e-06,
"cache_read_input_token_cost": 5e-07,
"input_cost_per_token": 5e-06,
"litellm_provider": "vertex_ai-anthropic_models",
"max_input_tokens": 1000000,
"max_output_tokens": 128000,
"max_tokens": 128000,
"mode": "chat",
"output_cost_per_token": 2.5e-05,
"search_context_cost_per_query": {
"search_context_size_high": 0.01,
"search_context_size_low": 0.01,
"search_context_size_medium": 0.01
},
"supports_assistant_prefill": false,
"supports_computer_use": true,
"supports_function_calling": true,
"supports_pdf_input": true,
"supports_prompt_caching": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_tool_choice": true,
"supports_vision": true,
"supports_xhigh_reasoning_effort": true,
"tool_use_system_prompt_tokens": 346
},
"vertex_ai/claude-sonnet-4-5": {
@@ -38236,4 +38539,4 @@
"supports_native_structured_output": true,
"supports_pdf_input": true
}
}
}
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py
+56
View File
@@ -18,6 +18,7 @@ from litellm.llms.bedrock.common_utils import (
normalize_tool_input_schema_types_for_bedrock_invoke,
remove_custom_field_from_tools,
)
from litellm.constants import BEDROCK_MIN_THINKING_BUDGET_TOKENS
from litellm.llms.bedrock.messages.invoke_transformations.anthropic_claude3_transformation import (
AmazonAnthropicClaudeMessagesConfig,
AmazonAnthropicClaudeMessagesStreamDecoder,
@@ -384,6 +385,61 @@ def test_bedrock_invoke_messages_transform_adds_name_when_tool_missing_name():
assert result["tools"][0]["name"] == "litellm_unnamed_tool_0"
def test_bedrock_invoke_messages_injects_thinking_for_clear_thinking_context_management():
"""
Bedrock requires extended thinking when ``clear_thinking_20251015`` appears in
``context_management`` (Claude Code sends CM without ``thinking``).
"""
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
optional_params = {
"max_tokens": 32000,
"stream": False,
"context_management": {
"edits": [{"type": "clear_thinking_20251015", "keep": "all"}]
},
}
result = cfg.transform_anthropic_messages_request(
model="global.anthropic.claude-sonnet-4-6-v1:0",
messages=[{"role": "user", "content": "hi"}],
anthropic_messages_optional_request_params=copy.deepcopy(optional_params),
litellm_params=GenericLiteLLMParams(),
headers={},
)
assert result["thinking"]["type"] == "enabled"
assert result["thinking"]["budget_tokens"] == BEDROCK_MIN_THINKING_BUDGET_TOKENS
betas = result.get("anthropic_beta") or []
assert "interleaved-thinking-2025-05-14" in betas
def test_bedrock_invoke_messages_skips_thinking_injection_when_already_enabled():
from litellm.types.router import GenericLiteLLMParams
cfg = AmazonAnthropicClaudeMessagesConfig()
optional_params = {
"max_tokens": 32000,
"stream": False,
"thinking": {"type": "enabled", "budget_tokens": 2048},
"context_management": {
"edits": [{"type": "clear_thinking_20251015", "keep": "all"}]
},
}
result = cfg.transform_anthropic_messages_request(
model="global.anthropic.claude-sonnet-4-6-v1:0",
messages=[{"role": "user", "content": "hi"}],
anthropic_messages_optional_request_params=copy.deepcopy(optional_params),
litellm_params=GenericLiteLLMParams(),
headers={},
)
# Claude 4.6/4.7 reject ``thinking.type=enabled``; legacy ``enabled`` is
# translated to ``adaptive`` (budget_tokens => output_config.effort) and the
# pre-4.6 ``interleaved-thinking-2025-05-14`` beta must not be attached.
assert result["thinking"]["type"] == "adaptive"
betas = result.get("anthropic_beta") or []
assert "interleaved-thinking-2025-05-14" not in betas
def test_bedrock_invoke_messages_transform_converts_custom_tool_schema_type_to_object():
"""
End-to-end: AmazonAnthropicClaudeMessagesConfig must emit Bedrock Invoke bodies
tests/test_litellm/test_utils.py
+1
View File
@@ -771,6 +771,7 @@ def test_aaamodel_prices_and_context_window_json_is_valid():
"supports_minimal_reasoning_effort": {"type": "boolean"},
"supports_none_reasoning_effort": {"type": "boolean"},
"supports_xhigh_reasoning_effort": {"type": "boolean"},
"supports_max_reasoning_effort": {"type": "boolean"},
"supports_service_tier": {"type": "boolean"},
"supports_preset": {"type": "boolean"},
"tool_use_system_prompt_tokens": {"type": "number"},