mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-09 00:26:01 +00:00
fix(anthropic): map reasoning_effort to output_config for Claude 4.6 models
Claude 4.6 models use output_config as a stable API feature. This commit: - Maps reasoning_effort to output_config for 4.6 models (minimal → low) - Restricts effort="max" to Opus 4.6 only - Skips beta header injection for 4.6 models - Updates docs for Claude 4.6 effort support
This commit is contained in:
@@ -4,6 +4,8 @@ import TabItem from '@theme/TabItem';
|
||||
# Anthropic
|
||||
LiteLLM supports all anthropic models.
|
||||
|
||||
- `claude-opus-4-6` (`claude-opus-4-6-20260205`)
|
||||
- `claude-sonnet-4-6`
|
||||
- `claude-sonnet-4-5-20250929`
|
||||
- `claude-opus-4-5-20251101`
|
||||
- `claude-opus-4-1-20250805`
|
||||
@@ -50,7 +52,7 @@ Check this in code, [here](../completion/input.md#translated-openai-params)
|
||||
**Notes:**
|
||||
- Anthropic API fails requests when `max_tokens` are not passed. Due to this litellm passes `max_tokens=4096` when no `max_tokens` are passed.
|
||||
- `response_format` is fully supported for Claude Sonnet 4.5 and Opus 4.1 models (see [Structured Outputs](#structured-outputs) section)
|
||||
- `reasoning_effort` is automatically mapped to `output_config={"effort": ...}` for Claude Opus 4.5 models (see [Effort Parameter](./anthropic_effort.md))
|
||||
- `reasoning_effort` is automatically mapped to `output_config={"effort": ...}` for Claude 4.6 and Opus 4.5 models (see [Effort Parameter](./anthropic_effort.md))
|
||||
|
||||
:::
|
||||
|
||||
|
||||
@@ -9,10 +9,11 @@ Control how many tokens Claude uses when responding with the `effort` parameter,
|
||||
|
||||
The `effort` parameter allows you to control how eager Claude is about spending tokens when responding to requests. This gives you the ability to trade off between response thoroughness and token efficiency, all with a single model.
|
||||
|
||||
**Note**: The effort parameter is currently in beta and only supported by Claude Opus 4.5. LiteLLM automatically adds the `effort-2025-11-24` beta header when:
|
||||
- `reasoning_effort` parameter is provided (for Claude Opus 4.5 only)
|
||||
**Supported models:**
|
||||
- **Claude 4.6** (Opus 4.6, Sonnet 4.6) — `output_config` is a stable API feature, no beta header needed. Opus 4.6 also supports `effort="max"`.
|
||||
- **Claude Opus 4.5** — requires the `effort-2025-11-24` beta header (automatically added by LiteLLM).
|
||||
|
||||
For Claude Opus 4.5, `reasoning_effort="medium"`—both are automatically mapped to the correct format.
|
||||
LiteLLM automatically maps `reasoning_effort` → `output_config={"effort": ...}` for all supported models.
|
||||
|
||||
## How Effort Works
|
||||
|
||||
@@ -35,6 +36,7 @@ This gives a much greater degree of control over efficiency.
|
||||
|
||||
| Level | Description | Typical use case |
|
||||
|-------|-------------|------------------|
|
||||
| `max` | Maximum capability beyond high — Claude uses even more tokens for the most thorough outcome. **Only supported by Claude Opus 4.6.** | The hardest reasoning problems, complex multi-step research |
|
||||
| `high` | Maximum capability—Claude uses as many tokens as needed for the best possible outcome. Equivalent to not setting the parameter. | Complex reasoning, difficult coding problems, agentic tasks |
|
||||
| `medium` | Balanced approach with moderate token savings. | Agentic tasks that require a balance of speed, cost, and performance |
|
||||
| `low` | Most efficient—significant token savings with some capability reduction. | Simpler tasks that need the best speed and lowest costs, such as subagents |
|
||||
@@ -49,16 +51,29 @@ This gives a much greater degree of control over efficiency.
|
||||
```python
|
||||
import litellm
|
||||
|
||||
# Works with Claude 4.6 models (no beta header needed)
|
||||
response = litellm.completion(
|
||||
model="anthropic/claude-sonnet-4-6",
|
||||
messages=[{
|
||||
"role": "user",
|
||||
"content": "Analyze the trade-offs between microservices and monolithic architectures"
|
||||
}],
|
||||
reasoning_effort="medium" # Automatically mapped to output_config
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
```python
|
||||
# Also works with Claude Opus 4.5 (beta header auto-injected)
|
||||
response = litellm.completion(
|
||||
model="anthropic/claude-opus-4-5-20251101",
|
||||
messages=[{
|
||||
"role": "user",
|
||||
"content": "Analyze the trade-offs between microservices and monolithic architectures"
|
||||
}],
|
||||
reasoning_effort="medium" # Automatically mapped to output_config for Opus 4.5
|
||||
reasoning_effort="medium"
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
@@ -71,8 +86,9 @@ const client = new Anthropic({
|
||||
apiKey: process.env.ANTHROPIC_API_KEY,
|
||||
});
|
||||
|
||||
// Claude 4.6 — output_config is a stable API feature (no beta header)
|
||||
const response = await client.messages.create({
|
||||
model: "claude-opus-4-5-20251101",
|
||||
model: "claude-sonnet-4-6",
|
||||
max_tokens: 4096,
|
||||
messages: [{
|
||||
role: "user",
|
||||
@@ -96,7 +112,29 @@ curl http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
-d '{
|
||||
"model": "anthropic/claude-opus-4-5-20251101",
|
||||
"model": "anthropic/claude-sonnet-4-6",
|
||||
"messages": [{
|
||||
"role": "user",
|
||||
"content": "Analyze the trade-offs between microservices and monolithic architectures"
|
||||
}],
|
||||
"reasoning_effort": "medium"
|
||||
}'
|
||||
```
|
||||
|
||||
### Direct Anthropic API Call
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="46" label="Claude 4.6 (stable)">
|
||||
|
||||
```bash
|
||||
# Claude 4.6 — no beta header needed
|
||||
curl https://api.anthropic.com/v1/messages \
|
||||
--header "x-api-key: $ANTHROPIC_API_KEY" \
|
||||
--header "anthropic-version: 2023-06-01" \
|
||||
--header "content-type: application/json" \
|
||||
--data '{
|
||||
"model": "claude-sonnet-4-6",
|
||||
"max_tokens": 4096,
|
||||
"messages": [{
|
||||
"role": "user",
|
||||
"content": "Analyze the trade-offs between microservices and monolithic architectures"
|
||||
@@ -107,9 +145,11 @@ curl http://localhost:4000/v1/chat/completions \
|
||||
}'
|
||||
```
|
||||
|
||||
### Direct Anthropic API Call
|
||||
</TabItem>
|
||||
<TabItem value="45" label="Claude Opus 4.5 (beta)">
|
||||
|
||||
```bash
|
||||
# Claude Opus 4.5 — requires beta header
|
||||
curl https://api.anthropic.com/v1/messages \
|
||||
--header "x-api-key: $ANTHROPIC_API_KEY" \
|
||||
--header "anthropic-version: 2023-06-01" \
|
||||
@@ -128,10 +168,19 @@ curl https://api.anthropic.com/v1/messages \
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Model Compatibility
|
||||
|
||||
The effort parameter is currently only supported by:
|
||||
- **Claude Opus 4.5** (`claude-opus-4-5-20251101`)
|
||||
The effort parameter is supported by:
|
||||
- **Claude Opus 4.6** (`claude-opus-4-6`) — supports `high`, `medium`, `low`, and `max`
|
||||
- **Claude Sonnet 4.6** (`claude-sonnet-4-6`) — supports `high`, `medium`, `low`
|
||||
- **Claude Opus 4.5** (`claude-opus-4-5-20251101`) — supports `high`, `medium`, `low`
|
||||
|
||||
:::info
|
||||
`effort="max"` is only available on Claude Opus 4.6. Using it with other models will raise a validation error.
|
||||
:::
|
||||
|
||||
## When Should I Adjust the Effort Parameter?
|
||||
|
||||
@@ -154,7 +203,7 @@ Example with tools:
|
||||
import litellm
|
||||
|
||||
response = litellm.completion(
|
||||
model="anthropic/claude-opus-4-5-20251101",
|
||||
model="anthropic/claude-sonnet-4-6",
|
||||
messages=[{
|
||||
"role": "user",
|
||||
"content": "Check the weather in multiple cities"
|
||||
@@ -173,9 +222,7 @@ response = litellm.completion(
|
||||
}
|
||||
}
|
||||
}],
|
||||
output_config={
|
||||
"effort": "low" # Will make fewer tool calls
|
||||
}
|
||||
reasoning_effort="low" # Mapped to output_config — will make fewer tool calls
|
||||
)
|
||||
```
|
||||
|
||||
@@ -187,18 +234,12 @@ The effort parameter works seamlessly with extended thinking. When both are enab
|
||||
import litellm
|
||||
|
||||
response = litellm.completion(
|
||||
model="anthropic/claude-opus-4-5-20251101",
|
||||
model="anthropic/claude-sonnet-4-6",
|
||||
messages=[{
|
||||
"role": "user",
|
||||
"content": "Solve this complex problem"
|
||||
}],
|
||||
thinking={
|
||||
"type": "enabled",
|
||||
"budget_tokens": 5000
|
||||
},
|
||||
output_config={
|
||||
"effort": "medium" # Affects both thinking and response tokens
|
||||
}
|
||||
reasoning_effort="medium" # Mapped to adaptive thinking + output_config for 4.6 models
|
||||
)
|
||||
```
|
||||
|
||||
@@ -218,14 +259,14 @@ response = litellm.completion(
|
||||
|
||||
The effort parameter is supported across all Anthropic-compatible providers:
|
||||
|
||||
- **Standard Anthropic API**: ✅ Supported (Claude Opus 4.5)
|
||||
- **Azure Anthropic / Microsoft Foundry**: ✅ Supported (Claude Opus 4.5)
|
||||
- **Amazon Bedrock**: ✅ Supported (Claude Opus 4.5)
|
||||
- **Google Cloud Vertex AI**: ✅ Supported (Claude Opus 4.5)
|
||||
- **Standard Anthropic API**: ✅ Supported (Claude 4.6, Opus 4.5)
|
||||
- **Azure Anthropic / Microsoft Foundry**: ✅ Supported (Claude 4.6, Opus 4.5)
|
||||
- **Amazon Bedrock**: ✅ Supported (Claude 4.6, Opus 4.5)
|
||||
- **Google Cloud Vertex AI**: ✅ Supported (Claude 4.6, Opus 4.5)
|
||||
|
||||
LiteLLM automatically handles:
|
||||
- Beta header injection (`effort-2025-11-24`) for all providers
|
||||
- Parameter mapping: `reasoning_effort` → `output_config={"effort": ...}` for Claude Opus 4.5
|
||||
- Parameter mapping: `reasoning_effort` → `output_config={"effort": ...}` for all supported models
|
||||
- Beta header injection (`effort-2025-11-24`) only for Claude Opus 4.5 (not needed for 4.6 models)
|
||||
|
||||
## Usage and Pricing
|
||||
|
||||
@@ -244,12 +285,13 @@ print(f"Total tokens: {response.usage.total_tokens}")
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Beta header not being added
|
||||
### Beta header not being added (Claude Opus 4.5)
|
||||
|
||||
LiteLLM automatically adds the `effort-2025-11-24` beta header when:
|
||||
- `reasoning_effort` parameter is provided (for Claude Opus 4.5 only)
|
||||
LiteLLM automatically adds the `effort-2025-11-24` beta header for Claude Opus 4.5 when `reasoning_effort` or `output_config` is provided.
|
||||
|
||||
If you're not seeing the header:
|
||||
**Note:** Claude 4.6 models do NOT need a beta header — `output_config` is a stable API feature for these models.
|
||||
|
||||
If you're not seeing the header for Opus 4.5:
|
||||
|
||||
1. Ensure you're using `reasoning_effort` parameter
|
||||
2. Verify the model is Claude Opus 4.5
|
||||
@@ -257,7 +299,7 @@ If you're not seeing the header:
|
||||
|
||||
### Invalid effort value error
|
||||
|
||||
Only three values are accepted: `"high"`, `"medium"`, `"low"`. Any other value will raise a validation error:
|
||||
Accepted values: `"high"`, `"medium"`, `"low"`, and `"max"` (Opus 4.6 only). Any other value will raise a validation error:
|
||||
|
||||
```python
|
||||
# ❌ This will raise an error
|
||||
@@ -265,11 +307,17 @@ output_config={"effort": "very_low"}
|
||||
|
||||
# ✅ Use one of the valid values
|
||||
output_config={"effort": "low"}
|
||||
|
||||
# ❌ This will raise an error (max only works on Opus 4.6)
|
||||
litellm.completion(model="anthropic/claude-sonnet-4-6", reasoning_effort="max", ...)
|
||||
|
||||
# ✅ max is only for Opus 4.6
|
||||
litellm.completion(model="anthropic/claude-opus-4-6", reasoning_effort="max", ...)
|
||||
```
|
||||
|
||||
### Model not supported
|
||||
|
||||
Currently, only Claude Opus 4.5 supports the effort parameter. Using it with other models may result in the parameter being ignored or an error.
|
||||
The effort parameter is supported by Claude Opus 4.6, Sonnet 4.6, and Opus 4.5. Using it with other models may result in the parameter being ignored or an error.
|
||||
|
||||
## Related Features
|
||||
|
||||
|
||||
@@ -186,6 +186,15 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
|
||||
)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _is_opus_4_6_model(model: str) -> bool:
|
||||
"""Check if the model is specifically Claude Opus 4.6."""
|
||||
model_lower = model.lower()
|
||||
return any(
|
||||
v in model_lower
|
||||
for v in ("opus-4-6", "opus_4_6", "opus-4.6", "opus_4.6")
|
||||
)
|
||||
|
||||
def get_supported_openai_params(self, model: str):
|
||||
params = [
|
||||
"stream",
|
||||
@@ -1006,6 +1015,11 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
|
||||
optional_params["thinking"] = AnthropicConfig._map_reasoning_effort(
|
||||
reasoning_effort=value, model=model
|
||||
)
|
||||
if AnthropicConfig._is_claude_4_6_model(model):
|
||||
# Map reasoning_effort to Anthropic's output_config for 4.6 models
|
||||
# "minimal" has no Anthropic equivalent → map to "low"
|
||||
anthropic_effort = value if value != "minimal" else "low"
|
||||
optional_params["output_config"] = {"effort": anthropic_effort}
|
||||
elif param == "web_search_options" and isinstance(value, dict):
|
||||
hosted_web_search_tool = self.map_web_search_tool(
|
||||
cast(OpenAIWebSearchOptions, value)
|
||||
@@ -1392,9 +1406,9 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
|
||||
raise ValueError(
|
||||
f"Invalid effort value: {effort}. Must be one of: 'high', 'medium', 'low', 'max'"
|
||||
)
|
||||
if effort == "max" and not self._is_claude_4_6_model(model):
|
||||
if effort == "max" and not self._is_opus_4_6_model(model):
|
||||
raise ValueError(
|
||||
f"effort='max' is only supported by Claude 4.6 models (Opus 4.6, Sonnet 4.6). Got model: {model}"
|
||||
f"effort='max' is only supported by Claude Opus 4.6. Got model: {model}"
|
||||
)
|
||||
data["output_config"] = output_config
|
||||
|
||||
|
||||
@@ -228,20 +228,35 @@ class AnthropicModelInfo(BaseLLMModelInfo):
|
||||
self, optional_params: Optional[dict], model: Optional[str] = None
|
||||
) -> bool:
|
||||
"""
|
||||
Check if effort parameter is being used.
|
||||
Check if effort parameter is being used and requires a beta header.
|
||||
|
||||
Returns True if effort-related parameters are present.
|
||||
Returns True if effort-related parameters are present and
|
||||
the model requires the effort beta header. Claude 4.6 models
|
||||
use output_config as a stable API feature — no beta header needed.
|
||||
"""
|
||||
if not optional_params:
|
||||
return False
|
||||
|
||||
# Claude 4.6 models use output_config as a stable API feature — no beta header needed
|
||||
if model:
|
||||
model_lower = model.lower()
|
||||
is_4_6 = any(
|
||||
v in model_lower
|
||||
for v in (
|
||||
"opus-4-6", "opus_4_6", "opus-4.6", "opus_4.6",
|
||||
"sonnet-4-6", "sonnet_4_6", "sonnet-4.6", "sonnet_4.6",
|
||||
)
|
||||
)
|
||||
if is_4_6:
|
||||
return False
|
||||
|
||||
# Check if reasoning_effort is provided for Claude Opus 4.5
|
||||
if model and ("opus-4-5" in model.lower() or "opus_4_5" in model.lower()):
|
||||
reasoning_effort = optional_params.get("reasoning_effort")
|
||||
if reasoning_effort and isinstance(reasoning_effort, str):
|
||||
return True
|
||||
|
||||
# Check if output_config is directly provided
|
||||
# Check if output_config is directly provided (for non-4.6 models)
|
||||
output_config = optional_params.get("output_config")
|
||||
if output_config and isinstance(output_config, dict):
|
||||
effort = output_config.get("effort")
|
||||
|
||||
@@ -1662,7 +1662,7 @@ def test_max_effort_rejected_for_opus_45():
|
||||
|
||||
messages = [{"role": "user", "content": "Test"}]
|
||||
|
||||
with pytest.raises(ValueError, match="effort='max' is only supported by Claude 4.6 models"):
|
||||
with pytest.raises(ValueError, match="effort='max' is only supported by Claude Opus 4.6"):
|
||||
optional_params = {"output_config": {"effort": "max"}}
|
||||
config.transform_request(
|
||||
model="claude-opus-4-5-20251101",
|
||||
@@ -2119,6 +2119,139 @@ def test_reasoning_effort_maps_to_budget_thinking_for_non_opus_4_6():
|
||||
assert "reasoning_effort" not in result
|
||||
|
||||
|
||||
def test_reasoning_effort_sets_output_config_for_46_models():
|
||||
"""
|
||||
Test that reasoning_effort generates output_config for Claude 4.6 models.
|
||||
|
||||
For Claude 4.6 models, reasoning_effort should produce both adaptive
|
||||
thinking AND output_config with the mapped effort level.
|
||||
"""
|
||||
config = AnthropicConfig()
|
||||
|
||||
for model in ["claude-opus-4-6-20250514", "claude-sonnet-4-6-20260219"]:
|
||||
for effort in ["low", "medium", "high"]:
|
||||
result = config.map_openai_params(
|
||||
non_default_params={"reasoning_effort": effort},
|
||||
optional_params={},
|
||||
model=model,
|
||||
drop_params=False,
|
||||
)
|
||||
|
||||
assert "output_config" in result, (
|
||||
f"output_config missing for {model} with effort={effort}"
|
||||
)
|
||||
assert result["output_config"]["effort"] == effort
|
||||
|
||||
|
||||
def test_reasoning_effort_minimal_maps_to_low_output_config_for_46():
|
||||
"""
|
||||
Test that reasoning_effort='minimal' maps to output_config effort='low'
|
||||
for 4.6 models, since 'minimal' has no Anthropic equivalent.
|
||||
"""
|
||||
config = AnthropicConfig()
|
||||
|
||||
result = config.map_openai_params(
|
||||
non_default_params={"reasoning_effort": "minimal"},
|
||||
optional_params={},
|
||||
model="claude-opus-4-6-20250514",
|
||||
drop_params=False,
|
||||
)
|
||||
|
||||
assert result["output_config"]["effort"] == "low"
|
||||
|
||||
|
||||
def test_reasoning_effort_does_not_set_output_config_for_older_models():
|
||||
"""
|
||||
Test that reasoning_effort does NOT generate output_config for pre-4.6 models.
|
||||
"""
|
||||
config = AnthropicConfig()
|
||||
|
||||
for model in [
|
||||
"claude-sonnet-4-5-20250929",
|
||||
"claude-3-7-sonnet-20250219",
|
||||
"claude-opus-4-5-20251101",
|
||||
]:
|
||||
result = config.map_openai_params(
|
||||
non_default_params={"reasoning_effort": "high"},
|
||||
optional_params={},
|
||||
model=model,
|
||||
drop_params=False,
|
||||
)
|
||||
|
||||
assert "output_config" not in result, (
|
||||
f"output_config should not be set for {model}"
|
||||
)
|
||||
|
||||
|
||||
def test_max_effort_rejected_for_sonnet_46():
|
||||
"""Test that effort='max' is rejected for Sonnet 4.6 (only Opus 4.6 supports max)."""
|
||||
config = AnthropicConfig()
|
||||
messages = [{"role": "user", "content": "Test"}]
|
||||
|
||||
with pytest.raises(ValueError, match="effort='max' is only supported by Claude Opus 4.6"):
|
||||
config.transform_request(
|
||||
model="claude-sonnet-4-6-20260219",
|
||||
messages=messages,
|
||||
optional_params={"output_config": {"effort": "max"}},
|
||||
litellm_params={},
|
||||
headers={},
|
||||
)
|
||||
|
||||
|
||||
def test_max_effort_accepted_for_opus_46():
|
||||
"""Test that effort='max' works for Opus 4.6."""
|
||||
config = AnthropicConfig()
|
||||
messages = [{"role": "user", "content": "Test"}]
|
||||
|
||||
result = config.transform_request(
|
||||
model="claude-opus-4-6-20250514",
|
||||
messages=messages,
|
||||
optional_params={"output_config": {"effort": "max"}},
|
||||
litellm_params={},
|
||||
headers={},
|
||||
)
|
||||
|
||||
assert result["output_config"]["effort"] == "max"
|
||||
|
||||
|
||||
def test_effort_beta_header_not_injected_for_46_models():
|
||||
"""
|
||||
Test that is_effort_used returns False for Claude 4.6 models.
|
||||
|
||||
Claude 4.6 models use output_config as a stable API feature —
|
||||
no beta header should be injected.
|
||||
"""
|
||||
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
|
||||
|
||||
model_info = AnthropicModelInfo()
|
||||
|
||||
for model in ["claude-opus-4-6-20250514", "claude-sonnet-4-6-20260219"]:
|
||||
# Even with output_config present, should return False for 4.6 models
|
||||
result = model_info.is_effort_used(
|
||||
optional_params={"output_config": {"effort": "high"}},
|
||||
model=model,
|
||||
)
|
||||
assert result is False, (
|
||||
f"is_effort_used should return False for {model}"
|
||||
)
|
||||
|
||||
|
||||
def test_effort_beta_header_still_injected_for_older_models():
|
||||
"""
|
||||
Test that is_effort_used still returns True for pre-4.6 models
|
||||
when output_config is present.
|
||||
"""
|
||||
from litellm.llms.anthropic.common_utils import AnthropicModelInfo
|
||||
|
||||
model_info = AnthropicModelInfo()
|
||||
|
||||
result = model_info.is_effort_used(
|
||||
optional_params={"output_config": {"effort": "low"}},
|
||||
model="claude-opus-4-5-20251101",
|
||||
)
|
||||
assert result is True
|
||||
|
||||
|
||||
def test_code_execution_tool_results_extraction():
|
||||
"""
|
||||
Test that code execution tool results (bash_code_execution_tool_result,
|
||||
|
||||
Reference in New Issue
Block a user