tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-13 17:10:08 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Feat] Add new anthropic web fetch tool support (#14951)

Browse Source 
* add web_fetch tool for ANTHROPIC_HOSTED_TOOLS

* add ANTHROPIC_BETA_HEADER_VALUES, ANTHROPIC_HOSTED_TOOLS

* feat: add web fetch tool anthropic

* test_anthropic_tool_use

* docs web fetch

* docs fix

* docs fix
This commit is contained in:
Ishaan Jaff
2025-09-26 11:04:11 -07:00
committed by GitHub
parent 6fe8c33448
commit d04c6d4eea
8 changed files with 374 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/docs/completion/web_fetch.md
+294
View File
@@ -0,0 +1,294 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Web Fetch
The web fetch tool allows LLMs to retrieve full content from specified web pages and PDF documents. This enables AI models to access real-time information from the internet and incorporate web content into their responses.
## Web Fetch vs Web Search
**Web Fetch** retrieves the full content from specific web pages that you provide URLs for, while **Web Search** performs internet searches to find relevant information based on your queries.
| Feature | Web Fetch | Web Search |
|---------|-----------|------------|
| **Purpose** | Retrieve content from specific URLs | Search the internet for information |
| **Input** | You provide exact URLs to fetch | You provide search queries/questions |
| **Output** | Full page content from specified URLs | Search results with relevant information |
| **Use Cases** | - Analyzing specific articles<br/>- Comparing content from known websites<br/>- Extracting data from particular pages | - Finding current news/events<br/>- Researching topics<br/>- Getting real-time information |
**Example Web Fetch**: "Fetch the content from https://example.com/pricing and summarize it"
**Example Web Search**: "What are the latest AI developments this week?"
**Supported Providers:**
- Anthropic API (`anthropic/`)
**Supported Tool Types:**
- `web_fetch_20250910` - Web content retrieval tool with usage limits, domain filtering, and citation support
## Quick Start
### LiteLLM Python SDK
```python
import os
from litellm import completion
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"
# Web fetch tool
tools = [
{
"type": "web_fetch_20250910",
"name": "web_fetch",
"max_uses": 5,
}
]
messages = [
{
"role": "user",
"content": "Please analyze the content at https://example.com/article and summarize the main points"
}
]
response = completion(
model="anthropic/claude-3-5-sonnet-latest",
messages=messages,
tools=tools,
)
print(response)
```
### LiteLLM Proxy
1. Define web fetch models on config.yaml
```yaml
model_list:
- model_name: claude-3-5-sonnet-latest # Anthropic claude-3-5-sonnet-latest
litellm_params:
model: anthropic/claude-3-5-sonnet-latest
api_key: os.environ/ANTHROPIC_API_KEY
```
2. Run proxy server
```bash
litellm --config config.yaml
```
3. Test it using the OpenAI Python SDK
```python
import os
from openai import OpenAI
client = OpenAI(
api_key="sk-1234", # your litellm proxy api key
base_url="http://0.0.0.0:4000"
)
response = client.chat.completions.create(
model="claude-3-5-sonnet-latest",
messages=[
{
"role": "user",
"content": "Please fetch and analyze the content from https://news.ycombinator.com and tell me about the top stories"
}
],
tools=[
{
"type": "web_fetch_20250910",
"name": "web_fetch",
"max_uses": 5,
}
]
)
print(response)
```
## Supported Models
Web fetch is available on the following Anthropic API models:
- `claude-opus-4-1-20250805` (Claude Opus 4.1)
- `claude-opus-4-20250514` (Claude Opus 4)
- `claude-sonnet-4-20250514` (Claude Sonnet 4)
- `claude-3-7-sonnet-20250219` (Claude Sonnet 3.7)
- `claude-3-5-sonnet-latest` (Claude Sonnet 3.5 v2 - deprecated)
- `claude-3-5-haiku-latest` (Claude Haiku 3.5)
:::note
The web fetch tool currently does not support websites dynamically rendered via JavaScript.
:::
## Usage Examples
### Basic Web Content Retrieval
```python
import os
from litellm import completion
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"
tools = [
{
"type": "web_fetch_20250910",
"name": "web_fetch",
"max_uses": 3,
}
]
messages = [
{
"role": "user",
"content": "Fetch the latest news from https://techcrunch.com and summarize the top 3 articles"
}
]
response = completion(
model="anthropic/claude-3-5-sonnet-latest",
messages=messages,
tools=tools,
)
print(response)
```
### Research and Analysis
```python
import os
from litellm import completion
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"
tools = [
{
"type": "web_fetch_20250910",
"name": "web_fetch",
"max_uses": 10,
}
]
messages = [
{
"role": "user",
"content": "Research the latest developments in AI by fetching content from multiple tech news websites and provide a comprehensive analysis"
}
]
response = completion(
model="anthropic/claude-3-5-sonnet-latest",
messages=messages,
tools=tools,
)
print(response)
```
### Content Comparison
```python
import os
from litellm import completion
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"
tools = [
{
"type": "web_fetch_20250910",
"name": "web_fetch",
"max_uses": 5,
}
]
messages = [
{
"role": "user",
"content": "Compare the pricing information from https://openai.com/pricing and https://anthropic.com/pricing and create a comparison table"
}
]
response = completion(
model="anthropic/claude-3-5-sonnet-latest",
messages=messages,
tools=tools,
)
print(response)
```
## Advanced Usage with Multiple Tools
You can combine web fetch with other tools like computer use or text editor:
```python
import os
from litellm import completion
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"
tools = [
{
"type": "web_fetch_20250910",
"name": "web_fetch",
"max_uses": 5,
},
{
"type": "text_editor_20250124",
"name": "str_replace_editor"
}
]
messages = [
{
"role": "user",
"content": "Fetch the latest AI research papers from arXiv, analyze them, and create a detailed report file with your findings"
}
]
response = completion(
model="anthropic/claude-3-5-sonnet-latest",
messages=messages,
tools=tools,
)
print(response)
```
## Spec
### Web Fetch Tool (`web_fetch_20250910`)
The web fetch tool supports the following parameters:
```json
{
"type": "web_fetch_20250910",
"name": "web_fetch",
// Optional: Limit the number of fetches per request
"max_uses": 10,
// Optional: Only fetch from these domains
"allowed_domains": ["example.com", "docs.example.com"],
// Optional: Never fetch from these domains
"blocked_domains": ["private.example.com"],
// Optional: Enable citations for fetched content
"citations": {
"enabled": true
},
// Optional: Maximum content length in tokens
"max_content_tokens": 100000
}
```
docs/my-website/docs/completion/web_search.md
+1 -1
View File
@@ -1,7 +1,7 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Using Web Search
# Web Search
Use web search with litellm
docs/my-website/docs/providers/vertex.md
+1 -1
View File
@@ -898,7 +898,7 @@ curl http://0.0.0.0:4000/chat/completions \
```
</TabItem>
</Tabs>
## Pre-requisites
* `pip install google-cloud-aiplatform` (pre-installed on proxy docker image)
docs/my-website/docs/proxy/native_litellm_prompt.md
+2 -6
View File
@@ -1,9 +1,5 @@
import '@theme/IdealImage'
import '@theme/TabItem'
import '@theme/Tabs'
import Image
import TabItem
import Tabs
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# LiteLLM Prompt Management (GitOps)
docs/my-website/sidebars.js
+11 -4
View File
@@ -518,12 +518,19 @@ const sidebars = {
type: "category",
label: "Guides",
items: [
{
type: "category",
label: "Tools",
items: [
"completion/computer_use",
"completion/web_search",
"completion/web_fetch",
"completion/function_call",
]
},
"completion/audio",
"completion/batching",
"completion/computer_use",
"completion/document_understanding",
"completion/drop_params",
"completion/function_call",
"completion/image_generation_chat",
"completion/json_mode",
"completion/knowledgebase",
@@ -538,8 +545,8 @@ const sidebars = {
"completion/stream",
"completion/provider_specific_params",
"completion/vision",
"completion/web_search",
"exception_mapping",
"completion/batching",
"guides/finetuned_models",
"guides/security_settings",
"proxy/veo_video_generation",
litellm/llms/anthropic/chat/transformation.py
+16 -4
View File
@@ -18,6 +18,8 @@ from litellm.litellm_core_utils.core_helpers import map_finish_reason
from litellm.llms.base_llm.base_utils import type_to_response_format_param
from litellm.llms.base_llm.chat.transformation import BaseConfig, BaseLLMException
from litellm.types.llms.anthropic import (
ANTHROPIC_BETA_HEADER_VALUES,
ANTHROPIC_HOSTED_TOOLS,
AllAnthropicMessageValues,
AllAnthropicToolsValues,
AnthropicCodeExecutionTool,
@@ -50,7 +52,10 @@ from litellm.types.utils import (
CompletionTokensDetailsWrapper,
)
from litellm.types.utils import Message as LitellmMessage
from litellm.types.utils import PromptTokensDetailsWrapper, ServerToolUse
from litellm.types.utils import (
PromptTokensDetailsWrapper,
ServerToolUse,
)
from litellm.utils import (
ModelResponse,
Usage,
@@ -70,9 +75,6 @@ else:
LoggingClass = Any
ANTHROPIC_HOSTED_TOOLS = ["web_search", "bash", "text_editor", "code_execution"]
class AnthropicConfig(AnthropicModelInfo, BaseConfig):
"""
Reference: https://docs.anthropic.com/claude/reference/messages_post
@@ -639,6 +641,14 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
)
)
return tools
def update_headers_with_optional_anthropic_beta(self, headers: dict, optional_params: dict) -> dict:
"""Update headers with optional anthropic beta."""
_tools = optional_params.get("tools", [])
for tool in _tools:
if tool.get("type", None) and tool.get("type").startswith(ANTHROPIC_HOSTED_TOOLS.WEB_FETCH.value):
headers["anthropic-beta"] = ANTHROPIC_BETA_HEADER_VALUES.WEB_FETCH_2025_09_10.value
return headers
def transform_request(
self,
@@ -675,6 +685,8 @@ class AnthropicConfig(AnthropicModelInfo, BaseConfig):
llm_provider="anthropic",
)
headers = self.update_headers_with_optional_anthropic_beta(headers=headers, optional_params=optional_params)
# Separate system prompt from rest of message
anthropic_system_message_list = self.translate_system_message(messages=messages)
# Handling anthropic API Prompt Caching
litellm/types/llms/anthropic.py
+14
View File
@@ -1,3 +1,4 @@
from enum import Enum
from typing import Any, Dict, Iterable, List, Optional, Union
from pydantic import BaseModel, validator
@@ -440,3 +441,16 @@ ANTHROPIC_API_ONLY_HEADERS = { # fails if calling anthropic on vertex ai / bedr
class AnthropicThinkingParam(TypedDict, total=False):
type: Literal["enabled"]
budget_tokens: int
class ANTHROPIC_HOSTED_TOOLS(str, Enum):
WEB_SEARCH = "web_search"
BASH = "bash"
TEXT_EDITOR = "text_editor"
CODE_EXECUTION = "code_execution"
WEB_FETCH = "web_fetch"
class ANTHROPIC_BETA_HEADER_VALUES(str, Enum):
"""
Known beta header values for Anthropic.
"""
WEB_FETCH_2025_09_10 = "web-fetch-2025-09-10"
tests/llm_translation/test_anthropic_completion.py
+35 -16
View File
@@ -329,32 +329,51 @@ def test_process_anthropic_headers_with_no_matching_headers():
assert result == expected_output, "Unexpected output for non-matching headers"
def test_anthropic_computer_tool_use():
from litellm import completion
tools = [
{
"type": "computer_20241022",
"function": {
"name": "computer",
"parameters": {
"display_height_px": 100,
"display_width_px": 100,
"display_number": 1,
@pytest.mark.parametrize(
"tool_type, tool_config, message_content",
[
(
"computer_20241022",
{
"type": "computer_20241022",
"function": {
"name": "computer",
"parameters": {
"display_height_px": 100,
"display_width_px": 100,
"display_number": 1,
},
},
},
}
]
"Save a picture of a cat to my desktop.",
),
(
"web_fetch_20250910",
{
"type": "web_fetch_20250910",
"name": "web_fetch",
"max_uses": 5,
},
"Please analyze the content at https://example.com/article",
),
],
)
def test_anthropic_tool_use(tool_type, tool_config, message_content):
"""Test Anthropic tool use with computer use and web fetch tools."""
from litellm import completion
litellm._turn_on_debug()
tools = [tool_config]
model = "claude-3-5-sonnet-20241022"
messages = [{"role": "user", "content": "Save a picture of a cat to my desktop."}]
messages = [{"role": "user", "content": message_content}]
try:
resp = completion(
model=model,
messages=messages,
tools=tools,
# headers={"anthropic-beta": "computer-use-2024-10-22"},
)
print(f"Tool type: {tool_type}")
print(resp)
except litellm.InternalServerError:
pass