mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-07 22:21:46 +00:00
litellm mcp support
This commit is contained in:
@@ -29,6 +29,8 @@ When MCP clients connect to LiteLLM they can follow this workflow:
|
||||
|
||||
### 1. Define your tools on mcp_tools
|
||||
|
||||
LiteLLM allows you to define your tools on the `mcp_tools` section in your config.yaml file. All tools listed here will be available to MCP clients (when they connect to LiteLLM and call `list_tools`).
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: gpt-4o
|
||||
@@ -54,13 +56,38 @@ mcp_tools:
|
||||
handler: "mcp_tools.get_current_time"
|
||||
```
|
||||
|
||||
### 2. Define a handler for your tool
|
||||
|
||||
Create a new file called `mcp_tools.py` and add this code. The key method here is `get_current_time` which gets executed when the `get_current_time` tool is called.
|
||||
|
||||
```python
|
||||
# mcp_tools.py
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
def get_current_time(format: str = "short"):
|
||||
"""
|
||||
Simple handler for the 'get_current_time' tool.
|
||||
|
||||
Args:
|
||||
format (str): The format of the time to return ('short').
|
||||
|
||||
Returns:
|
||||
str: The current time formatted as 'HH:MM'.
|
||||
"""
|
||||
# Get the current time
|
||||
current_time = datetime.now()
|
||||
|
||||
# Format the time as 'HH:MM'
|
||||
return current_time.strftime('%H:%M')
|
||||
```
|
||||
|
||||
### 3. Start LiteLLM Gateway
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="docker" label="Docker Run">
|
||||
|
||||
Mount your `custom_logger.py` on the LiteLLM Docker container.
|
||||
Mount your `mcp_tools.py` on the LiteLLM Docker container.
|
||||
|
||||
```shell
|
||||
docker run -d \
|
||||
@@ -68,7 +95,7 @@ docker run -d \
|
||||
-e OPENAI_API_KEY=$OPENAI_API_KEY \
|
||||
--name my-app \
|
||||
-v $(pwd)/my_config.yaml:/app/config.yaml \
|
||||
-v $(pwd)/custom_logger.py:/app/custom_logger.py \
|
||||
-v $(pwd)/mcp_tools.py:/app/mcp_tools.py \
|
||||
my-app:latest \
|
||||
--config /app/config.yaml \
|
||||
--port 4000 \
|
||||
|
||||
Reference in New Issue
Block a user