diff --git a/docs/my-website/docs/proxy/auto_routing.md b/docs/my-website/docs/proxy/auto_routing.md index da11e93a1f..3169047a6a 100644 --- a/docs/my-website/docs/proxy/auto_routing.md +++ b/docs/my-website/docs/proxy/auto_routing.md @@ -127,6 +127,44 @@ The `router.json` file supports the following structure: - **score_threshold**: Minimum similarity score to trigger this route (0.0-1.0) - **metadata**: Additional metadata for the route + +## LiteLLM Proxy Server + +### Setup + +Navigate to the LiteLLM UI and go to **Models+Endpoints** > **Add Model** > **Auto Router Tab**. + +Configure the following required fields: + +- **Auto Router Name** - The model name that developers will use when making LLM API requests to LiteLLM +- **Default Model** - The fallback model used when no route is matched (e.g., if set to "gpt-4o-mini", unmatched requests will be routed to gpt-4o-mini) +- **Embedding Model** - The model used to generate embeddings for input messages. These embeddings are used to semantically match input against the utterances defined in your routes + +#### Route Configuration + +Auto Router Setup + +
+ +
+ +Click **Add Route** to create a new routing rule. Each route consists of utterances that are matched against input messages to determine the target model. + +Configure each route with: + +- **Utterances** - Example phrases that will trigger this route. Use placeholders in brackets for variables: + +```json +"how to code a program in [language]", +"can you explain this [language] code", +"can you explain this [language] script", +"can you convert this [language] code to [target_language]" +``` + +- **Description** - A human-readable description of what this route handles +- **Score Threshold** - The minimum similarity score (0.0-1.0) required to trigger this route + + ## How It Works 1. When a request comes in, LiteLLM generates embeddings for the input message diff --git a/docs/my-website/img/auto_router2.png b/docs/my-website/img/auto_router2.png new file mode 100644 index 0000000000..23c1032286 Binary files /dev/null and b/docs/my-website/img/auto_router2.png differ