tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-11 21:46:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs - add info about routing strategy on load balancing docs

Browse Source
This commit is contained in:
Ishaan Jaff
2024-07-25 17:41:16 -07:00
committed by Krrish Dholakia
parent fe3c57e1c1
commit 7d68536763
1 changed files with 11 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/docs/proxy/reliability.md
+11
View File
@@ -31,8 +31,19 @@ model_list:
api_base: https://openai-france-1234.openai.azure.com/
api_key: <your-azure-api-key>
rpm: 1440
routing_strategy: simple-shuffle # Literal["simple-shuffle", "least-busy", "usage-based-routing","latency-based-routing"], default="simple-shuffle"
model_group_alias: {"gpt-4": "gpt-3.5-turbo"} # all requests with `gpt-4` will be routed to models with `gpt-3.5-turbo`
num_retries: 2
timeout: 30 # 30 seconds
redis_host: <your redis host> # set this when using multiple litellm proxy deployments, load balancing state stored in redis
redis_password: <your redis password>
redis_port: 1992
```
:::info
Detailed information about [routing strategies can be found here](../routing)
:::
#### Step 2: Start Proxy with config
```shell