diff --git a/docs/my-website/docs/proxy/deploy.md b/docs/my-website/docs/proxy/deploy.md
index 496bde05f1..655d93b6b0 100644
--- a/docs/my-website/docs/proxy/deploy.md
+++ b/docs/my-website/docs/proxy/deploy.md
@@ -28,7 +28,7 @@ docker run ghcr.io/berriai/litellm:main-latest
-### Run with LiteLLM CLI args
+#### Run with LiteLLM CLI args
See all supported CLI args [here](https://docs.litellm.ai/docs/proxy/cli):
@@ -129,13 +129,25 @@ spec:
name: litellm-config-file
```
-> [!TIP]
-> To avoid issues with predictability, difficulties in rollback, and inconsistent environments, use versioning or SHA digests (for example, `litellm:main-v1.30.3` or `litellm@sha256:12345abcdef...`) instead of `litellm:main-latest`.
+:::info
+To avoid issues with predictability, difficulties in rollback, and inconsistent environments, use versioning or SHA digests (for example, `litellm:main-v1.30.3` or `litellm@sha256:12345abcdef...`) instead of `litellm:main-latest`.
+:::
+**That's it ! That's the quick start to deploy litellm**
+
+## Options to deploy LiteLLM
+
+| Method | When to Use | Docs |
+| --- | --- | --- |
+| LiteLLM - Quick Start | call 100+ LLMs + Load Balancing | [Quick Start](#quick-start) |
+| LiteLLM Database container + PostgresDB | + use Virtual Keys + Track Spend | [Deploy with Database](#deploy-with-database) |
+| LiteLLM container + Redis | + load balance across multiple litellm containers | [LiteLLM container + Redis](#litellm-container--redis) |
+| LiteLLM Database container + PostgresDB + Redis | + use Virtual Keys + Track Spend + load balance across multiple litellm containers | [LiteLLM container + Redis](#litellm-container--redis) |
+
## Deploy with Database
We maintain a [seperate Dockerfile](https://github.com/BerriAI/litellm/pkgs/container/litellm-database) for reducing build time when running LiteLLM proxy with a connected Postgres Database
@@ -159,7 +171,7 @@ Your OpenAI proxy server is now running on `http://0.0.0.0:4000`.
-### Step 1. Create deployment.yaml
+#### Step 1. Create deployment.yaml
```yaml
apiVersion: apps/v1
@@ -188,7 +200,7 @@ Your OpenAI proxy server is now running on `http://0.0.0.0:4000`.
kubectl apply -f /path/to/deployment.yaml
```
-### Step 2. Create service.yaml
+#### Step 2. Create service.yaml
```yaml
apiVersion: v1
@@ -209,7 +221,7 @@ spec:
kubectl apply -f /path/to/service.yaml
```
-### Step 3. Start server
+#### Step 3. Start server
```
kubectl port-forward service/litellm-service 4000:4000
@@ -220,13 +232,13 @@ Your OpenAI proxy server is now running on `http://0.0.0.0:4000`.
-### Step 1. Clone the repository
+#### Step 1. Clone the repository
```bash
git clone https://github.com/BerriAI/litellm.git
```
-### Step 2. Deploy with Helm
+#### Step 2. Deploy with Helm
```bash
helm install \
@@ -235,7 +247,7 @@ helm install \
deploy/charts/litellm
```
-### Step 3. Expose the service to localhost
+#### Step 3. Expose the service to localhost
```bash
kubectl \
@@ -249,6 +261,38 @@ Your OpenAI proxy server is now running on `http://127.0.0.1:8000`.
+## LiteLLM container + Redis
+Use Redis when you need litellm to load balance across multiple litellm containers
+
+The only change required is setting Redis on your `config.yaml`
+LiteLLM Proxy supports sharing rpm/tpm shared across multiple litellm instances, pass `redis_host`, `redis_password` and `redis_port` to enable this. (LiteLLM will use Redis to track rpm/tpm usage )
+
+```yaml
+model_list:
+ - model_name: gpt-3.5-turbo
+ litellm_params:
+ model: azure/
+ api_base:
+ api_key:
+ rpm: 6 # Rate limit for this deployment: in requests per minute (rpm)
+ - model_name: gpt-3.5-turbo
+ litellm_params:
+ model: azure/gpt-turbo-small-ca
+ api_base: https://my-endpoint-canada-berri992.openai.azure.com/
+ api_key:
+ rpm: 6
+router_settings:
+ redis_host:
+ redis_password:
+ redis_port: 1992
+```
+
+Start docker container with config
+
+```shell
+docker run ghcr.io/berriai/litellm:main-latest --config your_config.yaml
+```
+
## Best Practices for Deploying to Production
### 1. Switch of debug logs in production
don't use [`--detailed-debug`, `--debug`](https://docs.litellm.ai/docs/proxy/debugging#detailed-debug) or `litellm.set_verbose=True`. We found using debug logs can add 5-10% latency per LLM API call
diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js
index 33720b43eb..62a2d38424 100644
--- a/docs/my-website/sidebars.js
+++ b/docs/my-website/sidebars.js
@@ -42,10 +42,6 @@ const sidebars = {
"proxy/team_based_routing",
"proxy/ui",
"proxy/budget_alerts",
- "proxy/model_management",
- "proxy/health",
- "proxy/debugging",
- "proxy/pii_masking",
{
"type": "category",
"label": "🔥 Load Balancing",
@@ -54,6 +50,10 @@ const sidebars = {
"proxy/reliability",
]
},
+ "proxy/model_management",
+ "proxy/health",
+ "proxy/debugging",
+ "proxy/pii_masking",
"proxy/caching",
{
"type": "category",