From 5b1fda02fbbf1e82b1f8d85256dd7b4dd9d7398e Mon Sep 17 00:00:00 2001
From: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Date: Fri, 19 Dec 2025 13:44:36 -0800
Subject: [PATCH] Add infrastructure recommendations to benchmarks
 documentation (#18264)

Added concise PostgreSQL and Redis specifications based on benchmark results and industry standards for API gateway deployments. Includes tiered recommendations for different RPS workloads, configuration best practices, and scaling guidelines.
---
 docs/my-website/docs/benchmarks.md | 52 ++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/docs/my-website/docs/benchmarks.md b/docs/my-website/docs/benchmarks.md
index 76b61d4c2b..640212808b 100644
--- a/docs/my-website/docs/benchmarks.md
+++ b/docs/my-website/docs/benchmarks.md
@@ -60,6 +60,58 @@ Each machine deploying LiteLLM had the following specs:
 - Database: PostgreSQL
 - Redis: Not used
 
+## Infrastructure Recommendations
+
+Recommended specifications based on benchmark results and industry standards for API gateway deployments.
+
+### PostgreSQL
+
+Required for authentication, key management, and usage tracking.
+
+| Workload | CPU | RAM | Storage | Connections |
+|----------|-----|-----|---------|-------------|
+| 1-2K RPS | 4-8 cores | 16GB | 200GB SSD (3000+ IOPS) | 100-200 |
+| 2-5K RPS | 8 cores | 16-32GB | 500GB SSD (5000+ IOPS) | 200-500 |
+| 5K+ RPS | 16+ cores | 32-64GB | 1TB+ SSD (10000+ IOPS) | 500+ |
+
+**Configuration:** Set `proxy_batch_write_at: 60` to batch writes and reduce DB load. Total connections = pool limit × instances.
+
+### Redis (Recommended)
+
+Redis was not used in these benchmarks but provides significant production benefits: 60-80% reduced DB load.
+
+| Workload | CPU | RAM |
+|----------|-----|-----|
+| 1-2K RPS | 2-4 cores | 8GB |
+| 2-5K RPS | 4 cores | 16GB |
+| 5K+ RPS | 8+ cores | 32GB+ |
+
+**Requirements:** Redis 7.0+, AOF persistence enabled, `allkeys-lru` eviction policy.
+
+**Configuration:**
+```yaml
+router_settings:
+  redis_host: os.environ/REDIS_HOST
+  redis_port: os.environ/REDIS_PORT
+  redis_password: os.environ/REDIS_PASSWORD
+
+litellm_settings:
+  cache: True
+  cache_params:
+    type: redis
+    host: os.environ/REDIS_HOST
+    port: os.environ/REDIS_PORT
+    password: os.environ/REDIS_PASSWORD
+```
+
+:::tip
+Use `redis_host`, `redis_port`, and `redis_password` instead of `redis_url` for ~80 RPS better performance.
+:::
+
+**Scaling:** DB connections scale linearly with instances. Consider PostgreSQL read replicas beyond 5K RPS.
+
+See [Production Configuration](./proxy/prod) for detailed best practices.
+
 ## Locust Settings
 
 - 1000 Users