From bb75d7e240d11f75fa816210f7439d401646e839 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Thu, 12 Jun 2025 19:58:38 -0700 Subject: [PATCH] docs(caching.md): remove batch redis get recommendation - old code path, no longer necessary --- docs/my-website/docs/proxy/caching.md | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/docs/my-website/docs/proxy/caching.md b/docs/my-website/docs/proxy/caching.md index 84e8c5f8d5..aec734e914 100644 --- a/docs/my-website/docs/proxy/caching.md +++ b/docs/my-website/docs/proxy/caching.md @@ -894,33 +894,6 @@ curl http://localhost:4000/v1/chat/completions \ - - -### Turn on `batch_redis_requests` - -**What it does?** -When a request is made: - -- Check if a key starting with `litellm:::` exists in-memory, if no - get the last 100 cached requests for this key and store it - -- New requests are stored with this `litellm:..` as the namespace - -**Why?** -Reduce number of redis GET requests. This improved latency by 46% in prod load tests. - -**Usage** - -```yaml -litellm_settings: - cache: true - cache_params: - type: redis - ... # remaining redis args (host, port, etc.) - callbacks: ["batch_redis_requests"] # 👈 KEY CHANGE! -``` - -[**SEE CODE**](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/hooks/batch_redis_get.py) - ## Supported `cache_params` on proxy config.yaml ```yaml