tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-15 12:27:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(caching.md): remove batch redis get recommendation - old code path, no longer necessary

Browse Source
This commit is contained in:
Krrish Dholakia
2025-06-12 19:59:45 -07:00
parent 61838bbbfc
commit bb75d7e240
1 changed files with 0 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/docs/proxy/caching.md
-27
View File
@@ -894,33 +894,6 @@ curl http://localhost:4000/v1/chat/completions \
</Tabs>
### Turn on `batch_redis_requests`
**What it does?**
When a request is made:
- Check if a key starting with `litellm:<hashed_api_key>:<call_type>:` exists in-memory, if no - get the last 100 cached requests for this key and store it
- New requests are stored with this `litellm:..` as the namespace
**Why?**
Reduce number of redis GET requests. This improved latency by 46% in prod load tests.
**Usage**
```yaml
litellm_settings:
cache: true
cache_params:
type: redis
... # remaining redis args (host, port, etc.)
callbacks: ["batch_redis_requests"] # 👈 KEY CHANGE!
```
[**SEE CODE**](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/hooks/batch_redis_get.py)
## Supported `cache_params` on proxy config.yaml
```yaml