mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-09 03:08:09 +00:00
docs(caching.md): add doc on enabling caching for just rate limiting features
This commit is contained in:
@@ -185,6 +185,8 @@ $ litellm --config /path/to/config.yaml
|
||||
</Tabs>
|
||||
|
||||
|
||||
|
||||
|
||||
## Using Caching - /chat/completions
|
||||
|
||||
<Tabs>
|
||||
@@ -230,6 +232,22 @@ curl --location 'http://0.0.0.0:4000/embeddings' \
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Set cache for proxy, but not on the actual llm api call
|
||||
|
||||
Use this if you just want to enable features like rate limiting, and loadbalancing across multiple instances.
|
||||
|
||||
Set `supported_call_types: []` to disable caching on the actual api call.
|
||||
|
||||
|
||||
```yaml
|
||||
litellm_settings:
|
||||
cache: True
|
||||
cache_params:
|
||||
type: redis
|
||||
supported_call_types: []
|
||||
```
|
||||
|
||||
|
||||
## Debugging Caching - `/cache/ping`
|
||||
LiteLLM Proxy exposes a `/cache/ping` endpoint to test if the cache is working as expected
|
||||
|
||||
|
||||
Reference in New Issue
Block a user