mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-27 02:28:09 +00:00
(docs) litellm semantic caching
This commit is contained in:
committed by
Krrish Dholakia
parent
f244e56893
commit
bf6e3f807b
@@ -104,7 +104,7 @@ litellm.cache = Cache(
|
||||
host=os.environ["REDIS_HOST"],
|
||||
port=os.environ["REDIS_PORT"],
|
||||
password=os.environ["REDIS_PASSWORD"],
|
||||
similarity_threshold=0.8,
|
||||
similarity_threshold=0.8, # similarity threshold for cache hits, 0 == no similarity, 1 = exact matches, 0.5 == 50% similarity
|
||||
redis_semantic_cache_embedding_model="text-embedding-ada-002", # this model is passed to litellm.embedding(), any litellm.embedding() model is supported here
|
||||
)
|
||||
response1 = completion(
|
||||
|
||||
@@ -7,6 +7,7 @@ Cache LLM Responses
|
||||
LiteLLM supports:
|
||||
- In Memory Cache
|
||||
- Redis Cache
|
||||
- Redis Semantic Cache
|
||||
- s3 Bucket Cache
|
||||
|
||||
## Quick Start - Redis, s3 Cache, Semantic Cache
|
||||
|
||||
Reference in New Issue
Block a user