(docs) litellm semantic caching

This commit is contained in:
ishaan-jaff
2024-02-06 13:32:03 -08:00
committed by Krrish Dholakia
parent f244e56893
commit bf6e3f807b
2 changed files with 2 additions and 1 deletions
+1 -1
View File
@@ -104,7 +104,7 @@ litellm.cache = Cache(
host=os.environ["REDIS_HOST"],
port=os.environ["REDIS_PORT"],
password=os.environ["REDIS_PASSWORD"],
similarity_threshold=0.8,
similarity_threshold=0.8, # similarity threshold for cache hits, 0 == no similarity, 1 = exact matches, 0.5 == 50% similarity
redis_semantic_cache_embedding_model="text-embedding-ada-002", # this model is passed to litellm.embedding(), any litellm.embedding() model is supported here
)
response1 = completion(
+1
View File
@@ -7,6 +7,7 @@ Cache LLM Responses
LiteLLM supports:
- In Memory Cache
- Redis Cache
- Redis Semantic Cache
- s3 Bucket Cache
## Quick Start - Redis, s3 Cache, Semantic Cache