litellm

mirror of https://github.com/tiennm99/litellm.git synced 2026-06-27 13:05:45 +00:00

Files

T

michelligabriele 8dc0c97958 fix(caching): check REDIS_CLUSTER_NODES env var in Cache and Router class selection (#22790 )

When Redis Cluster is configured via the REDIS_CLUSTER_NODES environment
variable, Cache.__init__() and Router._create_redis_cache() ignored the
env var and always created RedisCache instead of RedisClusterCache. This
caused the v3 rate limiter's cluster detection (_is_redis_cluster()) to
return False, skipping hash-slot key grouping. The resulting CROSSLOT
errors were silently caught, falling back to per-instance in-memory
counting — breaking RPM/TPM enforcement across multiple proxy instances.

Add REDIS_CLUSTER_NODES env var detection to both Cache.__init__() and
Router._create_redis_cache(), matching the existing pattern in
_redis.py:215-220. When the env var is set and no explicit startup_nodes
parameter is provided, parse it and create RedisClusterCache.

Fixes #22748
Related to #20836

2026-03-06 17:31:30 -08:00

test_azure_blob_cache.py

…

test_caching_handler.py

…

test_dual_cache.py

fix(cache): prevent DualCache async batch check-then-act race (#20986 )

2026-02-13 18:32:41 +05:30

test_gcs_cache.py

…

test_in_memory_cache.py

…