Krish Dholakia 39849627f7 feat(parallel_request_limiter_v2.py): add sliding window logic (#11283) ...

* feat(parallel_request_limiter_v2.py): add sliding window logic

allows rate limiting to work across minutes

* fix(parallel_request_limiter_v2.py): decrement usage on rate limit error

* fix(base_routing_strategy.py): fix merge from redis - preserve values in in-memory cache during gap b/w push to redis and read from redis

* fix(base_routing_strategy.py): catch the delta change during redis sync

ensures values are kept in sync

* fix(parallel_request_limiter_v2.py): update tpm tracking to use slot key logic

* fix: fix linting error

* test: update testing

* test: update tests

* test: skip on rate limit or internal server errors

* test: use pytest fixture instead

* test: bump mistral model

2025-05-31 10:06:42 -07:00

..

test_parallel_request_limiter_v2.py

feat(parallel_request_limiter_v2.py): add sliding window logic (#11283)

2025-05-31 10:06:42 -07:00

test_proxy_track_cost_callback.py

Litellm fix GitHub action testing (#11163)

2025-05-26 14:41:42 -07:00