Krish Dholakia 552b7e4013 Add customer + model per key level multi-instance tpm/rpm limiting (#10518) ...

* fix(redis_cache.py): handle multiple event loops

* fix(parallel_request_limiter_v2.py): add customer tpm limiting

* fix(parallel_request_limiter.py): add customer rpm limiting

* fix(parallel_request_limiter_v2.py): add model per key + customer tpm/rpm limiting

* fix(parallel_request_limiter_v2.py): make error more informative

* fix: fix ruff error

* fix: generate new poetry lock

2025-05-03 10:28:55 -07:00

..

enterprise_hooks

Add customer + model per key level multi-instance tpm/rpm limiting (#10518)

2025-05-03 10:28:55 -07:00

conftest.py

Add key-level multi-instance tpm/rpm/max parallel request limiting (#10458)

2025-04-30 21:32:31 -07:00