Yuneng Jiang
60d20ace5b
[Fix] test_ratelimit: skip over-limit cases that race with background RPM tracking
...
usage-based-routing tracks RPM in log_success_event which runs in a
background ThreadPoolExecutor. The cache update races with the next
call's routing check in both sync (tight loop) and async (concurrent
gather) modes, making over-limit detection non-deterministic.
Skip the over-limit parametrization (num_try_send > num_allowed_send).
The under-limit cases (2 sent, 3 allowed) still verify the routing
strategy works. Rate-limit enforcement is properly tested with mocks
in tests/test_litellm/test_router/test_enforce_model_rate_limits.py.
2026-04-11 13:08:59 -07:00
Ishaan Jaffer
c587aeaf3a
test_async_rate_limit
2025-11-15 10:34:24 -08:00
Ishaan Jaff and GitHub
8a235e7d38
(Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added ( #8112 )
...
* LoggingCallbackManager
* add logging_callback_manager
* use logging_callback_manager
* add add_litellm_failure_callback
* use add_litellm_callback
* use add_litellm_async_success_callback
* add_litellm_async_failure_callback
* linting fix
* fix logging callback manager
* test_duplicate_multiple_loggers_test
* use _reset_all_callbacks
* fix testing with dup callbacks
* test_basic_image_generation
* reset callbacks for tests
* fix check for _add_custom_logger_to_list
* fix test_amazing_sync_embedding
* fix _get_custom_logger_key
* fix batches testing
* fix _reset_all_callbacks
* fix _check_callback_list_size
* add callback_manager_test
* fix test gemini-2.0-flash-thinking-exp-01-21
2025-01-30 19:35:50 -08:00
Krrish Dholakia
e2eddac406
test(test_ratelimit.py): fix test to send below rpm
2024-04-30 19:35:21 -07:00
Krrish Dholakia
0d1cca9aa0
fix(router.py): make router async calls coroutine safe
...
uses pre-call checks to check if a call is below it's rpm limit, works even if multiple async calls are
made simultaneously
2024-04-06 17:31:26 -07:00
CLARKBENHAM
44cb0f352a
formating
2024-04-02 19:56:07 -07:00
CLARKBENHAM
6b0f3d3fd3
add routing strategy
2024-04-02 17:26:57 -07:00
CLARKBENHAM
2d7bafa12d
comments
2024-04-02 17:18:56 -07:00
CLARKBENHAM
75d029a91d
comments
2024-04-02 17:18:03 -07:00
CLARKBENHAM
f32d39aa93
check if in future None results are returned
2024-04-02 14:49:20 -07:00
CLARKBENHAM
a2bf891e48
test for rate limits
2024-04-02 13:49:13 -07:00
Yuneng Jiang