mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-20 16:19:55 +00:00
jimmychen-p72andGitHub
2fe615b373
* fix(s3): add retry with exponential backoff for transient S3 503/500 errors S3 occasionally returns 503 "Slow Down" during PUT operations when request rates spike above partition limits. The current code makes a single upload attempt via httpx — unlike boto3, httpx has no built-in retry for transient S3 errors. Failed uploads permanently lose the request's audit/logging data. Add exponential backoff retry (3 attempts, 1s/2s delays) for S3 500/503 responses in both async_upload_data_to_s3 and upload_data_to_s3. Logs a warning on each retry with the S3 object key for observability. In production we observed ~18 permanent S3 upload failures per day (124 over 7 days) — all transient 503s that would have succeeded on a single retry. * test(s3): add unit tests for S3 upload retry logic Tests cover: - Async retry on 503 (succeeds on second attempt) - Async retry on 500 - Exhausted retries on persistent 503 (calls handle_callback_failure) - No retry on 4xx errors (403) - Sync retry on 503 * style(s3): move time import to module level Address review feedback: move `import time` from inside upload_data_to_s3 to the top-level imports per project style guide.
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.