mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-10 05:15:19 +00:00
86ec2fbf84
Replace 11 separate asyncio.create_task() calls per request with a single batched task that runs all spend-update helpers sequentially. This reduces task scheduling overhead at high RPS (11,000 -> 1,000 tasks/sec at 1K RPS) and cuts 5 copy.deepcopy(payload) calls to 1 shared copy. Also fixes a mutation bug where the daily agent spend handler received the raw payload without deepcopy, unlike all other daily helpers.
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.