tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-16 00:23:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs fix

Browse Source
This commit is contained in:
Ishaan Jaffer
2025-10-04 17:29:06 -07:00
parent 8c495590b1
commit 02cb7a4c3b
1 changed files with 0 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/release_notes/v1.77.5-stable/index.md
-2
View File
@@ -61,8 +61,6 @@ pip install litellm==1.77.5
Throughput increased by 54% (1,040 → 1,602 RPS, aggregated) per instance while maintaining a 40 ms median overhead. The improvement comes from fixing major O(n²) inefficiencies in the router, primarily caused by repeated use of in statements inside loops over large arrays. Tests were run with a database-only setup (no cache hits). As a result, p95 latency improved by 30% (2,700 → 1,900 ms), enhancing overall stability and scalability under heavy load.
---
### Test Setup
All benchmarks were executed using Locust with 1,000 concurrent users and a ramp-up of 500. The environment was configured to stress the routing layer and eliminate caching as a variable.