mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-13 14:23:04 +00:00
Merge pull request #13899 from TeddyAmkie/docs/team-rate-limits-update
docs: clarify prerequisites and env var for team rate limits
This commit is contained in:
@@ -4,6 +4,12 @@ import TabItem from '@theme/TabItem';
|
||||
|
||||
# Setting Team Budgets
|
||||
|
||||
|
||||
# Pre-Requisites
|
||||
|
||||
- You must set up a Postgres database (e.g. Supabase, Neon, etc.)
|
||||
- To enable team member rate limits, set the environment variable `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING=true` **before starting the proxy server**. Without this, team member rate limits will not be enforced.
|
||||
|
||||
Track spend, set budgets for your Internal Team
|
||||
|
||||
## Setting Monthly Team Budgets
|
||||
|
||||
@@ -58,6 +58,9 @@ You can:
|
||||
|
||||
**Step-by step tutorial on setting, resetting budgets on Teams here (API or using Admin UI)**
|
||||
|
||||
> **Prerequisite:**
|
||||
> To enable team member rate limits, you must set the environment variable `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING=true` before starting the proxy server. Without this, team member rate limits will not be enforced.
|
||||
|
||||
👉 [https://docs.litellm.ai/docs/proxy/team_budgets](https://docs.litellm.ai/docs/proxy/team_budgets)
|
||||
|
||||
:::
|
||||
@@ -793,6 +796,11 @@ Expected Response:
|
||||
|
||||
Enable multi-instance rate limiting with the env var `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING="True"`
|
||||
|
||||
**Important Notes:**
|
||||
- Setting `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING="True"` is required for team member rate limits to function, not just for multi-instance scenarios.
|
||||
- **Rate limits do not apply to proxy admin users.**
|
||||
- When testing rate limits, use internal user roles (non-admin) to ensure limits are enforced as expected.
|
||||
|
||||
Changes:
|
||||
- This moves to using async_increment instead of async_set_cache when updating current requests/tokens.
|
||||
- The in-memory cache is synced with redis every 0.01s, to avoid calling redis for every request.
|
||||
|
||||
Reference in New Issue
Block a user