diff --git a/docs/my-website/docs/proxy/team_budgets.md b/docs/my-website/docs/proxy/team_budgets.md index 854d6edf30..66ba679c65 100644 --- a/docs/my-website/docs/proxy/team_budgets.md +++ b/docs/my-website/docs/proxy/team_budgets.md @@ -4,6 +4,12 @@ import TabItem from '@theme/TabItem'; # Setting Team Budgets + +# Pre-Requisites + +- You must set up a Postgres database (e.g. Supabase, Neon, etc.) +- To enable team member rate limits, set the environment variable `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING=true` **before starting the proxy server**. Without this, team member rate limits will not be enforced. + Track spend, set budgets for your Internal Team ## Setting Monthly Team Budgets diff --git a/docs/my-website/docs/proxy/users.md b/docs/my-website/docs/proxy/users.md index c812dccb19..d098e38de4 100644 --- a/docs/my-website/docs/proxy/users.md +++ b/docs/my-website/docs/proxy/users.md @@ -58,6 +58,9 @@ You can: **Step-by step tutorial on setting, resetting budgets on Teams here (API or using Admin UI)** +> **Prerequisite:** +> To enable team member rate limits, you must set the environment variable `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING=true` before starting the proxy server. Without this, team member rate limits will not be enforced. + 👉 [https://docs.litellm.ai/docs/proxy/team_budgets](https://docs.litellm.ai/docs/proxy/team_budgets) ::: @@ -793,6 +796,11 @@ Expected Response: Enable multi-instance rate limiting with the env var `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING="True"` +**Important Notes:** +- Setting `EXPERIMENTAL_MULTI_INSTANCE_RATE_LIMITING="True"` is required for team member rate limits to function, not just for multi-instance scenarios. +- **Rate limits do not apply to proxy admin users.** +- When testing rate limits, use internal user roles (non-admin) to ensure limits are enforced as expected. + Changes: - This moves to using async_increment instead of async_set_cache when updating current requests/tokens. - The in-memory cache is synced with redis every 0.01s, to avoid calling redis for every request.