diff --git a/docs/my-website/docs/proxy/db_deadlocks.md b/docs/my-website/docs/proxy/db_deadlocks.md index 39498a7ec1..0eb22eecfe 100644 --- a/docs/my-website/docs/proxy/db_deadlocks.md +++ b/docs/my-website/docs/proxy/db_deadlocks.md @@ -8,34 +8,49 @@ Resolve any Database Deadlocks you see in high traffic by using this setup ## What causes the problem? -LiteLLM writes `UPDATE` and `UPSERT` queries to the DB. When using 10+ pods of LiteLLM, these queries can cause deadlocks since each pod could simultaneously attempt to update the same `user_id`, `team_id`, `key` etc. +LiteLLM writes `UPDATE` and `UPSERT` queries to the DB. When using 10+ instances of LiteLLM, these queries can cause deadlocks since each instance could simultaneously attempt to update the same `user_id`, `team_id`, `key` etc. ## How the high availability setup fixes the problem -- All pods will write to a Redis queue instead of the DB. -- A single pod will acquire a lock on the DB and flush the redis queue to the DB. +- All instances will write to a Redis queue instead of the DB. +- A single instance will acquire a lock on the DB and flush the redis queue to the DB. ## How it works -### Stage 1. Each pod writes updates to redis +### Stage 1. Each instance writes updates to redis -Each pod will accumlate the spend updates for a key, user, team, etc and write the updates to a redis queue. +Each instance will accumlate the spend updates for a key, user, team, etc and write the updates to a redis queue. +

+Each instance writes updates to redis +

-### Stage 2. A single pod flushes the redis queue to the DB +### Stage 2. A single instance flushes the redis queue to the DB -A single pod will acquire a lock on the DB and flush all elements in the redis queue to the DB. +A single instance will acquire a lock on the DB and flush all elements in the redis queue to the DB. +

+A single instance flushes the redis queue to the DB +

-## Setup +## Usage + +## Required components + - Redis - Postgres +### Setup on LiteLLM config + +You can enable using the redis buffer by setting `use_redis_transaction_buffer: true` in the `general_settings` section of your `proxy_config.yaml` file. + +Note: This setup requires a redis instance to be running. + ```yaml showLineNumbers title="litellm proxy_config.yaml" general_settings: use_redis_transaction_buffer: true @@ -44,5 +59,5 @@ litellm_settings: cache: True cache_params: type: redis - supported_call_types: [] + supported_call_types: [] # Optional: Set cache for proxy, but not on the actual llm api call ```