From 816c925bdd955d58415a7b4e06eac40f70e58762 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Sat, 13 Apr 2024 22:25:26 -0700 Subject: [PATCH] docs(grafana_metrics.md): update for monitoring system health --- docs/my-website/docs/proxy/grafana_metrics.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/my-website/docs/proxy/grafana_metrics.md b/docs/my-website/docs/proxy/grafana_metrics.md index df92cd2853..c191aa1fc5 100644 --- a/docs/my-website/docs/proxy/grafana_metrics.md +++ b/docs/my-website/docs/proxy/grafana_metrics.md @@ -51,5 +51,21 @@ http://localhost:4000/metrics | `litellm_requests_metric` | Number of requests made, per `"user", "key", "model"` | | `litellm_spend_metric` | Total Spend, per `"user", "key", "model"` | | `litellm_total_tokens` | input + output tokens per `"user", "key", "model"` | + +## Monitor System Health + +To monitor the health of litellm adjacent services (redis / postgres), do: + +```yaml +model_list: + - model_name: gpt-3.5-turbo + litellm_params: + model: gpt-3.5-turbo +litellm_settings: + service_callback: ["prometheus_system"] +``` + +| Metric Name | Description | +|----------------------|--------------------------------------| | `litellm_redis_latency` | histogram latency for redis calls | | `litellm_redis_fails` | Number of failed redis calls |