diff --git a/docs/my-website/docs/proxy/alerting.md b/docs/my-website/docs/proxy/alerting.md index 5584267569..08030f4784 100644 --- a/docs/my-website/docs/proxy/alerting.md +++ b/docs/my-website/docs/proxy/alerting.md @@ -32,41 +32,33 @@ Get a slack webhook url from https://api.slack.com/messaging/webhooks You can also use Discord Webhooks, see [here](#using-discord-webhooks) -### Step 2: Update config.yaml -- Set `SLACK_WEBHOOK_URL` in your proxy env to enable Slack alerts. -- Just for testing purposes, let's save a bad key to our proxy. +Set `SLACK_WEBHOOK_URL` in your proxy env to enable Slack alerts. + +```bash +export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/<>/<>/<>" +``` + +### Step 2: Setup Proxy ```yaml -model_list: - model_name: "azure-model" - litellm_params: - model: "azure/gpt-35-turbo" - api_key: "my-bad-key" # 👈 bad key - general_settings: alerting: ["slack"] alerting_threshold: 300 # sends alerts if requests hang for 5min+ and responses take 5min+ - -environment_variables: - SLACK_WEBHOOK_URL: "https://hooks.slack.com/services/<>/<>/<>" - SLACK_DAILY_REPORT_FREQUENCY: "86400" # 24 hours; Optional: defaults to 12 hours ``` - -### Step 3: Start proxy - +Start proxy ```bash $ litellm --config /path/to/config.yaml ``` -## Testing Alerting is Setup Correctly -Make a GET request to `/health/services`, expect to see a test slack alert in your provided webhook slack channel +### Step 3: Test it! -```shell -curl -X GET 'http://localhost:4000/health/services?service=slack' \ - -H 'Authorization: Bearer sk-1234' + +```bash +curl -X GET 'http://0.0.0.0:4000/health/services?service=slack' \ +-H 'Authorization: Bearer sk-1234' ``` ## Advanced - Redacting Messages from Alerts @@ -84,7 +76,34 @@ litellm_settings: ``` +## Advanced - Add Metadata to alerts +Add alerting metadata to proxy calls for debugging. + +```python +import openai +client = openai.OpenAI( + api_key="anything", + base_url="http://0.0.0.0:4000" +) + +# request sent to model set on litellm proxy, `litellm --model` +response = client.chat.completions.create( + model="gpt-3.5-turbo", + messages = [], + extra_body={ + "metadata": { + "alerting_metadata": { + "hello": "world" + } + } + } +) +``` + +**Expected Response** + + ## Advanced - Opting into specific alert types diff --git a/docs/my-website/img/alerting_metadata.png b/docs/my-website/img/alerting_metadata.png new file mode 100644 index 0000000000..e75f0c72bf Binary files /dev/null and b/docs/my-website/img/alerting_metadata.png differ