From 73691fb373ed4aa82d42156fd00096a4d64c07e0 Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Mon, 2 Feb 2026 11:32:00 -0800 Subject: [PATCH] Model request tags documentation (#20290) * Add request tags documentation for spend tracking - Add new concise doc explaining how to tag model requests - Include Python SDK and cURL examples - Show where tags appear in spend logs - Add common use cases table (AWS accounts, teams, projects) - Include how to set default tags on API keys - Add to Spend Tracking section in sidebar Co-authored-by: ishaan * Simplify request tags doc for AI Gateway usage - Focus on config.yaml setup with default_key_generate_params - Show both request body and header methods for sending tags - Remove SDK examples, keep concise cURL examples - Streamline for quick reference Co-authored-by: ishaan * Update request tags doc to show model-level config - Set tags directly on model deployments in litellm_params - Requests just specify model, tags applied automatically - Use clear naming: AWS_IAM_PROD, AWS_IAM_DEV Co-authored-by: ishaan --------- Co-authored-by: Cursor Agent Co-authored-by: ishaan --- docs/my-website/docs/proxy/request_tags.md | 58 ++++++++++++++++++++++ docs/my-website/sidebars.js | 1 + 2 files changed, 59 insertions(+) create mode 100644 docs/my-website/docs/proxy/request_tags.md diff --git a/docs/my-website/docs/proxy/request_tags.md b/docs/my-website/docs/proxy/request_tags.md new file mode 100644 index 0000000000..c78c48229b --- /dev/null +++ b/docs/my-website/docs/proxy/request_tags.md @@ -0,0 +1,58 @@ +# Request Tags for Spend Tracking + +Add tags to model deployments to track spend by environment, AWS account, or any custom label. + +Tags appear in the `request_tags` field of LiteLLM spend logs. + +## Config Setup + +Set tags on model deployments in `config.yaml`: + +```yaml title="config.yaml" +model_list: + - model_name: gpt-4 + litellm_params: + model: azure/gpt-4-prod + api_key: os.environ/AZURE_PROD_API_KEY + api_base: https://prod.openai.azure.com/ + tags: ["AWS_IAM_PROD"] # 👈 Tag for production + + - model_name: gpt-4-dev + litellm_params: + model: azure/gpt-4-dev + api_key: os.environ/AZURE_DEV_API_KEY + api_base: https://dev.openai.azure.com/ + tags: ["AWS_IAM_DEV"] # 👈 Tag for development +``` + +## Make Request + +Requests just specify the model - tags are automatically applied: + +```bash +curl -X POST 'http://0.0.0.0:4000/chat/completions' \ + -H 'Authorization: Bearer sk-1234' \ + -H 'Content-Type: application/json' \ + -d '{ + "model": "gpt-4", + "messages": [{"role": "user", "content": "Hello"}] + }' +``` + +## Spend Logs + +The tag from the model config appears in `LiteLLM_SpendLogs`: + +```json +{ + "request_id": "chatcmpl-abc123", + "request_tags": ["AWS_IAM_PROD"], + "spend": 0.002, + "model": "gpt-4" +} +``` + +## Related + +- [Spend Tracking Overview](cost_tracking.md) +- [Tag Budgets](tag_budgets.md) - Set budget limits per tag diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js index 95a4412837..a9248d83dd 100644 --- a/docs/my-website/sidebars.js +++ b/docs/my-website/sidebars.js @@ -442,6 +442,7 @@ const sidebars = { label: "Spend Tracking", items: [ "proxy/cost_tracking", + "proxy/request_tags", "proxy/custom_pricing", "proxy/pricing_calculator", "proxy/provider_margins",