diff --git a/docs/my-website/docs/proxy/customer_usage.md b/docs/my-website/docs/proxy/customer_usage.md new file mode 100644 index 0000000000..8e366586b1 --- /dev/null +++ b/docs/my-website/docs/proxy/customer_usage.md @@ -0,0 +1,110 @@ +import Image from '@theme/IdealImage'; +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +# Customer Usage + +Track and visualize end-user spend directly in the dashboard. Monitor customer-level usage analytics, spend logs, and activity metrics to understand how your customers are using your LLM services. + +This feature is **available in v1.80.8-stable and above**. + +## Overview + +Customer Usage enables you to track spend and usage for individual customers (end users) by passing an ID in your API requests. This allows you to: + +- Track spend per customer automatically +- View customer-level usage analytics in the Admin UI +- Filter spend logs and activity metrics by customer ID +- Set budgets and rate limits per customer +- Monitor customer usage patterns and trends + + + +## How to Track Spend + +Track customer spend by including a `user` field in your API requests. The customer ID will be automatically tracked and associated with all spend from that request. + +### Example using cURL + +Make a `/chat/completions` call with the `user` field containing your customer ID: + +```bash showLineNumbers title="Track spend with customer ID" +curl -X POST 'http://0.0.0.0:4000/chat/completions' \ + --header 'Content-Type: application/json' \ + --header 'Authorization: Bearer sk-1234' \ # 👈 YOUR PROXY KEY + --data '{ + "model": "gpt-3.5-turbo", + "user": "customer-123", # 👈 CUSTOMER ID + "messages": [ + { + "role": "user", + "content": "What is the capital of France?" + } + ] + }' +``` + +The customer ID (`customer-123`) will be automatically upserted into the database with the new spend. If the customer ID already exists, spend will be incremented. + +### Example using OpenWebUI + +See the [Open WebUI tutorial](../tutorials/openweb_ui.md) for detailed instructions on connecting Open WebUI to LiteLLM and tracking customer usage. + +## How to View Spend + +### View Spend in Admin UI + +Navigate to the Customer Usage tab in the Admin UI to view customer-level spend analytics: + +#### 1. Access Customer Usage + +Go to the Usage page in the Admin UI (`PROXY_BASE_URL/ui/?login=success&page=new_usage`) and click on the **Customer Usage** tab. + + + +#### 2. View Customer Analytics + +The Customer Usage dashboard provides: + +- **Total spend per customer**: View aggregated spend across all customers +- **Daily spend trends**: See how customer spend changes over time +- **Model usage breakdown**: Understand which models each customer uses +- **Activity metrics**: Track requests, tokens, and success rates per customer + + + +#### 3. Filter by Customer + +Use the customer filter dropdown to view spend for specific customers: + +- Select one or more customer IDs from the dropdown +- View filtered analytics, spend logs, and activity metrics +- Compare spend across different customers + + + +## Use Cases + +### Customer Billing + +Track spend per customer to accurately bill your end users: + +- Monitor individual customer usage +- Generate invoices based on actual spend +- Set spending limits per customer + +### Usage Analytics + +Understand how different customers use your service: + +- Identify high-value customers +- Analyze usage patterns +- Optimize resource allocation + +--- + +## Related Features + +- [Customers / End-User Budgets](./customers.md) - Set budgets and rate limits for customers +- [Cost Tracking](./cost_tracking.md) - Comprehensive cost tracking and analytics +- [Billing](./billing.md) - Bill customers based on their usage diff --git a/docs/my-website/img/customer_usage.png b/docs/my-website/img/customer_usage.png new file mode 100644 index 0000000000..8e601c1f33 Binary files /dev/null and b/docs/my-website/img/customer_usage.png differ diff --git a/docs/my-website/img/customer_usage_analytics.png b/docs/my-website/img/customer_usage_analytics.png new file mode 100644 index 0000000000..443337d383 Binary files /dev/null and b/docs/my-website/img/customer_usage_analytics.png differ diff --git a/docs/my-website/img/customer_usage_filter.png b/docs/my-website/img/customer_usage_filter.png new file mode 100644 index 0000000000..d544cd9b25 Binary files /dev/null and b/docs/my-website/img/customer_usage_filter.png differ diff --git a/docs/my-website/img/customer_usage_ui_navigation.png b/docs/my-website/img/customer_usage_ui_navigation.png new file mode 100644 index 0000000000..2c92f7303b Binary files /dev/null and b/docs/my-website/img/customer_usage_ui_navigation.png differ diff --git a/docs/my-website/release_notes/v1.80.8-stable/index.md b/docs/my-website/release_notes/v1.80.8-stable/index.md index 3cf4aab61e..69ec590635 100644 --- a/docs/my-website/release_notes/v1.80.8-stable/index.md +++ b/docs/my-website/release_notes/v1.80.8-stable/index.md @@ -47,7 +47,7 @@ pip install litellm==1.80.8 - **Agent Gateway (A2A)** - [Invoke agents through the AI Gateway with request/response logging and access controls](../../docs/a2a) - **Guardrails API v2** - [Generic Guardrail API with streaming support, structured messages, and tool call checks](../../docs/adding_provider/generic_guardrail_api) -- **Customer (End User) Usage UI** - [Track and visualize end-user spend directly in the dashboard](../../docs/proxy/users) +- **Customer (End User) Usage UI** - [Track and visualize end-user spend directly in the dashboard](../../docs/proxy/customer_usage) - **vLLM Batch + Files API** - [Support for batch and files API with vLLM deployments](../../docs/batches) - **Dynamic Rate Limiting on Teams** - [Enable dynamic rate limits and priority reservation on team-level](../../docs/proxy/team_budgets) - **Google Cloud Chirp3 HD** - [New text-to-speech provider with Chirp3 HD voices](../../docs/text_to_speech) @@ -91,6 +91,23 @@ Get started with Agent Gateway here: [Agent Gateway Documentation](../../docs/a2 --- +### Customer (End User) Usage UI + + + +Users can now filter usage statistics by customers, providing the same granular filtering capabilities available for teams and organizations. + +**Details:** + +- Filter usage analytics, spend logs, and activity metrics by customer ID +- View customer-level breakdowns alongside existing team and user-level filters +- Consistent filtering experience across all usage and analytics views + +--- + ## New Providers and Endpoints ### New Providers (5 new providers)