diff --git a/docs/my-website/docs/proxy/cost_tracking.md b/docs/my-website/docs/proxy/cost_tracking.md index 019ca3da12..19e3344f21 100644 --- a/docs/my-website/docs/proxy/cost_tracking.md +++ b/docs/my-website/docs/proxy/cost_tracking.md @@ -14,12 +14,10 @@ LiteLLM automatically tracks spend for all known models. See our [model cost map 👉 [Setup LiteLLM with a Database](https://docs.litellm.ai/docs/proxy/virtual_keys#setup) - **Step2** Send `/chat/completions` request - ```python @@ -38,7 +36,7 @@ response = client.chat.completions.create( } ], user="palantir", # OPTIONAL: pass user to track spend by user - extra_body={ + extra_body={ "metadata": { "tags": ["jobID:214590dsff09fds", "taskName:run_page_classification"] # ENTERPRISE: pass tags to track spend by tags } @@ -47,6 +45,7 @@ response = client.chat.completions.create( print(response) ``` + @@ -71,6 +70,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \ } }' ``` + @@ -131,7 +131,7 @@ The following spend gets tracked in Table `LiteLLM_SpendLogs` ```json { "api_key": "fe6b0cab4ff5a5a8df823196cc8a450*****", # Hash of API Key used - "user": "default_user", # Internal User (LiteLLM_UserTable) that owns `api_key=sk-1234`. + "user": "default_user", # Internal User (LiteLLM_UserTable) that owns `api_key=sk-1234`. "team_id": "e8d1460f-846c-45d7-9b43-55f3cc52ac32", # Team (LiteLLM_TeamTable) that owns `api_key=sk-1234` "request_tags": ["jobID:214590dsff09fds", "taskName:run_page_classification"],# Tags sent in request "end_user": "palantir", # Customer - the `user` sent in the request @@ -152,7 +152,7 @@ Navigate to the Usage Tab on the LiteLLM UI (found on https://your-proxy-endpoin -### Allowing Non-Proxy Admins to access `/spend` endpoints +### Allowing Non-Proxy Admins to access `/spend` endpoints Use this when you want non-proxy admins to access `/spend` endpoints @@ -162,8 +162,10 @@ Schedule a [meeting with us to get your Enterprise License](https://calendly.com ::: -##### Create Key -Create Key with with `permissions={"get_spend_routes": true}` +##### Create Key + +Create Key with with `permissions={"get_spend_routes": true}` + ```shell curl --location 'http://0.0.0.0:4000/key/generate' \ --header 'Authorization: Bearer sk-1234' \ @@ -176,22 +178,24 @@ curl --location 'http://0.0.0.0:4000/key/generate' \ ##### Use generated key on `/spend` endpoints Access spend Routes with newly generate keys + ```shell curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30' \ -H 'Authorization: Bearer sk-H16BKvrSNConSsBYLGc_7A' ``` - - #### Reset Team, API Key Spend - MASTER KEY ONLY Use `/global/spend/reset` if you want to: + - Reset the Spend for all API Keys, Teams. The `spend` for ALL Teams and Keys in `LiteLLM_TeamTable` and `LiteLLM_VerificationToken` will be set to `spend=0` - LiteLLM will maintain all the logs in `LiteLLMSpendLogs` for Auditing Purposes -##### Request +##### Request + Only the `LITELLM_MASTER_KEY` you set can access this route + ```shell curl -X POST \ 'http://localhost:4000/global/spend/reset' \ @@ -205,6 +209,68 @@ curl -X POST \ {"message":"Spend for all API Keys and Teams reset successfully","status":"success"} ``` +## Total spend per user + +Assuming you have been issuing keys for end users, and setting their `user_id` on the key, you can check their usage. + +```shell title="Total for a user API" showLineNumbers +curl -L -X GET 'http://localhost:4000/user/info?user_id=jane_smith' \ +-H 'Authorization: Bearer sk-...' +``` + +```json title="Total for a user API Response" showLineNumbers +{ + "user_id": "jane_smith", + "user_info": { + "spend": 0.1 + }, + "keys": [ + { + "token": "6e952b0efcafbb6350240db25ed534b4ec6011b3e1ba1006eb4f903461fd36f6", + "key_name": "sk-...KE_A", + "key_alias": "user-01882d6b-e090-776a-a587-21c63e502670-01983ddb-872f-71a3-8b3a-f9452c705483", + "soft_budget_cooldown": false, + "spend": 0.1, + "expires": "2025-07-31T19:14:13.968000+00:00", + "models": [], + "aliases": {}, + "config": {}, + "user_id": "01982d6b-e090-776a-a587-21c63e502660", + "team_id": "f2044fde-2293-482f-bf35-a8dab4e85c5f", + "permissions": {}, + "max_parallel_requests": null, + "metadata": {}, + "blocked": null, + "tpm_limit": null, + "rpm_limit": null, + "max_budget": null, + "budget_duration": null, + "budget_reset_at": null, + "allowed_cache_controls": [], + "allowed_routes": [], + "model_spend": {}, + "model_max_budget": {}, + "budget_id": null, + "organization_id": null, + "object_permission_id": null, + "created_at": "2025-07-24T19:14:13.970000Z", + "created_by": "582b168f-fc11-4e14-ad6a-cf4bb3656ddc", + "updated_at": "2025-07-24T19:14:13.970000Z", + "updated_by": "582b168f-fc11-4e14-ad6a-cf4bb3656ddc", + "litellm_budget_table": null, + "litellm_organization_table": null, + "object_permission": null, + "team_alias": null + } + ], + "teams": [] +} +``` + +**Warning** +End users can provide the `user` parameter in their request bodies, doing this will increment the cost reported via `/customer/info?end_user_id=self-declared-user`, and not for the user that owns the key as reported by that API. This means users could "avoid" having their spend tracked, through their method. +This means if you need to track user spend, and are giving end users API keys, you must always set user_id when creating their api keys, and use keys issued for that user every time you're making LLM calls on their behalf in backend services. This will track their spend. + ## Daily Spend Breakdown API Retrieve granular daily usage data for a user (by model, provider, and API key) with a single endpoint. @@ -257,16 +323,14 @@ See our [Swagger API](https://litellm-api.up.railway.app/#/Budget%20%26%20Spend% ## Custom Tags -Requirements: +Requirements: - Virtual Keys & a database should be set up, see [virtual keys](https://docs.litellm.ai/docs/proxy/virtual_keys) -**Note:** By default, LiteLLM will track `User-Agent` as a custom tag for cost tracking. This enables viewing usage for tools like Claude Code, Gemini CLI, etc. - +**Note:** By default, LiteLLM will track `User-Agent` as a custom tag for cost tracking. This enables viewing usage for tools like Claude Code, Gemini CLI, etc. - ### Client-side spend tag @@ -331,32 +395,32 @@ response = client.chat.completions.create( print(response) ``` - + ```js -const openai = require('openai'); +const openai = require("openai"); async function runOpenAI() { const client = new openai.OpenAI({ - apiKey: 'sk-1234', - baseURL: 'http://0.0.0.0:4000' + apiKey: "sk-1234", + baseURL: "http://0.0.0.0:4000", }); try { const response = await client.chat.completions.create({ - model: 'gpt-3.5-turbo', + model: "gpt-3.5-turbo", messages: [ { - role: 'user', - content: "this is a test request, write a short poem" + role: "user", + content: "this is a test request, write a short poem", }, ], metadata: { - tags: ["model-anthropic-claude-v2.1", "app-ishaan-prod"] // 👈 Key Change - } + tags: ["model-anthropic-claude-v2.1", "app-ishaan-prod"], // 👈 Key Change + }, }); console.log(response); } catch (error) { @@ -368,6 +432,7 @@ async function runOpenAI() { // Call the asynchronous function runOpenAI(); ``` + @@ -388,6 +453,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \ "metadata": {"tags": ["model-anthropic-claude-v2.1", "app-ishaan-prod"]} }' ``` + @@ -427,8 +493,6 @@ print(response) - - ### Add custom headers to spend tracking You can add custom headers to the request to track spend and usage. @@ -447,7 +511,8 @@ You can disable user-agent tracking by setting `litellm_settings.disable_user_ag litellm_settings: disable_user_agent_tracking: true ``` -## ✨ (Enterprise) Generate Spend Reports + +## ✨ (Enterprise) Generate Spend Reports Use this to charge other teams, customers, users @@ -467,6 +532,7 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end ``` #### Example Response + @@ -511,7 +577,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end ] ``` - @@ -548,6 +613,7 @@ for row in spend_report: ``` Output from script + ```shell # Date: 2024-05-11T00:00:00+00:00 # Team: local_test_team @@ -570,21 +636,19 @@ Output from script # Metadata: [{'model': 'gpt-3.5-turbo', 'spend': 0.0005715000000000001, 'api_key': 'b94d5e0bc3a71a573917fe1335dc0c14728c7016337451af9714924ff3a729db', 'total_tokens': 423}] ``` - - :::info Customer [this is `user` passed to `/chat/completions` request](#how-to-track-spend-with-litellm) -- [LiteLLM API key](virtual_keys.md) +- [LiteLLM API key](virtual_keys.md) ::: @@ -592,7 +656,6 @@ Customer [this is `user` passed to `/chat/completions` request](#how-to-track-sp 👉 Key Change: Specify `group_by=customer` - ```shell curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30&group_by=customer' \ -H 'Authorization: Bearer sk-1234' @@ -600,7 +663,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end #### Example Response - ```shell [ { @@ -641,15 +703,12 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end ] ``` - - 👉 Key Change: Specify `api_key=sk-1234` - ```shell curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30&api_key=sk-1234' \ -H 'Authorization: Bearer sk-1234' @@ -657,7 +716,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end #### Example Response - ```shell [ { @@ -693,10 +751,8 @@ Internal User (Key Owner): This is the value of `user_id` passed when calling [` ::: - 👉 Key Change: Specify `internal_user_id=ishaan` - ```shell curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-12-30&internal_user_id=ishaan' \ -H 'Authorization: Bearer sk-1234' @@ -704,7 +760,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end #### Example Response - ```shell [ { @@ -768,46 +823,43 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end - ## 📊 Spend Logs API - Individual Transaction Logs The `/spend/logs` endpoint now supports a `summarize` parameter to control data format when using date filters. ### Key Parameters -| Parameter | Description | -|-----------|-------------| +| Parameter | Description | +| ----------- | -------------------------------------------------------------------------------------------- | | `summarize` | **New parameter**: `true` (default) = aggregated data, `false` = individual transaction logs | ### Examples **Get individual transaction logs:** + ```bash curl -X GET "http://localhost:4000/spend/logs?start_date=2024-01-01&end_date=2024-01-02&summarize=false" \ -H "Authorization: Bearer sk-1234" ``` **Get summarized data (default):** + ```bash curl -X GET "http://localhost:4000/spend/logs?start_date=2024-01-01&end_date=2024-01-02" \ -H "Authorization: Bearer sk-1234" ``` **Use Cases:** + - `summarize=false`: Analytics dashboards, ETL processes, detailed audit trails - `summarize=true`: Daily spending reports, high-level cost tracking (legacy behavior) - ## ✨ Custom Spend Log metadata Log specific key,value pairs as part of the metadata for a spend log -:::info +:::info Logging specific key,value pairs in spend logs metadata is an enterprise feature. [See here](./enterprise.md#tracking-spend-with-custom-metadata) ::: - - - -