mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-11 00:25:15 +00:00
chore: Improve docs for cost tracking (#12976)
This commit is contained in:
Richard Tweed
@@ -14,12 +14,10 @@ LiteLLM automatically tracks spend for all known models. See our [model cost map
|
||||
|
||||
👉 [Setup LiteLLM with a Database](https://docs.litellm.ai/docs/proxy/virtual_keys#setup)
|
||||
|
||||
|
||||
**Step2** Send `/chat/completions` request
|
||||
|
||||
<Tabs>
|
||||
|
||||
|
||||
<TabItem value="openai" label="OpenAI Python v1.0.0+">
|
||||
|
||||
```python
|
||||
@@ -38,7 +36,7 @@ response = client.chat.completions.create(
|
||||
}
|
||||
],
|
||||
user="palantir", # OPTIONAL: pass user to track spend by user
|
||||
extra_body={
|
||||
extra_body={
|
||||
"metadata": {
|
||||
"tags": ["jobID:214590dsff09fds", "taskName:run_page_classification"] # ENTERPRISE: pass tags to track spend by tags
|
||||
}
|
||||
@@ -47,6 +45,7 @@ response = client.chat.completions.create(
|
||||
|
||||
print(response)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="Curl" label="Curl Request">
|
||||
@@ -71,6 +70,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="langchain" label="Langchain">
|
||||
|
||||
@@ -131,7 +131,7 @@ The following spend gets tracked in Table `LiteLLM_SpendLogs`
|
||||
```json
|
||||
{
|
||||
"api_key": "fe6b0cab4ff5a5a8df823196cc8a450*****", # Hash of API Key used
|
||||
"user": "default_user", # Internal User (LiteLLM_UserTable) that owns `api_key=sk-1234`.
|
||||
"user": "default_user", # Internal User (LiteLLM_UserTable) that owns `api_key=sk-1234`.
|
||||
"team_id": "e8d1460f-846c-45d7-9b43-55f3cc52ac32", # Team (LiteLLM_TeamTable) that owns `api_key=sk-1234`
|
||||
"request_tags": ["jobID:214590dsff09fds", "taskName:run_page_classification"],# Tags sent in request
|
||||
"end_user": "palantir", # Customer - the `user` sent in the request
|
||||
@@ -152,7 +152,7 @@ Navigate to the Usage Tab on the LiteLLM UI (found on https://your-proxy-endpoin
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
### Allowing Non-Proxy Admins to access `/spend` endpoints
|
||||
### Allowing Non-Proxy Admins to access `/spend` endpoints
|
||||
|
||||
Use this when you want non-proxy admins to access `/spend` endpoints
|
||||
|
||||
@@ -162,8 +162,10 @@ Schedule a [meeting with us to get your Enterprise License](https://calendly.com
|
||||
|
||||
:::
|
||||
|
||||
##### Create Key
|
||||
Create Key with with `permissions={"get_spend_routes": true}`
|
||||
##### Create Key
|
||||
|
||||
Create Key with with `permissions={"get_spend_routes": true}`
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/key/generate' \
|
||||
--header 'Authorization: Bearer sk-1234' \
|
||||
@@ -176,22 +178,24 @@ curl --location 'http://0.0.0.0:4000/key/generate' \
|
||||
##### Use generated key on `/spend` endpoints
|
||||
|
||||
Access spend Routes with newly generate keys
|
||||
|
||||
```shell
|
||||
curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30' \
|
||||
-H 'Authorization: Bearer sk-H16BKvrSNConSsBYLGc_7A'
|
||||
```
|
||||
|
||||
|
||||
|
||||
#### Reset Team, API Key Spend - MASTER KEY ONLY
|
||||
|
||||
Use `/global/spend/reset` if you want to:
|
||||
|
||||
- Reset the Spend for all API Keys, Teams. The `spend` for ALL Teams and Keys in `LiteLLM_TeamTable` and `LiteLLM_VerificationToken` will be set to `spend=0`
|
||||
|
||||
- LiteLLM will maintain all the logs in `LiteLLMSpendLogs` for Auditing Purposes
|
||||
|
||||
##### Request
|
||||
##### Request
|
||||
|
||||
Only the `LITELLM_MASTER_KEY` you set can access this route
|
||||
|
||||
```shell
|
||||
curl -X POST \
|
||||
'http://localhost:4000/global/spend/reset' \
|
||||
@@ -205,6 +209,68 @@ curl -X POST \
|
||||
{"message":"Spend for all API Keys and Teams reset successfully","status":"success"}
|
||||
```
|
||||
|
||||
## Total spend per user
|
||||
|
||||
Assuming you have been issuing keys for end users, and setting their `user_id` on the key, you can check their usage.
|
||||
|
||||
```shell title="Total for a user API" showLineNumbers
|
||||
curl -L -X GET 'http://localhost:4000/user/info?user_id=jane_smith' \
|
||||
-H 'Authorization: Bearer sk-...'
|
||||
```
|
||||
|
||||
```json title="Total for a user API Response" showLineNumbers
|
||||
{
|
||||
"user_id": "jane_smith",
|
||||
"user_info": {
|
||||
"spend": 0.1
|
||||
},
|
||||
"keys": [
|
||||
{
|
||||
"token": "6e952b0efcafbb6350240db25ed534b4ec6011b3e1ba1006eb4f903461fd36f6",
|
||||
"key_name": "sk-...KE_A",
|
||||
"key_alias": "user-01882d6b-e090-776a-a587-21c63e502670-01983ddb-872f-71a3-8b3a-f9452c705483",
|
||||
"soft_budget_cooldown": false,
|
||||
"spend": 0.1,
|
||||
"expires": "2025-07-31T19:14:13.968000+00:00",
|
||||
"models": [],
|
||||
"aliases": {},
|
||||
"config": {},
|
||||
"user_id": "01982d6b-e090-776a-a587-21c63e502660",
|
||||
"team_id": "f2044fde-2293-482f-bf35-a8dab4e85c5f",
|
||||
"permissions": {},
|
||||
"max_parallel_requests": null,
|
||||
"metadata": {},
|
||||
"blocked": null,
|
||||
"tpm_limit": null,
|
||||
"rpm_limit": null,
|
||||
"max_budget": null,
|
||||
"budget_duration": null,
|
||||
"budget_reset_at": null,
|
||||
"allowed_cache_controls": [],
|
||||
"allowed_routes": [],
|
||||
"model_spend": {},
|
||||
"model_max_budget": {},
|
||||
"budget_id": null,
|
||||
"organization_id": null,
|
||||
"object_permission_id": null,
|
||||
"created_at": "2025-07-24T19:14:13.970000Z",
|
||||
"created_by": "582b168f-fc11-4e14-ad6a-cf4bb3656ddc",
|
||||
"updated_at": "2025-07-24T19:14:13.970000Z",
|
||||
"updated_by": "582b168f-fc11-4e14-ad6a-cf4bb3656ddc",
|
||||
"litellm_budget_table": null,
|
||||
"litellm_organization_table": null,
|
||||
"object_permission": null,
|
||||
"team_alias": null
|
||||
}
|
||||
],
|
||||
"teams": []
|
||||
}
|
||||
```
|
||||
|
||||
**Warning**
|
||||
End users can provide the `user` parameter in their request bodies, doing this will increment the cost reported via `/customer/info?end_user_id=self-declared-user`, and not for the user that owns the key as reported by that API. This means users could "avoid" having their spend tracked, through their method.
|
||||
This means if you need to track user spend, and are giving end users API keys, you must always set user_id when creating their api keys, and use keys issued for that user every time you're making LLM calls on their behalf in backend services. This will track their spend.
|
||||
|
||||
## Daily Spend Breakdown API
|
||||
|
||||
Retrieve granular daily usage data for a user (by model, provider, and API key) with a single endpoint.
|
||||
@@ -257,16 +323,14 @@ See our [Swagger API](https://litellm-api.up.railway.app/#/Budget%20%26%20Spend%
|
||||
|
||||
## Custom Tags
|
||||
|
||||
Requirements:
|
||||
Requirements:
|
||||
|
||||
- Virtual Keys & a database should be set up, see [virtual keys](https://docs.litellm.ai/docs/proxy/virtual_keys)
|
||||
|
||||
**Note:** By default, LiteLLM will track `User-Agent` as a custom tag for cost tracking. This enables viewing usage for tools like Claude Code, Gemini CLI, etc.
|
||||
|
||||
**Note:** By default, LiteLLM will track `User-Agent` as a custom tag for cost tracking. This enables viewing usage for tools like Claude Code, Gemini CLI, etc.
|
||||
|
||||
<Image img={require('../../img/claude_cli_tag_usage.png')} />
|
||||
|
||||
|
||||
### Client-side spend tag
|
||||
|
||||
<Tabs>
|
||||
@@ -331,32 +395,32 @@ response = client.chat.completions.create(
|
||||
|
||||
print(response)
|
||||
```
|
||||
</TabItem>
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="openai js" label="OpenAI JS">
|
||||
|
||||
```js
|
||||
const openai = require('openai');
|
||||
const openai = require("openai");
|
||||
|
||||
async function runOpenAI() {
|
||||
const client = new openai.OpenAI({
|
||||
apiKey: 'sk-1234',
|
||||
baseURL: 'http://0.0.0.0:4000'
|
||||
apiKey: "sk-1234",
|
||||
baseURL: "http://0.0.0.0:4000",
|
||||
});
|
||||
|
||||
try {
|
||||
const response = await client.chat.completions.create({
|
||||
model: 'gpt-3.5-turbo',
|
||||
model: "gpt-3.5-turbo",
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: "this is a test request, write a short poem"
|
||||
role: "user",
|
||||
content: "this is a test request, write a short poem",
|
||||
},
|
||||
],
|
||||
metadata: {
|
||||
tags: ["model-anthropic-claude-v2.1", "app-ishaan-prod"] // 👈 Key Change
|
||||
}
|
||||
tags: ["model-anthropic-claude-v2.1", "app-ishaan-prod"], // 👈 Key Change
|
||||
},
|
||||
});
|
||||
console.log(response);
|
||||
} catch (error) {
|
||||
@@ -368,6 +432,7 @@ async function runOpenAI() {
|
||||
// Call the asynchronous function
|
||||
runOpenAI();
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="Curl" label="Curl Request">
|
||||
@@ -388,6 +453,7 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
|
||||
"metadata": {"tags": ["model-anthropic-claude-v2.1", "app-ishaan-prod"]}
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="langchain" label="Langchain">
|
||||
|
||||
@@ -427,8 +493,6 @@ print(response)
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
|
||||
### Add custom headers to spend tracking
|
||||
|
||||
You can add custom headers to the request to track spend and usage.
|
||||
@@ -447,7 +511,8 @@ You can disable user-agent tracking by setting `litellm_settings.disable_user_ag
|
||||
litellm_settings:
|
||||
disable_user_agent_tracking: true
|
||||
```
|
||||
## ✨ (Enterprise) Generate Spend Reports
|
||||
|
||||
## ✨ (Enterprise) Generate Spend Reports
|
||||
|
||||
Use this to charge other teams, customers, users
|
||||
|
||||
@@ -467,6 +532,7 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end
|
||||
```
|
||||
|
||||
#### Example Response
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="response" label="Expected Response">
|
||||
@@ -511,7 +577,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="py-script" label="Script to Parse Response (Python)">
|
||||
@@ -548,6 +613,7 @@ for row in spend_report:
|
||||
```
|
||||
|
||||
Output from script
|
||||
|
||||
```shell
|
||||
# Date: 2024-05-11T00:00:00+00:00
|
||||
# Team: local_test_team
|
||||
@@ -570,21 +636,19 @@ Output from script
|
||||
# Metadata: [{'model': 'gpt-3.5-turbo', 'spend': 0.0005715000000000001, 'api_key': 'b94d5e0bc3a71a573917fe1335dc0c14728c7016337451af9714924ff3a729db', 'total_tokens': 423}]
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
|
||||
</Tabs>
|
||||
|
||||
</TabItem>
|
||||
|
||||
|
||||
<TabItem value="per customer" label="Spend Per Customer">
|
||||
|
||||
:::info
|
||||
|
||||
Customer [this is `user` passed to `/chat/completions` request](#how-to-track-spend-with-litellm)
|
||||
- [LiteLLM API key](virtual_keys.md)
|
||||
|
||||
- [LiteLLM API key](virtual_keys.md)
|
||||
|
||||
:::
|
||||
|
||||
@@ -592,7 +656,6 @@ Customer [this is `user` passed to `/chat/completions` request](#how-to-track-sp
|
||||
|
||||
👉 Key Change: Specify `group_by=customer`
|
||||
|
||||
|
||||
```shell
|
||||
curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30&group_by=customer' \
|
||||
-H 'Authorization: Bearer sk-1234'
|
||||
@@ -600,7 +663,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end
|
||||
|
||||
#### Example Response
|
||||
|
||||
|
||||
```shell
|
||||
[
|
||||
{
|
||||
@@ -641,15 +703,12 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="per key" label="Spend for Specific API Key">
|
||||
|
||||
|
||||
👉 Key Change: Specify `api_key=sk-1234`
|
||||
|
||||
|
||||
```shell
|
||||
curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-06-30&api_key=sk-1234' \
|
||||
-H 'Authorization: Bearer sk-1234'
|
||||
@@ -657,7 +716,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end
|
||||
|
||||
#### Example Response
|
||||
|
||||
|
||||
```shell
|
||||
[
|
||||
{
|
||||
@@ -693,10 +751,8 @@ Internal User (Key Owner): This is the value of `user_id` passed when calling [`
|
||||
|
||||
:::
|
||||
|
||||
|
||||
👉 Key Change: Specify `internal_user_id=ishaan`
|
||||
|
||||
|
||||
```shell
|
||||
curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end_date=2024-12-30&internal_user_id=ishaan' \
|
||||
-H 'Authorization: Bearer sk-1234'
|
||||
@@ -704,7 +760,6 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end
|
||||
|
||||
#### Example Response
|
||||
|
||||
|
||||
```shell
|
||||
[
|
||||
{
|
||||
@@ -768,46 +823,43 @@ curl -X GET 'http://localhost:4000/global/spend/report?start_date=2024-04-01&end
|
||||
|
||||
</Tabs>
|
||||
|
||||
|
||||
## 📊 Spend Logs API - Individual Transaction Logs
|
||||
|
||||
The `/spend/logs` endpoint now supports a `summarize` parameter to control data format when using date filters.
|
||||
|
||||
### Key Parameters
|
||||
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| Parameter | Description |
|
||||
| ----------- | -------------------------------------------------------------------------------------------- |
|
||||
| `summarize` | **New parameter**: `true` (default) = aggregated data, `false` = individual transaction logs |
|
||||
|
||||
### Examples
|
||||
|
||||
**Get individual transaction logs:**
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:4000/spend/logs?start_date=2024-01-01&end_date=2024-01-02&summarize=false" \
|
||||
-H "Authorization: Bearer sk-1234"
|
||||
```
|
||||
|
||||
**Get summarized data (default):**
|
||||
|
||||
```bash
|
||||
curl -X GET "http://localhost:4000/spend/logs?start_date=2024-01-01&end_date=2024-01-02" \
|
||||
-H "Authorization: Bearer sk-1234"
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
|
||||
- `summarize=false`: Analytics dashboards, ETL processes, detailed audit trails
|
||||
- `summarize=true`: Daily spending reports, high-level cost tracking (legacy behavior)
|
||||
|
||||
|
||||
## ✨ Custom Spend Log metadata
|
||||
|
||||
Log specific key,value pairs as part of the metadata for a spend log
|
||||
|
||||
:::info
|
||||
:::info
|
||||
|
||||
Logging specific key,value pairs in spend logs metadata is an enterprise feature. [See here](./enterprise.md#tracking-spend-with-custom-metadata)
|
||||
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user