docs: document input/output/total tokens behaviour

Closes https://github.com/BerriAI/litellm/issues/17480
This commit is contained in:
Krrish Dholakia
2026-01-20 10:45:47 -08:00
parent 1377721715
commit f95f5563ea
+20
View File
@@ -545,6 +545,26 @@ You can set:
- max parallel requests
- rpm / tpm limits per model for a given key
### TPM Rate Limit Type (Input/Output/Total)
By default, TPM (tokens per minute) rate limits count **total tokens** (input + output). You can configure this to count only input tokens or only output tokens instead.
Set `token_rate_limit_type` in your `config.yaml`:
```yaml
general_settings:
master_key: sk-1234
token_rate_limit_type: "output" # Options: "input", "output", "total" (default)
```
| Value | Description |
|-------|-------------|
| `total` | Count total tokens (prompt + completion). **Default behavior.** |
| `input` | Count only prompt/input tokens |
| `output` | Count only completion/output tokens |
This setting applies globally to all TPM rate limit checks (keys, users, teams, etc.).
<Tabs>
<TabItem value="per-team" label="Per Team">