From 14c2932387b13634fdef557fc16335e2a15a54ea Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Tue, 13 Jan 2026 16:44:02 +0530 Subject: [PATCH] Add docs on Zero-Cost Models --- docs/my-website/docs/proxy/custom_pricing.md | 46 ++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/docs/my-website/docs/proxy/custom_pricing.md b/docs/my-website/docs/proxy/custom_pricing.md index 8f4a4c450f..b61da85bb1 100644 --- a/docs/my-website/docs/proxy/custom_pricing.md +++ b/docs/my-website/docs/proxy/custom_pricing.md @@ -9,6 +9,7 @@ LiteLLM provides flexible cost tracking and pricing customization for all LLM pr - **Custom Pricing** - Override default model costs or set pricing for custom models - **Cost Per Token** - Track costs based on input/output tokens (most common) - **Cost Per Second** - Track costs based on runtime (e.g., Sagemaker) +- **Zero-Cost Models** - Bypass budget checks for free/on-premises models by setting costs to 0 - **[Provider Discounts](./provider_discounts.md)** - Apply percentage-based discounts to specific providers - **[Provider Margins](./provider_margins.md)** - Add fees/margins to LLM costs for internal billing - **Base Model Mapping** - Ensure accurate cost tracking for Azure deployments @@ -106,6 +107,51 @@ There are other keys you can use to specify costs for different scenarios and mo These keys evolve based on how new models handle multimodality. The latest version can be found at [https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json). +## Zero-Cost Models (Bypass Budget Checks) + +**Use Case**: You have on-premises or free models that should be accessible even when users exceed their budget limits. + +**Solution** ✅: Set both `input_cost_per_token` and `output_cost_per_token` to `0` (explicitly) to bypass all budget checks for that model. + +:::info + +When a model is configured with zero cost, LiteLLM will automatically skip ALL budget checks (user, team, team member, end-user, organization, and global proxy budget) for requests to that model. + +**Important**: Both costs must be **explicitly set to 0**. If costs are `null` or undefined, the model will be treated as having cost and budget checks will apply. + +::: + +### Configuration Example + +```yaml +model_list: + # On-premises model - free to use + - model_name: on-prem-llama + litellm_params: + model: ollama/llama3 + api_base: http://localhost:11434 + model_info: + input_cost_per_token: 0 # 👈 Explicitly set to 0 + output_cost_per_token: 0 # 👈 Explicitly set to 0 + + # Paid cloud model - budget checks apply + - model_name: gpt-4 + litellm_params: + model: gpt-4 + api_key: os.environ/OPENAI_API_KEY + # No model_info - uses default pricing from cost map +``` + +### Behavior + +With the above configuration: + +- **User over budget** → Can still use `on-prem-llama` ✅, but blocked from `gpt-4` ❌ +- **Team over budget** → Can still use `on-prem-llama` ✅, but blocked from `gpt-4` ❌ +- **End-user over budget** → Can still use `on-prem-llama` ✅, but blocked from `gpt-4` ❌ + +This ensures your free/on-premises models remain accessible regardless of budget constraints, while paid models are still properly governed. + ## Set 'base_model' for Cost Tracking (e.g. Azure deployments) **Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking