docs(model_access_guide.md): document how model access works on litellm

This commit is contained in:
Krrish Dholakia
2025-11-12 13:43:13 -08:00
parent c5496fd61c
commit c6d2714c52
3 changed files with 78 additions and 1 deletions
@@ -0,0 +1,77 @@
# How Model Access Works
## Concept
Each model onboarded is a "model deployment" in LiteLLM.
These model deployments are assigned to a "model group", via the "model_name" field in the config.yaml.
## Example
```yaml
model_list:
- model_name: my-custom-model
litellm_params:
model: openai/gpt-4o
api_key: os.environ/OPENAI_API_KEY
```
In here, we onboard a model deployment for the model `gpt-4o` and assign it to the model group `my-custom-model`.
## Access Control
When you give access to a key/user/team, you are giving them access to a "model group".
Example:
```bash
curl --location 'http://localhost:4000/key/generate' \
--header 'Authorization: Bearer <your-master-key>' \
--header 'Content-Type: application/json' \
--data-raw '{"models": ["my-custom-model"]}'
```
## Loadbalancing
You can add multiple model deployments to a single "model group". LiteLLM will automatically load balance requests across the model deployments in the group.
Example:
```yaml
model_list:
- model_name: my-custom-model
litellm_params:
model: openai/gpt-4o
api_key: os.environ/OPENAI_API_KEY
- model_name: my-custom-model
litellm_params:
model: azure/gpt-4o
api_key: os.environ/AZURE_API_KEY
api_base: os.environ/AZURE_API_BASE
api_version: os.environ/AZURE_API_VERSION
```
This way, you can maximize your rate limits across multiple model deployments.
## Fallbacks
You can fallback across model groups. This is useful, if all "model deployments" in a "model group" are down (e.g. raising 429 errors).
Example:
```yaml
model_list:
- model_name: my-custom-model
litellm_params:
model: openai/gpt-4o-mini
api_key: os.environ/OPENAI_API_KEY
- model_name: my-other-model
litellm_params:
model: openai/gpt-4o
api_key: os.environ/OPENAI_API_KEY
litellm_settings:
fallbacks: [{"my-custom-model": ["my-other-model"]}]
```
Fallbacks are done sequentially, so the first model group in the list will be tried first. If it fails, the next model group will be tried.
+1 -1
View File
@@ -246,6 +246,7 @@ const sidebars = {
type: "category",
label: "Model Access",
items: [
"proxy/model_access_guide",
"proxy/model_access",
"proxy/team_model_add"
]
@@ -792,7 +793,6 @@ const sidebars = {
"projects/GPTLocalhost",
"projects/HolmesGPT",
"projects/Railtracks",
"projects/Softgen",
],
},
"extras/code_quality",