mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-05 04:22:19 +00:00
docs gemini cli x litellm
This commit is contained in:
@@ -76,6 +76,99 @@ The CLI will now use LiteLLM Proxy as the backend, giving you access to LiteLLM'
|
||||
- Cost tracking
|
||||
- Model routing and fallbacks
|
||||
|
||||
|
||||
## Advanced
|
||||
|
||||
### Use Anthropic, OpenAI, Bedrock, etc. models on gemini-cli
|
||||
|
||||
In order to use non-gemini models on gemini-cli, you need to set a `model_group_alias` in the LiteLLM Proxy config. This tells LiteLLM that requests with model = `gemini-2.5-pro` should be routed to your desired model from any provider.
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="anthropic" label="Anthropic">
|
||||
|
||||
Route `gemini-2.5-pro` requests to Claude Sonnet:
|
||||
|
||||
```yaml showLineNumbers title="proxy_config.yaml"
|
||||
model_list:
|
||||
- model_name: claude-sonnet-4-20250514
|
||||
litellm_params:
|
||||
model: anthropic/claude-3-5-sonnet-20241022
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
|
||||
router_settings:
|
||||
model_group_alias: {"gemini-2.5-pro": "claude-sonnet-4-20250514"}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="openai" label="OpenAI">
|
||||
|
||||
Route `gemini-2.5-pro` requests to GPT-4o:
|
||||
|
||||
```yaml showLineNumbers title="proxy_config.yaml"
|
||||
model_list:
|
||||
- model_name: gpt-4o-model
|
||||
litellm_params:
|
||||
model: gpt-4o
|
||||
api_key: os.environ/OPENAI_API_KEY
|
||||
|
||||
router_settings:
|
||||
model_group_alias: {"gemini-2.5-pro": "gpt-4o-model"}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="bedrock" label="Bedrock">
|
||||
|
||||
Route `gemini-2.5-pro` requests to Claude on Bedrock:
|
||||
|
||||
```yaml showLineNumbers title="proxy_config.yaml"
|
||||
model_list:
|
||||
- model_name: bedrock-claude
|
||||
litellm_params:
|
||||
model: bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
|
||||
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
|
||||
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
|
||||
aws_region_name: us-east-1
|
||||
|
||||
router_settings:
|
||||
model_group_alias: {"gemini-2.5-pro": "bedrock-claude"}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="multi-provider" label="Multi-Provider Load Balancing">
|
||||
|
||||
All deployments with model_name=`anthropic-claude` will be load balanced. In this example we load balance between Anthropic and Bedrock.
|
||||
|
||||
```yaml showLineNumbers title="proxy_config.yaml"
|
||||
model_list:
|
||||
- model_name: anthropic-claude
|
||||
litellm_params:
|
||||
model: anthropic/claude-3-5-sonnet-20241022
|
||||
api_key: os.environ/ANTHROPIC_API_KEY
|
||||
- model_name: anthropic-claude
|
||||
litellm_params:
|
||||
model: bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
|
||||
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
|
||||
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
|
||||
aws_region_name: us-east-1
|
||||
|
||||
router_settings:
|
||||
model_group_alias: {"gemini-2.5-pro": "anthropic-claude"}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
With this configuration, when you use `gemini-2.5-pro` in the CLI, LiteLLM will automatically route your requests to the configured provider(s) with load balancing and fallbacks.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you encounter issues:
|
||||
|
||||
Reference in New Issue
Block a user