docs gemini cli x litellm

This commit is contained in:
Ishaan Jaff
2025-06-28 22:06:42 -07:00
parent cec7e4941a
commit 79ffb8a89e
@@ -76,6 +76,99 @@ The CLI will now use LiteLLM Proxy as the backend, giving you access to LiteLLM'
- Cost tracking
- Model routing and fallbacks
## Advanced
### Use Anthropic, OpenAI, Bedrock, etc. models on gemini-cli
In order to use non-gemini models on gemini-cli, you need to set a `model_group_alias` in the LiteLLM Proxy config. This tells LiteLLM that requests with model = `gemini-2.5-pro` should be routed to your desired model from any provider.
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<Tabs>
<TabItem value="anthropic" label="Anthropic">
Route `gemini-2.5-pro` requests to Claude Sonnet:
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
- model_name: claude-sonnet-4-20250514
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: os.environ/ANTHROPIC_API_KEY
router_settings:
model_group_alias: {"gemini-2.5-pro": "claude-sonnet-4-20250514"}
```
</TabItem>
<TabItem value="openai" label="OpenAI">
Route `gemini-2.5-pro` requests to GPT-4o:
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
- model_name: gpt-4o-model
litellm_params:
model: gpt-4o
api_key: os.environ/OPENAI_API_KEY
router_settings:
model_group_alias: {"gemini-2.5-pro": "gpt-4o-model"}
```
</TabItem>
<TabItem value="bedrock" label="Bedrock">
Route `gemini-2.5-pro` requests to Claude on Bedrock:
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
- model_name: bedrock-claude
litellm_params:
model: bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
aws_region_name: us-east-1
router_settings:
model_group_alias: {"gemini-2.5-pro": "bedrock-claude"}
```
</TabItem>
<TabItem value="multi-provider" label="Multi-Provider Load Balancing">
All deployments with model_name=`anthropic-claude` will be load balanced. In this example we load balance between Anthropic and Bedrock.
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
- model_name: anthropic-claude
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: anthropic-claude
litellm_params:
model: bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
aws_region_name: us-east-1
router_settings:
model_group_alias: {"gemini-2.5-pro": "anthropic-claude"}
```
</TabItem>
</Tabs>
With this configuration, when you use `gemini-2.5-pro` in the CLI, LiteLLM will automatically route your requests to the configured provider(s) with load balancing and fallbacks.
## Troubleshooting
If you encounter issues: