diff --git a/docs/my-website/docs/tutorials/litellm_gemini_cli.md b/docs/my-website/docs/tutorials/litellm_gemini_cli.md index 82d504803e..bf8e2cb44c 100644 --- a/docs/my-website/docs/tutorials/litellm_gemini_cli.md +++ b/docs/my-website/docs/tutorials/litellm_gemini_cli.md @@ -76,6 +76,99 @@ The CLI will now use LiteLLM Proxy as the backend, giving you access to LiteLLM' - Cost tracking - Model routing and fallbacks + +## Advanced + +### Use Anthropic, OpenAI, Bedrock, etc. models on gemini-cli + +In order to use non-gemini models on gemini-cli, you need to set a `model_group_alias` in the LiteLLM Proxy config. This tells LiteLLM that requests with model = `gemini-2.5-pro` should be routed to your desired model from any provider. + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + + + + +Route `gemini-2.5-pro` requests to Claude Sonnet: + +```yaml showLineNumbers title="proxy_config.yaml" +model_list: + - model_name: claude-sonnet-4-20250514 + litellm_params: + model: anthropic/claude-3-5-sonnet-20241022 + api_key: os.environ/ANTHROPIC_API_KEY + +router_settings: + model_group_alias: {"gemini-2.5-pro": "claude-sonnet-4-20250514"} +``` + + + + +Route `gemini-2.5-pro` requests to GPT-4o: + +```yaml showLineNumbers title="proxy_config.yaml" +model_list: + - model_name: gpt-4o-model + litellm_params: + model: gpt-4o + api_key: os.environ/OPENAI_API_KEY + +router_settings: + model_group_alias: {"gemini-2.5-pro": "gpt-4o-model"} +``` + + + + +Route `gemini-2.5-pro` requests to Claude on Bedrock: + +```yaml showLineNumbers title="proxy_config.yaml" +model_list: + - model_name: bedrock-claude + litellm_params: + model: bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0 + aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID + aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY + aws_region_name: us-east-1 + +router_settings: + model_group_alias: {"gemini-2.5-pro": "bedrock-claude"} +``` + + + + +All deployments with model_name=`anthropic-claude` will be load balanced. In this example we load balance between Anthropic and Bedrock. + +```yaml showLineNumbers title="proxy_config.yaml" +model_list: + - model_name: anthropic-claude + litellm_params: + model: anthropic/claude-3-5-sonnet-20241022 + api_key: os.environ/ANTHROPIC_API_KEY + - model_name: anthropic-claude + litellm_params: + model: bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0 + aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID + aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY + aws_region_name: us-east-1 + +router_settings: + model_group_alias: {"gemini-2.5-pro": "anthropic-claude"} +``` + + + + +With this configuration, when you use `gemini-2.5-pro` in the CLI, LiteLLM will automatically route your requests to the configured provider(s) with load balancing and fallbacks. + + + + + + + ## Troubleshooting If you encounter issues: