* Add concise Claude Code + LiteLLM Gateway tutorial
- Create focused tutorial matching existing tutorial style
- Step-by-step guide from installation to advanced configurations
- Multi-provider configuration examples (AWS Bedrock, Azure OpenAI, Load Balancing)
- Based on Anthropic's official LiteLLM configuration documentation
- Added to sidebar with clean title 'Use LiteLLM with Claude Code'
- Fixed sidebar reference from 'secret' to 'set_keys' for proper document resolution
* Update config_settings.md to correct documentation links for key management and Hashicorp Vault settings. Changed references from 'secret.md' to 'set_keys.md' for improved clarity and accuracy.
* Update sidebar and config_settings.md to reflect changes in key management documentation. Changed sidebar reference from 'set_keys' to 'secret' and updated links in config_settings.md for Hashicorp Vault settings to point to 'secret.md' for improved accuracy.
* Remove extra tutorial and update sidebar accordingly
* Update tutorial title from 'WebUI' to 'Open WebUI' for clarity and consistency in documentation.
* Remove Python version requirement from Claude Responses API tutorial for clarity and to align with updated prerequisites.
* feat: add input_fidelity parameter for OpenAI image generation
- Add input_fidelity to OpenAIImageGenerationOptionalParams type
- Update image_generation function signature to accept input_fidelity
- Add input_fidelity to default_params in get_optional_params_image_gen
- Include input_fidelity in openai_params list for proper handling
- Update documentation with input_fidelity parameter description
- Add test for input_fidelity parameter functionality
This enables control over how closely the model follows the input prompt
for gpt-image-1 model, improving prompt adherence and image quality.
* feat: add input_fidelity to optional parameters for image generation
- Include input_fidelity in the list of OpenAIImageGenerationOptionalParams
- This addition enhances the flexibility of image generation by allowing control over input fidelity.
* test: enhance test for gpt-image-1 with input_fidelity parameter
- Update test_gpt_image_1_with_input_fidelity to include mocking of OpenAI response
- Validate that the OpenAI client is called with correct parameters, including input_fidelity
- Improve response validation to ensure expected output structure and values
* Add comprehensive GitHub Copilot + LiteLLM integration tutorial
- Complete setup guide from installation to production deployment
- Multiple configuration examples including authentication, load balancing, and cost tracking
- Docker and Kubernetes deployment configurations
- Troubleshooting section with common issues and solutions
- Best practices for security, monitoring, and reliability
- Usage examples for code completion, chat interface, and direct API integration
* Add concise GitHub Copilot + LiteLLM tutorial
- Create focused tutorial matching Gemini CLI style
- Step-by-step guide from installation to production deployment
- Multi-provider configuration examples (OpenAI, Anthropic, Bedrock)
- Load balancing and fallback configuration
- Docker deployment instructions
- Troubleshooting section with common issues
- Updated sidebar with clean title 'Use LiteLLM with GitHub Copilot'
* Refactor GitHub Copilot integration tutorial
- Removed outdated production deployment and direct API usage sections
- Streamlined troubleshooting steps for clarity
- Ensured documentation aligns with current best practices and configurations
* Add proper credit to Sergio Pino for GitHub Copilot tutorial
- Reference original DEV.to article in info box
- Add credits section acknowledging foundational work
- Maintain attribution to original author's guide
- Add ttl parameter to cache_control for Gemini models
- Support Google's TTL format (e.g., '3600s', '7200s')
- Implement robust TTL extraction and validation
- Extract TTL before system message transformation to handle all cases
- Add comprehensive test suite with 17 test cases in tests/test_litellm/
- Update documentation with TTL usage examples
- Maintain backward compatibility with existing cache_control usage
Fixes#9810
* feat(route_checks.py): allow admin to disable proxy management endpoints on instance
useful for preventing multiple instances from doing admin actions
* docs(scaling_multiple_instances.md): add architecture doc on scaling multiple litellm instances
provide guidance on scaling proxy
* docs(scaling_multiple_instances.md): add doc on scaling across multiple regions for litellm
* fix(route_checks.py): allow disabling llm api endpoints on an instance
allows pure admin instance to exist
* refactor(enterprise/route_checks.py): refactor env var checks
* refactor: finish refactoring
* docs(control_plane_and_data_plane.md): refactor docs
* test: update tests
* [Feat] New LLM API Integration - Add Moonshot API (Kimi) (#12551)
* Add Moonshot AI provider support to LiteLLM
Co-authored-by: ishaan <ishaan@berri.ai>
* Refactor Moonshot provider params handling and transformation logic
Co-authored-by: ishaan <ishaan@berri.ai>
* fix constants
* add Moonshot AI
* fix get_supported_openai_params
* handle kimi temp
* add tool choice handling
* test moonshot unit tests
* fix kimi
* fix linting
* Add pricing information for Moonshot AI's kimi-k2 model (#12566)
* Add pricing information for Moonshot AI's kimi-k2 model
* Update model name to kimi-k2-0711-preview
- Changed model name from moonshot/kimi-k2 to moonshot/kimi-k2-0711-preview
- This reflects the specific model version as requested
* Update moonshot_models list to match model_context JSON
---------
Co-authored-by: openhands <openhands@all-hands.dev>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
* update docs
* docs moonshot
* fixes model cost map
* fix map_openai_params
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
* Get the basics of the integration working.
* Cleanup bytez integration.
* Update user agent for Bytez integration.
* Use the config class directly. Create the start of the docs.
* Finish up bytez documentation. Include a provider integration guide.
* Fix typing bug in custom_logger_utils. Add tests for bytez integration.
* Add token tracking for model usage for Bytez integration.
* Create a units test for the Bytez config.
* Make changes to Bytez transformation code per PR feedback.
* Cleanup coment in Bytez transformation test.
* Remove LRU usage for bytez integration.
* Consolidate Bytez tests into a single file. Conform to project structure for tests.
* Fix linting error with Bytez impl.
* Add Bytez to the list of providers in the docs.
* just use 1 param for mcp groups
* fix just use 1 param for access groups
* test_get_tools_from_mcp_servers
* docs access groups
* group MCPs
* test fix
* fix screenshots on docs
* TestMCPAccessGroupsE2E
* update img
* fix MCP connect
* Get the basics of the integration working.
* Cleanup bytez integration.
* Update user agent for Bytez integration.
* Use the config class directly. Create the start of the docs.
* Finish up bytez documentation. Include a provider integration guide.
* Fix typing bug in custom_logger_utils. Add tests for bytez integration.
* Add token tracking for model usage for Bytez integration.
* Create a units test for the Bytez config.
* Make changes to Bytez transformation code per PR feedback.
* Cleanup coment in Bytez transformation test.
* Remove LRU usage for bytez integration.
* Consolidate Bytez tests into a single file. Conform to project structure for tests.
* Fix linting error with Bytez impl.
* Update Vertex Model Garden documentation to deploy + use OpenAI Chat Completion tutorial.
* update anchor description
* update to LiteLLM specific tutorial
* update the right doc
* Reverted docs/my-website/docs/providers/vertex.md to match main
* revert to main vertex tutorial
* fix(internal_user_endpoints.py): initial commit removing logic to create new budget for new user if default max budget in team set
* feat(proxy_setting_endpoints.py): update team member budget when set via default internal user endpoint
removes need to create a unique budget per user
* feat(proxy_server.py): set team max member budget on startup, if set on config.yaml
* fix(prometheus.py): support custom tags for tracking on prometheus
Allows tracking user agent values on prometheus metrics
* test(test_internal_user_endpoints.py): fix test
* update types
* add STDIO to client
* allow using STDIO with MCP manager
* add Stdio-specific fields to schema.prisma
* fixes for MCP mgmt
* fix for adding stdio MCP to DB
* ui - allow adding stdio MCPs
* fix MCP server manager
* docs stdio MCP
* add_stdio_mcp.png
* new stdio tests
* allow adding MCPs through config.yaml
* fix tool test panel
* use TestMCPClient
* ui fixes for testing circle ci mcp
* feat(custom_guardrail.py): allow admin to run guardrails on specific modes based on request tags
Enables pre-call checks for openwebui but logging for claude code guardrails
* refactor(custom_guardrail.py): move tag based mode to enterprise
* test(test_custom_guardrail.py): update test
* feat(quick_start.md): add doc on tag-based guardrail mode
* docs(quick_start.md): cleanup doc
* docs: cleanup
* docs(openweb_ui.md): add tutorial to doc
* fix(custom_guardrail.py): ensure user is premium user
* test: update test
* Added dashscope as a provider
* Fix some leftover references on nebius
* Porting the dashscope api endpoit international version
* explicit tool_choice = True in config
* add support of bearer token for bedrock integration
* fix linting issue
* fix type checking issue
* reoder arguments to address type checking issue
* switch to use get_secret_str to fetch env variable
Co-authored-by: 0x-fang <fanggong@amazon.com>
* fix(auth_checks.py): resolve a model group alias when key has access to underlying model
Fixes LIT-293
* feat(anthropic/): add mock_response to anthropic /v1/messages
makes it easy to test fallback logic
* fix(router.py): support fallbacks on /v1/messages
adds working fallbacks on generic api route
* refactor(router.py): point _ageneric_api_call_with_fallbacks to updated function
* test: add unit test for new helper on router
* fix(router.py): use correct metadata variable name
* fix(router.py): use correct metadata field
* docs(config_settings.md): document new param
* fix(db_spend_update_writer.py): fix db query
* fix(litellm_pre_call_utils.py): support passing anthropic-beta headers when 'forward_client_headers_to_llm_api' is True
allows user to pass along extra headers to vertex ai anthropic models
* docs(config_settings.md): update docs
* add _get_mcp_auth_header_from_headers
* test_process_mcp_request_with_custom_auth_header
* Using a different Authentication Header
* fix customize MCP Auth header name
Ishaan Jaff