- Add new tutorial for integrating Qwen Code CLI with LiteLLM Proxy
- Update sidebar to include Qwen Code CLI in both AI Tools and main Tutorials sections
- Document environment variables for OpenAI-compatible configuration
- Include examples for routing to various providers (Anthropic, OpenAI, Bedrock)
* feat: Add Pillar Security guardrail integration
Implements comprehensive LLM security guardrails using Pillar Security API with support for prompt injection detection, PII/secret detection, content moderation, and multi-mode execution (pre_call, during_call, post_call). Includes complete documentation, testing, and configurable actions on flagged content.
* fix: Resolve MyPy type error in Pillar guardrail config
Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
* fix: Resolve MyPy type error in Pillar guardrail config
Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
* fix docs
* fix docs
* improved docs
* fix examples, READY
* fix(main.py): fix async retryer
Fixes https://github.com/BerriAI/litellm/issues/12830
* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers
clientside content-type != proxy content type, can cause requests to hang
* test(tests/): update tests
- Add comprehensive documentation for Model Armor integration
- Include configuration examples and parameter descriptions
- Add Model Armor to sidebars navigation
- Document authentication methods and error handling
* feat: add Morph provider support
- Add MorphChatConfig implementation for OpenAI-compatible API
- Support morph-v3-fast and morph-v3-large models
- Add pricing: morph-v3-fast (/bin/zsh.8/.2 per 1M tokens), morph-v3-large (/bin/zsh.9/.9 per 1M tokens)
- Both models support 16k context window and system messages
- Add comprehensive documentation and unit tests
- Update all necessary integration points (constants, init, provider logic)
* feat: Add Morph provider support in ProviderConfigManager
- Extend ProviderConfigManager to include MorphChatConfig for the Morph LLM provider.
- Update MorphChatConfig by removing unused parameters from the configuration.
- Add Hyperbolic as a new OpenAI-compatible provider
- Implement HyperbolicChatConfig inheriting from OpenAILikeChatConfig
- Register Hyperbolic in provider lists and constants
- Add comprehensive model configurations with pricing for:
- DeepSeek models (V3, R1, etc.)
- Qwen models (2.5, 3, QwQ, etc.)
- Meta Llama models (3.1, 3.2, 3.3)
- Other models like Kimi K2, Hermes 3, etc.
- Configure default API base URL: https://api.hyperbolic.xyz/v1
- Add provider documentation with usage examples
- Create unit tests for provider functionality
- Support all standard OpenAI parameters
Hyperbolic provides low-cost inference with OpenAI-compatible APIs,
supporting latest models without infrastructure overhead.
* feat: add Lambda AI provider support
Add support for Lambda AI (lambda.ai) as a new LLM provider in LiteLLM. Lambda AI provides access to a wide range of open-source models through their cloud GPU infrastructure.
Changes:
- Add Lambda AI provider implementation (OpenAI-compatible)
- Register 20 Lambda AI models with accurate pricing and 131k context windows
- Add comprehensive tests for Lambda AI integration
- Add detailed documentation with usage examples
- Use "lambda_ai" as provider name to avoid Python keyword conflict
Models include Llama 3.x, DeepSeek, Hermes, Qwen, and specialized models for coding and vision tasks.
* fix(tests): ensure lambda_ai_models list is repopulated after model cost reload
Updated test cases to clear and repopulate the lambda_ai_models list after reloading the model cost map. This ensures that the tests accurately reflect the current state of available models.
* feat: add Lambda AI chat configuration support
Added support for Lambda AI chat configuration in the ProviderConfigManager. This enhancement allows the integration of Lambda AI as a provider, expanding the capabilities of LiteLLM.
* feat: add v0 provider support to LiteLLM
- Add v0 as a new OpenAI-compatible provider
- Support all three v0 models: v0-1.0-md, v0-1.5-md, v0-1.5-lg
- Configure correct token limits and pricing for each model
- Enable vision support for all v0 models (multimodal)
- Add provider detection for v0/ prefix and api.v0.dev endpoint
- Include comprehensive unit tests for the provider
The v0 provider uses the standard OpenAI-compatible implementation
and supports all standard features including streaming, function
calling, and system messages.
* fix: add v0 provider to ProviderConfigManager
Add V0ChatConfig to the get_provider_chat_config method to fix
test_supports_tool_choice test failure. The v0 provider needs to
be included in the provider config manager to return the correct
configuration for tool choice support detection.
* docs: add documentation for v0 provider
- Add comprehensive v0 provider documentation
- Cover all supported models and their capabilities
- Include examples for SDK usage, proxy configuration, and all features
- Document supported OpenAI parameters based on v0 API docs
- Add v0 to the providers sidebar navigation
* fix: correct v0 supported OpenAI parameters
Based on review feedback and v0 API documentation:
- v0 only supports: messages, model, stream, tools, tool_choice
- Remove unsupported parameters like temperature, max_tokens, etc.
- Update tests to verify correct parameter set
- Update documentation to reflect actual API capabilities
- Remove JSON mode example as response_format is not supported
Reference: https://v0.dev/docs/v0-model-api#request-body
* fix: remove supports_response_schema from v0 models
Remove the supports_response_schema property from all v0 models in the model configuration files as v0 does not support this feature.
Models updated:
- v0/v0-1.0-md
- v0/v0-1.5-md
- v0/v0-1.5-lg
* Add concise Claude Code + LiteLLM Gateway tutorial
- Create focused tutorial matching existing tutorial style
- Step-by-step guide from installation to advanced configurations
- Multi-provider configuration examples (AWS Bedrock, Azure OpenAI, Load Balancing)
- Based on Anthropic's official LiteLLM configuration documentation
- Added to sidebar with clean title 'Use LiteLLM with Claude Code'
- Fixed sidebar reference from 'secret' to 'set_keys' for proper document resolution
* Update config_settings.md to correct documentation links for key management and Hashicorp Vault settings. Changed references from 'secret.md' to 'set_keys.md' for improved clarity and accuracy.
* Update sidebar and config_settings.md to reflect changes in key management documentation. Changed sidebar reference from 'set_keys' to 'secret' and updated links in config_settings.md for Hashicorp Vault settings to point to 'secret.md' for improved accuracy.
* Remove extra tutorial and update sidebar accordingly
* Update tutorial title from 'WebUI' to 'Open WebUI' for clarity and consistency in documentation.
* Remove Python version requirement from Claude Responses API tutorial for clarity and to align with updated prerequisites.
* Add comprehensive GitHub Copilot + LiteLLM integration tutorial
- Complete setup guide from installation to production deployment
- Multiple configuration examples including authentication, load balancing, and cost tracking
- Docker and Kubernetes deployment configurations
- Troubleshooting section with common issues and solutions
- Best practices for security, monitoring, and reliability
- Usage examples for code completion, chat interface, and direct API integration
* Add concise GitHub Copilot + LiteLLM tutorial
- Create focused tutorial matching Gemini CLI style
- Step-by-step guide from installation to production deployment
- Multi-provider configuration examples (OpenAI, Anthropic, Bedrock)
- Load balancing and fallback configuration
- Docker deployment instructions
- Troubleshooting section with common issues
- Updated sidebar with clean title 'Use LiteLLM with GitHub Copilot'
* Refactor GitHub Copilot integration tutorial
- Removed outdated production deployment and direct API usage sections
- Streamlined troubleshooting steps for clarity
- Ensured documentation aligns with current best practices and configurations
* Add proper credit to Sergio Pino for GitHub Copilot tutorial
- Reference original DEV.to article in info box
- Add credits section acknowledging foundational work
- Maintain attribution to original author's guide
* feat(route_checks.py): allow admin to disable proxy management endpoints on instance
useful for preventing multiple instances from doing admin actions
* docs(scaling_multiple_instances.md): add architecture doc on scaling multiple litellm instances
provide guidance on scaling proxy
* docs(scaling_multiple_instances.md): add doc on scaling across multiple regions for litellm
* fix(route_checks.py): allow disabling llm api endpoints on an instance
allows pure admin instance to exist
* refactor(enterprise/route_checks.py): refactor env var checks
* refactor: finish refactoring
* docs(control_plane_and_data_plane.md): refactor docs
* test: update tests
* [Feat] New LLM API Integration - Add Moonshot API (Kimi) (#12551)
* Add Moonshot AI provider support to LiteLLM
Co-authored-by: ishaan <ishaan@berri.ai>
* Refactor Moonshot provider params handling and transformation logic
Co-authored-by: ishaan <ishaan@berri.ai>
* fix constants
* add Moonshot AI
* fix get_supported_openai_params
* handle kimi temp
* add tool choice handling
* test moonshot unit tests
* fix kimi
* fix linting
* Add pricing information for Moonshot AI's kimi-k2 model (#12566)
* Add pricing information for Moonshot AI's kimi-k2 model
* Update model name to kimi-k2-0711-preview
- Changed model name from moonshot/kimi-k2 to moonshot/kimi-k2-0711-preview
- This reflects the specific model version as requested
* Update moonshot_models list to match model_context JSON
---------
Co-authored-by: openhands <openhands@all-hands.dev>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
* update docs
* docs moonshot
* fixes model cost map
* fix map_openai_params
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
Co-authored-by: Xingyao Wang <xingyao@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
* Get the basics of the integration working.
* Cleanup bytez integration.
* Update user agent for Bytez integration.
* Use the config class directly. Create the start of the docs.
* Finish up bytez documentation. Include a provider integration guide.
* Fix typing bug in custom_logger_utils. Add tests for bytez integration.
* Add token tracking for model usage for Bytez integration.
* Create a units test for the Bytez config.
* Make changes to Bytez transformation code per PR feedback.
* Cleanup coment in Bytez transformation test.
* Remove LRU usage for bytez integration.
* Consolidate Bytez tests into a single file. Conform to project structure for tests.
* Fix linting error with Bytez impl.
* Add Bytez to the list of providers in the docs.
* Get the basics of the integration working.
* Cleanup bytez integration.
* Update user agent for Bytez integration.
* Use the config class directly. Create the start of the docs.
* Finish up bytez documentation. Include a provider integration guide.
* Fix typing bug in custom_logger_utils. Add tests for bytez integration.
* Add token tracking for model usage for Bytez integration.
* Create a units test for the Bytez config.
* Make changes to Bytez transformation code per PR feedback.
* Cleanup coment in Bytez transformation test.
* Remove LRU usage for bytez integration.
* Consolidate Bytez tests into a single file. Conform to project structure for tests.
* Fix linting error with Bytez impl.
* Added dashscope as a provider
* Fix some leftover references on nebius
* Porting the dashscope api endpoit international version
* explicit tool_choice = True in config
* Litellm dev 03 05 2025 contributor prs (#9079)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
* Fix#7629 - Add tzdata package to Dockerfile (#8915)
* Add tzdata package to Dockerfile
* Move tzdata to python requirement.txt
* feat: add support for copilot provider (#8577)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat: add model information for copilot models
* fix: fix linting errors
* test: remove integration test for github_copilot + fix misisng mock
* fix: use print to make sure the logger message shown
* test: remove debug print
* fix lint (#11112)
* Add init files to make test directories Python packages and update import paths in test_token_counter.py (#11119)
* Update litellm/model_prices_and_context_window_backup.json
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
---------
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* refactor github copilot
* test_github_copilot_transformation.py
* test_github_copilot_authenticator.py
* add GitHub Copilot
* fix order
* doc fix
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
* fix(docs): Remove unused dotenv dependency from docusaurus config
The dotenv package was being required in docusaurus.config.js but was listed as
a devDependency, causing build failures. Since no environment variables are
actually used in the config, removed the unnecessary import.
* fix(docs): Remove reference to non-existent spending_monitoring doc
The sidebars.js file was referencing proxy/spending_monitoring which was deleted
in commit ba7463b9c. This was causing the documentation build to fail with missing
document errors.
* docs: add Elasticsearch logging tutorial and update sidebar
* docs: update Elasticsearch logging tutorial to include OpenTelemetry setup and configuration
* docs: remove sections from Elasticsearch logging tutorial
* docs: remove analytics examples from Elasticsearch logging tutorial
* Update Elasticsearch version and logging exporter configuration in the Elasticsearch logging tutorial
* Add visualization instructions for LLM telemetry data in Kibana to Elasticsearch logging tutorial
* Add Elasticsearch demo image to documentation
* Move demo image for Elasticsearch logging tutorial
* fix: allow setting no-default-models and unsetting max budget
* docs(sso_self_serve.md): add e2e tutorial of onboarding users for ai exploration
* docs: rename doc
* docs: track which items need docs
* docs(anthropic.md): add tool_choice="none" to docs
* docs: add docs for new anthropic + perplexity features
* docs: cleanup mistral reasoning docs
* docs: add links to docs
* docs(index.md): update docs
* docs: refactor to add a new 'integrations' tab to docs
* refactor(docs/): create separate tab for integrations
make it easier to highlight new integrations
* docs: sort sidebar
* docs: update
* feat: working claude code with openai codex mini
* docs: add responses api to docs
* feat(index.md): update docs
* fix: fix linting error
* docs: track which items need docs
* docs(anthropic.md): add tool_choice="none" to docs
* docs: add docs for new anthropic + perplexity features
* docs: cleanup mistral reasoning docs
* docs: add links to docs
* docs(index.md): update docs
* docs: refactor to add a new 'integrations' tab to docs
* refactor(docs/): create separate tab for integrations
make it easier to highlight new integrations
* docs: sort sidebar
* docs: update
* feat(langfuse_otel): add Langfuse OpenTelemetry integration for observability
- Introduced a new integration for Langfuse OpenTelemetry, allowing users to send LiteLLM traces and observability data.
- Updated sidebars to include documentation for the new integration.
- Added example usage and configuration details in the documentation.
- Implemented necessary classes and methods to handle OpenTelemetry attributes and configuration.
- Included tests to validate the integration functionality and environment variable handling.
Still WIP
* Remove example script for Langfuse OpenTelemetry integration with LiteLLM
* Feature/lasso guardrail (#9002)
* first version of lasso guardrail in litellm
* update to the new Lasso API
* change prod api_base and kill the request when lasso detect issue.
* change test for now api, local test pass
* add async tests
* all tests pass
* add docs for the new lasso guardrail
* Remove support for modes other than pre_call in Lasso guardrail
* code structure and naming
* only pre_call docs
* fix lint errors
* move test to the new location follows the same directory structure as litellm/.
* add lasso guard
* docs lasso docs
* add lasso guardrail
* fix lasso guardrail
---------
Co-authored-by: oroxenberg <oro@lasso.security>
Ishaan Jaff