* feat(proxy_server.py): support batch polling interval
allows admin to control batch polling interval (default is 3600s)
easier debugging
* fix(proxy_settings_endpoint.py): ensure value is actually set before updating env var
- Add comprehensive documentation for Model Armor integration
- Include configuration examples and parameter descriptions
- Add Model Armor to sidebars navigation
- Document authentication methods and error handling
* feat: add Morph provider support
- Add MorphChatConfig implementation for OpenAI-compatible API
- Support morph-v3-fast and morph-v3-large models
- Add pricing: morph-v3-fast (/bin/zsh.8/.2 per 1M tokens), morph-v3-large (/bin/zsh.9/.9 per 1M tokens)
- Both models support 16k context window and system messages
- Add comprehensive documentation and unit tests
- Update all necessary integration points (constants, init, provider logic)
* feat: Add Morph provider support in ProviderConfigManager
- Extend ProviderConfigManager to include MorphChatConfig for the Morph LLM provider.
- Update MorphChatConfig by removing unused parameters from the configuration.
- Add Hyperbolic as a new OpenAI-compatible provider
- Implement HyperbolicChatConfig inheriting from OpenAILikeChatConfig
- Register Hyperbolic in provider lists and constants
- Add comprehensive model configurations with pricing for:
- DeepSeek models (V3, R1, etc.)
- Qwen models (2.5, 3, QwQ, etc.)
- Meta Llama models (3.1, 3.2, 3.3)
- Other models like Kimi K2, Hermes 3, etc.
- Configure default API base URL: https://api.hyperbolic.xyz/v1
- Add provider documentation with usage examples
- Create unit tests for provider functionality
- Support all standard OpenAI parameters
Hyperbolic provides low-cost inference with OpenAI-compatible APIs,
supporting latest models without infrastructure overhead.
* feat: add Lambda AI provider support
Add support for Lambda AI (lambda.ai) as a new LLM provider in LiteLLM. Lambda AI provides access to a wide range of open-source models through their cloud GPU infrastructure.
Changes:
- Add Lambda AI provider implementation (OpenAI-compatible)
- Register 20 Lambda AI models with accurate pricing and 131k context windows
- Add comprehensive tests for Lambda AI integration
- Add detailed documentation with usage examples
- Use "lambda_ai" as provider name to avoid Python keyword conflict
Models include Llama 3.x, DeepSeek, Hermes, Qwen, and specialized models for coding and vision tasks.
* fix(tests): ensure lambda_ai_models list is repopulated after model cost reload
Updated test cases to clear and repopulate the lambda_ai_models list after reloading the model cost map. This ensures that the tests accurately reflect the current state of available models.
* feat: add Lambda AI chat configuration support
Added support for Lambda AI chat configuration in the ProviderConfigManager. This enhancement allows the integration of Lambda AI as a provider, expanding the capabilities of LiteLLM.
* feat(model_hub_table.tsx): add ability for admin to add links to model hub
allows admin to add model + key request access forms to model hub
makes it easier to request access to specific models
* refactor(ui/): cleanup ui - consistent styling
* fix(useful_links_management.tsx): make tab collapsible and explain purpose
* fix(ui/): fix ui linting errors
* fix: fix linting error
* Feature/track bedrock gov cloud models (#12771)
* feat: add AWS Bedrock GovCloud model support (LIT-257)
- Added 18 GovCloud-specific model entries (9 per region) to model_prices_and_context_window.json
- Updated is_bedrock_pricing_only_model() to allow GovCloud models (us-gov-east-1, us-gov-west-1)
- Added comprehensive test suite for GovCloud model support
- Ensures GovCloud models use appropriate APIs (Converse for Claude/Llama, Invoke for Titan)
Models added:
- Claude 3.5 Sonnet and Claude 3 Haiku (FedRAMP/IL4/5 approved)
- Llama 3 8B and 70B (FedRAMP/IL4/5 approved)
- Amazon Titan Text and Embedding models
* fix: add bedrock_converse GovCloud model mappings for Claude models
Added missing bedrock_converse model entries for AWS GovCloud regions:
- bedrock_converse/us-gov-east-1/anthropic.claude-3-5-sonnet-20240620-v1:0
- bedrock_converse/us-gov-east-1/anthropic.claude-3-haiku-20240307-v1:0
- bedrock_converse/us-gov-west-1/anthropic.claude-3-5-sonnet-20240620-v1:0
- bedrock_converse/us-gov-west-1/anthropic.claude-3-haiku-20240307-v1:0
This fixes test failures where supports_tool_choice() returned True but
the models weren't properly mapped in the configuration files.
* fix: correct AWS GovCloud Bedrock model pricing and configurations
- Fix Claude 3.5 Sonnet pricing (3.6e-06 input, 1.8e-05 output)
- Fix Claude 3 Haiku pricing (3e-07 input, 1.5e-06 output)
- Update Claude 3.5 Sonnet max_tokens from 4096 to 8192
- Add bedrock_converse entries for Llama models with correct token limits
- Add Amazon Nova Pro model for both GovCloud regions
- Add supports_pdf_input flag to Claude models
* fix: handle bedrock_converse prefix in get_non_litellm_routing_model_name
Fixes test failure where bedrock_converse/region/model paths were not properly
stripped to get the base model name, causing supports_function_calling to
return false for regional bedrock_converse models.
* revert: reset bedrock/common_utils.py to match main branch
Remove bedrock_converse prefix handling from get_non_litellm_routing_model_name
to align with main branch implementation.
* revert: reset litellm/__init__.py to match main branch
- Remove public_model_groups variables
- Remove GovCloud exception handling in is_bedrock_pricing_only_model
- Fix comment formatting
* revert: reset litellm/__init__.py to exact main branch content
Copy exact content from origin/main with no modifications
* fix: remove bedrock_converse prefixed models from pricing files
- Remove 10 bedrock_converse entries from model_prices_and_context_window.json
- Remove 4 bedrock_converse entries from litellm/model_prices_and_context_window_backup.json
- These were GovCloud-specific entries that are no longer needed
* fix: correct AWS GovCloud Bedrock model pricing and configurations
- Fix Anthropic Claude 3.5 Sonnet pricing: $3.60/$18.00 per million tokens (was $3.00/$15.00)
- Fix Anthropic Claude 3 Haiku pricing: $0.30/$1.50 per million tokens (was $0.25/$1.25)
- Fix Claude 3.5 Sonnet max_tokens: 8192 (was 4096)
- Fix Llama model max_tokens: 2048 (was 8192) and max_input_tokens: 8000 (was 8192)
- Fix Llama3-8b output pricing: $2.65 per million tokens (was $0.60)
- Add missing Amazon Nova Pro models for both GovCloud regions
- Add supports_pdf_input flag to Llama models
Based on official AWS Bedrock pricing documentation for GovCloud regions
* test: fix GovCloud bedrock models test to match implementation
Update test_govcloud_model_in_bedrock_models_list to correctly verify that
GovCloud models are excluded from bedrock_models list as they are
pricing-only models following the bedrock/<region>/<model> pattern.
---------
Co-authored-by: Cole McIntosh <colemcintosh6@gmail.com>
Co-authored-by: Cole McIntosh <82463175+colesmcintosh@users.noreply.github.com>
* add tests
* add tests
* Added test costs
* Added test costs
---------
Co-authored-by: Cole McIntosh <colemcintosh6@gmail.com>
Co-authored-by: Cole McIntosh <82463175+colesmcintosh@users.noreply.github.com>
Krish Dholakia