mirror of
https://github.com/tiennm99/litellm.git
synced 2026-06-18 21:35:06 +00:00
ryanh-ai
374c3458d5
* feat: add sagemaker_nova provider for Nova models on SageMaker Add support for custom/fine-tuned Amazon Nova models (Nova Micro, Nova Lite, Nova 2 Lite) deployed on SageMaker Inference real-time endpoints. Nova uses OpenAI-compatible request/response format with additional Nova-specific parameters (top_k, reasoning_effort, allowed_token_ids, truncate_prompt_tokens) and requires stream:true in the request body. Nova endpoints also reject 'model' in the request body. Changes: - New provider: sagemaker_nova/<endpoint-name> - SagemakerNovaConfig inherits from SagemakerChatConfig - Override transform_request to strip 'model' from request body - Override supports_stream_param_in_request_body (True for Nova) - Extend get_supported_openai_params with Nova-specific params - Refactored SagemakerChatConfig to use custom_llm_provider param instead of hardcoded strings (backwards-compatible) - Consolidated main.py routing for sagemaker_chat and sagemaker_nova - 22 unit tests + 9 integration tests (skip-gated) - Documentation with SDK, streaming, multimodal, and proxy examples - All tests verified against live SageMaker Nova endpoint * fix: move integration tests to tests/local_testing/ per test directory policy * fix: remove unused module-level SagemakerNovaConfig instance The sagemaker_nova_config singleton was never imported or used — the ProviderConfigManager creates its own instance via the lambda registered in utils.py. Removing this leftover boilerplate. --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.