mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-25 00:26:26 +00:00
Cole McIntoshandGitHub
506dc80b15
* feat: integrate Google Cloud Model Armor guardrails (LIT-298) - Add ModelArmorGuardrail class that extends CustomGuardrail and VertexBase - Support for both pre-call (sanitizeUserPrompt) and post-call (sanitizeModelResponse) sanitization - Integrate with existing Vertex AI authentication using VertexBase - Add configuration model for Model Armor in guardrail types - Register Model Armor in guardrail initializers and registry - Include comprehensive test suite for Model Armor functionality - Support content masking for both requests and responses - Handle streaming responses with content sanitization This integration allows LiteLLM to use Google Cloud Model Armor API for content moderation and sanitization, providing similar functionality to Bedrock Guards but using Google Cloud's security infrastructure. * fix: remove unused imports flagged by ruff linter - Remove unused asyncio import at top level (moved to local import where needed) - Remove unused TextCompletionResponse import * fix: remove additional unused imports - Remove unused TextChoices import from line 34 - Remove duplicate asyncio import from line 384 - Replace asyncio.iscoroutine() with hasattr check for __await__ * fix: remove final unused imports - Remove unused 'import sys' from line 9 - Remove unused 'StreamingChoices' from imports * fix: remove unused import from model_armor.py - Remove unused 'import os' from the top of the file * fix: remove commented-out header from model_armor.py - Eliminate unnecessary comments at the top of the file to improve code clarity. * feat(guardrails): Add Model Armor UI support - Convert model_armor.py to a directory structure with __init__.py for dynamic discovery - Add get_config_model() method to ModelArmorGuardrail class for UI integration - Add ui_friendly_name() to ModelArmorConfigModel returning "Google Cloud Model Armor" - Remove manual registration from guardrail_registry.py to use dynamic discovery - Model Armor now appears in the UI guardrails dropdown with proper configuration fields This enables users to configure Model Armor guardrails through the LiteLLM UI interface. * fix(guardrails): Fix undefined name 'GuardrailConfigModel' in Model Armor - Import TYPE_CHECKING and GuardrailConfigModel from base module - Fixes F821 linting error for undefined name in type annotation - Follows same pattern as other guardrail implementations * fix(guardrails): Fix Model Armor type errors and config model inheritance - Create ModelArmorGuardrailConfigModel that properly inherits from GuardrailConfigModel base class - Move config model to litellm/types/proxy/guardrails/guardrail_hooks/model_armor.py following convention - Update get_config_model() to return the properly typed config model - Remove ModelArmorConfigModel from LitellmParams inheritance chain - Add template_id field to BaseLitellmParams instead This fixes the mypy type errors and follows the same pattern as other guardrail implementations. * fix(guardrails): Add missing Model Armor fields to BaseLitellmParams - Add location, credentials, api_endpoint, and fail_on_error fields - Fixes mypy errors about missing attributes in LitellmParams - All Model Armor configuration parameters are now properly defined * fix: Apply PR review feedback for Model Armor guardrail - Move test file to tests/test_litellm/ for GitHub Actions - Extract only last consecutive user messages to avoid context limits - Use get_content_from_model_response helper for response extraction - Handle non-ModelResponse types (e.g., TTS) gracefully - Maintain newline separation for multi-part content * refactor: Simplify message content extraction in ModelArmorGuardrail - Removed the custom _extract_content_from_messages method. - Integrated get_last_user_message helper for improved content extraction. - Updated test to reflect changes in content formatting. * refactor: Remove unused import in model_armor.py - Deleted the unused import of AllMessageValues to clean up the codebase. * Add unit tests for Model Armor guardrail functionality - Implement tests for pre-call and post-call hooks, including content sanitization and blocking behavior. - Validate error handling for API responses and credential management. - Test streaming responses and handling of list content in user messages. - Ensure proper assertions for API interactions and response sanitization. * Add comprehensive test coverage for Model Armor guardrail - Add test for requests with no messages field - Add test for empty message content handling - Add test for system/assistant-only messages - Add test for fail_on_error=False behavior - Add test for custom API endpoint configuration - Add test for dictionary credentials (non-file path) - Add test for action=NONE response handling - Add test for missing sanitized_text field fallback - Add test for non-text response types (TTS/image) - Add test for auth token refresh behavior All tests ensure robust edge case handling and proper error management. * feat: improve Model Armor handling of non-ModelResponse types - Add debug logging when skipping non-text responses (TTS, images, etc.) - Improve docstring to clarify behavior for non-text responses - Add test coverage for non-ModelResponse handling - Ensure guardrail gracefully skips processing for response types it cannot handle
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/litellm
This folder can only run mock tests.