mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-19 12:18:35 +00:00
+18









Cesar Garcia
GitHub
An Tang
janfrederickk
Zhenting Huang
Darien Kindlund
Claude Opus 4.6
yuneng-jiang
Ryan Crabbe
Krish Dholakia
LeeJuOh
Monesh Ram
Trevor Prater
The Mavik
Edwin Isac
milan-berri
Ishaan Jaff
Sameer Kankute
Harshit Jain
Harshit Jain
Ephrim Stanley
TomAlon
Julio Quinteros Pro
greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
ryan-crabbe
Ron Zhong
Arindam Majumder
Lei Nie
9495f4e941
* auth_with_role_name add region_name arg for cross-account sts * update tests to include case with aws_region_name for _auth_with_aws_role * Only pass region_name to STS client when aws_region_name is set * Add optional aws_sts_endpoint to _auth_with_aws_role * Parametrize ambient-credentials test for no opts, region_name, and aws_sts_endpoint * consistently passing region and endpoint args into explicit credentials irsa * fix env var leakage * fix: bedrock openai-compatible imported-model should also have model arn encoded * feat: show proxy url in ModelHub (#21660) * fix(bedrock): correct modelInput format for Converse API batch models (#21656) * fix(proxy): add model_ids param to access group endpoints for precise deployment tagging (#21655) POST /access_group/new and PUT /access_group/{name}/update now accept an optional model_ids list that targets specific deployments by their unique model_id, instead of tagging every deployment that shares a model_name. When model_ids is provided it takes priority over model_names, giving API callers the same single-deployment precision that the UI already has via PATCH /model/{model_id}/update. Backward compatible: model_names continues to work as before. Closes #21544 * feat(proxy): add custom favicon support\n\nAdd ability to configure a custom favicon for the litellm proxy UI.\n\n- Add favicon_url field to UIThemeConfig model\n- Add LITELLM_FAVICON_URL env var support\n- Add /get_favicon endpoint to serve custom favicons\n- Update ThemeContext to dynamically set favicon\n- Add favicon URL input to UI theme settings page\n- Add comprehensive tests\n\nCloses #8323 (#21653) * fix(bedrock): prevent double UUID in create_file S3 key (#21650) In create_file for Bedrock, get_complete_file_url is called twice: once in the sync handler (generating UUID-1 for api_base) and once inside transform_create_file_request (generating UUID-2 for the actual S3 upload). The Bedrock provider correctly writes UUID-2 into litellm_params["upload_url"], but the sync handler unconditionally overwrites it with api_base (UUID-1). This causes the returned file_id to point to a non-existent S3 key. Fix: only set upload_url to api_base when transform_create_file_request has not already set it, preserving the Bedrock provider's value. Closes #21546 * feat(semantic-cache): support configurable vector dimensions for Qdrant (#21649) Add vector_size parameter to QdrantSemanticCache and expose it through the Cache facade as qdrant_semantic_cache_vector_size. This allows users to use embedding models with dimensions other than the default 1536, enabling cheaper/stronger models like Stella (1024d), bge-en-icl (4096d), voyage, cohere, etc. The parameter defaults to QDRANT_VECTOR_SIZE (env var or 1536) for backward compatibility. When creating new collections, the configured vector_size is used instead of the hardcoded constant. Closes #9377 * fix(utils): normalize camelCase thinking param keys to snake_case (#21762) Clients like OpenCode's @ai-sdk/openai-compatible send budgetTokens (camelCase) instead of budget_tokens in the thinking parameter, causing validation errors. Add early normalization in completion(). * feat: add optional digest mode for Slack alert types (#21683) Adds per-alert-type digest mode that aggregates duplicate alerts within a configurable time window and emits a single summary message with count, start/end timestamps. Configuration via general_settings.alert_type_config: alert_type_config: llm_requests_hanging: digest: true digest_interval: 86400 Digest key: (alert_type, request_model, api_base) Default interval: 24 hours Window type: fixed interval Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add blog_posts.json and local backup * feat: add GetBlogPosts utility with GitHub fetch and local fallback Adds GetBlogPosts class that fetches blog posts from GitHub with a 1-hour in-process TTL cache, validates the response, and falls back to the bundled blog_posts_backup.json on any network or validation failure. * test: add cache reset fixture and LITELLM_LOCAL_BLOG_POSTS test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add GET /public/litellm_blog_posts endpoint Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: log fallback warning in blog posts endpoint and tighten test * feat: add disable_show_blog to UISettings Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add useUISettings and useDisableShowBlog hooks * fix: rename useUISettings to useUISettingsFlags to avoid naming collision * fix: use existing useUISettings hook in useDisableShowBlog to avoid cache duplication Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add BlogDropdown component with react-query and error/retry state Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: enforce 5-post limit in BlogDropdown and add cap test * fix: add retry, stable post key, enabled guard in BlogDropdown Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add BlogDropdown to navbar after Docs link * feat: add network_mock transport for benchmarking proxy overhead without real API calls Intercepts at httpx transport layer so the full proxy path (auth, routing, OpenAI SDK, response transformation) is exercised with zero-latency responses. Activated via `litellm_settings: { network_mock: true }` in proxy config. * Litellm dev 02 19 2026 p2 (#21871) * feat(ui/): new guardrails monitor 'demo mock representation of what guardrails monitor looks like * fix: ui updates * style(ui/): fix styling * feat: enable running ai monitor on individual guardrails * feat: add backend logic for guardrail monitoring * fix(guardrails/usage_endpoints.py): fix usage dashboard * fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo (#21754) * fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo * fix(budget): update stale docstring on get_budget_reset_time * fix: add missing return type annotations to iterator protocol methods in streaming_handler (#21750) * fix: add return type annotations to iterator protocol methods in streaming_handler Add missing return type annotations to __iter__, __aiter__, __next__, and __anext__ methods in CustomStreamWrapper and related classes. - __iter__(self) -> Iterator["ModelResponseStream"] - __aiter__(self) -> AsyncIterator["ModelResponseStream"] - __next__(self) -> "ModelResponseStream" - __anext__(self) -> "ModelResponseStream" Also adds AsyncIterator and Iterator to typing imports. Fixes issue with PLR0915 noqa comments and ensures proper type checking support. Related to: BerriAI/litellm#8304 * fix: add ruff PLR0915 noqa for files with too many statements * Add gollem Go agent framework cookbook example (#21747) Show how to use gollem, a production Go agent framework, with LiteLLM proxy for multi-provider LLM access including tool use and streaming. * fix: avoid mutating caller-owned dicts in SpendUpdateQueue aggregation (#21742) * fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870) * server root path regression doc * fixing syntax * fix: replace Zapier webhook with Google Form for survey submission (#21621) * Replace Zapier webhook with Google Form for survey submission * Add back error logging for survey submission debugging --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> * Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth" This reverts commit0e1db3f7e4, reversing changes made to7e2d6f2355. * test_vertex_ai_gemini_2_5_pro_streaming * UI new build * fix rendering * ui new build * docs fix * docs fix * docs fix * docs fix * docs fix * docs fix * docs fix * docs fix * release note docs * docs * adding image * fix(vertex_ai): enable context-1m-2025-08-07 beta header The `context-1m-2025-08-07` Anthropic beta header was set to `null` for vertex_ai, causing it to be filtered out when users set `extra_headers: {anthropic-beta: context-1m-2025-08-07}`. This prevented using Claude's 1M context window feature via Vertex AI, resulting in `prompt is too long: 460500 tokens > 200000 maximum` errors. Fixes #21861 --------- Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: milan-berri <milan@berri.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> * Revert "fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)" (#21876) This reverts commitbce078a796. * docs(ui): add pre-PR checklist to UI contributing guide Add testing and build verification steps per maintainer feedback from @yjiang-litellm. Contributors should run their related tests per-file and ensure npm run build passes before opening PRs. * Fix entries with fast and us/ * Add tests for fast and us * Add support for Priority PayGo for vertex ai and gemini * Add model pricing * fix: ensure arrival_time is set before calculating queue time * Fix: Anthropic model wildcard access issue * Add incident report * Add ability to see which model cost map is getting used * Fix name of title * Readd tpm limit * State management fixes for CheckBatchCost * Fix PR review comments * State management fixes for CheckBatchCost - Address greptile comments * fix mypy issues: * Add Noma guardrails v2 based on custom guardrails (#21400) * Fix code qa issues * Fix mypy issues * Fix mypy issues * Fix test_aaamodel_prices_and_context_window_json_is_valid * fix: update calendly on repo * fix(tests): use counter-based mock for time.time in prisma self-heal test The test used a fixed side_effect list for time.time(), but the number of calls varies by Python version, causing StopIteration on 3.12 and AssertionError on 3.14. Replace with an infinite counter-based callable and assert the timestamp was updated rather than checking for an exact value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tests): use absolute path for model_prices JSON in validation test The test used a relative path 'litellm/model_prices_and_context_window.json' which only works when pytest runs from a specific working directory. Use os.path based on __file__ to resolve the path reliably. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update tests/test_litellm/test_utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(tests): use os.path instead of Path to avoid NameError Path is not imported at module level. Use os.path.join which is already available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * clean up mock transport: remove streaming, add defensive parsing * docs: add Google GenAI SDK tutorial (JS & Python) (#21885) * docs: add Google GenAI SDK tutorial for JS and Python Add tutorial for using Google's official GenAI SDK (@google/genai for JS, google-genai for Python) with LiteLLM proxy. Covers pass-through and native router endpoints, streaming, multi-turn chat, and multi-provider routing via model_group_alias. Also updates pass-through docs to use the new SDK replacing the deprecated @google/generative-ai. * fix(docs): correct Python SDK env var name in GenAI tutorial GOOGLE_GENAI_API_KEY does not exist in the google-genai SDK. The correct env var is GEMINI_API_KEY (or GOOGLE_API_KEY). Also note that the Python SDK has no base URL env var. * fix(docs): replace non-existent GOOGLE_GENAI_BASE_URL env var in interactions.md The Python google-genai SDK does not read GOOGLE_GENAI_BASE_URL. Use http_options={"base_url": "..."} in code instead. * docs: add network mock benchmarking section * docs: tweak benchmarks wording * fix: add auth headers and empty latencies guard to benchmark script * refactor: use method-level import for MockOpenAITransport * fix: guard print_aggregate against empty latencies * fix: add INCOMPLETE status to Interactions API enum and test Google added INCOMPLETE to the Interactions API OpenAPI spec status enum. Update both the Status3 enum in the SDK types and the test's expected values to match. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Guardrail Monitor - measure guardrail reliability in prod (#21944) * fix: fix log viewer for guardrail monitoring * feat(ui/): fix rendering logs per guardrail * fix: fix viewing logs on overview tab of guardrail * fix: log viewer * fix: fix naming to align with metric * docs: add performance & reliability section to v1.81.14 release notes * fix(tests): make RPM limit test sequential to avoid race condition Concurrent requests via run_in_executor + asyncio.gather caused a race condition where more requests slipped through the rate limiter than expected, leading to flaky test failures (e.g. 3 successes instead of 2 with rpm_limit=2). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: Singapore guardrail policies (PDPA + MAS AI Risk Management) (#21948) * feat: Singapore PDPA PII protection guardrail policy template Add Singapore Personal Data Protection Act (PDPA) guardrail support: Regex patterns (patterns.json): - sg_nric: NRIC/FIN detection ([STFGM] + 7 digits + checksum letter) - sg_phone: Singapore phone numbers (+65/0065/65 prefix) - sg_postal_code: 6-digit postal codes (contextual) - passport_singapore: Passport numbers (E/K + 7 digits, contextual) - sg_uen: Unique Entity Numbers (3 formats) - sg_bank_account: Bank account numbers (dash format, contextual) YAML policy templates (5 sub-guardrails): - sg_pdpa_personal_identifiers: s.13 Consent - sg_pdpa_sensitive_data: Advisory Guidelines - sg_pdpa_do_not_call: Part IX DNC Registry - sg_pdpa_data_transfer: s.26 overseas transfers - sg_pdpa_profiling_automated_decisions: Model AI Governance Framework Policy template entry in policy_templates.json with 9 guardrail definitions (4 regex-based + 5 YAML conditional keyword matching). Tests: - test_sg_patterns.py: regex pattern unit tests - test_sg_pdpa_guardrails.py: conditional keyword matching tests (100+ cases) * feat: MAS AI Risk Management Guidelines guardrail policy template Add Monetary Authority of Singapore (MAS) AI Risk Management Guidelines guardrail support for financial institutions: YAML policy templates (5 sub-guardrails): - sg_mas_fairness_bias: Blocks discriminatory financial AI (credit/loans/insurance by protected attributes) - sg_mas_transparency_explainability: Blocks opaque/unexplainable AI for consequential financial decisions - sg_mas_human_oversight: Blocks fully automated financial decisions without human-in-the-loop - sg_mas_data_governance: Blocks unauthorized sharing/mishandling of financial customer data - sg_mas_model_security: Blocks adversarial attacks, model poisoning, inversion on financial AI Policy template entry in policy_templates.json with 5 guardrail definitions. Aligned with MAS FEAT Principles, Project MindForge, and NIST AI RMF. Tests: - test_sg_mas_ai_guardrails.py: conditional keyword matching tests (100+ cases) * fix: address SG pattern review feedback - Update NRIC lowercase test for IGNORECASE runtime behavior - Add keyword context guard to sg_uen pattern to reduce false positives * docs: clarify MAS AIRM timeline references - Explicitly mark MAS AIRM as Nov 2025 consultation draft - Add 2018 qualifier for FEAT principles in MAS policy descriptions - Update MAS guardrail wording to avoid release-year ambiguity * chore: commit resolved MAS policy conflicts * test: * chore: * Add OpenAI Agents SDK tutorial with LiteLLM Proxy to docs (#21221) * Add OpenAI Agents SDK tutorial to docs * Update OpenAI Agents SDK tutorial to use LiteLLM environment variables * Enhance OpenAI Agents SDK tutorial with built-in LiteLLM extension details and updated configuration steps. Adjust section headings for clarity and improve the flow of information regarding model setup and usage. * adjust blog posts to fetch from github first * feat(videos): add variant parameter to video content download (#21955) openai videos models support the features to download variants. See more details here: https://developers.openai.com/api/docs/guides/video-generation#use-image-references. Plumb variant (e.g. "thumbnail", "spritesheet") through the full video content download chain: avideo_content → video_content → video_content_handler → transform_video_content_request. OpenAI appends ?variant=<value> to the GET URL; other providers accept the parameter in their signature but ignore it. * fixing path * adjust blog post path * Revert duplicate issue checker to text-based matching, remove duplicate PR workflow Remove the Claude Code-powered duplicate PR detection workflow and revert the duplicate issue checker back to wow-actions/potential-duplicates with text similarity matching. * ui changes * adding tests * adjust default aggregation threshold * fix(videos): pass api_key from litellm_params to video remix handlers (#21965) video_remix_handler and async_video_remix_handler were not falling back to litellm_params.api_key when the api_key parameter was None, causing Authorization: Bearer None to be sent to the provider. This matches the pattern already used by async_video_generation_handler. * adding testing coverage + fixing flaky tests * fix(ollama): thread api_base through get_model_info and add graceful fallback When users pass api_base to litellm.completion() for Ollama, the model info fetch (context window, function_calling support) was ignoring the user's api_base and only reading OLLAMA_API_BASE env var or defaulting to localhost:11434. This caused confusing errors in logs when Ollama runs on a remote server. Thread api_base from litellm_params through the get_model_info call chain so OllamaConfig.get_model_info() uses the correct server. Also return safe defaults instead of raising when the server is unreachable. Fixes #21967 --------- Co-authored-by: An Tang <ta@stripe.com> Co-authored-by: janfrederickk <75388864+janfrederickk@users.noreply.github.com> Co-authored-by: Zhenting Huang <3061613175@qq.com> Co-authored-by: Darien Kindlund <darien@kindlund.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu> Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: LeeJuOh <56071126+LeeJuOh@users.noreply.github.com> Co-authored-by: Monesh Ram <31161039+WhoisMonesh@users.noreply.github.com> Co-authored-by: Trevor Prater <trevor.prater@gmail.com> Co-authored-by: The Mavik <179817126+themavik@users.noreply.github.com> Co-authored-by: Edwin Isac <33712823+edwiniac@users.noreply.github.com> Co-authored-by: milan-berri <milan@berri.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: Harshit Jain <harshitjain0562@gmail.com> Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com> Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com> Co-authored-by: TomAlon <tom@noma.security> Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com> Co-authored-by: Ron Zhong <ron-zhong@hotmail.com> Co-authored-by: Arindam Majumder <109217591+Arindam200@users.noreply.github.com> Co-authored-by: Lei Nie <lenie@quora.com>
1626 lines
64 KiB
Python
1626 lines
64 KiB
Python
import asyncio
|
|
import json
|
|
import os
|
|
import sys
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
import pytest
|
|
|
|
sys.path.insert(
|
|
0, os.path.abspath("../..")
|
|
) # Adds the parent directory to the system path
|
|
|
|
import litellm
|
|
from litellm.cost_calculator import default_video_cost_calculator
|
|
from litellm.integrations.custom_logger import CustomLogger
|
|
from litellm.litellm_core_utils.litellm_logging import Logging as LitellmLogging
|
|
from litellm.llms.custom_httpx.http_handler import AsyncHTTPHandler
|
|
from litellm.llms.custom_httpx.llm_http_handler import BaseLLMHTTPHandler
|
|
from litellm.llms.gemini.videos.transformation import GeminiVideoConfig
|
|
from litellm.llms.openai.videos.transformation import OpenAIVideoConfig
|
|
from litellm.types.videos.main import VideoObject, VideoResponse
|
|
from litellm.videos import main as videos_main
|
|
from litellm.videos.main import (
|
|
avideo_generation,
|
|
avideo_status,
|
|
video_generation,
|
|
video_status,
|
|
)
|
|
|
|
|
|
class TestVideoGeneration:
|
|
"""Test suite for video generation functionality."""
|
|
|
|
def test_video_generation_basic(self):
|
|
"""Test basic video generation functionality."""
|
|
# Use mock_response parameter for reliable testing
|
|
response = video_generation(
|
|
prompt="Show them running around the room",
|
|
model="sora-2",
|
|
seconds="8",
|
|
size="720x1280",
|
|
mock_response={
|
|
"id": "video_123",
|
|
"object": "video",
|
|
"status": "queued",
|
|
"created_at": 1712697600,
|
|
"model": "sora-2",
|
|
"size": "720x1280",
|
|
"seconds": "8"
|
|
}
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_123"
|
|
assert response.status == "queued"
|
|
assert response.model == "sora-2"
|
|
assert response.size == "720x1280"
|
|
assert response.seconds == "8"
|
|
|
|
def test_video_generation_with_mock_response(self):
|
|
"""Test video generation with mock response."""
|
|
mock_data = {
|
|
"id": "video_456",
|
|
"object": "video",
|
|
"status": "completed",
|
|
"created_at": 1712697600,
|
|
"completed_at": 1712697660,
|
|
"model": "sora-2",
|
|
"size": "1280x720",
|
|
"seconds": "10"
|
|
}
|
|
|
|
response = video_generation(
|
|
prompt="A beautiful sunset over the ocean",
|
|
model="sora-2",
|
|
seconds="10",
|
|
size="1280x720",
|
|
mock_response=mock_data
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_456"
|
|
assert response.status == "completed"
|
|
assert response.model == "sora-2"
|
|
assert response.size == "1280x720"
|
|
assert response.seconds == "10"
|
|
|
|
def test_video_generation_async(self):
|
|
"""Test async video generation functionality."""
|
|
mock_response = VideoObject(
|
|
id="video_async_123",
|
|
object="video",
|
|
status="processing",
|
|
created_at=1712697600,
|
|
model="sora-2",
|
|
progress=50
|
|
)
|
|
|
|
# Mock the async_video_generation_handler to return the mock_response
|
|
async_mock = AsyncMock(return_value=mock_response)
|
|
with patch.object(videos_main.base_llm_http_handler, 'async_video_generation_handler', async_mock):
|
|
with patch.object(videos_main.base_llm_http_handler, 'video_generation_handler', side_effect=lambda **kwargs: async_mock(**kwargs)):
|
|
import asyncio
|
|
|
|
async def test_async():
|
|
response = await avideo_generation(
|
|
prompt="A cat playing with a ball",
|
|
model="sora-2",
|
|
seconds="5",
|
|
size="720x1280"
|
|
)
|
|
return response
|
|
|
|
response = asyncio.run(test_async())
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_async_123"
|
|
assert response.status == "processing"
|
|
assert response.progress == 50
|
|
|
|
def test_video_generation_parameter_validation(self):
|
|
"""Test video generation parameter validation."""
|
|
# Test with minimal required parameters
|
|
response = video_generation(
|
|
prompt="Test video",
|
|
model="sora-2",
|
|
mock_response={"id": "test", "object": "video", "status": "queued", "created_at": 1712697600}
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "test"
|
|
|
|
def test_video_generation_error_handling(self):
|
|
"""Test video generation error handling."""
|
|
with patch.object(videos_main.base_llm_http_handler, 'video_generation_handler', side_effect=Exception("API Error")):
|
|
with pytest.raises(Exception):
|
|
video_generation(
|
|
prompt="Test video",
|
|
model="sora-2"
|
|
)
|
|
|
|
def test_video_generation_provider_config(self):
|
|
"""Test video generation provider configuration."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Test supported parameters
|
|
supported_params = config.get_supported_openai_params("sora-2")
|
|
assert "prompt" in supported_params
|
|
assert "model" in supported_params
|
|
assert "seconds" in supported_params
|
|
assert "size" in supported_params
|
|
|
|
def test_video_generation_request_transformation(self):
|
|
"""Test video generation request transformation."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Test request transformation
|
|
data, files, returned_api_base = config.transform_video_create_request(
|
|
model="sora-2",
|
|
prompt="Test video prompt",
|
|
api_base="https://api.openai.com/v1/videos",
|
|
video_create_optional_request_params={
|
|
"seconds": "8",
|
|
"size": "720x1280"
|
|
},
|
|
litellm_params=MagicMock(),
|
|
headers={}
|
|
)
|
|
|
|
assert data["model"] == "sora-2"
|
|
assert data["prompt"] == "Test video prompt"
|
|
assert data["seconds"] == "8"
|
|
assert data["size"] == "720x1280"
|
|
assert files == []
|
|
assert returned_api_base == "https://api.openai.com/v1/videos"
|
|
|
|
def test_video_generation_response_transformation(self):
|
|
"""Test video generation response transformation."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Mock HTTP response
|
|
mock_http_response = MagicMock()
|
|
mock_http_response.json.return_value = {
|
|
"id": "video_789",
|
|
"object": "video",
|
|
"status": "completed",
|
|
"created_at": 1712697600,
|
|
"model": "sora-2",
|
|
"size": "1280x720",
|
|
"seconds": "12"
|
|
}
|
|
|
|
response = config.transform_video_create_response(
|
|
model="sora-2",
|
|
raw_response=mock_http_response,
|
|
logging_obj=MagicMock()
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_789"
|
|
assert response.status == "completed"
|
|
assert response.model == "sora-2"
|
|
|
|
def test_video_generation_cost_calculation(self):
|
|
"""Test video generation cost calculation."""
|
|
import json
|
|
import os
|
|
|
|
# Try to load the local model cost map, skip if not found
|
|
cost_map_path = "model_prices_and_context_window.json"
|
|
if not os.path.exists(cost_map_path):
|
|
# Try alternative paths
|
|
alt_paths = [
|
|
os.path.join(os.path.dirname(__file__), "..", "..", cost_map_path),
|
|
os.path.join(os.path.dirname(__file__), "..", "..", "..", cost_map_path),
|
|
]
|
|
for path in alt_paths:
|
|
if os.path.exists(path):
|
|
cost_map_path = path
|
|
break
|
|
else:
|
|
pytest.skip("model_prices_and_context_window.json not found")
|
|
|
|
with open(cost_map_path, "r") as f:
|
|
litellm.model_cost = json.load(f)
|
|
|
|
# Test with sora-2 model
|
|
cost = default_video_cost_calculator(
|
|
model="openai/sora-2",
|
|
duration_seconds=10.0,
|
|
custom_llm_provider="openai"
|
|
)
|
|
|
|
# Should calculate cost based on duration (10 seconds * $0.10 per second = $1.00)
|
|
assert cost == 1.0
|
|
|
|
def test_video_generation_cost_calculation_unknown_model(self):
|
|
"""Test video generation cost calculation for unknown model."""
|
|
with pytest.raises(Exception, match="Model not found in cost map"):
|
|
default_video_cost_calculator(
|
|
model="unknown-model",
|
|
duration_seconds=5.0,
|
|
custom_llm_provider="openai"
|
|
)
|
|
|
|
def test_video_generation_cost_with_custom_model_info(self):
|
|
"""Test that custom model_info pricing is applied for video generation.
|
|
|
|
When a deployment has custom pricing via model_info, it should be used
|
|
instead of looking up the global litellm.model_cost map.
|
|
|
|
Related: https://github.com/BerriAI/litellm/issues/21907
|
|
"""
|
|
model_info = {
|
|
"output_cost_per_video_per_second": 0.05,
|
|
}
|
|
cost = default_video_cost_calculator(
|
|
model="my-custom-video-model",
|
|
duration_seconds=10.0,
|
|
model_info=model_info,
|
|
)
|
|
assert cost == 0.5
|
|
|
|
def test_video_generation_cost_custom_model_info_fallback_to_per_second(self):
|
|
"""Test that output_cost_per_second is used as fallback when
|
|
output_cost_per_video_per_second is not set in custom model_info.
|
|
|
|
Related: https://github.com/BerriAI/litellm/issues/21907
|
|
"""
|
|
model_info = {
|
|
"output_cost_per_second": 0.10,
|
|
}
|
|
cost = default_video_cost_calculator(
|
|
model="my-custom-video-model",
|
|
duration_seconds=5.0,
|
|
model_info=model_info,
|
|
)
|
|
assert cost == 0.5
|
|
|
|
def test_video_generation_cost_custom_pricing_through_completion_cost(self):
|
|
"""Test that custom video pricing flows through completion_cost via litellm_logging_obj.
|
|
|
|
This tests the full cost calculation path: completion_cost extracts model_info
|
|
from litellm_logging_obj.litellm_params.metadata.model_info and passes it to
|
|
the video cost calculator.
|
|
|
|
Related: https://github.com/BerriAI/litellm/issues/21907
|
|
"""
|
|
from litellm.cost_calculator import completion_cost
|
|
|
|
# Create mock response with usage containing duration_seconds
|
|
mock_response = MagicMock()
|
|
mock_response.usage = MagicMock()
|
|
mock_response.usage.duration_seconds = 10.0
|
|
type(mock_response)._hidden_params = {}
|
|
|
|
# Create mock litellm_logging_obj with custom pricing
|
|
mock_logging_obj = MagicMock()
|
|
mock_logging_obj.litellm_params = {
|
|
"metadata": {
|
|
"model_info": {
|
|
"output_cost_per_video_per_second": 0.05,
|
|
}
|
|
}
|
|
}
|
|
|
|
cost = completion_cost(
|
|
completion_response=mock_response,
|
|
model="openai/hunyuanvideo",
|
|
call_type="create_video",
|
|
custom_llm_provider="openai",
|
|
custom_pricing=True,
|
|
litellm_logging_obj=mock_logging_obj,
|
|
)
|
|
assert cost == 0.5
|
|
|
|
def test_video_generation_with_files(self):
|
|
"""Test video generation with file uploads."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Mock file data
|
|
mock_file = MagicMock()
|
|
mock_file.read.return_value = b"fake_image_data"
|
|
|
|
data, files, returned_api_base = config.transform_video_create_request(
|
|
model="sora-2",
|
|
prompt="Test video with image",
|
|
api_base="https://api.openai.com/v1/videos",
|
|
video_create_optional_request_params={
|
|
"input_reference": mock_file,
|
|
"seconds": "8",
|
|
"size": "720x1280"
|
|
},
|
|
litellm_params=MagicMock(),
|
|
headers={}
|
|
)
|
|
|
|
assert data["model"] == "sora-2"
|
|
assert data["prompt"] == "Test video with image"
|
|
assert len(files) > 0 # Should have files when input_reference is provided
|
|
|
|
def test_video_generation_environment_validation(self):
|
|
"""Test video generation environment validation."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Test environment validation
|
|
headers = config.validate_environment(
|
|
headers={},
|
|
model="sora-2",
|
|
api_key="test-api-key"
|
|
)
|
|
|
|
assert "Authorization" in headers
|
|
assert headers["Authorization"] == "Bearer test-api-key"
|
|
|
|
def test_video_generation_uses_api_key_from_litellm_params(self):
|
|
"""Test that video generation handler uses api_key from litellm_params when function parameter is None."""
|
|
handler = BaseLLMHTTPHandler()
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Mock the validate_environment method to capture the api_key passed to it
|
|
with patch.object(config, 'validate_environment') as mock_validate:
|
|
mock_validate.return_value = {"Authorization": "Bearer deployment-api-key"}
|
|
|
|
# Mock the transform and HTTP client
|
|
with patch.object(config, 'transform_video_create_request') as mock_transform:
|
|
mock_transform.return_value = ({"model": "sora-2", "prompt": "test"}, [], "https://api.openai.com/v1/videos")
|
|
|
|
# Mock the transform_video_create_response to avoid needing a real response
|
|
with patch.object(config, 'transform_video_create_response') as mock_transform_response:
|
|
mock_video_object = MagicMock()
|
|
mock_video_object.id = "video_123"
|
|
mock_video_object.object = "video"
|
|
mock_video_object.status = "queued"
|
|
mock_transform_response.return_value = mock_video_object
|
|
|
|
mock_response = MagicMock()
|
|
mock_response.json.return_value = {
|
|
"id": "video_123",
|
|
"object": "video",
|
|
"status": "queued",
|
|
"created_at": 1712697600,
|
|
"model": "sora-2"
|
|
}
|
|
mock_response.status_code = 200
|
|
|
|
mock_client = MagicMock()
|
|
mock_client.post.return_value = mock_response
|
|
|
|
with patch(
|
|
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
|
|
return_value=mock_client,
|
|
):
|
|
result = handler.video_generation_handler(
|
|
model="sora-2",
|
|
prompt="test prompt",
|
|
video_generation_provider_config=config,
|
|
video_generation_optional_request_params={},
|
|
custom_llm_provider="openai",
|
|
litellm_params={"api_key": "deployment-api-key", "api_base": "https://api.openai.com/v1"},
|
|
logging_obj=MagicMock(),
|
|
timeout=5.0,
|
|
api_key=None, # Function parameter is None
|
|
_is_async=False,
|
|
)
|
|
|
|
# Verify validate_environment was called with api_key from litellm_params
|
|
mock_validate.assert_called_once()
|
|
call_args = mock_validate.call_args
|
|
assert call_args.kwargs["api_key"] == "deployment-api-key"
|
|
|
|
def test_video_generation_url_generation(self):
|
|
"""Test video generation URL generation."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Test URL generation
|
|
url = config.get_complete_url(
|
|
model="sora-2",
|
|
api_base="https://api.openai.com/v1",
|
|
litellm_params={}
|
|
)
|
|
|
|
assert url == "https://api.openai.com/v1/videos"
|
|
|
|
def test_video_generation_parameter_mapping(self):
|
|
"""Test video generation parameter mapping."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Test parameter mapping
|
|
mapped_params = config.map_openai_params(
|
|
video_create_optional_params={
|
|
"seconds": "8",
|
|
"size": "720x1280",
|
|
"user": "test-user"
|
|
},
|
|
model="sora-2",
|
|
drop_params=False
|
|
)
|
|
|
|
assert mapped_params["seconds"] == "8"
|
|
assert mapped_params["size"] == "720x1280"
|
|
assert mapped_params["user"] == "test-user"
|
|
|
|
def test_video_generation_unsupported_parameters(self):
|
|
"""Test video generation with provider-specific parameters via extra_body."""
|
|
from litellm.videos.utils import VideoGenerationRequestUtils
|
|
|
|
# Test that provider-specific parameters can be passed via extra_body
|
|
# This allows support for Vertex AI and Gemini specific parameters
|
|
result = VideoGenerationRequestUtils.get_optional_params_video_generation(
|
|
model="sora-2",
|
|
video_generation_provider_config=OpenAIVideoConfig(),
|
|
video_generation_optional_params={
|
|
"seconds": "8",
|
|
"extra_body": {
|
|
"vertex_ai_param": "value",
|
|
"gemini_param": "value2"
|
|
}
|
|
}
|
|
)
|
|
|
|
# extra_body params should be merged into the result
|
|
assert result["seconds"] == "8"
|
|
assert result["vertex_ai_param"] == "value"
|
|
assert result["gemini_param"] == "value2"
|
|
# extra_body itself should be removed from the result
|
|
assert "extra_body" not in result
|
|
|
|
def test_video_generation_types(self):
|
|
"""Test video generation type definitions."""
|
|
# Test VideoObject
|
|
video_obj = VideoObject(
|
|
id="test_id",
|
|
object="video",
|
|
status="completed",
|
|
created_at=1712697600,
|
|
model="sora-2"
|
|
)
|
|
|
|
assert video_obj.id == "test_id"
|
|
assert video_obj.object == "video"
|
|
assert video_obj.status == "completed"
|
|
|
|
# Test dictionary-like access
|
|
assert video_obj["id"] == "test_id"
|
|
assert video_obj["status"] == "completed"
|
|
assert "id" in video_obj
|
|
assert video_obj.get("id") == "test_id"
|
|
assert video_obj.get("nonexistent", "default") == "default"
|
|
|
|
# Test JSON serialization
|
|
json_data = video_obj.json()
|
|
assert json_data["id"] == "test_id"
|
|
assert json_data["object"] == "video"
|
|
|
|
def test_video_generation_response_types(self):
|
|
"""Test video generation response types."""
|
|
# Test VideoResponse
|
|
video_obj = VideoObject(
|
|
id="test_id",
|
|
object="video",
|
|
status="completed",
|
|
created_at=1712697600
|
|
)
|
|
|
|
response = VideoResponse(data=[video_obj])
|
|
|
|
assert len(response.data) == 1
|
|
assert response.data[0].id == "test_id"
|
|
|
|
# Test dictionary-like access
|
|
assert response["data"][0]["id"] == "test_id"
|
|
assert "data" in response
|
|
assert response.get("data")[0]["id"] == "test_id"
|
|
|
|
# Test JSON serialization
|
|
json_data = response.json()
|
|
assert len(json_data["data"]) == 1
|
|
assert json_data["data"][0]["id"] == "test_id"
|
|
|
|
def test_video_status_basic(self):
|
|
"""Test basic video status functionality."""
|
|
# Use mock_response parameter for reliable testing
|
|
response = video_status(
|
|
video_id="video_123",
|
|
model="sora-2",
|
|
mock_response={
|
|
"id": "video_123",
|
|
"object": "video",
|
|
"status": "completed",
|
|
"created_at": 1712697600,
|
|
"completed_at": 1712697660,
|
|
"model": "sora-2",
|
|
"progress": 100,
|
|
"size": "720x1280",
|
|
"seconds": "8"
|
|
}
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_123"
|
|
assert response.status == "completed"
|
|
assert response.progress == 100
|
|
assert response.model == "sora-2"
|
|
|
|
def test_video_status_with_mock_response(self):
|
|
"""Test video status with mock response."""
|
|
mock_data = {
|
|
"id": "video_456",
|
|
"object": "video",
|
|
"status": "processing",
|
|
"created_at": 1712697600,
|
|
"model": "sora-2",
|
|
"progress": 75,
|
|
"size": "1280x720",
|
|
"seconds": "10"
|
|
}
|
|
|
|
response = video_status(
|
|
video_id="video_456",
|
|
model="sora-2",
|
|
mock_response=mock_data
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_456"
|
|
assert response.status == "processing"
|
|
assert response.progress == 75
|
|
assert response.model == "sora-2"
|
|
|
|
def test_video_status_async(self):
|
|
"""Test async video status functionality."""
|
|
mock_response = VideoObject(
|
|
id="video_async_123",
|
|
object="video",
|
|
status="queued",
|
|
created_at=1712697600,
|
|
model="sora-2",
|
|
progress=0
|
|
)
|
|
|
|
# Mock the async_video_status_handler to return the mock_response
|
|
async_mock = AsyncMock(return_value=mock_response)
|
|
with patch.object(videos_main.base_llm_http_handler, 'async_video_status_handler', async_mock):
|
|
with patch.object(videos_main.base_llm_http_handler, 'video_status_handler', side_effect=lambda **kwargs: async_mock(**kwargs)):
|
|
import asyncio
|
|
|
|
async def test_async():
|
|
response = await avideo_status(
|
|
video_id="video_async_123",
|
|
model="sora-2"
|
|
)
|
|
return response
|
|
|
|
response = asyncio.run(test_async())
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_async_123"
|
|
assert response.status == "queued"
|
|
assert response.progress == 0
|
|
|
|
def test_video_status_parameter_validation(self):
|
|
"""Test video status parameter validation."""
|
|
# Test with minimal required parameters
|
|
response = video_status(
|
|
video_id="test_video_id",
|
|
model="sora-2",
|
|
mock_response={"id": "test", "object": "video", "status": "completed", "created_at": 1712697600}
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "test"
|
|
|
|
def test_video_status_error_handling(self):
|
|
"""Test video status error handling."""
|
|
with patch.object(videos_main.base_llm_http_handler, 'video_status_handler', side_effect=Exception("API Error")):
|
|
with pytest.raises(Exception):
|
|
video_status(
|
|
video_id="test_video_id",
|
|
model="sora-2"
|
|
)
|
|
|
|
def test_video_status_request_transformation(self):
|
|
"""Test video status request transformation."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Test request transformation
|
|
url, data = config.transform_video_status_retrieve_request(
|
|
video_id="video_123",
|
|
api_base="https://api.openai.com/v1/videos",
|
|
litellm_params=MagicMock(),
|
|
headers={}
|
|
)
|
|
|
|
assert url == "https://api.openai.com/v1/videos/video_123"
|
|
assert data == {}
|
|
|
|
def test_video_status_response_transformation(self):
|
|
"""Test video status response transformation."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Mock HTTP response
|
|
mock_http_response = MagicMock()
|
|
mock_http_response.json.return_value = {
|
|
"id": "video_789",
|
|
"object": "video",
|
|
"status": "completed",
|
|
"created_at": 1712697600,
|
|
"completed_at": 1712697660,
|
|
"model": "sora-2",
|
|
"progress": 100,
|
|
"size": "1280x720",
|
|
"seconds": "12"
|
|
}
|
|
|
|
response = config.transform_video_status_retrieve_response(
|
|
raw_response=mock_http_response,
|
|
logging_obj=MagicMock()
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_789"
|
|
assert response.status == "completed"
|
|
assert response.progress == 100
|
|
assert response.model == "sora-2"
|
|
|
|
def test_video_status_different_states(self):
|
|
"""Test video status with different video states."""
|
|
# Test queued state
|
|
queued_response = video_status(
|
|
video_id="video_queued",
|
|
model="sora-2",
|
|
mock_response={
|
|
"id": "video_queued",
|
|
"object": "video",
|
|
"status": "queued",
|
|
"created_at": 1712697600,
|
|
"model": "sora-2",
|
|
"progress": 0
|
|
}
|
|
)
|
|
assert queued_response.status == "queued"
|
|
assert queued_response.progress == 0
|
|
|
|
# Test processing state
|
|
processing_response = video_status(
|
|
video_id="video_processing",
|
|
model="sora-2",
|
|
mock_response={
|
|
"id": "video_processing",
|
|
"object": "video",
|
|
"status": "processing",
|
|
"created_at": 1712697600,
|
|
"model": "sora-2",
|
|
"progress": 50
|
|
}
|
|
)
|
|
assert processing_response.status == "processing"
|
|
assert processing_response.progress == 50
|
|
|
|
# Test completed state
|
|
completed_response = video_status(
|
|
video_id="video_completed",
|
|
model="sora-2",
|
|
mock_response={
|
|
"id": "video_completed",
|
|
"object": "video",
|
|
"status": "completed",
|
|
"created_at": 1712697600,
|
|
"completed_at": 1712697660,
|
|
"model": "sora-2",
|
|
"progress": 100
|
|
}
|
|
)
|
|
assert completed_response.status == "completed"
|
|
assert completed_response.progress == 100
|
|
|
|
def test_video_status_with_remix_info(self):
|
|
"""Test video status with remix information."""
|
|
mock_data = {
|
|
"id": "video_remix_123",
|
|
"object": "video",
|
|
"status": "completed",
|
|
"created_at": 1712697600,
|
|
"completed_at": 1712697660,
|
|
"model": "sora-2",
|
|
"progress": 100,
|
|
"remixed_from_video_id": "video_original_123",
|
|
"size": "720x1280",
|
|
"seconds": "8"
|
|
}
|
|
|
|
response = video_status(
|
|
video_id="video_remix_123",
|
|
model="sora-2",
|
|
mock_response=mock_data
|
|
)
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_remix_123"
|
|
assert response.status == "completed"
|
|
assert hasattr(response, 'remixed_from_video_id')
|
|
assert response.remixed_from_video_id == "video_original_123"
|
|
|
|
def test_video_status_async_inside_async_function(self):
|
|
"""Test that sync video_status works inside async functions (no asyncio.run issues)."""
|
|
import asyncio
|
|
|
|
async def test_sync_in_async():
|
|
# This should work without asyncio.run() issues
|
|
# Use mock_response parameter for reliable testing
|
|
response = video_status(
|
|
video_id="video_sync_in_async",
|
|
model="sora-2",
|
|
mock_response={
|
|
"id": "video_sync_in_async",
|
|
"object": "video",
|
|
"status": "completed",
|
|
"created_at": 1712697600,
|
|
"model": "sora-2",
|
|
"progress": 100
|
|
}
|
|
)
|
|
return response
|
|
|
|
response = asyncio.run(test_sync_in_async())
|
|
|
|
assert isinstance(response, VideoObject)
|
|
assert response.id == "video_sync_in_async"
|
|
assert response.status == "completed"
|
|
|
|
def test_video_status_url_construction(self):
|
|
"""Test video status URL construction."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Test with different API bases
|
|
test_cases = [
|
|
("https://api.openai.com/v1/videos", "video_123", "https://api.openai.com/v1/videos/video_123"),
|
|
("https://api.openai.com/v1/videos/", "video_123", "https://api.openai.com/v1/videos/video_123"),
|
|
("https://custom-api.com/v1/videos", "video_456", "https://custom-api.com/v1/videos/video_456"),
|
|
]
|
|
|
|
for api_base, video_id, expected_url in test_cases:
|
|
url, data = config.transform_video_status_retrieve_request(
|
|
video_id=video_id,
|
|
api_base=api_base,
|
|
litellm_params=MagicMock(),
|
|
headers={}
|
|
)
|
|
assert url == expected_url
|
|
assert data == {}
|
|
|
|
|
|
class TestVideoLogging:
|
|
"""Test video generation logging functionality."""
|
|
|
|
class TestVideoLogger(CustomLogger):
|
|
def __init__(self):
|
|
self.standard_logging_payload = None
|
|
|
|
async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
|
|
self.standard_logging_payload = kwargs.get("standard_logging_object")
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_video_generation_logging(self):
|
|
"""Test that video generation creates proper logging payload with cost tracking.
|
|
|
|
Note: Uses AsyncMock with side_effect pattern for reliable parallel execution.
|
|
"""
|
|
custom_logger = self.TestVideoLogger()
|
|
litellm.logging_callback_manager._reset_all_callbacks()
|
|
litellm.callbacks = [custom_logger]
|
|
|
|
# Mock video generation response
|
|
mock_response = VideoObject(
|
|
id="video_test_123",
|
|
object="video",
|
|
status="queued",
|
|
created_at=1712697600,
|
|
model="sora-2",
|
|
size="720x1280",
|
|
seconds="8"
|
|
)
|
|
|
|
# Create async mock function to return the mock_response
|
|
async def mock_async_handler(*args, **kwargs):
|
|
return mock_response
|
|
|
|
# Patch the async_video_generation_handler method on base_llm_http_handler
|
|
with patch.object(videos_main.base_llm_http_handler, 'async_video_generation_handler', side_effect=mock_async_handler):
|
|
response = await litellm.avideo_generation(
|
|
prompt="A cat running in a garden",
|
|
model="sora-2",
|
|
seconds="8",
|
|
size="720x1280"
|
|
)
|
|
|
|
await asyncio.sleep(1) # Allow logging to complete
|
|
|
|
# Verify logging payload was created
|
|
assert custom_logger.standard_logging_payload is not None
|
|
|
|
payload = custom_logger.standard_logging_payload
|
|
|
|
# Verify basic logging fields
|
|
assert payload["call_type"] == "avideo_generation"
|
|
assert payload["status"] == "success"
|
|
assert payload["model"] == "sora-2"
|
|
assert payload["custom_llm_provider"] == "openai"
|
|
|
|
# Verify response object is recognized for logging
|
|
assert payload["response"] is not None
|
|
assert payload["response"]["id"] == "video_test_123"
|
|
assert payload["response"]["object"] == "video"
|
|
|
|
# Verify cost tracking is present (may be 0 in test environment)
|
|
assert payload["response_cost"] is not None
|
|
# Note: Cost calculation may not work in test environment due to mocking
|
|
# The important thing is that the logging payload is created and recognized
|
|
|
|
|
|
def test_openai_transform_video_content_request_empty_params():
|
|
"""OpenAI content transform should return empty params to ensure GET is used."""
|
|
config = OpenAIVideoConfig()
|
|
url, params = config.transform_video_content_request(
|
|
video_id="video_123",
|
|
api_base="https://api.openai.com/v1/videos",
|
|
litellm_params={},
|
|
headers={},
|
|
)
|
|
|
|
assert url == "https://api.openai.com/v1/videos/video_123/content"
|
|
assert params == {}
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"variant,expected_suffix",
|
|
[
|
|
("thumbnail", "?variant=thumbnail"),
|
|
("spritesheet", "?variant=spritesheet"),
|
|
],
|
|
)
|
|
def test_openai_transform_video_content_request_with_variant(variant, expected_suffix):
|
|
"""OpenAI content transform should append ?variant= when variant is provided."""
|
|
config = OpenAIVideoConfig()
|
|
url, params = config.transform_video_content_request(
|
|
video_id="video_123",
|
|
api_base="https://api.openai.com/v1/videos",
|
|
litellm_params={},
|
|
headers={},
|
|
variant=variant,
|
|
)
|
|
|
|
assert url == f"https://api.openai.com/v1/videos/video_123/content{expected_suffix}"
|
|
assert params == {}
|
|
|
|
|
|
def test_openai_transform_video_content_request_variant_none_no_query_param():
|
|
"""OpenAI content transform should NOT append ?variant= when variant is None."""
|
|
config = OpenAIVideoConfig()
|
|
url, params = config.transform_video_content_request(
|
|
video_id="video_123",
|
|
api_base="https://api.openai.com/v1/videos",
|
|
litellm_params={},
|
|
headers={},
|
|
variant=None,
|
|
)
|
|
|
|
assert "variant" not in url
|
|
assert url == "https://api.openai.com/v1/videos/video_123/content"
|
|
|
|
|
|
def test_video_content_handler_passes_variant_to_url():
|
|
"""HTTP handler should pass variant through to the final URL."""
|
|
from litellm.llms.custom_httpx.http_handler import HTTPHandler
|
|
from litellm.types.router import GenericLiteLLMParams
|
|
|
|
if hasattr(litellm, "in_memory_llm_clients_cache"):
|
|
litellm.in_memory_llm_clients_cache.flush_cache()
|
|
|
|
handler = BaseLLMHTTPHandler()
|
|
config = OpenAIVideoConfig()
|
|
|
|
mock_client = MagicMock(spec=HTTPHandler)
|
|
mock_response = MagicMock()
|
|
mock_response.content = b"thumbnail-bytes"
|
|
mock_client.get.return_value = mock_response
|
|
|
|
with patch(
|
|
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
|
|
return_value=mock_client,
|
|
):
|
|
result = handler.video_content_handler(
|
|
video_id="video_abc",
|
|
video_content_provider_config=config,
|
|
custom_llm_provider="openai",
|
|
litellm_params=GenericLiteLLMParams(
|
|
api_base="https://api.openai.com/v1"
|
|
),
|
|
logging_obj=MagicMock(),
|
|
timeout=5.0,
|
|
api_key="sk-test",
|
|
client=mock_client,
|
|
_is_async=False,
|
|
variant="thumbnail",
|
|
)
|
|
|
|
assert result == b"thumbnail-bytes"
|
|
called_url = mock_client.get.call_args.kwargs["url"]
|
|
assert called_url == "https://api.openai.com/v1/videos/video_abc/content?variant=thumbnail"
|
|
|
|
|
|
def test_video_content_handler_uses_get_for_openai():
|
|
"""HTTP handler must use GET (not POST) for OpenAI content download."""
|
|
from litellm.llms.custom_httpx.http_handler import HTTPHandler
|
|
from litellm.types.router import GenericLiteLLMParams
|
|
|
|
# Clear the HTTP client cache to prevent test isolation issues
|
|
# In CI, a cached real HTTPHandler from a previous test might bypass the mock
|
|
if hasattr(litellm, 'in_memory_llm_clients_cache'):
|
|
litellm.in_memory_llm_clients_cache.flush_cache()
|
|
|
|
handler = BaseLLMHTTPHandler()
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Use spec=HTTPHandler so isinstance(mock_client, HTTPHandler) returns True,
|
|
# ensuring the handler uses our mock directly instead of creating a new client.
|
|
mock_client = MagicMock(spec=HTTPHandler)
|
|
mock_response = MagicMock()
|
|
mock_response.content = b"mp4-bytes"
|
|
mock_client.get.return_value = mock_response
|
|
|
|
# Patch _get_httpx_client to ensure no real HTTP client is created
|
|
# This prevents test isolation issues where isinstance check might fail
|
|
with patch('litellm.llms.custom_httpx.llm_http_handler._get_httpx_client') as mock_get_client:
|
|
mock_get_client.return_value = mock_client
|
|
|
|
result = handler.video_content_handler(
|
|
video_id="video_abc",
|
|
video_content_provider_config=config,
|
|
custom_llm_provider="openai",
|
|
litellm_params=GenericLiteLLMParams(api_base="https://api.openai.com/v1"),
|
|
logging_obj=MagicMock(),
|
|
timeout=5.0,
|
|
api_key="sk-test",
|
|
client=mock_client,
|
|
_is_async=False,
|
|
)
|
|
|
|
assert result == b"mp4-bytes"
|
|
mock_client.get.assert_called_once()
|
|
assert not mock_client.post.called
|
|
called_url = mock_client.get.call_args.kwargs["url"]
|
|
assert called_url == "https://api.openai.com/v1/videos/video_abc/content"
|
|
|
|
|
|
def test_video_content_respects_api_base_and_api_key_from_kwargs():
|
|
"""Test that video_content respects api_base and api_key from kwargs (simulating database entry)."""
|
|
from litellm.videos.main import video_content
|
|
|
|
# Mock the handler to capture litellm_params
|
|
captured_litellm_params = None
|
|
|
|
def capture_litellm_params(*args, **kwargs):
|
|
nonlocal captured_litellm_params
|
|
captured_litellm_params = kwargs.get("litellm_params")
|
|
return b"mp4-bytes"
|
|
|
|
with patch('litellm.videos.main.base_llm_http_handler') as mock_handler:
|
|
mock_handler.video_content_handler = capture_litellm_params
|
|
|
|
# Call video_content with api_base and api_key in kwargs (simulating database entry)
|
|
# This simulates how the router passes model config from database via **kwargs
|
|
result = video_content(
|
|
video_id="video_test_123",
|
|
custom_llm_provider="azure",
|
|
api_base="https://test-resource.openai.azure.com/", # Passed via kwargs by router
|
|
api_key="test-api-key-from-db", # Passed via kwargs by router
|
|
)
|
|
|
|
# Verify that api_base and api_key from kwargs were included in litellm_params
|
|
assert captured_litellm_params is not None
|
|
assert captured_litellm_params.get("api_base") == "https://test-resource.openai.azure.com/"
|
|
assert captured_litellm_params.get("api_key") == "test-api-key-from-db"
|
|
assert result == b"mp4-bytes"
|
|
|
|
|
|
def test_openai_video_config_has_async_transform():
|
|
"""Ensure OpenAIVideoConfig exposes async_transform_video_content_response at runtime."""
|
|
cfg = OpenAIVideoConfig()
|
|
assert callable(getattr(cfg, "async_transform_video_content_response", None))
|
|
|
|
|
|
def test_gemini_video_config_has_async_transform():
|
|
"""Ensure GeminiVideoConfig exposes async_transform_video_content_response at runtime."""
|
|
cfg = GeminiVideoConfig()
|
|
assert callable(getattr(cfg, "async_transform_video_content_response", None))
|
|
|
|
|
|
def test_encode_video_id_with_provider_handles_azure_video_prefix():
|
|
"""
|
|
Test that encode_video_id_with_provider correctly encodes Azure/OpenAI video IDs
|
|
that start with 'video_' prefix.
|
|
|
|
This test verifies the fix for the issue where Azure returns video IDs like
|
|
'video_69323201cf6081909263f751f89991e6', which were previously skipped
|
|
from encoding, causing video status retrieval to default to 'openai' provider.
|
|
"""
|
|
from litellm.types.videos.utils import (
|
|
decode_video_id_with_provider,
|
|
encode_video_id_with_provider,
|
|
)
|
|
|
|
# Test case: Azure returns a video ID starting with 'video_'
|
|
raw_azure_video_id = "video_69323201cf6081909263f751f89991e6"
|
|
provider = "azure"
|
|
model_id = "azure/sora-2"
|
|
|
|
# Encode the video ID with provider information
|
|
encoded_id = encode_video_id_with_provider(
|
|
video_id=raw_azure_video_id,
|
|
provider=provider,
|
|
model_id=model_id
|
|
)
|
|
|
|
# Verify the ID was encoded (should be different from the original)
|
|
assert encoded_id != raw_azure_video_id
|
|
assert encoded_id.startswith("video_")
|
|
|
|
# Decode the encoded ID to verify provider information is preserved
|
|
decoded = decode_video_id_with_provider(encoded_id)
|
|
assert decoded.get("custom_llm_provider") == provider
|
|
assert decoded.get("model_id") == model_id
|
|
assert decoded.get("video_id") == raw_azure_video_id
|
|
|
|
# Verify that encoding an already-encoded ID doesn't double-encode it
|
|
encoded_twice = encode_video_id_with_provider(
|
|
video_id=encoded_id,
|
|
provider=provider,
|
|
model_id=model_id
|
|
)
|
|
assert encoded_twice == encoded_id # Should return the same encoded ID
|
|
|
|
class TestVideoListTransformation:
|
|
"""Tests for video list request/response transformation with provider ID encoding."""
|
|
|
|
def test_transform_video_list_response_encodes_first_id_and_last_id(self):
|
|
"""Verify that first_id and last_id are encoded with provider metadata."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
mock_http_response = MagicMock()
|
|
mock_http_response.json.return_value = {
|
|
"object": "list",
|
|
"data": [
|
|
{
|
|
"id": "video_aaa",
|
|
"object": "video",
|
|
"model": "sora-2",
|
|
"status": "completed",
|
|
},
|
|
{
|
|
"id": "video_bbb",
|
|
"object": "video",
|
|
"model": "sora-2",
|
|
"status": "completed",
|
|
},
|
|
],
|
|
"first_id": "video_aaa",
|
|
"last_id": "video_bbb",
|
|
"has_more": False,
|
|
}
|
|
|
|
result = config.transform_video_list_response(
|
|
raw_response=mock_http_response,
|
|
logging_obj=MagicMock(),
|
|
custom_llm_provider="azure",
|
|
)
|
|
|
|
from litellm.types.videos.utils import decode_video_id_with_provider
|
|
|
|
# data[].id should be encoded
|
|
for item in result["data"]:
|
|
decoded = decode_video_id_with_provider(item["id"])
|
|
assert decoded["custom_llm_provider"] == "azure"
|
|
|
|
# first_id and last_id should also be encoded
|
|
first_decoded = decode_video_id_with_provider(result["first_id"])
|
|
assert first_decoded["custom_llm_provider"] == "azure"
|
|
assert first_decoded["video_id"] == "video_aaa"
|
|
assert first_decoded["model_id"] == "sora-2"
|
|
|
|
last_decoded = decode_video_id_with_provider(result["last_id"])
|
|
assert last_decoded["custom_llm_provider"] == "azure"
|
|
assert last_decoded["video_id"] == "video_bbb"
|
|
assert last_decoded["model_id"] == "sora-2"
|
|
|
|
def test_transform_video_list_response_no_provider_leaves_ids_unchanged(self):
|
|
"""When custom_llm_provider is None, all IDs should remain unchanged."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
mock_http_response = MagicMock()
|
|
mock_http_response.json.return_value = {
|
|
"object": "list",
|
|
"data": [
|
|
{"id": "video_aaa", "object": "video", "model": "sora-2", "status": "completed"},
|
|
],
|
|
"first_id": "video_aaa",
|
|
"last_id": "video_aaa",
|
|
"has_more": False,
|
|
}
|
|
|
|
result = config.transform_video_list_response(
|
|
raw_response=mock_http_response,
|
|
logging_obj=MagicMock(),
|
|
custom_llm_provider=None,
|
|
)
|
|
|
|
assert result["data"][0]["id"] == "video_aaa"
|
|
assert result["first_id"] == "video_aaa"
|
|
assert result["last_id"] == "video_aaa"
|
|
|
|
def test_transform_video_list_response_missing_pagination_fields(self):
|
|
"""first_id / last_id may be absent or null; should not raise."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
mock_http_response = MagicMock()
|
|
mock_http_response.json.return_value = {
|
|
"object": "list",
|
|
"data": [
|
|
{"id": "video_aaa", "object": "video", "model": "sora-2", "status": "completed"},
|
|
],
|
|
"has_more": False,
|
|
}
|
|
|
|
result = config.transform_video_list_response(
|
|
raw_response=mock_http_response,
|
|
logging_obj=MagicMock(),
|
|
custom_llm_provider="azure",
|
|
)
|
|
|
|
# data[].id should still be encoded
|
|
from litellm.types.videos.utils import decode_video_id_with_provider
|
|
|
|
decoded = decode_video_id_with_provider(result["data"][0]["id"])
|
|
assert decoded["custom_llm_provider"] == "azure"
|
|
|
|
# first_id / last_id should not be present
|
|
assert "first_id" not in result
|
|
assert "last_id" not in result
|
|
|
|
def test_transform_video_list_request_decodes_after_parameter(self):
|
|
"""Encoded 'after' cursor should be decoded back to the raw provider ID."""
|
|
from litellm.types.videos.utils import encode_video_id_with_provider
|
|
|
|
config = OpenAIVideoConfig()
|
|
|
|
raw_id = "video_69888baee890819086dd3366bfc372fe"
|
|
encoded_id = encode_video_id_with_provider(raw_id, "azure", "sora-2")
|
|
|
|
url, params = config.transform_video_list_request(
|
|
api_base="https://my-resource.openai.azure.com/openai/v1/videos",
|
|
litellm_params=MagicMock(),
|
|
headers={},
|
|
after=encoded_id,
|
|
limit=10,
|
|
)
|
|
|
|
assert params["after"] == raw_id
|
|
assert params["limit"] == "10"
|
|
|
|
def test_transform_video_list_request_passes_through_plain_after(self):
|
|
"""A plain (non-encoded) 'after' value should pass through unchanged."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
url, params = config.transform_video_list_request(
|
|
api_base="https://api.openai.com/v1/videos",
|
|
litellm_params=MagicMock(),
|
|
headers={},
|
|
after="video_plain_id",
|
|
)
|
|
|
|
assert params["after"] == "video_plain_id"
|
|
|
|
def test_transform_video_list_roundtrip(self):
|
|
"""first_id from list response should decode correctly when used as after parameter."""
|
|
config = OpenAIVideoConfig()
|
|
|
|
# Simulate a list response
|
|
mock_http_response = MagicMock()
|
|
mock_http_response.json.return_value = {
|
|
"object": "list",
|
|
"data": [
|
|
{"id": "video_aaa", "object": "video", "model": "sora-2", "status": "completed"},
|
|
{"id": "video_bbb", "object": "video", "model": "sora-2", "status": "completed"},
|
|
],
|
|
"first_id": "video_aaa",
|
|
"last_id": "video_bbb",
|
|
"has_more": True,
|
|
}
|
|
|
|
list_result = config.transform_video_list_response(
|
|
raw_response=mock_http_response,
|
|
logging_obj=MagicMock(),
|
|
custom_llm_provider="azure",
|
|
)
|
|
|
|
# Use the encoded last_id as the 'after' cursor for the next page
|
|
_, params = config.transform_video_list_request(
|
|
api_base="https://my-resource.openai.azure.com/openai/v1/videos",
|
|
litellm_params=MagicMock(),
|
|
headers={},
|
|
after=list_result["last_id"],
|
|
)
|
|
|
|
# The after param sent to the upstream API should be the raw video ID
|
|
assert params["after"] == "video_bbb"
|
|
|
|
|
|
class TestVideoEndpointsProxyLitellmParams:
|
|
"""Test that video proxy endpoints (status, content, remix) respect litellm_params from proxy config."""
|
|
|
|
@pytest.fixture
|
|
def client_with_vertex_config(self, monkeypatch):
|
|
"""Create a test client with a proxy config that includes Vertex AI model with litellm_params."""
|
|
import asyncio
|
|
import tempfile
|
|
|
|
import yaml
|
|
from fastapi import FastAPI
|
|
from fastapi.testclient import TestClient
|
|
|
|
from litellm.proxy.proxy_server import (
|
|
cleanup_router_config_variables,
|
|
initialize,
|
|
router,
|
|
)
|
|
from litellm.proxy.video_endpoints.endpoints import router as video_router
|
|
|
|
# Clean up any existing router config
|
|
cleanup_router_config_variables()
|
|
|
|
# Create inline config
|
|
config = {
|
|
"model_list": [
|
|
{
|
|
"model_name": "vertex-ai-sora-2",
|
|
"litellm_params": {
|
|
"model": "vertex_ai/veo-2.0-generate-001",
|
|
"vertex_project": "test-project-123",
|
|
"vertex_location": "global",
|
|
"vertex_credentials": "/path/to/test-credentials.json",
|
|
}
|
|
}
|
|
]
|
|
}
|
|
|
|
# Write config to temporary file
|
|
with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
|
|
yaml.dump(config, f)
|
|
config_fp = f.name
|
|
|
|
try:
|
|
# Initialize the proxy with the test config
|
|
app = FastAPI()
|
|
asyncio.run(initialize(config=config_fp, debug=True))
|
|
app.include_router(router)
|
|
app.include_router(video_router)
|
|
|
|
return TestClient(app)
|
|
finally:
|
|
# Clean up temporary file
|
|
import os
|
|
if os.path.exists(config_fp):
|
|
os.unlink(config_fp)
|
|
|
|
@pytest.fixture
|
|
def mock_video_generation_response(self):
|
|
"""Mock video generation response with encoded video_id."""
|
|
from litellm.types.videos.utils import encode_video_id_with_provider
|
|
|
|
# Create an encoded video_id that includes provider and model_id
|
|
original_video_id = "projects/test-project-123/locations/global/publishers/google/models/veo-2.0-generate-001/operations/test-operation-123"
|
|
encoded_video_id = encode_video_id_with_provider(
|
|
video_id=original_video_id,
|
|
provider="vertex_ai",
|
|
model_id="veo-2.0-generate-001",
|
|
)
|
|
|
|
return VideoObject(
|
|
id=encoded_video_id,
|
|
object="video",
|
|
status="processing",
|
|
created_at=1712697600,
|
|
model="vertex_ai/veo-2.0-generate-001",
|
|
)
|
|
|
|
@pytest.fixture
|
|
def mock_video_status_response(self):
|
|
"""Mock video status response."""
|
|
return VideoObject(
|
|
id="video_test_123",
|
|
object="video",
|
|
status="completed",
|
|
created_at=1712697600,
|
|
completed_at=1712697660,
|
|
model="vertex_ai/veo-2.0-generate-001",
|
|
progress=100,
|
|
)
|
|
|
|
@pytest.fixture
|
|
def mock_video_content_response(self):
|
|
"""Mock video content response (raw bytes)."""
|
|
return b"fake_video_content_bytes"
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_video_status_respects_litellm_params(
|
|
self, client_with_vertex_config, mock_video_generation_response, mock_video_status_response
|
|
):
|
|
"""Test that video_status endpoint uses litellm_params from proxy config."""
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
# Create an encoded video_id
|
|
encoded_video_id = mock_video_generation_response.id
|
|
|
|
# Mock the router instance
|
|
mock_router_instance = MagicMock()
|
|
mock_router_instance.resolve_model_name_from_model_id.return_value = "vertex-ai-sora-2"
|
|
mock_router_instance.model_names = {"vertex-ai-sora-2"}
|
|
mock_router_instance.has_model_id.return_value = False
|
|
|
|
# Mock route_request to capture the data being passed
|
|
# route_request should return a coroutine (not await it), so we return a coroutine
|
|
async def mock_route_request_func(*args, **kwargs):
|
|
return mock_video_status_response
|
|
|
|
# Create a coroutine that will be added to tasks
|
|
def create_mock_coroutine(*args, **kwargs):
|
|
return mock_route_request_func(*args, **kwargs)
|
|
|
|
with patch("litellm.proxy.proxy_server.llm_router", mock_router_instance):
|
|
with patch("litellm.proxy.common_request_processing.route_request", side_effect=create_mock_coroutine) as mock_route_request:
|
|
# Make request to video_status endpoint
|
|
response = client_with_vertex_config.get(
|
|
f"/v1/videos/{encoded_video_id}",
|
|
headers={"Authorization": "Bearer sk-1234"},
|
|
)
|
|
|
|
# Verify the endpoint was called
|
|
assert response.status_code == 200, f"Response: {response.text}"
|
|
|
|
# Verify that route_request was called
|
|
assert mock_route_request.called
|
|
call_args = mock_route_request.call_args
|
|
# route_request is called with data as a keyword argument
|
|
data_passed = call_args.kwargs.get("data", {}) if call_args.kwargs else (call_args.args[0] if call_args.args and len(call_args.args) > 0 else {})
|
|
|
|
# Verify that model was resolved and added to data
|
|
assert data_passed.get("model") == "vertex-ai-sora-2", (
|
|
f"Expected model to be 'vertex-ai-sora-2', got '{data_passed.get('model')}'. "
|
|
f"Full data: {data_passed}, call_args: {call_args}"
|
|
)
|
|
# Verify that custom_llm_provider is set from decoded video_id
|
|
assert data_passed.get("custom_llm_provider") == "vertex_ai", (
|
|
f"Expected custom_llm_provider to be 'vertex_ai', got '{data_passed.get('custom_llm_provider')}'. "
|
|
f"Full data: {data_passed}"
|
|
)
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_video_content_respects_litellm_params(
|
|
self, client_with_vertex_config, mock_video_generation_response, mock_video_content_response
|
|
):
|
|
"""Test that video_content endpoint uses litellm_params from proxy config."""
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
# Create an encoded video_id
|
|
encoded_video_id = mock_video_generation_response.id
|
|
|
|
# Mock the router instance
|
|
mock_router_instance = MagicMock()
|
|
mock_router_instance.resolve_model_name_from_model_id.return_value = "vertex-ai-sora-2"
|
|
mock_router_instance.model_names = {"vertex-ai-sora-2"}
|
|
mock_router_instance.has_model_id.return_value = False
|
|
|
|
# Mock route_request to capture the data being passed
|
|
# route_request should return a coroutine (not await it), so we return a coroutine
|
|
async def mock_route_request_func(*args, **kwargs):
|
|
return mock_video_content_response
|
|
|
|
# Create a coroutine that will be added to tasks
|
|
def create_mock_coroutine(*args, **kwargs):
|
|
return mock_route_request_func(*args, **kwargs)
|
|
|
|
with patch("litellm.proxy.proxy_server.llm_router", mock_router_instance):
|
|
with patch("litellm.proxy.common_request_processing.route_request", side_effect=create_mock_coroutine) as mock_route_request:
|
|
# Make request to video_content endpoint
|
|
response = client_with_vertex_config.get(
|
|
f"/v1/videos/{encoded_video_id}/content",
|
|
headers={"Authorization": "Bearer sk-1234"},
|
|
)
|
|
|
|
# Verify the endpoint was called
|
|
assert response.status_code == 200, f"Response: {response.text}"
|
|
|
|
# Verify that route_request was called
|
|
assert mock_route_request.called
|
|
call_args = mock_route_request.call_args
|
|
# route_request is called with data as a keyword argument
|
|
data_passed = call_args.kwargs.get("data", {}) if call_args.kwargs else (call_args.args[0] if call_args.args and len(call_args.args) > 0 else {})
|
|
|
|
# Verify that model was resolved and added to data
|
|
assert data_passed.get("model") == "vertex-ai-sora-2", (
|
|
f"Expected model to be 'vertex-ai-sora-2', got '{data_passed.get('model')}'. "
|
|
f"Full data: {data_passed}, call_args: {call_args}"
|
|
)
|
|
# Verify that custom_llm_provider is correctly set from decoded video_id (not "openai")
|
|
assert data_passed.get("custom_llm_provider") == "vertex_ai", (
|
|
f"Expected custom_llm_provider to be 'vertex_ai', got '{data_passed.get('custom_llm_provider')}'. "
|
|
f"Full data: {data_passed}"
|
|
)
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_video_content_preserves_custom_llm_provider_from_decoded_id(
|
|
self, client_with_vertex_config, mock_video_generation_response, mock_video_content_response
|
|
):
|
|
"""Test that video_content preserves custom_llm_provider from decoded video_id."""
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
|
|
# Create an encoded video_id
|
|
encoded_video_id = mock_video_generation_response.id
|
|
|
|
# Mock the router instance
|
|
mock_router_instance = MagicMock()
|
|
mock_router_instance.resolve_model_name_from_model_id.return_value = "vertex-ai-sora-2"
|
|
mock_router_instance.model_names = {"vertex-ai-sora-2"}
|
|
mock_router_instance.has_model_id.return_value = False
|
|
|
|
# Mock route_request to capture the data being passed
|
|
# route_request should return a coroutine (not await it), so we return a coroutine
|
|
async def mock_route_request_func(*args, **kwargs):
|
|
return mock_video_content_response
|
|
|
|
# Create a coroutine that will be added to tasks
|
|
def create_mock_coroutine(*args, **kwargs):
|
|
return mock_route_request_func(*args, **kwargs)
|
|
|
|
with patch("litellm.proxy.proxy_server.llm_router", mock_router_instance):
|
|
with patch("litellm.proxy.common_request_processing.route_request", side_effect=create_mock_coroutine) as mock_route_request:
|
|
# Make request to video_content endpoint
|
|
response = client_with_vertex_config.get(
|
|
f"/v1/videos/{encoded_video_id}/content",
|
|
headers={"Authorization": "Bearer sk-1234"},
|
|
)
|
|
|
|
# Verify the endpoint was called
|
|
assert response.status_code == 200, f"Response: {response.text}"
|
|
|
|
# Verify that route_request was called
|
|
assert mock_route_request.called
|
|
call_args = mock_route_request.call_args
|
|
# route_request is called with data as a keyword argument
|
|
data_passed = call_args.kwargs.get("data", {}) if call_args.kwargs else (call_args.args[0] if call_args.args and len(call_args.args) > 0 else {})
|
|
|
|
# Most importantly: verify that custom_llm_provider is "vertex_ai" not "openai"
|
|
# This was the bug we fixed - it was defaulting to "openai" before
|
|
assert data_passed.get("custom_llm_provider") == "vertex_ai", (
|
|
f"Expected custom_llm_provider to be 'vertex_ai', "
|
|
f"but got '{data_passed.get('custom_llm_provider')}'. "
|
|
f"Full data: {data_passed}, call_args: {call_args}"
|
|
)
|
|
|
|
|
|
def test_video_remix_handler_uses_api_key_from_litellm_params():
|
|
"""Sync remix handler should fall back to litellm_params api_key when api_key param is None."""
|
|
handler = BaseLLMHTTPHandler()
|
|
config = OpenAIVideoConfig()
|
|
|
|
with patch.object(config, "validate_environment") as mock_validate:
|
|
mock_validate.return_value = {"Authorization": "Bearer deployment-key"}
|
|
|
|
with patch.object(config, "transform_video_remix_request") as mock_transform:
|
|
mock_transform.return_value = ("https://api.openai.com/v1/videos/video_123/remix", {"prompt": "remix it"})
|
|
|
|
with patch.object(config, "transform_video_remix_response") as mock_resp:
|
|
mock_resp.return_value = MagicMock()
|
|
|
|
mock_client = MagicMock()
|
|
mock_client.post.return_value = MagicMock(status_code=200)
|
|
|
|
with patch(
|
|
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
|
|
return_value=mock_client,
|
|
):
|
|
handler.video_remix_handler(
|
|
video_id="video_123",
|
|
prompt="remix it",
|
|
video_remix_provider_config=config,
|
|
custom_llm_provider="openai",
|
|
litellm_params={"api_key": "deployment-key", "api_base": "https://api.openai.com/v1"},
|
|
logging_obj=MagicMock(),
|
|
timeout=5.0,
|
|
api_key=None,
|
|
_is_async=False,
|
|
)
|
|
|
|
mock_validate.assert_called_once()
|
|
assert mock_validate.call_args.kwargs["api_key"] == "deployment-key"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_async_video_remix_handler_uses_api_key_from_litellm_params():
|
|
"""Async remix handler should fall back to litellm_params api_key when api_key param is None."""
|
|
handler = BaseLLMHTTPHandler()
|
|
config = OpenAIVideoConfig()
|
|
|
|
with patch.object(config, "validate_environment") as mock_validate:
|
|
mock_validate.return_value = {"Authorization": "Bearer deployment-key"}
|
|
|
|
with patch.object(config, "transform_video_remix_request") as mock_transform:
|
|
mock_transform.return_value = ("https://api.openai.com/v1/videos/video_123/remix", {"prompt": "remix it"})
|
|
|
|
with patch.object(config, "transform_video_remix_response") as mock_resp:
|
|
mock_resp.return_value = MagicMock()
|
|
|
|
mock_client = MagicMock(spec=AsyncHTTPHandler)
|
|
mock_response = MagicMock(status_code=200)
|
|
mock_client.post = AsyncMock(return_value=mock_response)
|
|
|
|
with patch(
|
|
"litellm.llms.custom_httpx.llm_http_handler.get_async_httpx_client",
|
|
return_value=mock_client,
|
|
):
|
|
await handler.async_video_remix_handler(
|
|
video_id="video_123",
|
|
prompt="remix it",
|
|
video_remix_provider_config=config,
|
|
custom_llm_provider="openai",
|
|
litellm_params={"api_key": "deployment-key", "api_base": "https://api.openai.com/v1"},
|
|
logging_obj=MagicMock(),
|
|
timeout=5.0,
|
|
api_key=None,
|
|
)
|
|
|
|
mock_validate.assert_called_once()
|
|
assert mock_validate.call_args.kwargs["api_key"] == "deployment-key"
|
|
|
|
|
|
def test_video_remix_handler_prefers_explicit_api_key():
|
|
"""Sync remix handler should prefer explicit api_key over litellm_params."""
|
|
handler = BaseLLMHTTPHandler()
|
|
config = OpenAIVideoConfig()
|
|
|
|
with patch.object(config, "validate_environment") as mock_validate:
|
|
mock_validate.return_value = {"Authorization": "Bearer explicit-key"}
|
|
|
|
with patch.object(config, "transform_video_remix_request") as mock_transform:
|
|
mock_transform.return_value = ("https://api.openai.com/v1/videos/video_123/remix", {"prompt": "remix it"})
|
|
|
|
with patch.object(config, "transform_video_remix_response") as mock_resp:
|
|
mock_resp.return_value = MagicMock()
|
|
|
|
mock_client = MagicMock()
|
|
mock_client.post.return_value = MagicMock(status_code=200)
|
|
|
|
with patch(
|
|
"litellm.llms.custom_httpx.llm_http_handler._get_httpx_client",
|
|
return_value=mock_client,
|
|
):
|
|
handler.video_remix_handler(
|
|
video_id="video_123",
|
|
prompt="remix it",
|
|
video_remix_provider_config=config,
|
|
custom_llm_provider="openai",
|
|
litellm_params={"api_key": "deployment-key", "api_base": "https://api.openai.com/v1"},
|
|
logging_obj=MagicMock(),
|
|
timeout=5.0,
|
|
api_key="explicit-key",
|
|
_is_async=False,
|
|
)
|
|
|
|
mock_validate.assert_called_once()
|
|
assert mock_validate.call_args.kwargs["api_key"] == "explicit-key"
|
|
|
|
|
|
if __name__ == "__main__":
|
|
pytest.main([__file__])
|