- Add curl examples for avideo_edit and avideo_extension APIs
- Explain how LiteLLM encodes/decodes managed character IDs
- Show metadata included in character IDs (provider, model_id)
- Detail transparent router-first routing benefits
Made-with: Cursor
- Clear examples for SDK and proxy usage
- Feature highlights: router support, encoding, error handling
- Best practices for character uploads and prompting
- Available from LiteLLM v1.83.0+
- Troubleshooting guide for common issues
Made-with: Cursor
Add avideo_create_character and avideo_get_character to the list of video endpoints
that use router-first routing when a model is provided (either from decoded IDs or
target_model_names).
Previously only avideo_edit and avideo_extension were in the router-first block.
This ensures both character endpoints benefit from multi-deployment load balancing
and model resolution, making them consistent with the other video operations.
This allows:
- avideo_create_character: Router picks among multiple deployments when target_model_names is set
- avideo_get_character: Router assists with multi-model environments for consistency
Made-with: Cursor
Add response.raise_for_status() before transform_*_response() calls in all eight
video character/edit/extension handler methods (sync and async):
- video_create_character_handler / async_video_create_character_handler
- video_get_character_handler / async_video_get_character_handler
- video_edit_handler / async_video_edit_handler
- video_extension_handler / async_video_extension_handler
Without these checks, httpx does not raise on 4xx/5xx responses, so provider
errors (e.g., 401 Unauthorized) pass directly to Pydantic model constructors,
causing ValidationError instead of meaningful HTTP errors. The raise_for_status()
ensures the exception handler receives proper HTTPStatusError for translation into
actionable messages.
Made-with: Cursor
- Remove duplicate DecodedCharacterId TypedDict from litellm/types/videos/main.py
- Remove dead LITELLM_MANAGED_VIDEO_CHARACTER_COMPLETE_STR constant from litellm/types/utils.py
- Add FastAPI Form validation for name field in video_create_character endpoint
Made-with: Cursor
Use typed character response models and video multipart helpers so /videos/characters forwards uploaded MP4 files with video/* content type.
Made-with: Cursor
Support missing base64 padding in managed character/video IDs so copied encoded IDs still decode to the original upstream character ID.
Made-with: Cursor
Part1 had 4 test files combined (was originally 2), causing cross-file
state pollution under xdist. Reverted to original grouping.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The xdist-conditional reload (manual reset in xdist mode) was missing
attributes that importlib.reload resets, causing Azure connection errors.
The original conftest used importlib.reload unconditionally (even under
xdist) and that worked on main. Restore that behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GCS cache tests (test_gcs_cache_unit_tests.py) rely on module-level state
(vertex_chat_completion singleton, credential caches) that importlib.reload
resets but the xdist-safe function-scoped fixture does not. Removing -n 4
from this job restores single-process execution where module reload properly
resets all state before each test, while CI-level parallelism (parallelism: 2)
still splits test files across nodes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old conftest never flushed HTTP client cache. Adding flush_cache() before
every test forces new TCP connections to external APIs, causing transient
connection failures under xdist parallelism. Global state isolation is already
handled by _SCALAR_DEFAULTS reset.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
test_rerank.py sets litellm.api_base = "http://localhost:4000" which leaked
to all subsequent tests on the same xdist worker, causing connection failures
across every provider (Cohere, Azure, OpenAI, etc.).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The revert of 9711e3adfe left xdist tests without proper state isolation.
Module-level assignments like `litellm.num_retries = 3` in 12+ test files
pollute shared globals, and the fixture was saving/restoring contaminated
values instead of resetting to true defaults.
- Capture true litellm defaults at conftest import time and reset before
each test (local_testing + llm_translation)
- Make llm_translation/conftest.py xdist-safe (skip reload under xdist,
add state isolation)
- Replace asyncio.sleep(2) with polling in cooldown handler tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve db migration failure messaging and fix pyright errors in proxy_cli
- Clarify --skip_db_migration_check messaging so users know how to opt
into warn-and-continue behavior when database setup fails
- Fix pyright reportArgumentType error by casting get_secret result to str
- Fix pyright reportPossiblyUnboundVariable by initializing litellm_settings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace --skip_db_migration_check with --enforce_prisma_migration_check
Flip the default behavior: database migration failures now warn and
continue by default. Only when --enforce_prisma_migration_check (or
ENFORCE_PRISMA_MIGRATION_CHECK=true) is explicitly set will the proxy
exit on migration failure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Fixes https://github.com/BerriAI/litellm/issues/23185
* fix(responses/main.py): ensure litellm metadata custom cost works
* refactor: move all logging updates to a common function, to have just 1 place to update logging kwarg updates
aurelio_sdk imports requests_toolbelt at load time, so it needs its deps.
Unlike semantic_router, aurelio_sdk has no conflict with openai>=2, so
--no-deps is unnecessary. Verified via uv dry-run locally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch from expensive Linux machine (medium) to docker xlarge executor.
Drop miniconda, manual Docker CLI install, and manual PostgreSQL container
in favor of cimg/python:3.13, setup_remote_docker, and service container.
Use uv + cache for dependency installation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
semantic_router imports aurelio_sdk at module load time, so it must be
installed even when using --no-deps.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Downgrade langfuse, assistants, and python 3.13 install jobs to medium (were defaulting to large at ~25% CPU).
Bump enterprise and image_gen xdist workers to -n 4 on explicit large instances.
Drop coverage collection and persist_to_workspace for 4 jobs that no longer need it.
Downgrade verbosity from -vv to -v across all 5 jobs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The conftest fixtures were saving/restoring the current (potentially
contaminated) values of litellm globals like num_retries instead of
resetting to true defaults. Under xdist, module-level assignments
(e.g. `litellm.num_retries = 3` in 12+ test files) pollute the
shared module state and leak across tests in the same worker.
- Capture true litellm defaults at conftest import time and reset
before each test (local_testing + llm_translation)
- Make llm_translation/conftest.py xdist-safe (skip reload, add
state isolation)
- Replace asyncio.sleep(2) with polling in cooldown handler tests
- Add @pytest.mark.flaky to tests making real API calls under xdist
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch pip install -r requirements.txt to uv pip install --system -r requirements.txt
for all docker-based jobs that use the main requirements.txt. This applies the same
optimization already proven in the mapped test jobs to the rest of the CI pipeline.
Also adds --no-deps to semantic_router installs in guardrails_testing and
litellm_mapped_enterprise_tests to avoid uv's strict resolution conflict with openai>=2.
Skipped: machine executor + conda jobs (security, proxy_spend_accuracy,
proxy_multi_instance, proxy_store_model_in_db) and Group B jobs using
.circleci/requirements.txt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
uv's strict resolver rejects transitive dep conflicts (semantic-router
wants openai<2, llm-sandbox wants pydantic>=2.11.5). Use uv for the
heavy requirements.txt install and pip for the small test dep batch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Switch setup_litellm_test_deps from pip to uv with batched installs
- Cache installed site-packages (~/.local/lib, ~/.local/bin) instead of
pip download cache for near-instant installs on cache hit
- Remove unused coverage collection from 6 mapped test jobs (only mcps
coverage is consumed by the coverage combine step)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Router tests: expand conftest save/restore to cover all globals mutated
by router tests (default_fallbacks, tag_budget_config, request_timeout,
enable_azure_ad_token_refresh, num_retries_per_request, model_cost,
token_counter). These were leaking across xdist workers.
Proxy tests: move test_proxy_utils.py (169 parametrized) and
test_proxy_server.py (72 parametrized) from part2 to part1, balancing
~370 vs ~360 tests (was ~129 vs ~600).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase pytest-xdist parallelism to match available CPU on I/O-bound and
CPU-bound test jobs. Drop coverage collection from 8 jobs (still collected
by ~15 other jobs). Add dependency caching to 4 uncached jobs. Reduce
verbose output (-vv to -v) and remove -s/--log-cli-level overhead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Batching pip installs surfaced more hidden conflicts:
- respx==0.22.0 requires httpx>=0.25.0, conflicting with httpx==0.24.1
- traceloop-sdk==0.21.1 requires otel-semantic-conventions<0.46,
conflicting with opentelemetry-sdk==1.25.0 (needs ==0.46b0)
These were masked before because separate pip install calls let later
installs silently override earlier pins. Dropping the pins lets pip
resolve compatible versions. Verified with pip --dry-run locally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Batching pip installs exposed a dependency conflict: langfuse==2.59.7
requires anyio>=4.4.0, which conflicts with the anyio==4.2.0 pin.
Dropping the pin lets pip resolve a compatible version.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Several test jobs were underutilizing their CPU allocation (~25%) because
they were either missing pytest-xdist -n or using -n 2 on 4-vCPU machines.
Batching individual pip install calls into single commands reduces resolver
overhead and saves ~30-60s per job.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test intermittently fails in CI due to Redis cache write propagation
delays, causing the second call to miss the cache and hit OpenAI directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
LLM responses are non-deterministic and ClaudeAgentOptions doesn't expose
temperature control. The model occasionally returns unexpected short responses
(e.g. just "!") instead of the expected greeting. Add up to 3 retries with a
more explicit prompt to make the test deterministic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test waited a fixed 3s for async callbacks to populate log_queue.
Under xdist -n 4, CPU contention can delay the GLOBAL_LOGGING_WORKER
background task beyond 3s. Replace fixed sleeps with polling loops
(up to 10s) that break as soon as the expected condition is met.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Module-level mutations (litellm.num_retries=3 in test_langfuse_e2e_test.py
and test_amazing_s3_logs.py, litellm.success_callback=['langfuse']) run
at import time, BEFORE any function fixture. The save/restore pattern
captured these polluted values as 'originals' and kept restoring them.
Fix: capture litellm defaults when conftest.py is first imported (before
test modules), then reset to those true defaults before each test instead
of saving/restoring the current (potentially polluted) state.
ThrottlingException is a transient AWS rate-limit error unrelated to code
correctness. Skip the test instead of failing the CI pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Clear _in_memory_loggers before/after each test to prevent cached logger
instances (LangsmithLogger, SlackAlerting, etc.) from leaking stale state
- Add pre_call_rules, post_call_rules to list attrs save/restore
- Add vector_store_registry to scalar attrs save/restore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_avertex_batch_prediction: Add google.auth.default mock and env vars
so the test doesn't depend on real GCP credentials (was already a unit
test with mocked HTTP, just missing auth mock)
- test_async_create_batch[openai]: Add DNS pre-check that skips gracefully
when api.openai.com is unreachable instead of failing after 4 retries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The logging tests mutate many more litellm globals than guardrails tests
(turn_off_message_logging, s3_callback_params, datadog_params, service_callback,
etc.). The initial save/restore list only covered callbacks and a few basics,
causing state leaks like redaction settings bleeding across tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add LITELLM_LOG=WARNING to suppress verbose DEBUG log output
- Remove -s flag to stop capturing all stdout
- Bump xdist workers from -n 2 to -n 4
- Add --timeout=120 for safety
- Rewrite conftest.py to use save/restore pattern (matching guardrails_tests)
instead of per-function importlib.reload + event loop creation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The retrieve_batch endpoint sets batch status to "complete" but never set
batch_processed=True, permanently blocking file deletion. CheckBatchCost
(the safety net) also excluded completed batches from its primary query,
so batch_processed was never set by either path.
Three fixes:
1. update_batch_in_database sets batch_processed=True when status reaches
"complete", with old-schema fallback retry
2. CheckBatchCost primary query no longer excludes complete/completed
(batch_processed=False filter prevents reprocessing)
3. retrieve_batch early-return now includes "complete" (DB-normalized
spelling) to avoid unnecessary provider re-polls
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
