* fix(test): add missing mocks for test_streamable_http_mcp_handler_mock
The test was missing mocks for extract_mcp_auth_context and set_auth_context,
causing the handler to fail silently in the except block instead of reaching
session_manager.handle_request. This mirrors the fix already applied to the
sibling test_sse_mcp_handler_mock.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(ci): route OpenAI models through chat completions in pass-through tests
The test_anthropic_messages_openai_model_streaming_cost_injection test fails
because the OpenAI Responses API returns 400 for requests routed through the
Anthropic Messages endpoint. Setting LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES=true
routes OpenAI models through the stable chat completions path instead.
Cost injection still works since it happens at the proxy level.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(ci): fix assemblyai custom auth and router wildcard test flakiness
1. custom_auth_basic.py: Add user_role='proxy_admin' so the custom auth
user can access management endpoints like /key/generate. The test
test_assemblyai_transcribe_with_non_admin_key was hidden behind an
earlier -x failure and was never reached before.
2. test_router_utils.py: Add flaky(retries=3) and increase sleep from 1s
to 2s for test_router_get_model_group_usage_wildcard_routes. The async
callback needs time to write usage to cache, and 1s is insufficient on
slower CI hardware.
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* ci: retrigger CI pipeline
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix(mypy): use LitellmUserRoles enum instead of raw string in custom_auth_basic
Fixes mypy error: Argument 'user_role' has incompatible type 'str'; expected 'LitellmUserRoles | None'
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* fix: don't close HTTP/SDK clients on LLMClientCache eviction (#22926)
* fix: don't close HTTP/SDK clients on LLMClientCache eviction
Removing the _remove_key override that eagerly called aclose()/close()
on evicted clients. Evicted clients may still be held by in-flight
streaming requests; closing them causes:
RuntimeError: Cannot send a request, as the client has been closed.
This is a regression from commit fb72979432. Clients that are no longer
referenced will be garbage-collected naturally. Explicit shutdown cleanup
happens via close_litellm_async_clients().
Fixes production crashes after the 1-hour cache TTL expires.
* test: update LLMClientCache unit tests for no-close-on-eviction behavior
Flip the assertions: evicted clients must NOT be closed. Replace
test_remove_key_closes_async_client → test_remove_key_does_not_close_async_client
and equivalents for sync/eviction paths.
Add test_remove_key_removes_plain_values for non-client cache entries.
Remove test_background_tasks_cleaned_up_after_completion (no more _background_tasks).
Remove test_remove_key_no_event_loop variant that depended on old behavior.
* test: add e2e tests for OpenAI SDK client surviving cache eviction
Add two new e2e tests using real AsyncOpenAI clients:
- test_evicted_openai_sdk_client_stays_usable: verifies size-based eviction
doesn't close the client
- test_ttl_expired_openai_sdk_client_stays_usable: verifies TTL expiry
eviction doesn't close the client
Both tests sleep after eviction so any create_task()-based close would
have time to run, making the regression detectable.
Also expand the module docstring to explain why the sleep is required.
* docs(AGENTS.md): add rule — never close HTTP/SDK clients on cache eviction
* docs(CLAUDE.md): add HTTP client cache safety guideline
* [Fix] Install bsdmainutils for column command in security scans
The security_scans.sh script uses `column` to format vulnerability
output, but the package wasn't installed in the CI environment.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: handle string callback values in prometheus multiproc setup
When callbacks are configured as a plain string (e.g., `callbacks: "my_callback"`)
instead of a list, the proxy crashes on startup with:
TypeError: can only concatenate str (not "list") to str
Normalize each callback setting to a list before concatenating.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* bump: version 1.82.2 → 1.82.3
* fix(test): update test_startup_fails_when_db_setup_fails for opt-in enforcement
The --enforce_prisma_migration_check flag is now required to trigger
sys.exit(1) on DB migration failure, after #23675 flipped the default
behavior to warn-and-continue.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(cost_calculator): use model name for per-request custom pricing when router_model_id has no pricing
When custom pricing is passed as per-request kwargs (input_cost_per_token/output_cost_per_token),
completion() registers pricing under the model name, but _select_model_name_for_cost_calc was
selecting the router deployment hash (which has no pricing data), causing response_cost to be 0.0.
Now checks whether the router_model_id entry actually has pricing before preferring it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Non-admin users (INTERNAL_USER) could call /key/block and /key/unblock on
arbitrary keys, and modify max_budget on their own keys via /key/update.
These endpoints are now restricted to proxy admins, team admins, or org admins.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added "Web Search Integration" to the integrations sidebar for better navigation.
- Updated authors in multiple blog posts to use shorthand references for consistency.
- Corrected links in various documentation files to ensure proper navigation.
- Improved clarity in load test documentation and related settings.
These changes aim to streamline user experience and maintain consistency across the documentation.
Model-level guardrails (litellm_params.guardrails on a deployment) were
only merged into request metadata in the streaming post_call path
(async_post_call_streaming_hook) but not in the non-streaming path
(post_call_success_hook). This caused should_run_guardrail to skip the
guardrail because the guardrail name was never added to metadata.guardrails.
Add the same _check_and_merge_model_level_guardrails call to
post_call_success_hook before the guardrail callback loop, mirroring the
streaming path.
Fixes model-level guardrails silently not firing for non-streaming
post_call requests.
Tests added for: UiLoadingSpinner, HashicorpVaultEmptyPlaceholder,
PageVisibilitySettings, errorUtils, and mcpToolCrudClassification.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The entry was accidentally dropped in commit 6bd7cd7 during a merge
conflict resolution. The model is deprecated but still accessible for
existing users until its shutdown date of 2026-03-26 per OpenAI docs.
Fixes#23738
Convert all 8 new video methods from @abstractmethod to concrete implementations
that raise NotImplementedError. This prevents breaking external third-party
BaseVideoConfig subclasses at import time.
Methods affected:
- transform_video_create_character_request/response
- transform_video_get_character_request/response
- transform_video_edit_request/response
- transform_video_extension_request/response
External integrators can now upgrade without instantiation errors; NotImplementedError
is only raised when operations are actually called on unsupported providers.
This restores backward compatibility with the project's policy.
Made-with: Cursor
Add avideo_create_character, avideo_get_character, avideo_edit, and avideo_extension
to the skip condition since Azure video calls don't use initialize_azure_sdk_client.
Tests now properly skip with expected behavior instead of failing:
- test_ensure_initialize_azure_sdk_client_always_used[avideo_create_character] ✓
- test_ensure_initialize_azure_sdk_client_always_used[avideo_get_character] ✓
- test_ensure_initialize_azure_sdk_client_always_used[avideo_edit] ✓
- test_ensure_initialize_azure_sdk_client_always_used[avideo_extension] ✓
Made-with: Cursor
- Add curl examples for avideo_edit and avideo_extension APIs
- Explain how LiteLLM encodes/decodes managed character IDs
- Show metadata included in character IDs (provider, model_id)
- Detail transparent router-first routing benefits
Made-with: Cursor
- Clear examples for SDK and proxy usage
- Feature highlights: router support, encoding, error handling
- Best practices for character uploads and prompting
- Available from LiteLLM v1.83.0+
- Troubleshooting guide for common issues
Made-with: Cursor
Add avideo_create_character and avideo_get_character to the list of video endpoints
that use router-first routing when a model is provided (either from decoded IDs or
target_model_names).
Previously only avideo_edit and avideo_extension were in the router-first block.
This ensures both character endpoints benefit from multi-deployment load balancing
and model resolution, making them consistent with the other video operations.
This allows:
- avideo_create_character: Router picks among multiple deployments when target_model_names is set
- avideo_get_character: Router assists with multi-model environments for consistency
Made-with: Cursor
Add response.raise_for_status() before transform_*_response() calls in all eight
video character/edit/extension handler methods (sync and async):
- video_create_character_handler / async_video_create_character_handler
- video_get_character_handler / async_video_get_character_handler
- video_edit_handler / async_video_edit_handler
- video_extension_handler / async_video_extension_handler
Without these checks, httpx does not raise on 4xx/5xx responses, so provider
errors (e.g., 401 Unauthorized) pass directly to Pydantic model constructors,
causing ValidationError instead of meaningful HTTP errors. The raise_for_status()
ensures the exception handler receives proper HTTPStatusError for translation into
actionable messages.
Made-with: Cursor
- Remove duplicate DecodedCharacterId TypedDict from litellm/types/videos/main.py
- Remove dead LITELLM_MANAGED_VIDEO_CHARACTER_COMPLETE_STR constant from litellm/types/utils.py
- Add FastAPI Form validation for name field in video_create_character endpoint
Made-with: Cursor
Use typed character response models and video multipart helpers so /videos/characters forwards uploaded MP4 files with video/* content type.
Made-with: Cursor
Support missing base64 padding in managed character/video IDs so copied encoded IDs still decode to the original upstream character ID.
Made-with: Cursor
Ryan Crabbe