Commit Graph
27304 Commits
Author SHA1 Message Date
Krrish Dholakia c14afa93b0 fix(responses_id_security.py): fix issue when no master key set 2025-11-12 14:09:11 -08:00
f4a5196728 fix: Improve Azure auth parameter handling for None values (#14436)
* fix: Improve Azure auth parameter handling for None values

Previously, litellm_params.get() with default fallbacks could ignore
environment variables when the param existed but was None. Now explicitly
checks for None values before falling back to environment variables.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix lint

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-11-12 14:01:55 -08:00
Krrish Dholakia 48579f7539 feat(anthropic/chat/transformation.py): add minimal reasoning effort thinking support
Closes Slack issue
2025-11-12 13:57:14 -08:00
YutaSaitoandGitHub 2843dab7fe fix: allow tool call even when server name prefix is missing (#16425)
* fix: allow tool call even when server name prefix is missing

* fix: test

* fix: test

* fix: test
2025-11-12 13:50:52 -08:00
Krrish Dholakia ae3178d5d4 docs(deploy.md): document how to disable pulling live model prices (faster startup time) on docker deployment 2025-11-12 13:49:44 -08:00
Krrish Dholakia 020a66c01f docs(model_access_guide.md): explain how model access works on litellm 2025-11-12 13:44:38 -08:00
Krrish Dholakia c6d2714c52 docs(model_access_guide.md): document how model access works on litellm 2025-11-12 13:43:13 -08:00
yuneng-jiangandGitHub c5496fd61c [Feature] UI - Move Budgets out of Experimental (#16544)
* Move budget out of experimental and delete budget with friction

* Fixed tests

* Revert non test change
2025-11-12 13:27:02 -08:00
Cesar GarciaandGitHub 20350fa094 docs: update broken Slack invite links to support page (#16546)
Replace broken Slack links (litellmossslack.slack.com and expired invite URLs)
with the correct support page URL (https://www.litellm.ai/support) across all
documentation files.

Files updated:
- CONTRIBUTING.md
- docs/my-website/docs/contact.md
- docs/my-website/docs/proxy/docker_quick_start.md
- docs/my-website/docs/troubleshoot.md
- docs/my-website/src/pages/contact.md
v1.79.3.dev5
2025-11-12 12:41:55 -08:00
Ishaan JaffandGitHub 061b7b2713 [Fix] Bedrock Embeddings - Ensure correct aws_region is used when provided dynamically (#16547)
* test_bedrock_embedding_uses_correct_region_when_specified

* fix aws_region_name in bedrock embeddings
2025-11-12 12:38:58 -08:00
Ishaan JaffandGitHub abde56391b [Fix] - Bedrock Knowledge Bases - add support for filtering kb queries (#16543)
* test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call_with_tools_and_filters

* fix vs registry

* fix merging params

* test_bedrock_kb_request_body_has_transformed_filters

* fix typing / linting
2025-11-12 12:38:50 -08:00
Mubashir OsmaniandGitHub e6bbf3ce81 fix: app_roles missing from jwt payload (#16448)
* fix: jwt app_roles missing

* add test
2025-11-12 12:07:04 -08:00
Jean-Adrien DucastaingandGitHub 086e557d78 fix: use vllm passthrough config for hosted vllm provider instead of raising error (#16537)
* fix(passthrough): use VLLM passthrough config for hosted VLLM provider instead of raising an error

* test(passthrough): add tests for hosted VLLM passthrough configuration and routing

* fix account linking for CLA
2025-11-12 09:12:14 -08:00
Jón LevyandGitHub 63445f2f37 fix(agentcore): Convert SSE stream iterator to async for proper streaming support (#16293)
* fix(agentcore): support async agentcore runtime streaming

* revert: CLAUDE.md

* revert: .gitignore

* fix: map runtimeUserId to X-Amzn-Bedrock-AgentCore-Runtime-User-Id header for runtime oauth support
2025-11-11 19:21:53 -08:00
Sameer KankuteandGitHub 517eb0ee10 Use safe loading of creds (#16479) 2025-11-11 19:16:29 -08:00
Francesco BertolottiandGitHub 06efc7631d Update model logging format for custom LLM provider (#16485)
This avoids the annoying red message 
```
Provider List: https://docs.litellm.ai/docs/providers
```
when there is a cache hit.
2025-11-11 19:13:42 -08:00
Andrew MaguireandGitHub bd15250960 fix: Add atexit handlers to flush callbacks for async completions (#16487)
Fixes #16486

## Problem
Callbacks configured via litellm.success_callback (e.g., PostHog, LangSmith)
were not being invoked for litellm.acompletion() in short-lived scripts.
The callbacks worked correctly for synchronous completions but async
completions would queue callbacks that were lost when the script exited
before GLOBAL_LOGGING_WORKER could process them.

Root cause: asyncio.run() closes the event loop immediately after the
async function completes, preventing the background worker from processing
queued callbacks.

## Solution
Implemented a two-level atexit handler approach:

1. GLOBAL_LOGGING_WORKER atexit handler (logging_worker.py):
   - Processes remaining callbacks from queue before exit
   - Creates new event loop to run pending coroutines synchronously
   - Applies time and iteration limits to prevent blocking shutdown

2. Integration-specific atexit handlers (posthog.py as example):
   - Flushes internal queue to external service
   - Uses synchronous HTTP client for reliable delivery
   - Each integration needs its own handler due to varying sync APIs

## Changes
- litellm/litellm_core_utils/logging_worker.py:
  - Added _flush_on_exit() method
  - Registered atexit handler in __init__
  - Processes up to MAX_ITERATIONS_TO_CLEAR_QUEUE events
  - Time-limited to MAX_TIME_TO_CLEAR_QUEUE seconds

- litellm/integrations/posthog.py:
  - Added _flush_on_exit() method
  - Registered atexit handler in __init__
  - Groups events by credentials for batch sending
  - Uses sync_client for reliable HTTP delivery

- tests/logging_callback_tests/test_posthog.py:
  - Added test_async_callback_atexit_handler_exists()
  - Added test_posthog_atexit_flushes_internal_queue()
  - Added test_sync_callback_not_affected_by_atexit()

## Testing
- All existing tests pass
- Manual end-to-end testing confirms fix:
  - Async events now arrive in PostHog
  - Sync events continue working (no regression)
- Unit tests verify atexit handlers registered and functional

## Impact
- Fixes async callback delivery for ALL integrations using GLOBAL_LOGGING_WORKER
- No breaking changes - only adds missing functionality
- Sync path unchanged - no performance impact
2025-11-11 19:12:53 -08:00
Matt CowgerandGitHub 51dc4a8476 fix: add new models, delete repeat models, update pricing. (#16491)
Added 4 new OpenRouter models:

openrouter/deepseek/deepseek-v3.2-exp
openrouter/minimax/minimax-m2
openrouter/z-ai/glm-4.6
openrouter/z-ai/glm-4.6:exacto ]

Updated openrouter/qwen/qwen3-coder:

Significantly reduced input/output costs (1e-06 → 2.2e-7 / 5e-06 → 9.5e-7)
Reduced max token limits (1,000,000 → 262,100)
Added supports_function_calling flag

Removed 2 duplicated Vertex AI Qwen models (just removed duplicates):

vertex_ai/qwen/qwen3-next-80b-a3b-instruct-maas
vertex_ai/qwen/qwen3-next-80b-a3b-thinking-maas
2025-11-11 19:11:22 -08:00
Alan PonnachanandGitHub 8cb610b8b2 Sanitize null token values in provider responses (#16493) 2025-11-11 19:08:00 -08:00
Anthony MonacoandGitHub a1748ad550 Documentation Code Example corrections (#16502)
* Update quick_start.md

changed -D to -d

* Update users.md

Changed in a number of locations:

 "budget_duration": 10s,

to 

 "budget_duration": "10s",

* Update users.md

Changed all 10s to 30s to keep in line with the example
2025-11-11 19:06:44 -08:00
Krrish Dholakia 6931d3013a docs(readme.md): document 8ms p95 latency 2025-11-11 19:05:50 -08:00
yuneng-jiangandGitHub ba674f241c [Fix] Use user budget instead of key budget when creating new team (#16074)
* Use end user budget instead of key budget when creating new team

* Fixed implementation to use user's max budget from the UserTable instead of EndUserTable
2025-11-11 18:55:05 -08:00
yuneng-jiangandGitHub 67478a9074 [Fix] Litellm tags usage add request_id (#16111)
* Add request_id into tag spend

* Linting
2025-11-11 18:53:48 -08:00
yuneng-jiangandGitHub b9759b4bfa [Feature] UI - Add LiteLLM Params to Edit Model (#16496)
* Add LiteLLM Params to Edit Model

* Fixed tests
2025-11-11 18:52:11 -08:00
yuneng-jiangandGitHub 7dd76bc4e3 Usage indicator Near Limit Fix (#16504) 2025-11-11 18:50:15 -08:00
Ishaan JaffandGitHub 50b5cf5215 [Feat] New Provider - Add RunwayML Provider for video generations (#16505)
* add RUNWAYML

* init folders

* add RunwayMLVideoConfig

* add RUNWAYML_DEFAULT_API_VERSION

* add RunwayMLVideoConfig

* fix getting status

* add async_transform_video_content_response

* add runwayml transform_video_content_response

* fix config.yaml

* add runwayml docs

* add runwayml to videos

* docs runwayml video gen

* add new models to model cost map

* TestRunwayMLVideoTransformation

* fix linting errors
2025-11-11 18:48:23 -08:00
Pedro AzevedoandGitHub 663f2d7e7f docs: remove enterprise restriction from guardrails list endpoint (#15333)
- Remove enterprise-only label from 'View Available Guardrails' section
- The /guardrails/list endpoint appears to be available in OSS version
- Makes documentation more accurate for OSS users
2025-11-11 18:45:26 -08:00
yuneng-jiangandGitHub c623ff916f Add Tags To Edit Key Flow (#16500) 2025-11-11 17:44:29 -08:00
443bada425 Add Zscaler AI Guard hook (#15691)
* Add Zscaler AI Guard hook

Co-authored-by: Angela Tao <atao@zscaler.com>

* Fix lint error, update document

* Fix lint error, update document

* update document

* fix mypy type error

* fix mypy issue

* fix test

* fix test

* improve document

* remove unuseful code

* use litellm httphandler

* update test cases

* revover guardrail_initializers.py and guardrail_registry.py

* remove unuse import

* app apply_guardrail

* remove functions repleased by apply_guardrail, update test and doc

* remove functions repleased by apply_guardrail, update test and doc

---------

Co-authored-by: Angela Tao <atao@zscaler.com>
2025-11-11 15:34:27 -08:00
yuneng-jiangandGitHub 627463b21f [Infra] CI/CD - Bump up docker version for e2e ui testing (#16506)
* Bump up docker version for e2e ui testing CICD

* Fixing config file
2025-11-11 15:07:30 -08:00
Krrish Dholakia a987e3ca46 fix(proxy_cli.py): check for env var for IAM_TOKEN_DB_AUTH v1.79.dev.1 2025-11-11 11:56:58 -08:00
Ishaan Jaffer be05324645 docs fix MAX_LANGFUSE_INITIALIZED_CLIENTS 2025-11-11 11:26:33 -08:00
Ishaan JaffandGitHub 5c9f50d584 [AI Gateway] - End User Budgets - Allow pointing max_end_user budget to an id, so the default ID applies to all end users (#16456)
* add _apply_budget_limits_to_end_user_params

* add _apply_budget_limits_to_end_user_params

* add _apply_budget_limits_to_end_user_params

* test_default_budget_applied_to_end_user_without_budget

* docs fix

* fix config
2025-11-11 08:20:13 -08:00
Ishaan JaffandGitHub e94186629d [Fix] Bedrock Knowledge bases - ensure users can access search_results for both stream + non stream response to /chat/completions (#16459)
* fix message with provider_specific_fields

* test_provider_specific_fields_in_proxy_http_response

* test_provider_specific_fields_in_proxy_http_response
2025-11-11 08:19:57 -08:00
YutaSaitoandGitHub aff1060512 fix: unable to delete MCP server from permission settings #16124 (#16407) 2025-11-10 19:50:13 -08:00
Daniel SabanovandGitHub 0ecc38519e [Bug] Updated spend would not be sent to CloudZero (#16201)
* Address a bug where cloudzero spend is not sent to cloudzero if a spend
update happens

* revert change unrelated to PR

* use polars for mocking instead of sqlite
2025-11-10 19:43:15 -08:00
yellowsubmarine372andGitHub b6dbd4fa28 fix: apply provided timeout value to ClientTimeout.total (#16395) 2025-11-10 19:37:42 -08:00
Alan PonnachanandGitHub 5f12e4be1e fix(langfuse): Handle null usage values to prevent validation errors (#16396)
* langfuse null validation fix

* formatting
2025-11-10 19:34:56 -08:00
Alan PonnachanandGitHub 7b292ccdf5 router fallback for unknown models (#16419) 2025-11-10 19:33:01 -08:00
Matt CowgerandGitHub e0d6774ec1 fix: update model_cost_map_url to use environment variable (#16429)
* fix: update model_cost_map_url to use environment variable, to match behavior to documentation in docs/my-website/docs/proxy/sync_models_github.md

* Add an appropriate test.
2025-11-10 19:30:44 -08:00
Alan PonnachanandGitHub df33f36c06 Correctly handle date filters in /spend/logs endpoint (#16443) 2025-11-10 19:26:55 -08:00
Val MiscenkoandGitHub 19e6e60d30 fix: remove strict master_key check in add_deployment (#16453)
Allows proxy to save spend logs without requiring master_key.
  Decryption now gracefully handles both encrypted and unencrypted values.
2025-11-10 19:23:13 -08:00
Sameer KankuteandGitHub aaa8cba00b Add docs for tracking callback failure (#16474) 2025-11-10 19:15:14 -08:00
Sameer KankuteandGitHub 6cab77f53f Added thinking streaming support for mistral (#16434) 2025-11-10 18:41:45 -08:00
Sameer KankuteandGitHub bf363cdf10 Add sdk focused examples (#16441) 2025-11-10 18:39:34 -08:00
yuneng-jiangandGitHub 8f1f5825aa Invite User Searchable Team Select (#16454) 2025-11-10 18:38:23 -08:00
Sameer KankuteandGitHub be3c09e6d5 Add GET list of providers endpoint (#16432) 2025-11-10 18:38:09 -08:00
yuneng-jiangandGitHub af68763f5d Only show models based on selected endpoint (#16452) 2025-11-10 18:38:01 -08:00
Ishaan Jaffer b8eda0ef55 proxy_store_model_in_db_tests 2025-11-10 18:20:52 -08:00
Ishaan JaffandGitHub 8140d85d28 [Bug Fix] - LiteLLM Usage shows key_hash- (#16471)
* test_get_logging_payload_api_key_preserved_when_standard_logging_payload_is_none

* fix get_logging_payload

* test_api_key_preserved_through_failure_hook_to_database
2025-11-10 18:15:52 -08:00