* feat(proxy): add max_iterations limiter for agent session loops (#22058)
Adds a new proxy hook that enforces a per-session cap on the number of
LLM calls an agentic loop can make. Callers send a session_id with each
request, and the hook counts calls per session, returning 429 when the
configured max_iterations limit is exceeded.
- Uses Redis Lua script for atomic increment (multi-instance safe)
- Falls back to in-memory cache when Redis unavailable
- Follows parallel_request_limiter_v3 pattern
- Configurable via key metadata: {"max_iterations": 25}
- Session counters auto-expire via TTL (default 1hr)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add new code execution dataset
* feat(agent_endpoints/): allow giving agents keys
* fix: ui fixes
* feat: allow assigning mcp servers to agents
* fix: eliminate duplicate DB queries in MCP agent auth and N+1 in agent listing (#22110)
- Extract _get_agent_object_permission helper so _get_allowed_mcp_servers_for_agent
and _get_agent_tool_permissions_for_server share a single DB fetch instead of
each independently querying the same agent row (was 1+N queries per MCP request)
- Use include={"object_permission": True} on find_many in get_all_agents_from_db
to eagerly load permissions in one query instead of N+1
- Use include={"object_permission": True} on create/update/find_unique in all
agent CRUD operations, removing attach_object_permission_to_dict follow-up calls
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add a `request_duration_ms` column to `LiteLLM_SpendLogs` to track request
duration. New rows are computed at write time. Legacy rows use a COALESCE
fallback in the `/spend/logs/ui` query to compute duration on the fly from
`endTime - startTime`. The field is also sortable in the UI endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(migrations): add ensure_project_id_verification_token migration
Ensures project_id column exists on LiteLLM_VerificationToken. The original
migration (20251113000000_add_project_table) adds this column, but may have
been skipped if LiteLLM_ProjectTable already existed and the migration was
resolved as idempotent. Uses IF NOT EXISTS for safety.
* bump: litellm-proxy-extras 0.4.45 → 0.4.46
Replace find_many + Python-side aggregation with a single SQL GROUP BY
query via query_raw in get_daily_activity_aggregated. This collapses
rows across entities (users/teams/orgs) in the database, reducing ~150k
rows to ~2-3k grouped rows before transfer to Python.
Also adds composite indexes (entity_id, date) to all 6 daily spend
tables for faster filtered queries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Virtual keys only track created_at and updated_at, which don't indicate
when a key was last used. This adds a last_active field that gets updated
during the async batch spend update, giving admins visibility into which
keys are actively being used.
Changes:
- Add last_active DateTime? to VerificationToken and
DeletedVerificationToken in all 3 schema files and Python types
- Set last_active in the batch key spend update alongside spend increment
- Add Last Active column to virtual keys UI table with info popover
and hover tooltip showing full date/time with timezone
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(schema.prisma): add object permissions for end users
allows controlling if end user can call specific mcp servers
* feat: cleanup for customer_endpoints support of object permission id
* fix: cleanup str
* feat(customers/): enforce end user can only call allowed mcps - if configured
* docs: document customer/end user object permission usage
* feat: enforce end user permissions on MCP tool calls
This commit implements end user permission enforcement for MCP servers:
1. Always add server prefixes to MCP tool names
- Removed conditional logic that only added prefixes when multiple servers existed
- Now always adds server prefix for consistent tool naming across all scenarios
- Updated 5 locations in server.py (list_tools, get_prompts, get_resources,
get_resource_templates, get_prompt)
2. Created MCP End User Permission Guardrail Hook
- New guardrail hook: litellm/proxy/guardrails/guardrail_hooks/mcp_end_user_permission.py
- Runs on post_call to validate tool calls in LLM responses
- Extracts MCP server name from tool names (splits on first '-')
- Checks if end_user_id has permissions for the MCP server
- Raises GuardrailRaisedException if end user lacks permission
- Supports both streaming and non-streaming responses
3. Added comprehensive tests
- Test file: tests/test_litellm/proxy/guardrails/guardrail_hooks/test_mcp_end_user_permission.py
- Tests cover: authorized/unauthorized tools, non-MCP tools, no end_user scenarios
- Tests permission checking logic and exception raising
The hook integrates with the existing MCPRequestHandler._get_allowed_mcp_servers_for_end_user
to fetch end user permissions and enforce access control at the response level.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* refactor: remove redundant add_prefix variable assignments
Simplified the code by removing intermediate `add_prefix` variable
assignments and passing `True` directly to function calls since
we now always add server prefixes.
Changes:
- Removed `add_prefix = True` variable assignments in 5 locations
- Changed `add_prefix=add_prefix` to `add_prefix=True` in function calls
- Added inline comments to clarify the behavior
This makes the code more concise and clearer in intent.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(auth_utils.py): support safety_identifier as a valid way of passing the end user id for responses api
* feat(llms): ensure 'tools' is correctly updated for responses api
* fix: fix greptile feedback
* feat: transformation.py
proper responses api tool handling for guardrail translation layer
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: SSO PKCE support fails in multi-pod Kubernetes deployments
* fix: virutal key grace period from env/UI
* fix: refactor, race condition handle, fstring sql injection
* fix: add async call to avoid server pauses
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: add await in tests
* add modify test to perform async run
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix grace period with better error handling on frontend and as per best practices
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: as per request changes
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix errors when callbacks are invoked for file delete operations:
* Fix errors when callbacks are invoked for file operations
* Fix: pass deployment credentials to afile_retrieve in managed_files post-call hook
* Fix: bypass managed files access check in batch polling by calling afile_content directly
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: afile_retrieve returns unified ID for batch output files
* fix: batch retrieve returns unified input_file_id
* fix(chatgpt): drop unsupported responses params for Codex
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(chatgpt): ensure Codex request filters unsupported params
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix deleted managed files returning 403 instead of 404
* Add comments
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: thread deployment model_info through batch cost calculation
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
* fix(deps): add pytest-postgresql for db schema migration tests
The test_db_schema_migration.py test requires pytest-postgresql but it was
missing from dependencies, causing import errors:
ModuleNotFoundError: No module named 'pytest_postgresql'
Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection
errors in proxy_unit_tests.
This is a pre-existing issue, not related to PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): replace caplog with custom handler for parallel execution
The cost calculation log level tests were failing when run with pytest-xdist
parallel execution because caplog doesn't work reliably across worker processes.
This causes "ValueError: I/O operation on closed file" errors.
Solution: Replace caplog fixture with a custom LogRecordHandler that directly
attaches to the logger. This approach works correctly in parallel execution
because each worker process has its own handler instance.
Fixes test failures in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): correct async mock for video generation logging test
The test was failing with AuthenticationError because the mock wasn't
intercepting the actual HTTP handler calls. This caused real API calls
with no API key, resulting in 401 errors.
Root cause: The test was patching the wrong target using string path
'litellm.videos.main.base_llm_http_handler' instead of using patch.object
on the actual handler instance. Additionally, it was mocking the sync
method instead of async_video_generation_handler.
Solution: Use patch.object with side_effect pattern on the correct
async handler method, following the same pattern used in
test_video_generation_async().
Fixes test failure in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): add cleanup fixture and no_parallel mark for MCP tests
Two MCP server tests were failing when run with pytest-xdist parallel
execution (--dist=loadscope):
- test_mcp_routing_with_conflicting_alias_and_group_name
- test_oauth2_headers_passed_to_mcp_client
Both tests showed assertion failures where mocks weren't being called
(0 times instead of expected 1 time).
Root cause: These tests rely on global_mcp_server_manager singleton
state and complex async mocking that doesn't work reliably with
parallel execution. Each worker process can have different state
and patches may not apply correctly.
Solution:
1. Added autouse fixture to clean up global_mcp_server_manager registry
before and after each test for better isolation
2. Added @pytest.mark.no_parallel to these specific tests to ensure
they run sequentially, avoiding parallel execution issues
This approach maintains test reliability while allowing other tests
in the file to still benefit from parallelization.
Fixes test failures exposed by PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Regenerate poetry.lock with Poetry 2.3.2
Updated lock file to use Poetry 2.3.2 (matching main branch standard).
This addresses Greptile feedback about Poetry version mismatch.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove unused pytest import and add trailing newline
- Removed unused pytest import (caplog fixture was removed)
- Added missing trailing newline at end of file
Addresses Greptile feedback (minor style issues).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove redundant import inside test method
The module litellm.videos.main is already imported at the top of
the file (line 21), so the import inside the test method is redundant.
Addresses Greptile feedback (minor style issue).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix converse anthropic usage object according to v1/messages specs
* Add routing based on if reasoning is supported or not
* add fireworks_ai/accounts/fireworks/models/kimi-k2p5 in model map
* Removed stray .md file
* fix(bedrock): clamp thinking.budget_tokens to minimum 1024
Bedrock rejects thinking.budget_tokens values below 1024 with a 400
error. This adds automatic clamping in the LiteLLM transformation
layer so callers (e.g. router with reasoning_effort="low") don't
need to know about the provider-specific minimum.
Fixes#21297
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve Langfuse test isolation to prevent flaky failures (#21093)
The test was creating fresh mocks but not fully isolating from setUp state,
causing intermittent CI failures with 'Expected generation to be called once.
Called 0 times.'
Instead of creating fresh mocks, properly reset the existing setUp mocks to
ensure clean state while maintaining proper mock chain configuration.
* feat(s3): add support for virtual-hosted-style URLs (#21094)
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).
This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
* Addressed greptile comments to extract common helpers and return 404
* Allow effort="max" for Claude Opus 4.6 (#21112)
* fix(aiohttp): prevent closing shared ClientSession in AiohttpTransport (#21117)
When a shared ClientSession is passed to LiteLLMAiohttpTransport,
calling aclose() on the transport would close the shared session,
breaking other clients still using it.
Add owns_session parameter (default True for backwards compatibility)
to AiohttpTransport and LiteLLMAiohttpTransport. When a shared session
is provided in http_handler.py, owns_session=False is set to prevent
the transport from closing a session it does not own.
This aligns AiohttpTransport with the ownership pattern already used
in AiohttpHandler (aiohttp_handler.py).
* perf(spend): avoid duplicate daily agent transaction computation (#21187)
* fix: proxy/batches_endpoints/endpoints.py:309:11: PLR0915 Too many statements (54 > 50)
* fix mypy
* Add doc for OpenAI Agents SDK with LiteLLM
* Add doc for OpenAI Agents SDK with LiteLLM
* Update docs/my-website/sidebars.js
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix mypy
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add blog fffor Managing Anthropic Beta Headers
* Add blog fffor Managing Anthropic Beta Headers
* correct the time
* Fix: Exclude tool params for models without function calling support (#21125) (#21244)
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
* fix(index.md): cleanup str
* fix(proxy): handle missing DATABASE_URL in append_query_params (#21239)
* fix: handle missing database url in append_query_params
* Update litellm/proxy/proxy_cli.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(migrations): Make vector stores migration idempotent with IF NOT EXISTS
- Add IF NOT EXISTS to ALTER TABLE ADD COLUMN statements
- Add IF NOT EXISTS to CREATE INDEX statements
- Prevents migration failures when columns/indexes already exist from manual fixes
- Follows PostgreSQL best practices for idempotent migrations
---------
Co-authored-by: Harshit Jain <harshitjain0562@gmail.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com>
Co-authored-by: Jay Prajapati <79649559+jayy-77@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: mjkam <mjkam@naver.com>
Co-authored-by: Fly <48186978+tuzkiyoung@users.noreply.github.com>
Co-authored-by: Kristoffer Arlind <13228507+KristofferArlind@users.noreply.github.com>
Co-authored-by: Constantine <Runixer@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Atharva Jaiswal <92455570+AtharvaJaiswal005@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
* fix: SSO PKCE support fails in multi-pod Kubernetes deployments
* fix: virutal key grace period from env/UI
* fix: refactor, race condition handle, fstring sql injection
* fix: add async call to avoid server pauses
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: add await in tests
* add modify test to perform async run
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix grace period with better error handling on frontend and as per best practices
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: as per request changes
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Fix errors when callbacks are invoked for file delete operations:
* Fix errors when callbacks are invoked for file operations
* Fix: pass deployment credentials to afile_retrieve in managed_files post-call hook
* Fix: bypass managed files access check in batch polling by calling afile_content directly
* Update tests/test_litellm/proxy/management_endpoints/test_ui_sso.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: afile_retrieve returns unified ID for batch output files
* fix: batch retrieve returns unified input_file_id
* fix(chatgpt): drop unsupported responses params for Codex
Co-authored-by: Cursor <cursoragent@cursor.com>
* test(chatgpt): ensure Codex request filters unsupported params
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix deleted managed files returning 403 instead of 404
* Add comments
* Update litellm/proxy/utils.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: thread deployment model_info through batch cost calculation
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
* fix(deps): add pytest-postgresql for db schema migration tests
The test_db_schema_migration.py test requires pytest-postgresql but it was
missing from dependencies, causing import errors:
ModuleNotFoundError: No module named 'pytest_postgresql'
Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection
errors in proxy_unit_tests.
This is a pre-existing issue, not related to PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): replace caplog with custom handler for parallel execution
The cost calculation log level tests were failing when run with pytest-xdist
parallel execution because caplog doesn't work reliably across worker processes.
This causes "ValueError: I/O operation on closed file" errors.
Solution: Replace caplog fixture with a custom LogRecordHandler that directly
attaches to the logger. This approach works correctly in parallel execution
because each worker process has its own handler instance.
Fixes test failures in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): correct async mock for video generation logging test
The test was failing with AuthenticationError because the mock wasn't
intercepting the actual HTTP handler calls. This caused real API calls
with no API key, resulting in 401 errors.
Root cause: The test was patching the wrong target using string path
'litellm.videos.main.base_llm_http_handler' instead of using patch.object
on the actual handler instance. Additionally, it was mocking the sync
method instead of async_video_generation_handler.
Solution: Use patch.object with side_effect pattern on the correct
async handler method, following the same pattern used in
test_video_generation_async().
Fixes test failure in PR #21277 when running with --dist=loadscope.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix(test): add cleanup fixture and no_parallel mark for MCP tests
Two MCP server tests were failing when run with pytest-xdist parallel
execution (--dist=loadscope):
- test_mcp_routing_with_conflicting_alias_and_group_name
- test_oauth2_headers_passed_to_mcp_client
Both tests showed assertion failures where mocks weren't being called
(0 times instead of expected 1 time).
Root cause: These tests rely on global_mcp_server_manager singleton
state and complex async mocking that doesn't work reliably with
parallel execution. Each worker process can have different state
and patches may not apply correctly.
Solution:
1. Added autouse fixture to clean up global_mcp_server_manager registry
before and after each test for better isolation
2. Added @pytest.mark.no_parallel to these specific tests to ensure
they run sequentially, avoiding parallel execution issues
This approach maintains test reliability while allowing other tests
in the file to still benefit from parallelization.
Fixes test failures exposed by PR #21277.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Regenerate poetry.lock with Poetry 2.3.2
Updated lock file to use Poetry 2.3.2 (matching main branch standard).
This addresses Greptile feedback about Poetry version mismatch.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove unused pytest import and add trailing newline
- Removed unused pytest import (caplog fixture was removed)
- Added missing trailing newline at end of file
Addresses Greptile feedback (minor style issues).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Remove redundant import inside test method
The module litellm.videos.main is already imported at the top of
the file (line 21), so the import inside the test method is redundant.
Addresses Greptile feedback (minor style issue).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* Fix converse anthropic usage object according to v1/messages specs
* Add routing based on if reasoning is supported or not
* add fireworks_ai/accounts/fireworks/models/kimi-k2p5 in model map
* Removed stray .md file
* fix(bedrock): clamp thinking.budget_tokens to minimum 1024
Bedrock rejects thinking.budget_tokens values below 1024 with a 400
error. This adds automatic clamping in the LiteLLM transformation
layer so callers (e.g. router with reasoning_effort="low") don't
need to know about the provider-specific minimum.
Fixes#21297
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve Langfuse test isolation to prevent flaky failures (#21093)
The test was creating fresh mocks but not fully isolating from setUp state,
causing intermittent CI failures with 'Expected generation to be called once.
Called 0 times.'
Instead of creating fresh mocks, properly reset the existing setUp mocks to
ensure clean state while maintaining proper mock chain configuration.
* feat(s3): add support for virtual-hosted-style URLs (#21094)
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).
This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
* Addressed greptile comments to extract common helpers and return 404
* Allow effort="max" for Claude Opus 4.6 (#21112)
* fix(aiohttp): prevent closing shared ClientSession in AiohttpTransport (#21117)
When a shared ClientSession is passed to LiteLLMAiohttpTransport,
calling aclose() on the transport would close the shared session,
breaking other clients still using it.
Add owns_session parameter (default True for backwards compatibility)
to AiohttpTransport and LiteLLMAiohttpTransport. When a shared session
is provided in http_handler.py, owns_session=False is set to prevent
the transport from closing a session it does not own.
This aligns AiohttpTransport with the ownership pattern already used
in AiohttpHandler (aiohttp_handler.py).
* perf(spend): avoid duplicate daily agent transaction computation (#21187)
* fix: proxy/batches_endpoints/endpoints.py:309:11: PLR0915 Too many statements (54 > 50)
* fix mypy
* Add doc for OpenAI Agents SDK with LiteLLM
* Add doc for OpenAI Agents SDK with LiteLLM
* Update docs/my-website/sidebars.js
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix mypy
* Update tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add blog fffor Managing Anthropic Beta Headers
* Add blog fffor Managing Anthropic Beta Headers
* correct the time
* Fix: Exclude tool params for models without function calling support (#21125) (#21244)
* Fix tool params reported as supported for models without function calling (#21125)
JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.
The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.
* Style: move verbose_logger to module-level import, remove redundant try/except
Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
already handles exceptions internally via _supports_factory()
* fix(index.md): cleanup str
* fix(proxy): handle missing DATABASE_URL in append_query_params (#21239)
* fix: handle missing database url in append_query_params
* Update litellm/proxy/proxy_cli.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(mcp): revert StreamableHTTPSessionManager to stateless mode (#21323)
PR #19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.
Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.
Fixes#20242
* UI - Content Filters, help edit/view categories and 1-click add categories + go to next page (#21223)
* feat(ui/): allow viewing content filter categories on guardrail info
* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails
* feat(ui/): improve ux around adding new content filter categories
easy to skip adding a category, so make it a 1-click thing
* Fix OCI Grok output pricing (#21329)
* fix(proxy): fix master key rotation Prisma validation errors
_rotate_master_key() used jsonify_object() which converts Python dicts
to JSON strings. Prisma's Python client rejects strings for Json-typed
fields — it requires prisma.Json() wrappers or native dicts.
This affected three code paths:
- Model table (create_many): litellm_params and model_info converted to
strings, plus created_at/updated_at were None (non-nullable DateTime)
- Config table (update): param_value converted to string
- Credentials table (update): credential_values/credential_info
converted to strings
Fix: replace jsonify_object() with model_dump(exclude_none=True) +
prisma.Json() wrappers for all Json fields. Wrap model delete+insert
in a Prisma transaction for atomicity. Add try/except around MCP
server rotation to prevent non-critical failures from blocking the
entire rotation.
---------
Co-authored-by: Harshit Jain <harshitjain0562@gmail.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Ephrim Stanley <ephrim.stanley@point72.com>
Co-authored-by: Jay Prajapati <79649559+jayy-77@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Julio Quinteros Pro <jquinter@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: mjkam <mjkam@naver.com>
Co-authored-by: Fly <48186978+tuzkiyoung@users.noreply.github.com>
Co-authored-by: Kristoffer Arlind <13228507+KristofferArlind@users.noreply.github.com>
Co-authored-by: Constantine <Runixer@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Atharva Jaiswal <92455570+AtharvaJaiswal005@users.noreply.github.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* feat(ui/): allow viewing content filter categories on guardrail info
* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails
* feat(ui/): improve ux around adding new content filter categories
easy to skip adding a category, so make it a 1-click thing
yuneng-jiang