Commit Graph
34202 Commits
Author SHA1 Message Date
Sameer Kankute 65eba6b70e Add search api config 2026-03-04 13:54:25 +05:30
Sameer KankuteandGitHub 8541272629 Merge pull request #22685 from Varad2001/litellm_support_Qwen3.5-397B-A17B
feat(togetherai): add support for TogetherAI Qwen3.5-397B-A17B model
2026-03-04 08:31:49 +05:30
90eb6729d5 Agent Tracing - support context_id based trace id propogation + nested llm calls (#22626)
* style(ui/): distinguish agent calls from llm calls on ui

* feat: initial grouping working

* feat: set stable contextid for a2a calls - allows for easily passing to downstream llm/mcp calls

* feat(a2a_endpoints.py): fix tracing to avoid recreating logging objects for the same call

allows stable trace id usage

* fix(guardrail_endpoints): handle string ui_type values in _build_field_dict

_build_field_dict unconditionally called .value on ui_type, which crashes
for guardrail configs that use plain strings (e.g. BlockCodeExecutionGuardrailConfigModel
uses "multiselect" and "percentage"). Now checks with hasattr before calling .value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: propagate trace/session id from headers in MCP server calls

Cherry-picked mcp_server/server.py fixes from 6feb9bab: adds
get_chain_id_from_headers to extract x-litellm-trace-id /
x-litellm-session-id from raw headers, and uses it in call_tool
and list_tools to keep spend logs and tracing consistent with A2A.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:19:12 -08:00
yuneng-jiangandGitHub ba7a6d9bfd Merge pull request #22476 from BerriAI/litellm_audit_pagination_fix
[Fix] UI - Audit Logs: Server-side pagination, filtering, and drawer view
2026-03-03 16:52:33 -08:00
yuneng-jiangandGitHub ab101671c7 Merge pull request #22725 from BerriAI/litellm_team_admin_reset_spend
[Fix] Team Admin Reset Key Spend
2026-03-03 16:50:46 -08:00
yuneng-jiangandClaude Sonnet 4.6 5fa0c6e994 fix(ui): Copyable JSON blocks in audit drawer; custom table spinner
- Replace plain JsonBlock with CopyableJsonBlock: header row with label
  and copy icon (CheckOutlined on success), pre block below — matches
  the spend logs RequestResponsePanel pattern
- Key-table rows use the same card chrome for visual consistency
- Remove separate "Changes" section label (now redundant with block headers)
- Table loading spinner replaced with custom Spin + LoadingOutlined

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 16:00:46 -08:00
yuneng-jiangandClaude Sonnet 4.6 326ff422f1 fix(ui): Audit logs table and drawer polish
Table:
- Use DefaultProxyAdminTag for changed_by column
- Remove tooltips on Object ID and API Key columns
- Rename API Key column to API Key (Hash)
- Move pagination controls to upper-right of filter bar; add icon-only refresh button

Drawer:
- Object ID is now copyable
- API Key (Hash) is copyable and no longer truncated
- Changed By uses DefaultProxyAdminTag
- Expand JSON view boxes from max-h-72 to max-h-96
- Remove unnecessary vertical scrollbar (drop overflow-auto h-full from body div;
  use flex column layout so header and content flow naturally)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:46 -08:00
yuneng-jiangandClaude Sonnet 4.6 0a1b2635d7 fix: allow team admins to access /key/{key}/reset_spend route
The route-level auth check was blocking internal_user role (team admins)
from reaching /key/{key}/reset_spend because KEY_RESET_SPEND was missing
from key_management_routes. Added it so team admins pass the route check
and the endpoint's existing _check_proxy_or_team_admin_for_key enforces
actual authorization.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:54:34 -08:00
milan-berriandGitHub 98b9bc8b72 fix: resolve base_model in /cost/estimate for Azure custom deployments (#22724)
The _resolve_model_for_cost_lookup function was only checking
litellm_params.model when resolving model names from the router.
For Azure custom deployment names (e.g. azure/openai/gpt-5.3-codex),
this deployment name doesn't exist in the model cost map, so cost
returned /bin/zsh.

Now checks model_info.base_model and litellm_params.base_model first,
falling back to litellm_params.model only if no base_model is set.
This matches how the router resolves base_model everywhere else.
2026-03-03 15:43:02 -08:00
yuneng-jiangandGitHub 661f1e16cf Merge pull request #22715 from BerriAI/litellm_reset_key_spend
[Feature] UI - Virtual Keys: Manual Spend Reset
2026-03-03 15:40:09 -08:00
yuneng-jiangandClaude Sonnet 4.6 5b2110ddb5 Polish Reset Spend button and modal
- Move Reset Spend button after Regenerate Key in header
- Make modal OK button danger style with text "Reset"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:16:47 -08:00
yuneng-jiangandClaude Sonnet 4.6 98b91080ca Update Reset Spend button style and modal copy
- Use TransactionOutlined icon and danger style to match Delete Key button
- Rewrite modal description: remove assumption about key being blocked,
  clarify spend history is preserved in logs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:12:37 -08:00
Julio Quinteros ProandGitHub d9a349f7d2 Merge pull request #22720 from BerriAI/fix/proxy-utils-mock-request-state
fix: use real State in mock_request for proxy_utils tests
2026-03-03 20:10:53 -03:00
Julio Quinteros ProandClaude Opus 4.6 740cdc5c20 fix: use real State object in mock_request to fix _safe_get_request_headers
The _safe_get_request_headers caching (commit e7175a52) uses
request.state._cached_headers. With Mock(spec=Request), getattr on
state returns a Mock (truthy), causing RedactedDict to receive a Mock
instead of a dict. Using a real starlette State object fixes this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 20:05:20 -03:00
Julio Quinteros ProandGitHub 2415035fcd Merge pull request #22719 from BerriAI/fix/vertex-response-format-test
fix: account for vertex_ai schema diff in response_format test
2026-03-03 19:59:24 -03:00
Julio Quinteros ProandGitHub 5dfa96f62f Merge pull request #22718 from BerriAI/fix/jwt-test-mock-robustness
fix: use direct AsyncMock in JWT OIDC discovery tests
2026-03-03 19:59:15 -03:00
Julio Quinteros ProandClaude Opus 4.6 a07d041881 fix: apply same AsyncMock pattern to remaining OIDC discovery test
Address Greptile review: test_resolve_jwks_url_resolves_oidc_discovery_document
also used the inconsistent patch.object pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:56:51 -03:00
Julio Quinteros ProandClaude Opus 4.6 9b92ea16ab fix: update response_format test for vertex_ai's intentional schema diff
Vertex AI / Gemini uses Pydantic's model_json_schema() which omits
additionalProperties: False (Gemini rejects it). The test expected
the same schema for all providers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:55:18 -03:00
Julio Quinteros ProandClaude Opus 4.6 eb658693a3 fix: use direct AsyncMock assignment instead of patch.object in JWT tests
The patch.object with new_callable=AsyncMock can behave inconsistently
across Python versions, causing mock_response.status_code to return a
MagicMock instead of the assigned value. Direct assignment is simpler
and more reliable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:51:09 -03:00
Julio Quinteros ProandGitHub e6a1194c28 Merge pull request #22717 from BerriAI/fix/types-utils-video-tokens-test
fix: add video_tokens to completion_tokens_details test
2026-03-03 19:48:44 -03:00
Julio Quinteros ProandClaude Opus 4.6 2943df96ba fix: add video_tokens to expected completion_tokens_details in test
The CompletionTokensDetailsWrapper type now includes video_tokens field,
but this test's expected dict was not updated to include it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:46:20 -03:00
Julio Quinteros ProandGitHub fc9d06ceca Merge pull request #22716 from BerriAI/fix/vertex-function-response-tests
fix: update vertex AI tests for function_response role=user
2026-03-03 19:44:54 -03:00
Julio Quinteros ProandClaude Opus 4.6 5a0aba9fb7 fix: update vertex AI tests to expect role=user on function_response messages
The Gemini API requires role="user" on function_response content blocks
(added in commit 273cf12afa), but these tests were never updated to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 19:40:28 -03:00
yuneng-jiangandClaude Sonnet 4.6 5da7fa9ac1 [Feature] UI - Virtual Keys: Add manual spend reset to unblock keys
Adds a "Reset Spend" button to the key detail view so proxy admins and team
admins can immediately reset a key's spend to $0, unblocking keys that have
hit their budget limit without waiting for the next scheduled budget reset.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:39:39 -08:00
Julio Quinteros ProandGitHub 2c5c38333d Merge pull request #22701 from BerriAI/fix/streaming-and-azure-gpt5-test-failures
Fix Anthropic streaming sync and Azure GPT-5.1 logprobs tests
2026-03-03 19:36:44 -03:00
ryan-crabbeandGitHub 8aa7d53864 Merge pull request #22705 from BerriAI/litellm_feat_batch_expiry_setting_teams
Litellm feat batch expiry setting teams
2026-03-03 14:36:12 -08:00
ryan-crabbeGitHubgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
c8e6428eb7 Update litellm/proxy/openai_files_endpoints/files_endpoints.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-03 14:35:58 -08:00
Ryan Crabbe 35d2bc382f fix(batches): suppress PLR0915 lint for create_batch dispatch function 2026-03-03 14:28:25 -08:00
yuneng-jiangandClaude Sonnet 4.6 657a60ea5b fix(audit): AND semantics for combined JSON filters; remove unused allTeams prop
- Fix object_team_id + object_key_hash combining incorrectly as OR — each
  filter now adds an AND clause wrapping an internal OR over before_value
  and updated_values, so both conditions must be satisfied simultaneously
- Rename helper to _build_json_field_or_condition to reflect its purpose
- Remove allTeams from AuditLogsProps and its call site in index.tsx

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:24:55 -08:00
Ryan Crabbe 903ade4a1b fix(proxy): add anchor validation for file expiry, key validation for batch expiry
Validate anchor is "created_at" in enforced_file_expires_after (matching
user-provided path). Add key existence validation to batch endpoint for
enforced_batch_output_expires_after.
2026-03-03 13:51:41 -08:00
yuneng-jiangandGitHub 2d26209d80 Merge pull request #22698 from BerriAI/litellm_test_projects_hooks
[Test] UI - Projects: Add unit tests for project hooks
2026-03-03 13:42:28 -08:00
Ryan Crabbe d6ad312a4c fix(proxy): validate enforced_file_expires_after keys before access
Add key validation for enforced_file_expires_after to return a clear
400 error instead of an unhandled KeyError 500.
2026-03-03 13:37:11 -08:00
Julio Quinteros ProandClaude Opus 4.6 ab718444c5 Remove dead pending_new_content_block attribute
Cleanup per review: this class attribute is no longer used after the
__next__ refactor to queue-based approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:28:34 -03:00
Julio Quinteros ProandGitHub fbf916d71e Merge pull request #22699 from BerriAI/fix/mypy-count-tokens-signature
Fix mypy override errors in count_tokens signatures
2026-03-03 18:28:14 -03:00
Julio Quinteros ProandClaude Opus 4.6 6b4bc99202 Fix Anthropic streaming sync __next__ and Azure GPT-5.1 logprobs
Two independent fixes for pre-existing test failures on main:

1. Anthropic streaming: The sync __next__ method used a simple
   holding_chunk pattern that lost chunks when multiple events needed
   to be returned. Refactored to use the same chunk_queue approach as
   the async __anext__ method. Also fixed tests that used ModelResponse
   (which defaults finish_reason to 'stop') instead of ModelResponseStream.

2. Azure GPT-5.1 logprobs: The base OpenAI class includes logprobs for
   gpt-5.1+ models, but Azure hasn't verified support for gpt-5.1.
   Added explicit removal of logprobs/top_logprobs for gpt-5.1 (non-5.2)
   models in the Azure config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:16:11 -03:00
Julio Quinteros ProandClaude Opus 4.6 279b4f16cb Fix mypy override errors in count_tokens signatures
Replace **kwargs with explicit tools and system parameters to match
the BaseTokenCounter.count_tokens abstract method signature.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:15:27 -03:00
Harshit JainandGitHub b62a4be6b6 Merge pull request #22676 from BerriAI/litellm_feat_guardrail-list
feat: guardrail-mode-default-list
2026-03-04 02:36:57 +05:30
yuneng-jiangandClaude Sonnet 4.6 4edb1e00c6 [Test] UI - Add unit tests for project hooks
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 12:36:26 -08:00
Harshit28j d661419109 fix: support list of modes in Mode.default for tag-based guardrails 2026-03-04 01:50:28 +05:30
Ryan Crabbe 08613b24cb feat(proxy): add team-level file expiry enforcement 2026-03-03 11:03:14 -08:00
Ryan Crabbe 3d15bcdb11 feat(proxy): add team-level batch output expiry enforcement 2026-03-03 10:58:31 -08:00
yuneng-jiangandGitHub 4c1b15d685 Merge pull request #22687 from BerriAI/litellm_fix_spend_log_cleanup
[Fix] Spend Log Cleanup: lock tracking, integer retention, skip log level
2026-03-03 10:48:20 -08:00
Ryan Crabbe 43cec8c980 feat(batches): support output_expires_after passthrough 2026-03-03 10:46:45 -08:00
yuneng-jiangandClaude Sonnet 4.6 a1ba6c9fa6 Fix log levels: info for unconfigured, warning for misconfigured
Suppress noisy error log fired every cron tick when spend log cleanup
is simply not configured. _should_delete_spend_logs already logs the
specific reason at the right level (info for None, warning for
invalid value), so the redundant blanket error log in
cleanup_old_spend_logs is removed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 10:21:31 -08:00
yuneng-jiangandClaude Sonnet 4.6 224c617119 Fix spend log cleanup: lock tracking, integer retention, skip log level
- Only release distributed lock in finally if it was actually acquired;
  prevents spurious Redis release_lock calls on early returns
- Treat bare integer maximum_spend_logs_retention_period as days (e.g. 3 → "3d")
  instead of silently failing with a ValueError
- Elevate "Skipping cleanup" log from info to error so misconfigured
  retention settings are visible without verbose logging
- Add tests for all three fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 10:12:08 -08:00
Varad Khonde 7d65df351f feat(togetherai): add support for TogetherAI Qwen3.5-397B-A17B model 2026-03-03 23:40:41 +05:30
yuneng-jiangandGitHub b4980340fb Merge pull request #22680 from BerriAI/litellm_fix_mcp_server_dates
[Fix] MCP server: preserve created_at and updated_at timestamps
2026-03-03 09:53:37 -08:00
yuneng-jiangandClaude Sonnet 4.6 76e3dba0f8 fix mcp server created_at and updated_at timestamps being overwritten with current time
- Add created_at field to MCPServer type (was missing)
- Map created_at from LiteLLM_MCPServerTable in build_mcp_server_from_table()
- Use server.created_at and server.updated_at instead of datetime.now() in _build_mcp_server_table() and health check table builder
- Add regression tests to verify timestamps are preserved through round-trip conversions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 09:41:45 -08:00
Harshit28j 22e682b1e8 feat: guardrail-mode-default-list 2026-03-03 22:45:32 +05:30
Sameer KankuteandGitHub afc7b87b36 Merge pull request #22656 from BerriAI/litellm_add_vector_store_Ui
Add support for Attaching knowledge base to model via UI
2026-03-03 22:25:59 +05:30