Commit Graph
34882 Commits
Author SHA1 Message Date
Chesars 1be6b31e2f merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
Cesar GarciaandGitHub 8d2432b21b Merge pull request #23373 from Chesars/style/black-format-codebase
style: run black formatter on /litellm
2026-03-11 17:13:37 -03:00
Chesars 2e9f057fbd style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
d398d1f9b9 fix: use group_by instead of find_many(distinct) in /tag/list to avoid full table scan (#23136)
* fix: use group_by instead of find_many(distinct) in /tag/list to avoid full table scan

The list_tags endpoint used Prisma's find_many(distinct=["tag"]) to discover
dynamic tags from LiteLLM_DailyTagSpend. Prisma's distinct is a client-side
post-processing filter that fetches all rows with all columns before
deduplicating, causing a full table scan of 3M+ rows on every usage page load.

Replace with Prisma's group_by() which generates proper GROUP BY SQL, letting
the database handle deduplication efficiently. NULL tags are filtered out via
where={"tag": {"not": None}}, and created_at/updated_at are preserved via
_min/_max aggregates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback

- Remove unused LiteLLM_DailyTagSpendTable class and datetime import
- Simplify created_at/updated_at access (values guaranteed by spend system)
- Remove DB call assertions from tests, keep response-only assertions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: RheagalFire <arishalam121@gmail.com>
2026-03-12 00:32:37 +05:30
c7e3b11da8 fix: batch key queries in list_team to eliminate N+1 (#23152)
* fix: batch key queries in list_team to eliminate N+1

The list_team endpoint issued one DB query per team to fetch keys,
causing N+1 query overhead. Replace with a single batched IN query.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback on team list test

- Rename test to include "v1" for consistency with other test names
- Remove unnecessary DB call assertion, keep behavioral assertions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: RheagalFire <arishalam121@gmail.com>
2026-03-12 00:17:17 +05:30
Cesar GarciaandGitHub cb24b8b05e Merge pull request #19104 from Chesars/fix/vertex-ai-zai-org-global-region
feat(vertex_ai): route region for partner models and add GLM support
2026-03-11 15:19:33 -03:00
Chesars 3948513a4c fix(vertex-ai): warn on region override and remove dead is_global_only_vertex_model
Add verbose_logger.warning when user-specified region is overridden by
supported_regions. Remove now-unused is_global_only_vertex_model function
and its tests since get_vertex_region handles all region logic directly.
2026-03-11 15:12:53 -03:00
RJ DuffnerandGitHub 0c95d415e1 Add Abilty To Set minReadySeconds From values Files (#23173)
* Add Abilty To Set minReadySeconds From values Files

* typo

* uppercase Min as it comes after deployment

* Don't use defaults, just omit
2026-03-11 23:29:15 +05:30
Chesars f3ceb69e9f fix(vertex-ai): override unsupported user region for models with supported_regions
- get_vertex_region now overrides user-specified region when it's not in
  the model's supported_regions list (prevents 404 for users with a
  global VERTEXAI_LOCATION default hitting global-only models)
- Add supported_regions: ["global"] to glm-5-maas in both JSON files
- Update tests to cover the override behavior
2026-03-11 14:55:51 -03:00
Chesars 689cbaa6c1 fix(vertex-ai): update tests to match new get_vertex_region model_cost lookup
- Remove redundant get_vertex_region() call in partner models main.py
  (already called inside get_complete_vertex_url)
- Rewrite test mocks to use patch.dict(litellm.model_cost) instead of
  patching the removed is_global_only_vertex_model symbol
- Align test assertions with new behavior: user-specified region is
  preserved (not overridden) for global-only models
2026-03-11 14:25:08 -03:00
Cesar GarciaandGitHub ebe75a1bc6 Merge pull request #16594 from Chesars/feat/anthropic-files-api
feat: Add Anthropic Files API support
2026-03-11 13:45:53 -03:00
Aarish AlamandGitHub 3bbadca6e2 Clean up skills test: remove duplicate imports, parameterize mock HTTP method (#23360)
Address review comments from PR #23325:
- Remove duplicate `from unittest.mock import MagicMock, patch` and unused `json` import
- Parameterize HTTP method in `_make_mock_response` helper so mock requests
  use the correct method (GET/POST/DELETE) matching each test scenario
2026-03-11 22:10:27 +05:30
Chesars d7c2532137 fix: remove redundant import os, use Path().name instead 2026-03-11 13:37:52 -03:00
Joe ReynaGitHubAarish Alamgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
ca910a8fa0 Replace broken Anthropic Skills API integration test with unit tests (#23325)
* Replace broken Anthropic Skills API integration test with unit tests

* Update tests/test_litellm/test_anthropic_skills_transformation.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Aarish Alam <arishalam121@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-11 22:00:08 +05:30
Chesars abe0c40c1a fix: clarify purpose field docstring in AnthropicFilesConfig 2026-03-11 13:23:24 -03:00
Chesars 8d4e98faa1 fix(vertex-ai): remove unused import is_global_only_vertex_model 2026-03-11 13:19:12 -03:00
Chesars bb5d57645f docs: add VertexAI ZAI (GLM) documentation 2026-03-11 13:18:12 -03:00
Chesars c723d63e59 fix(vertex_ai): auto-resolve vertex_location from supported_regions
- Add call to get_vertex_region() in partner models to auto-detect region
- Improve get_vertex_region() to read supported_regions from model_cost
- User-specified vertex_location still takes priority
2026-03-11 13:18:12 -03:00
Chesars c63b7965bd fix(vertex_ai): add supported_regions for zai-org/glm-4.7-maas
Add `supported_regions: ["global"]` to the GLM-4.7 model configuration
so that LiteLLM automatically uses the correct global endpoint.

Without this, users must manually specify `vertex_location: global`,
otherwise LiteLLM defaults to us-central1 which returns 404.
2026-03-11 13:18:12 -03:00
Chesars 093cca9f57 chore: remove unused AnthropicFilesHandler import and instance 2026-03-11 13:12:40 -03:00
Chesars b601b531e7 fix: use calendar.timegm for UTC timestamps and reuse AnthropicError
Address Greptile review feedback:
- Use calendar.timegm() instead of time.mktime() for correct UTC parsing
- Import AnthropicError from common_utils instead of redefining it
2026-03-11 13:03:49 -03:00
Chesars 9eff611b1a feat(anthropic): add Files API support for SDK
Implement Anthropic Files API (upload, retrieve, list, delete, content)
using the BaseFilesConfig provider pattern. Adds multipart form-data
support to BaseLLMHTTPHandler for file uploads.
2026-03-11 12:45:19 -03:00
+2 3f18cd2fdc [Docs] Fix "Page Not Found" link for Anthropic endpoint (#23349)
* fix(anthropic): enforce type:'object' on tool input schemas

Anthropic's API requires all tool input_schema to have type:'object'
at the root level. When OpenAI-format tools have parameters with a
missing or non-'object' type field (common with MCP tool servers),
the schema was passed through unchanged, causing Anthropic to reject
with: 'tools.N.custom.input_schema.type: Input should be object'.

The existing default handles the case where parameters is entirely
missing, but does not normalize schemas that ARE provided with a
wrong or absent type field.

Fix: After extracting _input_schema in _map_tool_helper(), ensure
type is set to 'object' and properties exists. This matches the
normalization already done implicitly by the Bedrock handler.

Added 4 unit tests covering: missing type, wrong type, valid schema
(no-op), and entirely missing parameters.

Related issues: #12020, #64, #1671

* fix(anthropic): deduplicate tool_result messages by tool_call_id

Anthropic requires exactly one tool_result per tool_use. When
conversation history (e.g. from session resume/checkpoint restore)
contains duplicate tool result messages with the same tool_call_id,
the API rejects with: 'each tool_use must have a single result.
Found multiple tool_result blocks with id: <id>'.

This is already handled for Bedrock via _deduplicate_bedrock_tool_content()
but was missing from the Anthropic direct and Vertex AI partner paths,
which share sanitize_messages_for_tool_calling().

Fix: Add Case D to sanitize_messages_for_tool_calling() — after the
existing orphan detection passes, scan for duplicate tool_call_ids
and keep only the last occurrence (most complete result).

Added 3 unit tests: dedup with duplicates, no-op with unique IDs,
and behavior when modify_params=False.

Related issues: #11804, #11029, #6836, #1782, #151

* fix: shallow copy input_schema to avoid caller mutation + add mutation guard test

Addresses Greptile review:
- dict(_input_schema) before mutation prevents cross-provider state leakage
- Test asserts original tool parameters dict is unchanged after call

* feat: add qwen3.5 series for openrouter

* fix: typo on max_output_tokens and max_tokens from qwen3.5 series

* chore: fix

* chore: fix

* [Test] UI - Logs: Add unit tests for 5 untested view_logs components

Add vitest tests for TypeBadges, ErrorViewer, ConfigInfoMessage, TimeCell, and TruncatedValue covering rendering, user interactions, and edge cases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Rename 'Team-Based Guardrails' to 'Team Bring-Your-Own Guardrails' (#23307)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* feat(chat-ui): responses API + MCP tool execution in /chat (#23297)

* feat(ui): add Chat UI v0 — standalone LiteLLM-branded chat window

Adds a full chat UI accessible from the sidebar Chat link (opens in new tab).
- Standalone route at /chat (outside dashboard layout — no Navbar/Sidebar chrome)
- Claude.ai-style layout: model selector top-left, LiteLLM logo center, settings top-right
- Greeting with time-of-day, centered input card, suggestion chips (Write/Learn/Code/Brainstorm)
- Sliding conversation history sidebar with Cmd+K search, rename, delete, date grouping
- localStorage-backed conversation persistence (litellm_chat_history_v1)
- Streaming completions via makeOpenAIChatCompletionRequest with AbortController stop support
- MCP server picker (toggle servers on/off per conversation)
- LiteLLM aesthetic: white/light-gray background, Ant Design blue (#1677ff) primary, system font
- Sidebar2: Chat menu item opens in new tab via window.open

* feat(chat-ui): responses API + MCP tool execution display

- Switch /chat from chat completions to responses API (previous_response_id session chaining)
- Add MCP server picker with search filter in chat input bar
- Show MCP tool call events (list_tools + call_tool) inline in chat via MCPEventsDisplay
- Add tool chip strip showing available tools when MCP servers are selected
- Non-blocking MCP toggle: server added immediately, verification in background (works for no-auth MCPs like deepwiki)
- Add truncateAfterMessage to useChatHistory for edit/retry
- Sync activeConversationId on URL change (fixes stale conversation on new chat)
- Add "Open Chat" shortcut button to sidebar

* fix(chat-ui): switch to responses API, remove dead code, add tests

- Switch handleSend from makeOpenAIChatCompletionRequest to makeOpenAIResponsesRequest with previous_response_id session chaining
- Add responsesSessionId state; reset to null when starting a new conversation
- Remove unused ChatInputBar.tsx and ModelSelector.tsx (dead code)
- Add tests/test_litellm/test_chat_ui_responses_session.py covering previous_response_id forwarding and signature validation

* fix(chat-ui): address greptile review issues

- Reset responsesSessionId when activeConversationId changes (not just on new conversation)
- Wire onMCPEvent callback into makeOpenAIResponsesRequest; render MCPEventsDisplay below messages
- Clear mcpEvents on each new send
- Explicitly filter history to user/assistant roles only (no tool-role casting)
- Remove duplicate "Chat" menu item from sidebar (pinned button serves same purpose)
- Make Sider a flex column so "Open Chat" button actually pins to bottom
- Fix tests to intercept real HTTP requests and assert previous_response_id in body

* fix(chat-ui): address greptile review feedback (greploop iteration 1)

- Fix duplicate context: when responsesSessionId is set, only send the
  new user message as input (prior context is already server-side via
  session chaining). Full history is still sent on the first turn.
- Fix ephemeral MCP events: store events per-message in ChatMessage.mcpEvents
  instead of ephemeral component state. Events now survive across turns
  and render inline below each assistant response via MCPEventsDisplay.
- Remove stale mcpEvents useState and ephemeral panel at bottom of chat.

* fix(chat-ui): address greptile review feedback (greploop iteration 2)

- Fix stale session on edit/retry: derive previousResponseId as null when
  historyOverride is set so edit/retry always starts a fresh Responses API
  session rather than chaining off a now-invalid prior session
- Fix unsafe MCPEvent cast: import MCPEvent directly from MCPEventsDisplay
  into types.ts and type ChatMessage.mcpEvents as MCPEvent[], eliminating
  the bare 'as MCPEvent[]' cast in ChatMessages.tsx

* fix(chat-ui): fix MCPEvent layering, batch localStorage writes, module-level test imports

- Move MCPEvent interface definition into chat/types.ts (single source of truth)
- MCPEventsDisplay.tsx now imports MCPEvent from types.ts instead of defining it locally
- Batch MCP event localStorage writes: accumulate during stream, persist once in finally
- Move test imports to module level per PEP 8 convention

* fix(chat-ui): fix MCPEvent import path and rename truncateFromMessage

- responses_api.tsx now imports MCPEvent directly from chat/types (not via MCPEventsDisplay re-export)
- Remove the now-unnecessary MCPEvent re-export from MCPEventsDisplay.tsx
- Rename truncateAfterMessage → truncateFromMessage: the function removes the target message and all subsequent ones (not just what comes after), so the new name accurately describes the behavior

* fix(responses-api): fix whitespace token filter and MCP server URL construction

- Drop the delta.trim() whitespace filter that was silently swallowing spaces
  and newlines during streaming, causing words to concatenate and paragraphs
  to collapse. Only skip truly empty strings (delta.length > 0).
- Use proxyBaseUrl for MCP server_url construction instead of the hardcoded
  relative path "litellm_proxy/mcp", so non-root deployments route correctly.

* fix(responses-api): use unique server_label per MCP server to prevent tool routing collisions

* fix(chat-ui): move MCPEvent to shared mcp_tools/types, skip partial events on abort

- Move MCPEvent interface to mcp_tools/types.tsx (shared with MCPServer/MCPTool),
  eliminating the playground→chat cross-module dependency. chat/types.ts and
  both playground components now import from mcp_tools/types.
- Only persist accumulated MCP events when the stream completes cleanly; aborted
  or errored turns drop partial events to avoid showing incomplete tool calls.

* fix(responses-api): use server_name for MCP URL routing, fix test path

- Use server_name (not alias) as the URL path segment for MCP server_url;
  alias is a display name that may differ from the registered proxy route.
  URL-encode the path to handle names with spaces/special characters.
- Fix sys.path.insert in tests to use __file__-relative path so tests pass
  regardless of which directory pytest is invoked from.

* fix(chat-ui): fix stale session after failed edit, clean MCP event persistence, unique server_label

- Eagerly call setResponsesSessionId(null) when historyOverride is set so a
  failed/aborted edit does not leave a stale session contaminating the next turn
- Replace abort-signal check with streamCompletedCleanly flag to correctly skip
  MCP event persistence on both abort and non-abort errors (network/API failures)
- Use server_name (unique) as server_label instead of alias to prevent silent
  tool-routing failures when two MCP servers share the same display name

* [Feat] UI - Show logos on MCP Apps page (#23320)

* feat(ui): add MCP server logo support across admin and chat UIs

- New MCPLogoSelector component with grid of well-known logos (GitHub,
  Slack, Notion, Linear, Jira, etc.) and custom URL input
- Create MCP Server form: logo picker with preview, OpenAPI presets
  auto-fill logo from registry icon_url
- Edit MCP Server form: logo picker pre-populated from mcp_info.logo_url
- Admin table: logos rendered next to server name in Name column
- Chat MCPAppsPanel: logos on server cards (list + detail view) with
  graceful fallback to letter avatars
- Chat MCPConnectPicker: logos next to server names in toggle list
- Fix pre-existing bug: setTools -> clearTools in create form cancel
- All 321 vitest files / 3211 tests pass

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat(ui): use local SVG logos for MCP services, fix Chat UI rendering

- Add 15 new MCP service logo SVGs (Slack, Notion, Linear, Jira, Figma,
  Gmail, Stripe, Salesforce, Shopify, HubSpot, Twilio, Sentry, Zapier,
  GitLab, Google Drive) to both source and pre-built directories
- Switch MCPLogoSelector from CDN URLs (cdn.simpleicons.org) to local
  asset paths (/ui/assets/logos/) for reliable rendering
- Logos now served by the proxy itself, working from any page path
  including /ui/chat/ (absolute paths resolve correctly everywhere)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(codeql): remove ruby from language matrix (#23227)

* Add team-scoped MCP server filtering for key creation and fix UnboundLocalError

When creating a key, the MCP server list now filters by the selected team's
allowed servers. Also fixes UnboundLocalError on `is_restricted_virtual_key`
when `team_id` query param was provided to GET /v1/mcp/server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix cross-team MCP server info disclosure and restricted key bypass

The GET /v1/mcp/server endpoint allowed any authenticated user to pass
an arbitrary team_id and enumerate another team's MCP server config.
Restricted virtual keys could also use the team_id param to bypass
their access limitations. Add team membership check for non-admins
and block restricted keys from using the team_id filter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix mcp_tool_permissions JSON string deserialization in _resolve_team_allowed_mcp_servers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [Feature] UI - MCP Servers: Add per-server health recheck

Allow users to recheck health for individual MCP servers by clicking
the health status badge. On hover the badge text changes to "Recheck"
with a refresh icon, and the check runs only for that server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix Anthropic docs link for beta endpoint

Update the Anthropic /v1/messages beta endpoint docstring to point to
its current pass-through documentation.

This keeps the change scoped to the incorrect URL and avoids changing
unverified wording in the surrounding comment.

---------

Co-authored-by: netbrah <162479981+netbrah@users.noreply.github.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: Joe Reyna <joseph.reyna@gmail.com>
2026-03-11 20:17:41 +05:30
Cesar GarciaandGitHub 5c8e87a9a1 Merge pull request #17155 from Chesars/fix/xai-streaming-empty-chunk-bug
Fix (xai): streaming empty chunk bug for providers using BaseLLMHTTPHandler
2026-03-11 10:36:58 -03:00
dependabot[bot]GitHubKrish DholakiaCursor Agentdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a78bd9a468 build(deps): bump hono from 4.10.6 to 4.12.7 in /litellm-js/spend-logs (#23312)
* Rename 'Team-Based Guardrails' to 'Team Bring-Your-Own Guardrails' (#23307)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* build(deps): bump hono from 4.10.6 to 4.12.7 in /litellm-js/spend-logs

Bumps [hono](https://github.com/honojs/hono) from 4.10.6 to 4.12.7.
- [Release notes](https://github.com/honojs/hono/releases)
- [Commits](https://github.com/honojs/hono/compare/v4.10.6...v4.12.7)

---
updated-dependencies:
- dependency-name: hono
  dependency-version: 4.12.7
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-11 14:13:33 +05:30
yuneng-jiangandGitHub d9e6758655 Merge pull request #23301 from BerriAI/litellm_user_email_keys
[Feature] UI - Keys: Improve VirtualKeysTable Column Display
2026-03-10 17:11:53 -07:00
yuneng-jiangandGitHub 37f325898f Merge pull request #23063 from BerriAI/litellm_fix_mcp_health_check_trigger
[Fix] UI - MCP Servers: stop health checks triggering on server deletion
2026-03-10 17:05:05 -07:00
yuneng-jiangandClaude Opus 4.6 9d83a6f814 [Feature] UI - Keys: Improve VirtualKeysTable column display and usability
Consolidate User Email and User ID columns into a single "User" column with
fallback display (Alias > Email > ID) and hover popover with copyable values.
Resolve Team and Organization columns to show aliases instead of raw UUIDs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 16:55:48 -07:00
Ishaan JaffandGitHub 1c0c504877 fix(mcp): fix OpenAPI OAuth flow — transport mapping, error messages, discovery bypass (#23300)
* fix(mcp): fix OpenAPI OAuth flow — transport mapping, error messages, and discovery bypass

Three bugs fixed to make the end-to-end OAuth flow work for OpenAPI MCP servers:

1. **Transport mapping in getTemporaryPayload**: `TRANSPORT.OPENAPI` is a UI-only concept;
   the backend only accepts `"http"`, `"sse"`, or `"stdio"`. The pre-OAuth temp-session
   call was sending `transport: "openapi"` and getting a 422. Fixed by mapping to `"http"`.

2. **deriveErrorMessage handles FastAPI 422 arrays**: FastAPI validation errors return
   `detail` as an array of `{loc, msg, type}` objects. The shared error extractor was
   returning the array directly, causing `Error: [object Object]`. Fixed to map each
   item to its `.msg` field.

3. **Skip OAuth discovery when authorization_url already provided**: `build_mcp_server_from_table`
   was unconditionally calling `_descovery_metadata(server_url)` for OAuth servers. For
   OpenAPI servers the url is the spec JSON file, not the API base — this caused a timeout
   fetching e.g. the GitHub spec (2 MB). Fixed by skipping discovery when `authorization_url`
   is already set.

Also: collapsible auth section in MCP server form, "Create OAuth App →" link next to
Client ID when a docs URL is available (e.g. GitHub OAuth App creation page), and
`extractErrorMessage` helper in `useMcpOAuthFlow` for cleaner error display.

* refactor(mcp): extract needs_discovery flag and reduceStaticHeaders helper
2026-03-10 16:48:53 -07:00
Shivam RawatandGitHub cf331c4fc5 Merge pull request #23295 from BerriAI/revert-23287-docs_flow_builder
Revert "policy builder"
2026-03-10 15:39:10 -07:00
Shivam RawatandGitHub a71ba39b78 Revert "policy builder" 2026-03-10 15:38:59 -07:00
Cesar GarciaandGitHub 3bf91ed9fe Merge pull request #23258 from Chesars/docs/openai-tool-search
docs(responses): add tool_search & namespaces docs for gpt-5.4
2026-03-10 18:51:16 -03:00
Chesars e7a9c1e156 docs(responses): remove unused json import from tool search example 2026-03-10 18:41:54 -03:00
Ishaan JaffandGitHub 3b80b312a9 feat(ui): show tools per MCP server in chat panel (#23294)
- fix transport display: use handleTransport() instead of hardcoding HTTP/stdio based on server_url presence
- show available tools list when clicking into a server detail view
- preload tool counts per server card in parallel (one request per server, counts pop in independently)
- add skeleton loading indicator on each card while its tool count is fetching
- fix: pass server UUID (not name) to /mcp-rest/tools/list — name was always hitting access_denied
2026-03-10 14:40:28 -07:00
Ishaan JaffandGitHub 6faad2919a feat(chat-ui): add MCP OAuth2 value prop and server auth badges (#23291)
* feat(chat-ui): add MCP OAuth2 value prop and OAuth2 pill on server cards

- Add subtitle in chat empty state explaining MCP OAuth2 value prop with 'Open Apps' link
- Update MCPAppsPanel header copy to explain the flow more clearly
- Show OAuth2 pill badge on server cards where auth_type is oauth2

* fix(chat-ui): use AUTH_TYPE.OAUTH2 constant instead of hardcoded string
2026-03-10 14:37:32 -07:00
Cesar GarciaandGitHub 375b7665ad Merge pull request #23275 from Chesars/fix/add-gpt-5.3-model-entry
feat(openai): add missing supports_web_search to OpenAI models
2026-03-10 18:00:59 -03:00
Ishaan JaffandGitHub 8c12781487 feat(ui): OpenAPI MCP server support with popular API quick-picker (#23200)
* fix: guard prisma import in config_override_endpoints to fix proxy import without prisma

Top-level `from prisma.errors import RecordNotFoundError` was introduced in the
Hashicorp Vault feature PR and breaks `import litellm.proxy.proxy_server` when
prisma is not installed (e.g. plain `pip install litellm[proxy]` in CI).

Wraps the import in try/except ImportError so the module loads cleanly when
prisma is absent; the except branch aliases RecordNotFoundError to Exception,
which is safe because the code path that catches it only logs a debug message.

* fix: sync poetry.lock with pyproject.toml (litellm-proxy-extras 0.4.51 → 0.4.52)

poetry.lock was regenerated for 0.4.51 but pyproject.toml was subsequently
bumped to 0.4.52 without re-running poetry lock. This caused the
proxy_e2e_azure_batches_tests CI job to fail at the Install Dependencies step
('pyproject.toml changed significantly since poetry.lock was last generated'),
preventing all 3 tests in that job from running.

* Revert "fix: sync poetry.lock with pyproject.toml (litellm-proxy-extras 0.4.51 → 0.4.52)"

This reverts commit 249ec7c9c2.

* feat(ui): add OpenAPI MCP server support with popular API quick-picker

- New `openapi_registry.json` with 10 well-known APIs (GitHub, Atlassian, Figma, Google, Stripe, HubSpot, Notion, Slack, Shopify, Snowflake) — each with validated spec URLs and OAuth 2.0 endpoints
- Backend endpoint `GET /v1/mcp/registry.json` to serve the registry (reads fresh from disk)
- `OpenAPIQuickPicker` component: logo grid for popular APIs with letter fallback for broken images
- `OpenAPIFormSection` component: encapsulates picker + spec URL input as a clean unit
- When selecting a preset, spec URL and OAuth fields are pre-filled automatically
- Fixed `useTestMCPConnection`: for OpenAPI transport, tools load from the spec as soon as the URL is set — no auth type or OAuth token required
- Validated all spec URLs are reachable; removed Linear (GraphQL-only, no REST spec)

* feat(ui): add curated key tools preview for OpenAPI MCP servers

When selecting a popular API from the quick-picker, show the 8 most
useful MCP tools for that API in a collapsible preview card. First 4
are shown by default; clicking "expand" shows all 8 with descriptions
on hover.

- Added `key_tools` array (8 tools each) to all 10 APIs in openapi_registry.json
- New `KeyToolsPreview` component in OpenAPIFormSection with expand/collapse
- Extended `OpenAPIRegistryEntry` type with `key_tools?: OpenAPIKeyTool[]`

* fix(ui): move key tools preview inside Tool Configuration card

* fix(ui): pin suggested tools at top of tool list, fix TDZ crash, add per-section enable/disable

* fix: address greptile review - fix registry spec URLs, remove redundant dep, add error handling

* fix: restore enable/disable all buttons for non-preset MCP servers

* refactor: extract ToolRow component, move handlers to component body, fix key props

* fix: remove rewrites() from next.config.mjs (incompatible with output: export), fix OAuth field paths

* fix: enable/disable all operates on full tool set, not just filtered subset

* fix: cache openapi registry, make oauth optional, reset preset auto-select on new preset

* fix: use official Shopify API specs repo, move lru_cache error handling to caller

* fix: use Shopify 2023-10 REST spec, co-locate rest section header with tool rows

* fix: guard fuzzy-match against empty keywords, remove placeholder OAuth URLs for Shopify/Snowflake

* fix: lift useTestMCPConnection to parent to eliminate duplicate requests, clear keyTools on manual spec URL edit

* fix: clear stale OAuth fields when switching to non-OAuth preset, show expected tools on empty spec

* fix: gate registry fetch on modal visibility, use resetFields to clear OAuth fields
2026-03-10 13:59:52 -07:00
Ishaan JaffandGitHub 373e5e316b feat(mcp): BYOM — non-admin MCP server submission + admin review workflow (#23205)
* feat(mcp): add BYOM (Bring Your Own MCPs) submission + admin review workflow

Non-admins can now submit MCP servers for review via POST /v1/mcp/server/register.
Admins get a Submissions tab in the UI to approve or reject pending servers.
Approved servers enter the active runtime; rejected ones stay out with notes.

- DB: add approval_status, submitted_by, submitted_at, reviewed_at, review_notes
  to LiteLLM_MCPServerTable with migration
- Backend: new endpoints register, submissions, approve, reject
- reload_servers_from_database now only loads approval_status=active servers
- UI: Submissions tab with stat cards, card list, confirm dialogs; non-admin
  "Submit MCP Server" button wired to /register endpoint
- Fix get_mcp_submissions to filter by submitted_at IS NOT NULL (not submitted_by,
  which can be null for team-scoped keys without an associated user)

* feat(mcp): rename nav item to Team MCPs + add New badge

* fix(mcp): revert nav label, rename Submissions tab to Team MCPs + New badge

* feat(mcp): add MCP Standards — required fields config + CI-style checks on submissions

Adds a "Standards" tab (admin-only) to MCP Servers where admins define which
server fields are required for a submission to pass. Each submission card in
Team MCPs then shows a green ✓ or red ✗ for each required field, with a
summary "N/M checks" badge in the header — like GitHub CI status rows.

Also adds a `source_url` field (GitHub / Source URL) to the MCP server schema
so non-admins can link to the source repo when submitting a server.

- schema.prisma: add `source_url String?` to LiteLLM_MCPServerTable
- migration: 20260309000001_add_mcp_source_url
- _types.py: source_url on NewMCPServerRequest, UpdateMCPServerRequest, LiteLLM_MCPServerTable
- types.tsx: source_url on MCPServer interface
- create_mcp_server.tsx: GitHub/Source URL form field
- MCPStandardsSettings.tsx: new — toggle which fields are required (stored in general settings as mcp_required_fields)
- mcp_servers.tsx: Standards tab (admin-only)
- MCPSubmissionsTab.tsx: load required fields + CI-style check pills on each card

* refactor(mcp): move submission rules into Team MCPs tab, grouped free-form UI

Folds the Standards tab into Team MCPs. Submission Rules panel now lives at the
top of the Team MCPs tab — collapsible, shows active rules as chips when closed,
expands to a grouped checkbox editor (Documentation / Source / Connection /
Security). Removes the separate Standards tab from the nav.

MCPStandardsSettings.tsx is now constants-only (FIELD_GROUPS, MCP_REQUIRED_FIELD_DEFS,
SETTINGS_KEY) — the UI lives in MCPSubmissionsTab.

* feat(mcp): add mcp_required_fields to ConfigGeneralSettings + config/list endpoint

Registers mcp_required_fields as a proper general_settings field so the UI
can read/write it via /config/list and /config/field/update without the
"Invalid field" error. Also fixes a pre-existing pyright None-check issue
in _sync_ui_settings_to_general_settings.

* ui(mcp): GitHub-style PR checks panel on submission cards

* ui: rename Team MCPs -> Submitted Tools, Team Guardrails -> Submitted Guardrails

* address greptile review feedback (greploop iteration 1)

* fix: inline import, add approval workflow tests, rename Submitted MCPs

* fix(mcp): allow re-approval of rejected MCP server submissions

* fix(mcp): evict rejected servers from runtime; enforce mcp_required_fields on /register

* fix(mcp): sort submissions newest-first; force active status on admin-created servers

* fix(mcp): add missing mock in test, show Approve for rejected, clear submission metadata, drop spurious Content-Type

* fix(mcp/ui): show Reject for active servers; show submit form to non-admins with team-key note

* fix(mcp): conditional reload on reject; view-only admin for submissions; block admin from /register

* fix(mcp): match auth_type required-field validation to UI compliance check (reject 'none')

* fix(mcp): block view-only admin from /register; log settings failure; warn on active server reject

* fix(mcp): allow view-only admin to use /register; add _validate_mcp_required_fields tests

* fix(mcp): validate field names in mcp_required_fields; surface backend error in submit UI

* fix(mcp): fix falsy field check; add field-name validation; add take limit; document server-managed fields; close dialog on error
2026-03-10 13:58:59 -07:00
Cesar GarciaandGitHub 6a3b029066 Merge pull request #23271 from Chesars/docs/gpt54-reasoning-tools-limitation
docs(openai): document gpt-5.4 reasoning_effort + tools limitation
2026-03-10 17:57:31 -03:00
milan-berriGitHubgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
9100e16776 docs: pip venv upgrade workflow (#23290)
* docs: add pip/venv upgrade workflow guide

- Add comprehensive guide for upgrading LiteLLM proxy via pip
- Covers Prisma client regeneration and DB migration steps
- Includes verification commands and troubleshooting tips
- Links to existing Prisma migration troubleshooting doc

* docs: clarify Python version in prisma generate command

- Update example to show multiple Python versions (3.11, 3.12, 3.13)
- Make it clear LiteLLM supports multiple Python versions, not just 3.11

* docs: emphasize venv activation before running commands

- Add info box at top reminding users to activate venv
- Include venv activation step before starting proxy (both options)
- Add Windows activation command for cross-platform clarity
- Make it clear all commands assume activated venv

* docs: add pip_venv_upgrade to sidebar navigation

- Add new page to Troubleshooting section in sidebars.js
- Positioned after Performance/Latency category and before rollback
- Makes the upgrade guide discoverable through docs navigation

* docs: show explicit --schema flag in prisma migrate deploy

- Add explicit --schema path to Option B migration command
- Remove ambiguous instruction about running from litellm_proxy_extras
- Include path variable guidance for clarity
- Makes the command immediately runnable without directory navigation

* Update docs/my-website/docs/troubleshoot/pip_venv_upgrade.md

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update docs/my-website/docs/troubleshoot/pip_venv_upgrade.md

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: close code block and add missing section in pip_venv_upgrade.md

* docs: define schema-path placeholder in verification section

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-10 13:53:54 -07:00
Chesars e88dc2e428 feat(openai): add supports_web_search to o4-mini models
Tested and confirmed both o4-mini and o4-mini-2025-04-16 support
web_search_preview via the Responses API.
2026-03-10 17:49:39 -03:00
Shivam RawatandGitHub 97c92cc84e Merge pull request #23287 from BerriAI/docs_flow_builder
policy builder
2026-03-10 13:44:18 -07:00
Shivam RawatGitHubgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
592232e835 Update docs/my-website/docs/proxy/guardrails/guardrail_pipeline_flow_builder.md
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-10 13:42:46 -07:00
Shivam RawatGitHubgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
f3844d8356 Update docs/my-website/docs/proxy/guardrails/guardrail_pipeline_flow_builder.md
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-10 13:42:36 -07:00
Chesars effaef3cfb fix: remove duplicate supports_web_search keys
7 models already had supports_web_search from upstream, causing
duplicate JSON keys. Re-serialized to remove duplicates.
2026-03-10 16:49:45 -03:00
Chesars 7a0c8af458 merge: resolve conflicts with upstream/main
Keep both: supports_web_search additions and upstream's
supports_none_reasoning_effort / supports_xhigh_reasoning_effort fields.
2026-03-10 16:42:28 -03:00
Chesars 6bc4cc8f0f feat(openai): add supports_web_search to OpenAI models with Responses API
Add `supports_web_search: true` to 31 OpenAI models that support the
`web_search_preview` tool via the Responses API. This enables the Router
to correctly include these deployments when requests use web search tools.

Models excluded (tested, confirmed unsupported):
- o1-pro (Tool 'web_search_preview' is not supported)
- gpt-audio / gpt-audio-mini (not supported)
- gpt-4.1-nano (not supported)
- codex-mini-latest (model not found)

Also removes the invalid `gpt-5.3` entry added in prior commit
(model name does not exist in OpenAI API; use gpt-5.3-chat-latest).
2026-03-10 16:28:50 -03:00
shivam fa330ed96b policy builder 2026-03-10 12:09:00 -07:00
Chesars af297dc082 fix(openai): clean up gpt-5.3 model entry fields
- Remove dead fields: supports_none_reasoning_effort, supports_xhigh_reasoning_effort
  (not referenced anywhere in the codebase)
- Remove supports_web_search (inconsistent with other base models)
- Add supports_service_tier (consistent with gpt-5, gpt-5.1, gpt-5.2)
2026-03-10 16:04:31 -03:00
ffc89e4ef6 fix(mcp): add AWS SigV4 auth for Bedrock AgentCore MCP servers (#22782)
* fix(mcp): add AWS SigV4 auth for Bedrock AgentCore MCP servers

Add aws_sigv4 auth type to MCP client via httpx.Auth subclass that
signs each request with SigV4 using botocore. Enables mcp_servers
config to connect to AgentCore-hosted MCP servers.

* docs(mcp): add AWS SigV4 auth documentation for Bedrock AgentCore

Add dedicated docs page for configuring MCP servers with AWS SigV4
authentication, update MCP overview with aws_sigv4 auth type and
config example, and link from Bedrock AgentCore provider docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(mcp): address Greptile review — requires_request_body, full header signing, health check

- Add requires_request_body = True to MCPSigV4Auth so httpx buffers the
  request body before calling auth_flow (prevents empty body hash for
  streaming requests)
- Pass all request headers to AWSRequest for canonical SigV4 signing
  instead of only Content-Type
- Exclude aws_sigv4 from health check skip logic since it has its own
  credential fields (not authentication_token)
- Fix docs: mark aws_access_key_id/aws_secret_access_key as optional
  (falls back to boto3 credential chain)
- Add test for requires_request_body flag

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-10 11:11:20 -07:00