litellm

mirror of https://github.com/tiennm99/litellm.git synced 2026-07-14 20:22:20 +00:00

Files

T

Sameer Kankute 1294165768 feat(mcp): support MCP access group names in URL-based namespacing (#27726 )

* feat(mcp): support MCP access group names in URL-based namespacing

Extends dynamic_mcp_route to resolve /{name}/mcp requests where {name}
is an MCP access group tag or a comma-separated list of servers/groups,
matching what the documentation promised but the handler did not implement.

Resolution order: registered server alias → toolset → comma-separated
list → single access group tag (404 if none match).

Adds unit tests covering all four resolution paths plus 404 cases.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): address Greptile review comments on dynamic_mcp_route

- Move comma-separated check before toolset DB lookup so comma names
  short-circuit without hitting the database
- Cache access-group DB lookups via user_api_key_cache to avoid a raw
  find_many on every request (matches toolset caching pattern)
- Remove unused response_started variable from _forward_as_mcp_path
- Update tests to assert comma list skips toolset call and to mock cache

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(mcp): extract helpers to fix PLR0915 too-many-statements in dynamic_mcp_route

Extract _mcp_forward_as_path and _is_mcp_access_group_cached as
module-level helpers so dynamic_mcp_route stays under the 50-statement
limit. Update tests to patch the new module-level symbols directly.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Avoid caching missing MCP access groups

* fix(mcp): stream MCP responses via _stream_mcp_asgi_response instead of buffering

_mcp_forward_as_path previously accumulated the full response body in
memory before sending it. Replace the buffering custom_send pattern with
_stream_mcp_asgi_response, which uses an asyncio.Queue bridge so chunks
are yielded to the client as they arrive, preventing unbounded memory
growth on large or long-lived MCP responses.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(mcp): short-TTL negative cache for access-group existence lookup

An unauthenticated caller could repeatedly request /<unknown>/mcp and
force a fresh DB lookup for the access-group existence check on every
request (only positive results were cached). Cache negative results
for a short DEFAULT_MCP_ACCESS_GROUP_NEGATIVE_CACHE_TTL window (10s by
default) so the DB is shielded from flooding while a transient DB error
(which surfaces as an empty list) cannot hide a real group for long.

https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9

* fix(mcp): use plain int for access-group negative cache TTL

Drop the os.getenv wrapper around DEFAULT_MCP_ACCESS_GROUP_NEGATIVE_CACHE_TTL
to avoid the documentation_test_env_keys check failing on the new variable.
The negative-cache window is a small internal tuning constant, not a
user-facing knob, so a plain integer is clearer than an env override.

https://claude.ai/code/session_01SjyPmwfmrq8fveFgw9iHW9

* fix(mcp): validate, dedupe, and cap CSV tokens in dynamic MCP route

For /{name1,name2,...}/mcp, validate every token resolves to a known
server alias or access group, dedupe case-insensitively, and cap at
DEFAULT_MCP_NAMESPACE_CSV_MAX_TOKENS=16 before forwarding.

- Bounds the per-request DB / cache fan-out an authenticated caller can
  trigger by stuffing the path with tokens (raised by veria-ai).
- Returns 404 instead of forwarding when no token resolves, so the
  downstream server filter cannot silently fall back to the full
  allowed_mcp_servers list (raised by Cursor agentic security review).
- Forwards only the resolved subset, so unknown tokens cannot ride along
  into the downstream filter.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

* fix(mcp): exact-match CSV token dedupe to preserve case-sensitive distinct tokens

Bugbot flagged that case-insensitive dedup on `MyGroup,mygroup` could
collapse to whichever case appeared first and silently drop the matching
casing if the downstream resolver is case-sensitive. Switch to exact-match
dedup so distinct casings survive; whitespace-only differences still
collapse via the .strip() before comparison.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude <claude@anthropic.com>
Co-authored-by: mateo-berri <mateo@berri.ai>
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>