tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-16 10:17:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
30,058 Commits 2,072 Branches 1,425 Tags
ebeea46fc4e12ac9ee79acbf19bc8a8585322e9f
Commit Graph
76 Commits
Author SHA1 Message Date
Yuta Saito 35f9a75d55 feat: add UI support for configuring meta URLs 2026-01-02 15:07:37 +09:00
Yuta Saito 34b500c7f5 feat: support MCP stdio header env overrides 2025-12-22 12:58:31 +09:00
Yuta Saito ed5cbdac2f feat: add support for using MCPs on /chat/completions 2025-12-10 10:55:33 +09:00
YutaSaitoGitHubAfroz AhmadAfroz Ahmad
 		b5133c4c7d Feat/mcp preserve tool metadata calltoolresult (#17561) 
* feat(mcp): preserve tool metadata and full CallToolResult in MCP gateway

This PR fixes two issues that prevented ChatGPT from rendering MCP UI widgets
when proxied through LiteLLM:

1. Preserve Tool Metadata in tools/list
   - Modified _create_prefixed_tools() to mutate tools in place instead of
     reconstructing them, preserving all fields including metadata/_meta
   - This ensures ChatGPT can see 'openai/outputTemplate' URIs in tools/list
     and will call resources/read to fetch widgets

2. Preserve Full CallToolResult (structuredContent + metadata)
   - Changed call_mcp_tool() and _handle_managed_mcp_tool() to return full
     CallToolResult objects instead of just content
   - Updated error handlers to return CallToolResult with isError flag
   - Wrapped local tool results in CallToolResult objects
   - This preserves structuredContent and metadata fields needed for widget rendering

Files changed:
- litellm/proxy/_experimental/mcp_server/mcp_server_manager.py
- litellm/proxy/_experimental/mcp_server/server.py

Fixes issues where ChatGPT could not render MCP UI widgets when using
LiteLLM as an MCP gateway.

* feat(mcp): Preserve tool metadata and return full CallToolResult for ChatGPT UI widgets

- Preserve metadata and _meta fields when creating prefixed tools
- Return full CallToolResult instead of just content list
- Ensures ChatGPT can discover and render UI widgets via openai/outputTemplate
- Fixes metadata stripping that prevented widget rendering in ChatGPT

Changes:
- mcp_server_manager.py: Mutate tools in place to preserve all fields including metadata
- server.py: Return CallToolResult with structuredContent and metadata preserved
- Added test to verify metadata preservation

* fix: guard cost calculator when BaseModel lacks _hidden_params

---------

Co-authored-by: Afroz Ahmad <aahmad@Afrozs-MacBook-Pro.local>
Co-authored-by: Afroz Ahmad <aahmad@KNDMCPTMZH3.sephoraus.com>
 2025-12-05 16:15:22 -08:00
Ishaan Jaffer 6530749c89 test fixes 2025-11-26 11:42:12 -08:00
YutaSaitoandGitHub b72b49757e feat: add backend support for OAuth2 auth_type registration via UI (#17006) 2025-11-23 21:52:18 -08:00
YutaSaitoandGitHub f0b10b854b chore: remove unused MCP_PROTOCOL_VERSION_HEADER_NAME constant (#17008) 2025-11-23 21:51:11 -08:00
Ishaan Jaffer 0c28af8705 test MCP server 2025-11-22 10:02:15 -08:00
Ishaan Jaffer fc0eac2d10 test_get_tools_from_mcp_servers 2025-11-22 10:02:15 -08:00
YutaSaitoandGitHub 93affcb732 [Feat] mcp resources support (#16800) 
* feat: mcp prompts support

* feat: mcp resources support
 2025-11-20 14:53:44 -08:00
YutaSaitoandGitHub 0b586d26fc refactor: drop MCPClient.connect and use run_with_session lifecycle (#16696) 
Surface detailed connection errors by handling HTTP failures
 2025-11-15 17:54:27 -08:00
yuneng-jiangandGitHub 00ecb99f3e [Fix] UI MCP Tool Test Regression (#16695) 
* Fix UI MCP testing tool regression

* Fixed linting
 2025-11-15 15:31:04 -08:00
YutaSaitoandGitHub f487f4e3a9 feat: add dynamic OAuth2 metadata discovery for MCP servers (#16676) 
* feat: add dynamic OAuth2 metadata discovery for MCP servers

* fix: lint error
 2025-11-14 18:14:43 -08:00
YutaSaitoandGitHub 8e0b66a814 fix: exclude unauthorized MCP servers from allowed server list (#16551) 
* fix: exclude unauthorized MCP servers from allowed server list

* fix: test after resolving merge conflicts
 2025-11-13 12:33:54 -08:00
YutaSaitoandGitHub 2843dab7fe fix: allow tool call even when server name prefix is missing (#16425) 
* fix: allow tool call even when server name prefix is missing

* fix: test

* fix: test

* fix: test
 2025-11-12 13:50:52 -08:00
YutaSaitoandGitHub 6eb74bd62a Feat/persist mcp credentials in db (#16308) 
* feat: persist mcp credentials in db

* feat: remove Auth Value field from MCP Tool Testing Playground

* fix: test
 2025-11-07 19:22:49 -08:00
YutaSaitoandGitHub 8e27b6c0b4 [MCP] configure static mcp header (#16179) 
* feat: configure extra mcp headers in ui

* doc: static header

* build: add new migration file

* chore: add missing image file

* fix: test
 2025-11-03 21:06:36 -08:00
YutaSaitoandGitHub c0890e7d33 [Feat] add support for dynamic client registration (#15921) (enables Atlassian MCP to work via Oauth on LiteLLM) 
* feat: add support for dynamic client registration #13856

* fix: test

* feat: return 401 when oauth2_header is missing for OAuth2-based MCP servers
 2025-10-26 10:13:46 -07:00
Vinod SinghandGitHub d4aadda692 Auth Header Fix for MCP Tool Call (#15736) 
* fixed the Auth header for MCP Tool Call

* Final fix for Auth header

* testcase for mcp_auth_header_extraction, insensitive_alias_matching, insensitive_servername_matching added
 2025-10-21 13:58:03 -07:00
Yuta Saito 5359a0d6a6 fix: test 2025-09-30 07:25:32 +09:00
Yuta Saito be5010a675 test: fix tests 2025-09-29 07:21:32 +09:00
Krrish Dholakia 9403efa511 test: fix tests 2025-09-28 08:54:15 -07:00
Krrish Dholakia 1256dd53d9 test: update tests 2025-09-28 08:53:11 -07:00
Krrish Dholakia 340262c85f feat(server.py): only show allowed mcp tools 2025-09-27 20:57:12 -07:00
Krrish Dholakia 5d6889dc92 test: update tests 2025-09-27 10:40:06 -07:00
Ishaan Jaffer a6795a6560 test fix 2025-09-19 18:05:36 -07:00
Ishaan Jaffer dec15a80a2 test: mcp test fix 2025-09-19 17:52:32 -07:00
Ishaan Jaffer c3f150b13d mcp test fix 2025-09-19 17:41:15 -07:00
Yuta Saito 654f1d3290 fix: stop including spec_version in MCP server registration inserts 2025-09-19 07:06:15 +09:00
Yuta Saito 6c291093e9 fix: remove adding Mcp-Protocol-Version header (#14069) 
The Mcp-Protocol-Version header is already handled in the MCP Python SDK, so the explicit addition on LiteLLM Proxy was redundant.
 2025-09-19 07:05:20 +09:00
Ishaan JaffandGitHub a13aa4740a [Fixes] Bug fixes to using LiteLLM MCP Gateway (#14392) 
* fix: use _get_mcp_servers_in_path

* fix checks for using litellm_proxy as MCP tool provider

* fix: fix mcp_tools_with_litellm_proxy

* fix: fix aresponses_api_with_mcp

* aresponses_api_with_mcp

* test_mcp_allowed_tools_filtering

* fix: _filter_mcp_tools_by_allowed_tools

* fix: _filter_mcp_tools_by_allowed_tools

* test_streaming_responses_api_with_mcp_tools

* fixes: test tools transfrom MCP->OpenaI spec

* test_streaming_responses_api_with_mcp_tools

* fix: chat ui allow multi select with allowed tools

* fix: use correct MCP events with litellm proxy response API

* fix get_event_model_class

* fix litellm proxy MCP handler

* fix MCPEnhancedStreamingIterator

* chat ui show list tools result

* UI: show MCP events

* fix stream iterator

* fixes: litellm proxy mcp handler

* test responses + mcp

* fix: update responses api with mcp handling

* ruff check fix

* central: _process_mcp_tools_to_openai_format

* fix: refactor code

* test_mcp_allowed_tools_filtering

* test mcp with litellm proxy

* fix mcp call

* demo: video using MCP ui

* fixes for using stream iterator

* test_no_duplicate_mcp_tools_in_streaming_e2e

* docs fix

* fix code snippet
 2025-09-10 19:12:11 -07:00
Ishaan JaffandGitHub 1f42e41c8d [Bug]: Fix Authorization header not being sent to configured MCP servers (#14422) 
* test: test_mcp_server_config_auth_value_header_used

* fix: authentication_token

* docs: fix instructions on using responses api with MCPs

* mcp fixes
 2025-09-10 16:41:08 -07:00
Ishaan JaffandGitHub 68d250973c [Feature]: Support "Token" MCPAuth (#14359) 
* add MCPAuth.authorization type

* test: Token Auth MCP

* fix: _get_auth_headers to handle auth_type == MCPAuth.authorization

* docs auth types with MCP on config.yaml

* test_mcp_server_works_without_config_auth_value
 2025-09-08 18:25:41 -07:00
Ishaan JaffandGitHub b9132968b2 [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905) 
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f48935652f054777652938ad71976cce.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
 2025-08-23 13:13:23 -07:00
Jugal D. BhattandGitHub b8fe5f7b17 [MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630) 
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
 2025-08-14 16:32:18 -07:00
Jugal Bhatt f487816b9f [fix] Enhance MCPServerManager with access groups and description support 
* Added access_groups and description fields to MCPServerManager for better server configuration.
* Implemented tests to verify integration of config-based servers with database servers, ensuring correct handling of access_groups and description.
* Updated add_update_server method to accommodate new fields and validate server addition in the registry.
 2025-08-12 14:22:40 -07:00
Thiago SalvatoreandGitHub c2ad858c83 fix(access group): allow access group on mcp tool retrieval (#13425) 
* fix(access group): allow access group on mcp tool retrieval

* fix(test): fix broken tests and add test case for access group

* fix(mypy): fix typing issues
 2025-08-08 08:55:46 -07:00
Jugal D. BhattandGitHub b1a8968895 [MCP Gateway] fix auth on ui for bearer servers (#13312) 
* fix auth on ui for bearer servers

* add tests and fixes

* fix tests
 2025-08-06 09:46:10 -07:00
Jugal D. BhattandGitHub 900c7f45c0 [MCP Gateway] Litellm mcp pre and during guardrails (#13188) 
* add guardrail support

* add guardrail support

* guardrails for MCP

* added changes

* add mcp guardrails

* added test

* add ui

* fix guardrail form

* working with cursor

* remvoe print

* fix mcp servertests

* fix mypy and remove console logs

* fix mypy and remove console logs

* fix mypy tests
 2025-08-01 20:02:25 -07:00
Jugal D. BhattandGitHub 5db4862cbf [MCP Gateway] Litellm mcp client list fail (#13114) 
* fix headers

* fix test

* fix ruff

* added try except for catching errors which lead to client failures

* fix mypy

* fix ruff

* fix tests

* fix python error

* fix test

* fix test

* fixed the MCP Call Tool result
 2025-07-30 15:23:19 -07:00
Jugal D. BhattandGitHub eb8a338d9b [MCP Guardrails] move pre and during hooks to ProxyLoggin (#13109) 
* move pre and during hooks t o ProxyLoggin

* fix lint

* fix ruff

* fix tests
 2025-07-30 13:58:41 -07:00
Ishaan Jaff 1af0743e58 Revert "Revert "[MCP Gateway] Add protocol headers (#13062)"" 
This reverts commit acd915f2db.
 2025-07-29 18:14:31 -07:00
Ishaan Jaff acd915f2db Revert "[MCP Gateway] Add protocol headers (#13062)" 
This reverts commit 8de24bab7c.
 2025-07-29 17:26:00 -07:00
Jugal D. BhattandGitHub 8de24bab7c [MCP Gateway] Add protocol headers (#13062) 
* Add protocol headers

* fix mypy

* fix tests

* fix tests
 2025-07-29 16:03:23 -07:00
Jugal D. BhattandGitHub ae9a02d7d5 [MCP gateway] add pre and during call hooks init (#13067) 
* add hook init

* add during hook

* added logging
 2025-07-29 11:06:58 -07:00
Jugal D. BhattandGitHub b921e22594 [MCP Gateway] Litellm mcp multi header propagation (#13003) 
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* add multi server header support

* add and fix tests

* fix tests

* fix tests

* add a common util

* ruff fix

* fix ruff

* fix tests

* fix migration

* mypy fix

* change server py
 2025-07-26 11:45:14 -07:00
Jugal D. BhattandGitHub 10595ea077 [MCP Gateway] add Litellm mcp alias for prefixing (#12994) 
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* fix tests

* add a common util

* ruff fix

* fix migration
 2025-07-25 17:57:52 -07:00
Jugal D. BhattandGitHub a7d0b122b9 [MCP Gateway] List tools from access list for keys (#12657) 
* fix tools fetch for keys

* add unit test
 2025-07-16 14:28:53 -07:00
Jugal D. BhattandGitHub b3a214bce7 [MCP Gateway] Allow MCP access groups to be added via the config LIT-312 (#12654) 
* allow mcp access groups to be added via the config

* fix code and doc quality

* fix mypy

* create helpers:
 2025-07-16 13:57:43 -07:00
Ishaan JaffandGitHub 2d6751a396 [Feat] MCP Gateway - allow using MCPs with all LLM APIs when using /responses with LiteLLM (#12546) 
* add MCPResponsesAPIHelper

* rename LiteLLM_Proxy_MCP_Handler

* aresponses_api_with_mcp

* mock_responses_api_response

* test response with litellm proxy MCP

* add _should_use_litellm_mcp_gateway

* fix transform_mcp_tool_to_openai_responses_api_tool

* use correct _transform_mcp_tools_to_openai

* fix config.yaml

* fixes for native MCP handling

* docs MCP with litellm proxy

* aresponses_api_with_mcp

* fix linting

* fix mypy

* fix linting

* test_aresponses_api_with_mcp_mock_integration

* docs How it works when server_url="litellm_proxy"
 2025-07-15 14:06:31 -07:00