* added mcp tools on internal user and divide it by teams
* add support for server api call
* Added frontend for test key
* added tools used output
* fix ui for servers
* All servers to personal
* change columns format
* revert ui logic
* Added vertical align
* fix mapped tests
* fix lint
* fix lint
* remove extra file
* fix ui test
* comments fixes
* change query type
* change query type
* change query type
* Update model_checks.py
* add _get_mcp_auth_header_from_headers
* test_process_mcp_request_with_custom_auth_header
* Using a different Authentication Header
* fix customize MCP Auth header name
* fix security - mcp
* fix(router.py): validate model provider before adding deployment to pattern
prevents routing on pattern match to invalid deployment
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* Fix: Properly close aiohttp client sessions to prevent resource leaks (#12107)
- Add close() method to BaseLLMAIOHTTPHandler to properly close aiohttp ClientSession
- Create async_client_cleanup module with utility functions to close all cached async clients
- Register automatic cleanup at exit via atexit hook
- Export close_litellm_async_clients() function for manual cleanup
- Add comprehensive tests to verify resource cleanup
This fixes the "Unclosed client session" and "Unclosed connector" warnings when using acompletion with Gemini and other models that use aiohttp.
Fixes#12107
* Fix: Remove unused import to satisfy linter
* Fix: Extend cleanup to handle AsyncHTTPHandler instances used by Gemini
The original implementation only cleaned up BaseLLMAIOHTTPHandler instances,
but Gemini/Vertex AI providers use AsyncHTTPHandler objects which contain
httpx clients with aiohttp transports.
This commit extends the cleanup function to:
- Handle AsyncHTTPHandler instances by accessing their internal client
- Close both the aiohttp transport and httpx client
- Add generic fallback for any objects with aclose method
This properly fixes the resource leak warnings for all provider types.
* fix(common_daily_activity.py): initial commit with working mock BE endpoint for mcp usage
* feat(ui/): show mcp server activity on UI
allows admin to know which mcp's are being used
* feat(common_daily_activity.py): return activity by key
* feat(ui/): show top api keys for a given model / mcp server
allow user to know which key is driving spend
* fix(common_daily_activity.py): use known mcp server names
* feat(server.py): log the namespaced tool name (includes server prefix)
allow accurate cost tracking
* feat(db_spend_update_writer.py): log by mcp_namespaced_tool_name
store aggregate daily activity by mcp_namespaced_tool_name
Enables cost / usage tracking by mcp tool name
* fix(server.py): add key/user metadata to mcp calls
* refactor(common_daily_activity.py): update to return mcp activity in API
* fix(common_daily_activity.py): handle empty key
* fix(common_daily_activity.py): track when api key is empty
* test(test_spend_management_endpoints.py): update tests
* fix: fix ui linting error
* fix: fix linting errors
* test: add missing key
* build(schema.prisma): add mcp tool tracking
* fix(migration.sql): add schema migration file
* feat(server.py): add request logging for mcp calls
enables storing the mcp calls
* fix(new_usage.tsx): fix linting errors
* fix: fix code qa errors
* fix(activity_metrics.tsx): fix ui linting errors post-merge
* fix(types/utils.py): fix linting error
* fix(server.py): always have name
Fix the header to highlight the website guardrailsai.com which is the real name/website of this provider.
Guardrails.ai leads to some different website, which is confusing.
```bash
LiteLLM:ERROR: server.py:370 - Error handling MCP request: Task group is not initialized. Make sure to use run().
Traceback (most recent call last):
File "/usr/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/server.py", line 368, in handle_streamable_http_mcp
await session_manager.handle_request(scope, receive, send)
File "/usr/lib/python3.13/site-packages/mcp/server/streamable_http_manager.py", line 137, in handle_request
raise RuntimeError("Task group is not initialized. Make sure to use run().")
RuntimeError: Task group is not initialized. Make sure to use run().
```
The use of asyncio.create_task with an infinite loop is not ideal for integration with FastAPI's lifespan events. Now correctly handles this by manually invoking the __aenter__ and __aexit__ methods of the session managers' async context managers within the application's lifespan.
* Enhance chart readability with short-form notation for large numbers (#11571)
* format y-axis value for total tokens
* format y-axis for the rest of the charts on model-activity
* revert changes for requests per day
* labels modified to plain text
* added plain text label for api_requests and spend
* minor
* improve keyname display (#11573)
* fix tags overlap
- no need to fix decimal points for valueFormatterSpend() bcz we're only going to use it for values on the x and y axis of a chart
- show only top 5 values of tags
- format the values on the x-axis
* format spend values to short form
* format spend to short form
* showing top 5 tags data more perceptively
* remove mock data
* remove console.log
* prettier
* prettier + remove unused functions
* Add _transform_input_image_item_to_image_item() to handle image transformations
* Update content transformation to process input_image items from Responses API format to Chat Completions format
* fix(key_management_endpoints.py): add new `/key/service_account/generate` endpoint
enables controlling team member permissions for creating service account keys
* refactor(ui/): use new /key/service-account/generate endpoint when creating keys on UI
enables team member permissions for service accounts
* fix(permission_definitions.tsx): add permission for creating service accounts on UI
* fix(key_management_endpoints.py): ignore user id for service account keys
* fix(key_management_endpoints.py): improve docstring
* docs(service_accounts.md): update docs
* refactor(aim.py): refactor to support adding aim guardrails on UI
* fix(base.py): add ui_friendly_name to config model
* feat(ui/): support loading new guardrails from backend api call
removes need to onboard each guardrail to ui
* fix: don't show optional params if not set and don't show ui_friendly_name (internal param0
* fix(ui/add_guardrail_form.tsx): ensure dynamic provider value is used
* fix(ui/): just one-time update the provider map dictionary
* fix(ui/): show masked api base / api key on guardrail update
* refactor(aporia_ai/): refactor to show on UI
* feat(aporia_ai/): add aporia ai guardrail to UI
* refactor(guardrails_ai/): refactor to add via UI
* refactor(lasso.py): refactor to enable adding lasso guardrails via UI
* feat(pangea.py): add pangea guardrail on UI
* feat(panw): add panw prisma airs through UI
* test: update tests
* fix: fix ruff linting error
* test: update tests
* fix: add missing docs
* fix: fix guardrail init
* fix: suppress linting errors
* fix(proxy_server.py): fix linting error
Ishaan Jaff