Commit Graph
23283 Commits
Author SHA1 Message Date
Aaron VoglerandGitHub 2b2ba8a2b1 Add Bytez to the list of providers in the docs (#12588)
* Get the basics of the integration working.

* Cleanup bytez integration.

* Update user agent for Bytez integration.

* Use the config class directly. Create the start of the docs.

* Finish up bytez documentation. Include a provider integration guide.

* Fix typing bug in custom_logger_utils. Add tests for bytez integration.

* Add token tracking for model usage for Bytez integration.

* Create a units test for the Bytez config.

* Make changes to Bytez transformation code per PR feedback.

* Cleanup coment in Bytez transformation test.

* Remove LRU usage for bytez integration.

* Consolidate Bytez tests into a single file. Conform to project structure for tests.

* Fix linting error with Bytez impl.

* Add Bytez to the list of providers in the docs.
2025-07-14 14:17:56 -07:00
Krish DholakiaandGitHub 2290250c9f Litellm release notes 07 12 2025 (#12563)
* docs(index.md): add new models / updated models to new release note

* docs(index.md): add LLM api endpoints to release notes

* docs(index.md): refactor mcp into separate section '

* docs: doc improvements

* docs(index.md): more cleanup

* docs(index.md): more cleanup

* docs: update litellm enterprise trial link

* docs(index.md): cleanup

* docs(datadog.md): add separate datadog doc

* docs(index.md): cleanup

* docs(index.md): update docker version

* docs(index.md): add screenshots for new features

* docs(index.md): add doc links
2025-07-12 19:31:13 -07:00
Ishaan Jaff 08ce596c55 bump litellm enterprise pkg 2025-07-12 18:31:52 -07:00
Ishaan Jaff 2770e5b53e docs mcp fix 2025-07-12 17:38:43 -07:00
Ishaan Jaff 3e13f8dd0f openai.ConflictError 2025-07-12 17:07:21 -07:00
Ishaan Jaff 8bcdb07fcd ui new build v1.74.3.rc.1 2025-07-12 16:44:13 -07:00
Ishaan JaffandGitHub 50b106be11 [MCP Gateway] Ensure we use the same param for specifying groups (#12561)
* just use 1 param for mcp groups

* fix just use 1 param for access groups

* test_get_tools_from_mcp_servers

* docs access groups

* group MCPs

* test fix

* fix screenshots on docs

* TestMCPAccessGroupsE2E

* update img

* fix MCP connect
2025-07-12 16:41:58 -07:00
Ishaan JaffandGitHub 00ab400500 UI - v1.74.3-stable QA fixes (#12559)
* fix chat ui

* fixes for types

* ui - fix back button on MCP

* mcp - show existing groups when creating new mcps

* fix design

* fix color scheme

* fix linting

* fixes for mapped tests
2025-07-12 15:31:39 -07:00
Cole McIntoshandGitHub 7f5033d8a6 fix: handle missing 'env' attribute in MCP server Prisma model (#12560)
- Use getattr() with default value when accessing mcp_server.env
- Prevents AttributeError when Prisma doesn't include the field
- Fixes startup error: 'LiteLLM_MCPServerTable' object has no attribute 'env'

The issue occurs when Prisma fetches records and doesn't include
null/undefined fields in the returned object.
2025-07-12 15:23:12 -07:00
Jugal D. BhattandGitHub 65533ccf62 added mcp access group docs (#12557) 2025-07-12 14:49:52 -07:00
Eugene MyasyshchevandGitHub 23b04c350e Fixing issue to output github copilot verification uri immediately when running in docker. (#12558) 2025-07-12 14:48:58 -07:00
Krish DholakiaandGitHub d8ae044ec5 UI - Model Hub - refactor 'Make Public' to have a select + confirm form (#12555)
* fix(model_hub_table.tsx): fix null link

* fix(model_hub_table.tsx): add tooltip on disabled make public

tell user how to enable button

* feat(model_hub_table.tsx): make Model Hub OSS

allow more teams to share available models via litellm

* fix(model_hub_table_columns.tsx): make table view only model hub view, rename 'model' column to 'public model name', make model hub url copyable

* fix(page.tsx): fix logo re-rendering

* fix(public_model_hub.tsx): fix theme

* style(public_model_hub.tsx): move filters into same card as model table

filters are for table

* fix(page.tsx): fix ui linting error

* refactor(model_hub_table.tsx): refactor public model selection to inside the 'make public' button - modal

similar flow to adding guardrails

consistent flow for doing stuff on the proxy

* style(ui/): make 'make public' modal wider, to have all filters on same horizontal axis

* style(model_hub_table.tsx): have already public models be pre-selected

* fix(model_hub_table.tsx): ensure only admin can make models public via UI

* fix: fix linting errors

* fix(model_hub_table.tsx): make model hub purpose clearer

* fix(model_hub_table_columns.tsx): fix linting error
v1.74.3-stable-draft
2025-07-12 14:12:52 -07:00
Krish DholakiaandGitHub 1f5ab2ee74 UI - Model Hub Page - minor fixes + improvements (+ Make Model Hub OSS 🚀) (#12553)
* fix(model_hub_table.tsx): fix null link

* fix(model_hub_table.tsx): add tooltip on disabled make public

tell user how to enable button

* feat(model_hub_table.tsx): make Model Hub OSS

allow more teams to share available models via litellm

* fix(model_hub_table_columns.tsx): make table view only model hub view, rename 'model' column to 'public model name', make model hub url copyable

* fix(page.tsx): fix logo re-rendering

* fix(public_model_hub.tsx): fix theme

* style(public_model_hub.tsx): move filters into same card as model table

filters are for table

* fix(page.tsx): fix ui linting error
2025-07-12 14:10:03 -07:00
Jugal D. BhattandGitHub 5bd63c5a72 [MCP Gateway] access group fixes on UI for keys and teams (#12556)
* add changes to mcp servers in team and keys

* added access group in bracket

* added access group capitalise
2025-07-12 12:55:36 -07:00
Ishaan JaffandGitHub f0e87d4eb0 test test_redis_caching_multiple_namespaces (#12552) 2025-07-12 12:06:16 -07:00
Ishaan Jaff 57d9ca5f3a ci/cd remove flaky load testing scripts. We will do load testing on post releases 2025-07-12 11:17:54 -07:00
Ishaan Jaff a497f21345 test_anthropic_basic_completion_with_headers ci/cd fix 2025-07-12 11:08:03 -07:00
Ishaan Jaff cf5b4c0497 ruff check ./litellm --fix v1.74.3-nightly 2025-07-12 11:04:02 -07:00
Ishaan Jaff 8fe0007f3c ui new build 2025-07-12 11:01:20 -07:00
Ishaan Jaff 98da9c73dc fix bytez validate_environment 2025-07-12 10:57:18 -07:00
Ishaan Jaff cb4d3a88a1 ruff check ./litellm --fix 2025-07-12 10:53:56 -07:00
Aaron VoglerandGitHub 4c466ef157 Integration: Bytez as a model provider (#12121)
* Get the basics of the integration working.

* Cleanup bytez integration.

* Update user agent for Bytez integration.

* Use the config class directly. Create the start of the docs.

* Finish up bytez documentation. Include a provider integration guide.

* Fix typing bug in custom_logger_utils. Add tests for bytez integration.

* Add token tracking for model usage for Bytez integration.

* Create a units test for the Bytez config.

* Make changes to Bytez transformation code per PR feedback.

* Cleanup coment in Bytez transformation test.

* Remove LRU usage for bytez integration.

* Consolidate Bytez tests into a single file. Conform to project structure for tests.

* Fix linting error with Bytez impl.
2025-07-12 10:50:39 -07:00
tanjiroandGitHub 9f9534b537 Fix e2e test (#12549)
* user header removed

* fix test
2025-07-12 10:42:57 -07:00
Jugal D. BhattandGitHub 3a122bc48e [MCP Gateway] UI headers groups example on connect tab (#12550)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes

* fix test

* fixed double asterisks

* Litellm mcp groups UI (#12522)

* add ui for teams

* fix object permissions

* fix mcp servers test object permission

* remove print

* add helper method

* add tests + remove logs

* add mcp access group servers to test key

* add mcp access group support for headers

* lint fix

* add tests and helper  function

* fixed test

* change list -> List

* tests

* added ui for selecting servers
2025-07-12 10:42:43 -07:00
Jugal D. BhattandGitHub 9fe9e1cd6e [MCP Gateway] Allow mcp access groups on test key and tool calls (#12529)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes

* fix test

* fixed double asterisks

* Litellm mcp groups UI (#12522)

* add ui for teams

* fix object permissions

* fix mcp servers test object permission

* remove print

* add helper method

* add tests + remove logs

* add mcp access group servers to test key

* add mcp access group support for headers

* lint fix

* add tests and helper  function

* fixed test

* change list -> List

* tests
2025-07-12 10:36:42 -07:00
Krish DholakiaandGitHub d202ce229b Prevent writing default user setting updates to yaml (error in non-root env) + Use central team member budget when max_budget_in_team set on UI (#12533)
* fix(proxy_setting_endpoints.py): require store model in db is enabled for setting user default settings

* test(test_proxy_server.py): update test

* fix(reset_budget_job.py): initial commit adding reset budget logic for team members

* test: update unit testing

* test(test_proxy_budget_reset.py): validate team member budget was reset

* test(test_reset_budget_job.py): update unit tests

* test: update tests
2025-07-12 10:13:07 -07:00
Ishaan JaffandGitHub 6bc926e04b [Bug Fix] xai/ translation fix - ensure finish_reason includes tool calls when xai responses with tool calls (#12545)
* Helper to fix finish_reason for tool calls when XAI API returns empty string

* use llm http handler for Groq
2025-07-12 09:20:31 -07:00
Eliza HuangandGitHub 858c756d47 chore: Update Vertex AI Model Garden LiteLLM integration tutorial (#12428)
* Update Vertex Model Garden documentation to deploy + use OpenAI Chat Completion tutorial.

* update anchor description

* update to LiteLLM specific tutorial

* update the right doc

* Reverted docs/my-website/docs/providers/vertex.md to match main

* revert to main vertex tutorial
2025-07-12 09:10:41 -07:00
tanjiroandGitHub e3b2fd8c8d remove margin bottom css (#12538) 2025-07-12 09:08:09 -07:00
tanjiroandGitHub b4fbdeeaaa user header removed (#12544) 2025-07-12 08:51:49 -07:00
tanjiroandGitHub 2faba5f215 Consistent layout for Create and Back buttons on all the pages (#12542)
* create new model button

* cleanup + prettier

* prettier

* create new model button added for model and model info+ back button added

* prettier

* team and team info has create team button + back to teams button added in team_info

* prettier

* better prettier

* always show create new organization button and add back to organizations button

* minor, prettier

* prettier

* remove user header and prettier

* move top buttons to the left

* prettier

* back to all logs button style fixed

* prettier

* modify button text

* create new guardrail button style

* prettier

* button modifications

* guardrails page done

* prettier

* add new guardrail text button + prettier on mcp

* button related changes

* prettier

* mcp servers page done
2025-07-12 08:18:02 -07:00
Krish DholakiaandGitHub 749051105b Team Members - reset budget, if duration set + Prometheus - support tag based metrics (#12534)
* fix(internal_user_endpoints.py): initial commit removing logic to create new budget for new user if default max budget in team set

* feat(proxy_setting_endpoints.py): update team member budget when set via default internal user endpoint

removes need to create a unique budget per user

* feat(proxy_server.py): set team max member budget on startup, if set on config.yaml

* fix(prometheus.py): support custom tags for tracking on prometheus

Allows tracking user agent values on prometheus metrics

* test(test_internal_user_endpoints.py): fix test
2025-07-11 22:54:16 -07:00
Ishaan Jaff b83fbe6896 fix code qa check 2025-07-11 22:35:27 -07:00
Ishaan Jaff 12b16c1f0c bump: version 1.74.2 → 1.74.3 2025-07-11 22:26:26 -07:00
Krrish Dholakia 8b4673fe84 build(ui/): UI - Public Model Hub v2 (#12532)
Closes https://github.com/BerriAI/litellm/pull/12532
2025-07-11 22:24:31 -07:00
William EastonandGitHub 75d68a47d0 remove deprecated pydantic class Config (#12528) 2025-07-11 22:21:02 -07:00
Ishaan Jaff a1a979fae1 add back MCPConnectionStatus 2025-07-11 22:20:43 -07:00
Ema SurianoandGitHub 7b40d6adfe docs: Update github.md (#12509)
Fix wrong model names
2025-07-11 22:19:21 -07:00
Ishaan Jaff dbe07adf29 bump litellm proxy extras 2025-07-11 22:09:19 -07:00
Ishaan Jaff 7eb1a68854 fix test_qdrant_semantic_cache_acompletion 2025-07-11 22:07:19 -07:00
Ishaan Jaff 14f6d039dc add new migration 2025-07-11 22:07:09 -07:00
Ishaan Jaff e64bd66b1c ruff fix 2025-07-11 21:59:10 -07:00
Ishaan Jaff 1c91e22942 Groq - ensure service tier is OpenAI compatible 2025-07-11 21:55:05 -07:00
Ishaan Jaff bccbe921e4 build - sync all schema.prisma files 2025-07-11 20:25:26 -07:00
Ishaan Jaff 8e06a42c63 new ui build 2025-07-11 20:23:58 -07:00
Ishaan JaffandGitHub d931446a79 [MCP Gateway] Allow using stdio MCPs with LiteLLM (#12530)
* update types

* add STDIO to client

* allow using STDIO with MCP manager

* add Stdio-specific fields to schema.prisma

* fixes for MCP mgmt

* fix for adding stdio MCP to DB

* ui - allow adding stdio MCPs

* fix MCP server manager

* docs stdio MCP

* add_stdio_mcp.png

* new stdio tests

* allow adding MCPs through config.yaml

* fix tool test panel

* use TestMCPClient

* ui fixes for testing circle ci mcp
2025-07-11 20:21:02 -07:00
Jugal D. BhattandGitHub db57e765a5 [MCP Gateway] UI Quality check fixes (#12521)
* Add tooltip to mcp server access groups

* fix ui issues
2025-07-11 18:26:43 -07:00
Jugal D. BhattandGitHub 38c9b69e8f [MCP Gateway] access group UI object permission fix (#12523)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes

* fix test

* fixed double asterisks

* Litellm mcp groups UI (#12522)

* add ui for teams

* fix object permissions

* fix mcp servers test object permission

* remove print

* add helper method

* add tests + remove logs
2025-07-11 18:25:40 -07:00
Ishaan JaffandGitHub 4f9793951d [Bug Fix] - QA for MCP Gateway - show the cost config on the root of MCP Settings (#12526)
* show cost display

* fixes MCP cost display
2025-07-11 17:34:34 -07:00
Ishaan Jaff 7ac187b269 test_pre_call_hook_team_rpm_limits 2025-07-11 15:52:21 -07:00