Commit Graph

3875 Commits

Author SHA1 Message Date
Aaron Vogler 2b2ba8a2b1 Add Bytez to the list of providers in the docs (#12588)
* Get the basics of the integration working.

* Cleanup bytez integration.

* Update user agent for Bytez integration.

* Use the config class directly. Create the start of the docs.

* Finish up bytez documentation. Include a provider integration guide.

* Fix typing bug in custom_logger_utils. Add tests for bytez integration.

* Add token tracking for model usage for Bytez integration.

* Create a units test for the Bytez config.

* Make changes to Bytez transformation code per PR feedback.

* Cleanup coment in Bytez transformation test.

* Remove LRU usage for bytez integration.

* Consolidate Bytez tests into a single file. Conform to project structure for tests.

* Fix linting error with Bytez impl.

* Add Bytez to the list of providers in the docs.
2025-07-14 14:17:56 -07:00
Krish Dholakia 2290250c9f Litellm release notes 07 12 2025 (#12563)
* docs(index.md): add new models / updated models to new release note

* docs(index.md): add LLM api endpoints to release notes

* docs(index.md): refactor mcp into separate section '

* docs: doc improvements

* docs(index.md): more cleanup

* docs(index.md): more cleanup

* docs: update litellm enterprise trial link

* docs(index.md): cleanup

* docs(datadog.md): add separate datadog doc

* docs(index.md): cleanup

* docs(index.md): update docker version

* docs(index.md): add screenshots for new features

* docs(index.md): add doc links
2025-07-12 19:31:13 -07:00
Ishaan Jaff 2770e5b53e docs mcp fix 2025-07-12 17:38:43 -07:00
Ishaan Jaff 50b106be11 [MCP Gateway] Ensure we use the same param for specifying groups (#12561)
* just use 1 param for mcp groups

* fix just use 1 param for access groups

* test_get_tools_from_mcp_servers

* docs access groups

* group MCPs

* test fix

* fix screenshots on docs

* TestMCPAccessGroupsE2E

* update img

* fix MCP connect
2025-07-12 16:41:58 -07:00
Jugal D. Bhatt 65533ccf62 added mcp access group docs (#12557) 2025-07-12 14:49:52 -07:00
Aaron Vogler 4c466ef157 Integration: Bytez as a model provider (#12121)
* Get the basics of the integration working.

* Cleanup bytez integration.

* Update user agent for Bytez integration.

* Use the config class directly. Create the start of the docs.

* Finish up bytez documentation. Include a provider integration guide.

* Fix typing bug in custom_logger_utils. Add tests for bytez integration.

* Add token tracking for model usage for Bytez integration.

* Create a units test for the Bytez config.

* Make changes to Bytez transformation code per PR feedback.

* Cleanup coment in Bytez transformation test.

* Remove LRU usage for bytez integration.

* Consolidate Bytez tests into a single file. Conform to project structure for tests.

* Fix linting error with Bytez impl.
2025-07-12 10:50:39 -07:00
Eliza Huang 858c756d47 chore: Update Vertex AI Model Garden LiteLLM integration tutorial (#12428)
* Update Vertex Model Garden documentation to deploy + use OpenAI Chat Completion tutorial.

* update anchor description

* update to LiteLLM specific tutorial

* update the right doc

* Reverted docs/my-website/docs/providers/vertex.md to match main

* revert to main vertex tutorial
2025-07-12 09:10:41 -07:00
Krish Dholakia 749051105b Team Members - reset budget, if duration set + Prometheus - support tag based metrics (#12534)
* fix(internal_user_endpoints.py): initial commit removing logic to create new budget for new user if default max budget in team set

* feat(proxy_setting_endpoints.py): update team member budget when set via default internal user endpoint

removes need to create a unique budget per user

* feat(proxy_server.py): set team max member budget on startup, if set on config.yaml

* fix(prometheus.py): support custom tags for tracking on prometheus

Allows tracking user agent values on prometheus metrics

* test(test_internal_user_endpoints.py): fix test
2025-07-11 22:54:16 -07:00
Ema Suriano 7b40d6adfe docs: Update github.md (#12509)
Fix wrong model names
2025-07-11 22:19:21 -07:00
Ishaan Jaff d931446a79 [MCP Gateway] Allow using stdio MCPs with LiteLLM (#12530)
* update types

* add STDIO to client

* allow using STDIO with MCP manager

* add Stdio-specific fields to schema.prisma

* fixes for MCP mgmt

* fix for adding stdio MCP to DB

* ui - allow adding stdio MCPs

* fix MCP server manager

* docs stdio MCP

* add_stdio_mcp.png

* new stdio tests

* allow adding MCPs through config.yaml

* fix tool test panel

* use TestMCPClient

* ui fixes for testing circle ci mcp
2025-07-11 20:21:02 -07:00
Ishaan Jaff cb96f1d740 [Feat] - New guardrail - OpenAI Moderations API (#12519)
* add OpenAIModerationGuardrailConfigModel

* add OpenAIGuardrailBase

* _extract_response_text

* fixes for openai moderation guard

* docs OpenAI Moderation

* add OpenAIModerationGuardrailConfigModel

* use literal for model

* fix init openai moderation

* test_openai_moderation_guardrail_adds_to_litellm_callbacks

* show default on on UI

* fix code qa
2025-07-11 11:25:45 -07:00
Krrish Dholakia 68fe7d245d docs(index.md): update doc 2025-07-11 08:08:47 -07:00
Krish Dholakia 569ddde9fe [Enterprise] Support tag based mode for guardrails (#12508)
* feat(custom_guardrail.py): allow admin to run guardrails on specific modes based on request tags

Enables pre-call checks for openwebui but logging for claude code guardrails

* refactor(custom_guardrail.py): move tag based mode to enterprise

* test(test_custom_guardrail.py): update test

* feat(quick_start.md): add doc on tag-based guardrail mode

* docs(quick_start.md): cleanup doc

* docs: cleanup

* docs(openweb_ui.md): add tutorial to doc

* fix(custom_guardrail.py): ensure user is premium user

* test: update test
2025-07-10 22:27:30 -07:00
minghao bee63a1ae7 Added dashscope (alibaba's cloud - qwen) as a provider (#12361)
* Added dashscope as a provider

* Fix some leftover references on nebius

* Porting the dashscope api endpoit international version

* explicit tool_choice = True in config
2025-07-10 18:09:26 -07:00
Rayshard Thompson 8aa616698e Fix typo and import required types (#12487) 2025-07-10 18:03:42 -07:00
Ishaan Jaff b0003bd03c Feat(bedrock): support api key authentication for AWS Bedrock API (#12426) (#12495)
* add support of bearer token for bedrock integration

* fix linting issue

* fix type checking issue

* reoder arguments to address type checking issue

* switch to use get_secret_str to fetch env variable

Co-authored-by: 0x-fang <fanggong@amazon.com>
2025-07-10 15:12:17 -07:00
Ishaan Jaff 610d56ae5d docs stable release 2025-07-10 14:53:29 -07:00
Krish Dholakia 07e8609edb Resolve model group alias on Auth + /v1/messages Fallback support (#12440)
* fix(auth_checks.py): resolve a model group alias when key has access to underlying model

Fixes LIT-293

* feat(anthropic/): add mock_response to anthropic /v1/messages

makes it easy to test fallback logic

* fix(router.py): support fallbacks on /v1/messages

adds working fallbacks on generic api route

* refactor(router.py): point _ageneric_api_call_with_fallbacks to updated function

* test: add unit test for new helper on router

* fix(router.py): use correct metadata variable name

* fix(router.py): use correct metadata field

* docs(config_settings.md): document new param
2025-07-09 22:27:55 -07:00
Krish Dholakia 635367b020 Litellm dev 07 09 2025 p1 (#12462)
* fix(db_spend_update_writer.py): fix db query

* fix(litellm_pre_call_utils.py): support passing anthropic-beta headers when 'forward_client_headers_to_llm_api' is True

allows user to pass along extra headers to vertex ai anthropic models

* docs(config_settings.md): update docs
2025-07-09 21:46:15 -07:00
Ishaan Jaff 35bfa7c9ee [MCP Gateway] - Allow using custom post call MCP hook for cost tracking (#12469)
* add async_post_mcp_tool_call_hook to custom logger

* instrument call_mcp_tool

* add async_post_mcp_tool_call_hook

* fixes for MCP tests

* test fix MCP tests

* allow adding custom cost for MCP response

* fix LiteLLM_MCPTextContent

* add MCPPostCallResponseObject

* fix _handle_local_mcp_tool

* async_post_mcp_tool_call_hook

* fix typing loc

* fix call_mcp_tool

* fix server.py

* fix MCPPostCallResponseObject

* async_post_mcp_tool_call_hook

* docs mcp cost hook

* fix import path
2025-07-09 18:41:08 -07:00
Ishaan Jaff 17ea770ab5 [Feat] SSO - Allow users to run a custom sso login handler (#12465)
* fix return_ui_sso_redirect_response_after_sign_in

* add CustomSSOLoginHandler

* add handle_custom_ui_sso_sign_in

* fix handle_custom_ui_sso_sign_in

* fixes for /sso/key/generate

* add test_handle_custom_ui_sso_sign_in_success

* fixed for loading config

* add example custom handler

* test_custom_ui_sso_sign_in_handler_config_loading

* docs - custom sso

* move to enterprise folder

* EnterpriseCustomSSOHandler

* handle_custom_ui_sso_sign_in

* docs custom sso

* docs sso

* docs custom sso

* get_redirect_response_from_openid

* fix mock tests
2025-07-09 16:36:30 -07:00
Ishaan Jaff afd382d09f [Feat] MCP Gateway - Allow customizing what client side header to use (#12460)
* add _get_mcp_auth_header_from_headers

* test_process_mcp_request_with_custom_auth_header

* Using a different Authentication Header

* fix customize MCP Auth header name
2025-07-09 12:30:20 -07:00
Ishaan Jaff ce2934349f [Feat] Add XInference Image Generation API Provider (#12439)
* add XInferenceImageGenerationConfig

* add get_xinference_image_generation_config

* test_xinference_image_generation

* docs Image Generation xinference

* docs inference

* docs xinference

* fix xinference img gen
2025-07-08 21:17:38 -07:00
Dmitriy Alergant f7d5958dca Update guardrails_ai.md (#12356)
Fix the header to highlight the website guardrailsai.com which is the real name/website of this provider.

Guardrails.ai leads to some different website, which is confusing.
2025-07-08 16:23:42 -07:00
Krrish Dholakia 9fa9d547c0 docs(index.md): document known issues 2025-07-08 10:39:51 -07:00
Ishaan Jaff 08f3b06d82 [Feat] Bump langfuse python SDK version and LANGFUSE_TRACING_ENVIRONMENT (#12376)
* bump langfuse to 2.59.7

* _set_langfuse_specific_attributes

* fix files

* test_set_langfuse_environment_attribute
2025-07-07 15:53:43 -07:00
Krrish Dholakia 88a7b40f86 docs(index.md): update note with all guardrails now on UI 2025-07-06 21:55:06 -07:00
Krrish Dholakia d50a28d424 docs(index.md): add to release note 2025-07-06 21:54:02 -07:00
Krrish Dholakia 4d718f97ef docs(azure_content_guardrails.md): add doc on azure content guardrails 2025-07-06 21:42:58 -07:00
Krrish Dholakia 55ecb543f2 docs(azure_content_guardrails.md): add azure content guardrails doc 2025-07-06 21:39:03 -07:00
Ishaan Jaff 10c704ccb9 docs release notes 2025-07-05 21:24:09 -07:00
Krish Dholakia dbe37b542d Litellm dev 07 05 2025 p4 (#12350)
* fix(key_management_endpoints.py): add new `/key/service_account/generate` endpoint

enables controlling team member permissions for creating service account keys

* refactor(ui/): use new /key/service-account/generate endpoint when creating keys on UI

enables team member permissions for service accounts

* fix(permission_definitions.tsx): add permission for creating service accounts on UI

* fix(key_management_endpoints.py): ignore user id for service account keys

* fix(key_management_endpoints.py): improve docstring

* docs(service_accounts.md): update docs
2025-07-05 19:38:42 -07:00
Krish Dholakia bda9eecd45 Litellm dev 07 05 2025 p3 (#12349)
* refactor(aim.py): refactor to support adding aim guardrails on UI

* fix(base.py): add ui_friendly_name to config model

* feat(ui/): support loading new guardrails from backend api call

removes need to onboard each guardrail to ui

* fix: don't show optional params if not set and don't show ui_friendly_name (internal param0

* fix(ui/add_guardrail_form.tsx): ensure dynamic provider value is used

* fix(ui/): just one-time update the provider map dictionary

* fix(ui/): show masked api base / api key on guardrail update

* refactor(aporia_ai/): refactor to show on UI

* feat(aporia_ai/): add aporia ai guardrail to UI

* refactor(guardrails_ai/): refactor to add via UI

* refactor(lasso.py): refactor to enable adding lasso guardrails via UI

* feat(pangea.py): add pangea guardrail on UI

* feat(panw): add panw prisma airs through UI

* test: update tests

* fix: fix ruff linting error

* test: update tests

* fix: add missing docs

* fix: fix guardrail init

* fix: suppress linting errors

* fix(proxy_server.py): fix linting error
2025-07-05 18:44:00 -07:00
Ishaan Jaff 54eb00ddfb docs team / Key Based Logging on UI 2025-07-05 18:26:51 -07:00
Ishaan Jaff eabe0c5e7f docs - team based logging 2025-07-05 17:55:18 -07:00
Ishaan Jaff 2fc6896b0a docs refactor 2025-07-05 16:40:37 -07:00
Ishaan Jaff 8637142fd5 new img 2025-07-05 16:27:46 -07:00
Ishaan Jaff 303c4bd628 docs - 1.74.0.rc (#12347)
* doc updates

* doc fixes

* doc fixes

* docs fix

* doc fix

* doc clean up

* doc fixes

* cleanup
2025-07-05 13:17:51 -07:00
Ishaan Jaff 912e5084e5 docs fix 2025-07-05 11:25:31 -07:00
Ishaan Jaff a3c88c5207 docs add rerank 2025-07-05 11:19:24 -07:00
Ishaan Jaff 6ce8f5692e [Docs] vertex deepseek (#12345)
* docs vertex

* Vertex AI - Anthropic (Claude)
2025-07-05 11:14:41 -07:00
Ishaan Jaff 9991a813e2 doc fix 2025-07-05 10:54:00 -07:00
Ishaan Jaff 92a3f263d4 doc update 2025-07-05 10:53:22 -07:00
Ishaan Jaff 0e62803c0e v1.74.0.rc docs (#12344)
* cleanup

* fixes

* doc fix

* doc fix

* fixes release note

* docs

* docs

* v1.74.0
2025-07-05 10:44:38 -07:00
Ishaan Jaff eafad0ecc5 fix docker run cmd 2025-07-05 10:17:03 -07:00
Ishaan Jaff 738db9336e [Feat] JWT - Sync user roles and team memberships when JWT Auth is used (#11994)
* add JWTLiteLLMRoleMap

* test_sync_user_role_and_teams

* add sync_user_role_and_teams

* test_sync_user_role_and_teams

* fix types

* Sync User Roles and Teams with IDP

* Add test for JWT role mapping to LiteLLM roles
2025-07-05 08:58:34 -07:00
Ishaan Jaff 98db2d9fb9 add aiml 2025-07-05 07:56:51 -07:00
Jugal D. Bhatt 529f67f544 Add mcp server segregation comma separated support (#12326)
* add mcp comma separated support

* dont support legacy json array

* lint
2025-07-05 00:08:43 -07:00
Jugal D. Bhatt b2464a2b33 ssl_cert_error (#12327) 2025-07-04 13:21:29 -07:00
Ishaan Jaff 9da97eb939 add new env vars to docs 2025-07-04 13:13:34 -07:00