Commit Graph
80 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub a6527e5010 [Feat] Add litellm-proxy cli login for starting to use litellm proxy (#12216)
* add handlers for auth commands

* add login, logout, whoami

* refactor auth

* add CLI Authentication Flow

* add SSO sign in constants

* add itellm-session-token

* fixes for managing state with cli

* use locally stored context for cli session

* add litellm banner + interactive shell

* update main.py

* update auth to show commands

* fix ui sso render

* add TestCLISSOCallbackFunction

* update banner.py

* remove file

* fix cli sso success

* TestTokenUtilities

* fix code qa

* fix execute_command

* fix cli_sso_callback

* fix import

* Authentication using CLI
2025-07-01 18:11:19 -07:00
Jason RobertsandGitHub cc480f94c9 Fix/move panw prisma airs test file location per feedback on PR #12116 (#12175)
* Move PANW Prisma AIRS test per feedback on PR #12116

- Move test to tests/test_litellm/proxy/guardrails/guardrail_hooks/

* Remove test file from old location
2025-07-01 18:08:04 -07:00
Cole McIntoshandGitHub abe364c2bc Fix: Ensure exception is not None before checking its string representation (#12209)
The test_keys_delete_error_handling test was failing with:
- ConnectionError when the mock wasn't properly applied
- The test was checking str(result.exception) without first verifying exception exists

This fix adds an explicit check that result.exception is not None before
attempting to convert it to string, preventing potential AttributeError
and making the test more robust.
2025-07-01 12:30:31 -07:00
Krish DholakiaandGitHub ee9dd158dd Fix - handle empty config.yaml + Fix gemini /models - replace models/ as expected, instead of using 'strip' (#12189)
* fix(proxy_server.py): handle empty config yaml

Fixes https://github.com/BerriAI/litellm/issues/12163

* fix(gemini/common_utils.py): replace models/ as expected, instead of using 'strip'

Fixes https://github.com/BerriAI/litellm/issues/12160

* fix(anthropic/experimental_pass_through/messages/transformation.py): check for env var when selecting api key

* docs(config_settings.md): add api key to docs
2025-06-30 21:56:03 -07:00
Krish DholakiaandGitHub cec7e4941a UI QA Fixes - prevent team model reset on model add + return team-only models on /v2/model/info + render team member budget correctly (#12144)
* fix(team_endpoints.py): prevent overwriting current list of team models on new model add

* fix(networking.tsx): fix default proxy base url

* fix(proxy_server.py): include team only models when retrieving all deployments on `/v2/model/info` helper util

ensures team only models are shown to user

* fix(router.py): check model name by team public model name when team id given

Fixes issue where team member could not see team only models when clicking into that team on `Models + Endpoints`

* fix(team_member_view.tsx): fix rendering team member budget, when budget is set

* test: update tests

* test: update unit test
2025-06-28 22:04:49 -07:00
Cole McIntoshandGitHub 6578133bb7 Fix user-team association issues in LiteLLM proxy (#12082)
* Fix user-team association issues in LiteLLM proxy

- Update list_team function to properly filter teams using user's teams array instead of only checking members_with_roles field
- Add Field descriptions and docstring to TeamMemberAddRequest and related models for better Swagger/OpenAPI documentation
- Maintain backward compatibility with fallback to members_with_roles if user lookup fails

This ensures users created with teams parameter appear correctly in team views and improves API documentation.

* Fix duplicate member checking in team_member_add endpoint

- Enhanced team_member_add_duplication_check to check both user_id and user_email
- Added additional duplicate prevention logic after user creation/lookup
- Fixed issue where users added by email could be duplicated in teams
- Added logging for debugging duplicate detection

This addresses the bug where adding the same user by email multiple times would create duplicate entries in the team's members_with_roles array.

* Improve duplicate member prevention in team_member_add

- Enhanced early duplicate check to handle both user_id and user_email
- Added late-stage duplicate prevention after user lookup/creation
- Fixed issue where users could be added multiple times by email
- Cleaned up debug logging

Note: There's still an edge case where the duplicate prevention may not work
correctly in all scenarios. This needs further investigation and testing.

* Refactor team_member_add endpoint for improved member management

- Split team_member_add functionality into smaller, dedicated functions for permission validation, member processing, and team member list updates.
- Enhanced permission checks to ensure only authorized users can add members.
- Streamlined member addition logic to reduce redundancy and improve readability.
- Maintained existing functionality while improving code structure and maintainability.

* Add tests for team_member_add helper functions

- Add test for _validate_team_member_add_permissions with admin user
- Add test for _validate_team_member_add_permissions with non-admin user
- Add test for _process_team_members with single member
- Add test for _process_team_members with multiple members
- Add test for _update_team_members_list with new member
- Add test for _update_team_members_list duplicate prevention

These tests ensure the refactored helper functions work correctly
after fixing the PLR0915 linting error.
2025-06-27 21:29:49 -07:00
Krish DholakiaandGitHub e8d7537b57 Raise clearer error on Anthropic Unified route + allow setting new_key on /key/regenerate
* fix(llm_http_handler.py): raise clearer error on anthropic unified route

Fixes https://github.com/BerriAI/litellm/issues/12063

* fix(key_management_endpoints.py): add new param `new_key` for setting the regenerated key value

user request

* test: add unit tests

* fix(pass_through_endpoints.py): use data instead of json for passthrough requests

fixes bedrock latency issue

* Revert "fix(pass_through_endpoints.py): use data instead of json for passthrough requests"

This reverts commit 021dfd9165f837d37e6aad247ccbd0d2e8ca6043.
2025-06-26 21:56:03 -07:00
Cole McIntosh e4bc213bfe fix(proxy): Add managed_files hook to test_mock_create_audio_file
The test was failing because the managed_files hook was not configured.
This fix adds a DummyManagedFiles class and mocks the create_file
function as an AsyncMock to ensure proper test execution.
2025-06-26 10:42:25 -06:00
Krish DholakiaandGitHub e2f6fb2d7c Managed Files + Batches - filter deployments to only those where file was written + save all model file id mappings in DB (prev just 1st one) (#12048)
* test(test_router.py): initial unit test confirming router.afile_content uses dynamic api key / api base

* fix(managed_files.py): filter deployments for only those within file id mapping

ensure call works - only route to models where the file was written

* fix(proxy_server.py): fix loading in model ids from config, if config id is int

* fix(router.py): return all model file id mappings on create_file

if multiple deployments - this ensures all the file id mappings are bubbled up

Fixes issue when trying to use loadbalanced deployments - only 1 file id mapping was being stored
2025-06-25 21:27:06 -07:00
Ishaan Jaff de86246e14 test fixes 2025-06-25 18:42:06 -07:00
Krish DholakiaandGitHub 1a4ad8bf18 Update mistral 'supports_response_schema' field + Fix ollama embedding (#12024)
* build(model_prices_and_context_window.json): update all mistral models (besides codestral-mamba) to indicate support for response schema

Closes https://github.com/BerriAI/litellm/issues/12012

* fix(route_llm_request.py): if llm router is not initialized, go straight through to litellm sdk

Fixes https://github.com/BerriAI/litellm/issues/12008

* test: add unit test

* fix(ollama_embeddings): fix unecessary await

Fixes https://github.com/BerriAI/litellm/issues/11997

* test: update ollama embedding tests
2025-06-25 07:20:13 -07:00
Krish DholakiaandGitHub ac15ca3014 Teams - Support default key expiry + UI - support enforcing access for members of specific SSO Group (#12023)
* fix(team_endpoints.py): support setting default key expiry

allows admin to set key expiry on all team member keys

makes it easier to setup default team for experimentation

* feat(key_management_endpoints.py): allows admin to set duration for keys created by team members

* feat(team_endpoints.py): support team_member_key_duration on `/team/update`

allows setting max time team member keys are valid for

* fix(team_info.tsx): ui component to update team member key duration

* fix(team_info.tsx): support updating team member key duration, if set

* feat(teams.tsx): add team member key duration param ui component

allow admin to set this on UI

* feat(ui_sso.py): support restricting ui access by sso group

allows controlling who can/can't access the UI

* feat(ssomodals.tsx): add initial commit adding sso group access to admin ui

* feat(proxy_server.py): support reading + writing ui_access_mode from db

allows admin to configure allowed sso groups from UI

* feat(ui_sso.py): support enforcing all teams on sso jwt handler

if ui access mode set via ui, support reading the value and enforcing it

* feat(ui/): ui component for controlling sso access group

allow admin to only allow users within specific sso group to log into UI

* fix(uiaccesscontrolform.tsx): fix field names

* feat(ui_sso.py): return received sso response in the clientside error message - enables easier debugging

* test: add unit tests

* fix: minor fixes
2025-06-24 21:58:07 -07:00
Ishaan JaffandGitHub b9feb43dac [Bug Fix] SCIM - Ensure new user roles are applied (#12015)
* SCIM fix new user roles

* test_create_user_defaults_to_viewer

* test_create_user_uses_default_internal_user_params_role

* fix default user for SCIM

* fix linting error
2025-06-24 13:22:56 -07:00
Ishaan JaffandGitHub b5c48c8c22 [Feat] Add List Callbacks API Endpoint (#11987)
* add get_callbacks_by_type

* add list_callbacks

* fix _get_callback_string

* add callback_management_endpoints_router

* fix proxy config.yaml

* fixes list callbacks

* TestCallbackManagementEndpoints

* update docs

* docs Response Fields

* docs header format

* docs Dynamic Callback Management
2025-06-23 15:34:25 -07:00
Krish DholakiaandGitHub 7f3d8bb9c5 Management Fixes - don't apply default internal user settings to admins + preserve all model access for teams with empty model list, when team model added + /v2/model/info fixes (#11957)
* fix(internal_user_endpoints.py): don't apply default internal user params if role is admin

prevent internal user restrictions from being applied to admin

* fix(proxy_server.py): fix model info v2 endpoint check - handle user_id being none

* fix(team_endpoints.py): ensure team doesn't lose all model access if set as empty string and new team model added

* fix(proxy_server.py): ensure model with team id is only added as valid for team which has that id
2025-06-21 20:33:10 -07:00
Ishaan Jaff 7006cd4139 test ui sso 2025-06-21 15:58:48 -07:00
Ishaan Jaff 6cd2fe4879 test_team_endpoints.py 2025-06-21 15:57:58 -07:00
Krish DholakiaandGitHub 2404d3fb1b Show user all models they can call (Across teams) on UI (#11948)
* fix(proxy_cli.py): check for module not found error on proxy import

Fixes https://github.com/BerriAI/litellm/issues/11836

* feat(proxy_server.py): utility function to get all models across all teams user is in

Allows user to see all team models on UI

* feat(proxy_server.py): return models accessible via team id's in `/v2/model/info`

Allows UI to tell user which team they can use to access model

* feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub

allows user to know what teams they can access a model through

* Revert "feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub"

This reverts commit f844c79383ec6739ed712f59e33a524a26b3d35a.

* fix(proxy_server.py): backend model info endpoint improvements

* UI Improvements for Default User access (#11952)

* feat(ui/): add a 'current team' and 'view' filters to the models page

allow user to see what all models they have access to within a specific team

* feat: working ui for seeing models in teams

* fix(model_dashboard.tsx): make current team filter more prominent

* style(model_dashboard.tsx): add a helpful note telling user how to create a model for the team they've selected

* style(model_dashboard.tsx): only show helpful note when current view is team, not for global

* fix(team_dropdown.tsx): allow searching by team id on create key modal

* feat(create_key_button.tsx): add helpful message when team selection is required

* fix: fix linting checks

* fix: fix ui linting error

* docs(team_endpoints.py): document new param
2025-06-21 15:29:25 -07:00
Ishaan Jaff 1bfe6a1110 test_get_user_groups_pagination 2025-06-21 14:24:04 -07:00
Ishaan Jaff 18c93d99cd test ui sso 2025-06-21 14:17:57 -07:00
Krrish Dholakia 1e6d43e761 Squashed commit of the following:
commit 440bc027251d8180174d762d83d271d0f7b68cc5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 23:04:11 2025 -0700

    fix: fix check

commit 89a7451cb9ee26ff9f642335714dcc6f449d1fc2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 22:42:30 2025 -0700

    fix: fix test

commit 1322e3b3497e5d334fdcaa18f0cf7a98ea758df4
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 20:52:40 2025 -0700

    style: add more tooltips

commit 172738b98b7864aabcacf3334a394098b300283f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 20:51:09 2025 -0700

    feat(team_member_view.tsx): add a tooltip

commit 895eb28deb9127985e30b5e859e5bca8530951c9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:46:49 2025 -0700

    fix(teams.tsx): support setting team member budget on create

commit 003cc54a6dd0f65030c4f39a8487adc771b62e11
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:40:49 2025 -0700

    fix(team_member_view.tsx): style improvements

commit a627a044f21df788f80d92a4081212072be91632
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:40:01 2025 -0700

    fix(team_member_view.tsx): handle scientific notation in string

commit c5a3b7bd8419f6394e1b490849555d02d473baed
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:34:25 2025 -0700

    feat(team_membership_view.tsx): show team member spend + max budget on UI

commit e986d12ad5b07c676f4cac5e16745939d7473dee
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:28:06 2025 -0700

    feat(team_member_view.tsx): show team member spend + budget on team info

commit 8e398607b25f8a8f0bab41964810b5dd27c5e3f2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:18:16 2025 -0700

    feat(team_info.tsx): show team member budget on team info

commit 1f56886b5913dafefc0c00fbe741c0c9c01144a6
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:15:30 2025 -0700

    feat(team_endpoints.py): get team budget table on team info

    allows user to see max budget set for team members

commit 0a4320bbfa406c24ad32a420f82152da7bdd7323
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 18:10:06 2025 -0700

    feat(team_endpoints.py): return team member budget on team info

    allows ui to display this to admin / team member

commit 6a4e29f87b333ae9977e8f878960e63becd89150
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 17:57:20 2025 -0700

    fix(team_endpoints.py): support updating team budget on UI

commit 53f0fff34032977433dfe6935ce0a684a4141fd8
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 17:38:17 2025 -0700

    feat(proxy/_types.py): return team member spend

    update pydantic object to include spend

    Allows showing spend of team member within team on UI

commit ef2a1a43ecf7fecfb904042cbf47b3d56246edcb
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 16:31:42 2025 -0700

    feat(team_endpoints.py): support 'team_member_budget' param on `/team/update`

    enables budget working across all team members

commit 512999f1249b00a02a30f049a0cfa36e829ff989
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 16:20:04 2025 -0700

    test: add unit tests for default team member budget

commit 90fa3f61a2d63e12b9f3e1da9775f5c8b7294b5f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 15:37:51 2025 -0700

    feat(team_endpoints.py): support using default team member budget id, if set

    allows all team members to use the same budget id

commit acef5324b1a0935a482c71060f610c3d8823e8c3
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 15:22:30 2025 -0700

    feat(team_endpoints.py): support `team_member_budget` param on `/team/new`

    Allow creating 1 budget for all users within team (makes it easier to increase/reduce budget if needed for all team members)

commit 2e867ac70fbd8768e7c27cf3b078e6dc10e566b9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Fri Jun 20 13:45:06 2025 -0700

    fix(ui_sso.py): ensure user is added to team, if set via default internal settings

    allows users signed up via SSO to be added to default team
2025-06-20 23:11:53 -07:00
Ishaan JaffandGitHub dc2c1122bf [Feat] UI QA: Pass through endpoints (#11939)
* use ID for pass through management

* use id for pass through

* fix columns

* fix PassThroughInfoView

* cleanup

* working edit and delete pass through

* fix rendering id for pt row

* fixes for pt info view

* working delete pass through

* fix use NumericalInput

* fix alignment

* qa - creating pt

* show route preview

* fix show just 1 msg

* test_create_pass_through_endpoint

* fix ui linting
2025-06-20 18:35:38 -07:00
frankandGitHub 0d486120bc add ciphers in command and pass to hypercorn for proxy (#11916)
Signed-off-by: frankzye1 <frankzye@qq.com>
2025-06-20 14:45:48 -07:00
John TongandGitHub c36d0f667b Feat/add delete callback (#11654)
* feat(delete callback): add api and ui for deleting logging callbacks

* hotfix(delete callback): hotfix lint error

* move test case to test_litellm folder

* fix test case to use mock prisma client
2025-06-20 13:50:37 -07:00
Krish DholakiaandGitHub 308e82d885 LiteLLM SDK <-> Proxy improvement (don't transform message client-side) + Bedrock - handle qs:.. in base64 file data + Tag Management - support adding public model names (#11908)
* fix(factory.py): handle qs:.. in mime type

Fixes https://github.com/BerriAI/litellm/issues/11839

* feat(litellm_proxy/): don't transform messages client-side

leave litellm proxy messages untouched - allow proxy to handle transformation

 prevents double transformation

* feat(tag_management_endpoints.py): support adding models to tag by adding model_name

Closes https://github.com/BerriAI/litellm/issues/11884

* test(test_tag_management_endpoints.py): add unit tests for adding new model by public model name

* test: update test
2025-06-19 22:34:18 -07:00
b90d3ca8d0 [Feat] MCP - Allow connecting to MCP with authentication headers + Allow clients to specify MCP headers (#11890) (#11891)
* initial mcp auth with special header (#11890)

Co-authored-by: wagnerjt <wagnerjt@github.com>

* add mcp auth header

* fixes MCP client for litellm proxy

* fixes loc of MCP types

* fixes use MCP client for auth to MCPs

* fix organization

* fix mcp auth header

* add MCP auth header to litellm auth

* fixes for MCP auth

* Add MCP auth to list tools

* fix MCP call tool

* fixes for MCP auth header

* tests for MCP transport

* TestMCPClientUnitTests

* docs MCP auth

* fix types

* docs fix

* fix MCP auth import

* fix code qa check

* test fix mcp auth token check

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>
2025-06-19 20:07:08 -07:00
Ishaan Jaff 8818347acc test scim 2025-06-18 21:42:40 -07:00
Krish DholakiaandGitHub c8bdc552fb Revert "Revert "UI - allow setting default team for new users (#11874)" (#11876)" (#11877)
This reverts commit 8cfb3cfa94.
2025-06-18 21:30:59 -07:00
Krish DholakiaandGitHub 8cfb3cfa94 Revert "UI - allow setting default team for new users (#11874)" (#11876)
This reverts commit 65bda75f31.
2025-06-18 21:27:59 -07:00
Krish DholakiaandGitHub 65bda75f31 UI - allow setting default team for new users (#11874)
* feat(internal_user_endpoints.py): allow passing team-specific details when creating new user with team

allows assigning users to teams on creation

* feat(internal_user_endpoints.py): support for setting team values in default internal user params

Ensures admin can assign all new users to a default team

* test(test_internal_user_endpoints.py): add unit tests

* feat(ui/): allow setting default team on admin UI

easier onboarding for users

* fix(internal_user_endpoints.py): fix exception check
2025-06-18 21:27:32 -07:00
Ishaan Jaff c71d50fd09 test_handle_team_membership_changes_add_teams 2025-06-18 21:21:55 -07:00
Ishaan JaffandGitHub 3a1a8f5a78 [Bug Fix] - Ensure "Request" is tracked for pass through requests on LiteLLM Proxy (#11873)
* fixes for pt tracing

* test_pass_through_request_contains_proxy_server_request_in_kwargs
2025-06-18 21:15:09 -07:00
Ishaan Jaff 3043df0d30 fixes test scimv2 2025-06-18 16:38:36 -07:00
Ishaan JaffandGitHub 4782d435ed [Fix] SCIM - Add SCIM PATCH and PUT Ops for Users (#11863)
* fix SCIM memberships Patch

* fixes for SCIM updates

* fixes for SCIM

* working provisioning for teams on SCIM

* working user patch / PUT ops SCIM

* fixes SCIM

* test_scim_v2_endpoints.py

* handle_existing_user_by_email

* fixes for provisioning SCIMUser

* fixes SCIM provisioning

* test scim v2

* fixes for linting

* fix _apply_patch_ops

* fixes code QA check for team membership checks
2025-06-18 16:24:55 -07:00
dfdbfdd71c [Feat] Passthrough - Add support for setting custom cost per pass through request (#11870)
* use cost_per_request

* fix cost_per_request

* fixes cost_per_request

* fixes for cost per request for pass through

* ui fix param name

* fixes for _set_cost_per_request

* test cost per request pass through endpoints

* Update tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* tests pass through endpoints

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-18 16:24:38 -07:00
Cole McIntoshandGitHub e1fbdde289 Fix boto3 tracer wrapping for observability (#11869)
* feat: add summarize parameter to /spend/logs endpoint for individual transaction logs

- Introduced a new `summarize` parameter to control data format when querying spend logs.
- `summarize=true` (default) returns aggregated data, while `summarize=false` provides individual transaction logs.
- Updated documentation and added tests to validate the new functionality.

* fix: wrap boto3.Session() with tracer for observability - Add tracer.trace wrapper around boto3.Session() call in _get_aws_region_name method - Ensures all boto3 initializations in base_aws_llm.py are properly instrumented - Fixes test_boto3_init_tracer_wrapping test failure - Maintains consistency with other boto3 calls in the same file
2025-06-18 15:47:55 -07:00
Krish DholakiaandGitHub 7f8b2579a2 Minor Fixes (#11868)
* fix(litellm_pre_call_utils.py): add user agent tags to spend logs in standard logging payload logic

avoid clash when tag based routing is enabled

* test: remove redundant test

* test: rename oidc test to run earlier

quicker debuging

* fix(azure.py): return more detailed error message

* fix(azure/common_utils.py): use default scope, if scope is none

fixes oidc test

* fix: always default to cognitiveservices.azure.com

* test: update test
2025-06-18 14:12:59 -07:00
Ishaan JaffandGitHub ef336dcb38 [SCIM] Add Error handling for existing user on SCIM (#11862)
* fix handling existing user on SCIM

* test scim v2 fixes
2025-06-18 09:49:13 -07:00
Cole McIntoshandGitHub 7d1e0651d8 Implement health check backend API and storage functionality - fix ci/cd (#11852)
* feat: Add health check functionality and endpoints

- Introduced methods for saving health check results to the database, including validation and cleaning of data.
- Added new health check endpoints to retrieve health check history and latest health statuses for models.
- Updated model prices and context window configuration for new Azure transcription models.

* test: Add unit tests for health check functionality

- Introduced tests for PrismaClient health check methods, including saving results and retrieving health check history.
- Added tests for the _save_health_check_to_db function to ensure proper handling of healthy and unhealthy endpoints.
- Implemented mock objects to simulate database interactions and validate method behaviors.

* Refactor health endpoint model ID handling and improve logging

- Updated health endpoint to use `get_deployment` for retrieving model names based on model IDs, enhancing error handling for missing models.
- Changed health check result saving to the database to be non-blocking by using `asyncio.create_task`.
- Cleaned up code for better readability and maintainability.

* Refactor utility functions in proxy module for improved readability and error handling

- Removed unused imports and simplified exception handling in `_get_redoc_url` and `_get_docs_url` functions to manage circular imports.
- Cleaned up logging statements for consistency and clarity.
- Streamlined error message formatting in `handle_exception_on_proxy` function.

* Enhance type hinting and default values in ProxyUpdateSpend class for improved clarity and robustness

- Added type hints for `_end_user_list_transactions` to specify it as a dictionary mapping end user IDs to spend amounts.
- Updated default values for optional fields in `SpendLogsPayload` to ensure they are initialized properly, enhancing error handling.
- Refactored `_premium_user_check` function to improve model validation logic and error handling.

* Fix disable_spend_updates method to handle None return value gracefully

- Updated the disable_spend_updates method to return False if the environment variable DISABLE_SPEND_UPDATES is not set or is None, improving robustness in configuration handling.

* Refactor join_paths function in utils.py for improved path handling

- Enhanced the join_paths function to better manage leading and trailing slashes, ensuring correct path concatenation.
- Added logic to handle cases where either base_path or route is empty, improving robustness and usability.

* Enhance health check functionality and improve error handling

- Introduced a new method `_save_health_check_to_db` for saving health check results to the database, utilizing safe JSON functions for data integrity.
- Refactored existing health check methods to streamline the process and improve error logging.
- Updated email sending logic to ensure secure connections and better error handling.
- Improved spend update logic with batch processing and retry mechanisms for database operations.
- Added utility functions for projected spend calculations and enhanced validation for team configurations.

* Add health check methods for database interaction

- Introduced `save_health_check_result` method to save health check results with detailed logging and validation.
- Added `get_health_check_history` method for retrieving health check records with optional filtering.
- Implemented `get_all_latest_health_checks` method to fetch the latest health checks for each model.
- Enhanced error handling and logging for all new methods to improve reliability and traceability.

* Refactor health check result saving to use typed arguments

- Updated the `_save_health_check_to_db` function to call `save_health_check_result` with explicitly typed arguments instead of a dictionary spread, enhancing code clarity and type safety.
- Removed unused method bindings in the mock Prisma client tests to streamline the test setup.

* Remove unused `_save_health_check_to_db` function from utils.py to streamline code and improve maintainability.

* Implement response time validation and details cleaning in health check result saving

- Added `_validate_response_time` method to ensure response time values are valid and handle exceptions gracefully.
- Introduced `_clean_details` method to validate and clean details JSON, improving data integrity.
- Refactored `save_health_check_result` to utilize these new methods for optional fields, enhancing code clarity and maintainability.
- Updated tests to bind new methods to the mock Prisma client for comprehensive testing.

* Add health check utility functions and refactor existing endpoints

- Introduced `_convert_health_check_to_dict` to standardize health check record conversion to dictionary format for JSON responses.
- Added `_check_prisma_client` helper function to streamline database availability checks and improve error handling.
- Refactored health check endpoints to utilize the new utility functions, enhancing code clarity and maintainability.

* Refactor health check tests for improved clarity and coverage

- Simplified the mock PrismaClient setup by consolidating method bindings.
- Updated health check result saving tests to use parameterized scenarios for better coverage.
- Added tests for health check history retrieval and graceful handling when no database client is provided.
- Removed redundant mock functions to streamline the test suite.

* Implement helper function for health check and database saving

- Added `_perform_health_check_and_save` to encapsulate health check execution and optional database saving.
- Refactored health endpoint logic to utilize the new helper function, improving code clarity and reducing redundancy.
- Enhanced error handling and streamlined the process of saving health check results to the database.

* refactor: rename target_model parameter to model in health check function
2025-06-18 06:57:08 -07:00
Ishaan JaffandGitHub 0eb8a3de10 [Feat] v2 Pass through endpoints - Add support for subroutes for pass through endpoints + Cleaned up UI (#11827)
* fix: add construct_target_url_with_subpath

* add InitPassThroughEndpointHelpers

* added debugging for pass through routes

* add PassThroughGenericEndpoint to include subpath and input_cost_per_request

* polish page

* Add Pass-Through Endpoint

* polish pass through ui

* fixes for initialize_pass_through_endpoints

* PassThroughGenericEndpoint

* test_add_subpath_route

* test_initialize_pass_through_endpoints_with_include_subpath

* fix code QA check
2025-06-17 20:52:28 -07:00
Ishaan JaffandGitHub a8196159b9 [Bug Fixes] MCP - using MCPs defined on config.yaml + fix for MCP error Team doesn't exist in cache (#11824)
* fix - dont allow missing team to block MCP list response

* use HTTP as default transport

* _generate_stable_server_id

* fix list MCP servers

* ui fix

* test_list_mcp_servers_empty_result

* test_generate_stable_server_id

* _generate_stable_server_id

* fix code QA

* fix code QA
2025-06-17 16:25:12 -07:00
Krish DholakiaandGitHub 30b431681e JWT Auth - correctly return user email + UI Model Update - Allow editing model access group for existing model (#11783)
* fix(handle_jwt.py): check user object, if jwt user is proxy admin

correctly return user role - if jwt user has role updated in UI

* test(test_handle_jwt.py): add unit test for passing correct user role

* feat(model_info_view.tsx): separate UI component for updating edit model component

* feat(model_info_view.tsx): allow updating model access group on UI

show all available access groups in ui component

* docs: minor fixes
2025-06-16 22:11:04 -07:00
Krish DholakiaandGitHub 87ae2cf3d7 SSO - Allow passing additional headers + Spend Tags - automatically track spend by user agent (allows cost tracking for claude code) (#11781)
* feat(ui_sso.py): allow admin to specify additional headers for sso provider

some sso providers require special headers to return a json response

* test(test_ui_sso.py): add unit tests to ensure custom headers are respect3ed

* docs(config_settings.md): document new header param

* fix(litellm_pre_call_utils.py): add spend tag tracking by user agent

allows checking spend for cli tools like claude code

* feat(litellm_pre_call_utils.py): track spend by user agent part if user agent contains "/"

allows tracking spend across user agent versions

Better cost tracking for claude cod

* test(test_litellm_pre_call_utils.py): add testing for pre call utils, user agent parsing

* fix: fix linting check
2025-06-16 21:53:40 -07:00
35d74af5a0 [Fix] Pass through - Langfuse don't log request to Langfuse passthrough on Langfuse (#11768)
* dont log lf pass through request

* test_is_langfuse_route

* Update litellm/proxy/pass_through_endpoints/success_handler.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-16 17:30:23 -07:00
Ishaan JaffandGitHub c40580f892 [Fix] JWT - Fix error when team member already part of team (#11735)
* fix _check_member_duplication

* fix map_user_to_teams

* test_map_user_to_teams_handles_already_in_team_exception

* test_team_endpoints.py
2025-06-14 15:50:16 -07:00
Ishaan Jaff 1c1e41c51d Revert "Implement health check backend API and storage functionality (#11678)"
This reverts commit 5f34ceea1a.
2025-06-14 15:45:33 -07:00
Ishaan JaffandGitHub d4107dfd68 [Bug Fix] Ensure /redis/info works on GCP Redis (#11732)
* fix cache_redis_info

* mock_redis_client_list_restricted
2025-06-14 15:35:09 -07:00
Cole McIntoshandGitHub 5f34ceea1a Implement health check backend API and storage functionality (#11678)
* feat: Add health check functionality and endpoints

- Introduced methods for saving health check results to the database, including validation and cleaning of data.
- Added new health check endpoints to retrieve health check history and latest health statuses for models.
- Updated model prices and context window configuration for new Azure transcription models.

* test: Add unit tests for health check functionality

- Introduced tests for PrismaClient health check methods, including saving results and retrieving health check history.
- Added tests for the _save_health_check_to_db function to ensure proper handling of healthy and unhealthy endpoints.
- Implemented mock objects to simulate database interactions and validate method behaviors.

* Refactor health endpoint model ID handling and improve logging

- Updated health endpoint to use `get_deployment` for retrieving model names based on model IDs, enhancing error handling for missing models.
- Changed health check result saving to the database to be non-blocking by using `asyncio.create_task`.
- Cleaned up code for better readability and maintainability.

* Refactor utility functions in proxy module for improved readability and error handling

- Removed unused imports and simplified exception handling in `_get_redoc_url` and `_get_docs_url` functions to manage circular imports.
- Cleaned up logging statements for consistency and clarity.
- Streamlined error message formatting in `handle_exception_on_proxy` function.

* Enhance type hinting and default values in ProxyUpdateSpend class for improved clarity and robustness

- Added type hints for `_end_user_list_transactions` to specify it as a dictionary mapping end user IDs to spend amounts.
- Updated default values for optional fields in `SpendLogsPayload` to ensure they are initialized properly, enhancing error handling.
- Refactored `_premium_user_check` function to improve model validation logic and error handling.

* Fix disable_spend_updates method to handle None return value gracefully

- Updated the disable_spend_updates method to return False if the environment variable DISABLE_SPEND_UPDATES is not set or is None, improving robustness in configuration handling.

* Refactor join_paths function in utils.py for improved path handling

- Enhanced the join_paths function to better manage leading and trailing slashes, ensuring correct path concatenation.
- Added logic to handle cases where either base_path or route is empty, improving robustness and usability.

* Enhance health check functionality and improve error handling

- Introduced a new method `_save_health_check_to_db` for saving health check results to the database, utilizing safe JSON functions for data integrity.
- Refactored existing health check methods to streamline the process and improve error logging.
- Updated email sending logic to ensure secure connections and better error handling.
- Improved spend update logic with batch processing and retry mechanisms for database operations.
- Added utility functions for projected spend calculations and enhanced validation for team configurations.

* Add health check methods for database interaction

- Introduced `save_health_check_result` method to save health check results with detailed logging and validation.
- Added `get_health_check_history` method for retrieving health check records with optional filtering.
- Implemented `get_all_latest_health_checks` method to fetch the latest health checks for each model.
- Enhanced error handling and logging for all new methods to improve reliability and traceability.

* Refactor health check result saving to use typed arguments

- Updated the `_save_health_check_to_db` function to call `save_health_check_result` with explicitly typed arguments instead of a dictionary spread, enhancing code clarity and type safety.
- Removed unused method bindings in the mock Prisma client tests to streamline the test setup.

* Remove unused `_save_health_check_to_db` function from utils.py to streamline code and improve maintainability.

* Implement response time validation and details cleaning in health check result saving

- Added `_validate_response_time` method to ensure response time values are valid and handle exceptions gracefully.
- Introduced `_clean_details` method to validate and clean details JSON, improving data integrity.
- Refactored `save_health_check_result` to utilize these new methods for optional fields, enhancing code clarity and maintainability.
- Updated tests to bind new methods to the mock Prisma client for comprehensive testing.

* Add health check utility functions and refactor existing endpoints

- Introduced `_convert_health_check_to_dict` to standardize health check record conversion to dictionary format for JSON responses.
- Added `_check_prisma_client` helper function to streamline database availability checks and improve error handling.
- Refactored health check endpoints to utilize the new utility functions, enhancing code clarity and maintainability.

* Refactor health check tests for improved clarity and coverage

- Simplified the mock PrismaClient setup by consolidating method bindings.
- Updated health check result saving tests to use parameterized scenarios for better coverage.
- Added tests for health check history retrieval and graceful handling when no database client is provided.
- Removed redundant mock functions to streamline the test suite.

* Implement helper function for health check and database saving

- Added `_perform_health_check_and_save` to encapsulate health check execution and optional database saving.
- Refactored health endpoint logic to utilize the new helper function, improving code clarity and reducing redundancy.
- Enhanced error handling and streamlined the process of saving health check results to the database.
2025-06-14 14:27:11 -07:00
19a8b23644 [Feat] Add Authentication + Permission Management for MCP List, Call Tool Ops (#11682)
* add basic auth checks for MCP

* working MCP auth

* fix MCP

* add get_allowed_mcp_servers for key, team

* add get_allowed_mcp_servers

* TOOLS FROM MCP SERVERS

* fix - working context vars for handling auth to MCPs

* fix imports

* working _get_allowed_mcp_servers_for_key

* TestUserAPIKeyAuthMCP

* test_user_api_key_auth_mcp

* fix code qa check

* fix imports

* test_mcp_server.py

* fix: fix - working permission mgmt

* Update litellm/proxy/_experimental/mcp_server/auth/user_api_key_auth_mcp.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-13 10:13:51 -07:00
Alvaro SevillaandGitHub 196a7269f6 fix(vertex_ai): support global location in vertex ai passthrough (#11661)
* Support global location in vertex ai passthrough

* refactor
2025-06-13 09:51:04 -07:00