* refactor(aim.py): refactor to support adding aim guardrails on UI
* fix(base.py): add ui_friendly_name to config model
* feat(ui/): support loading new guardrails from backend api call
removes need to onboard each guardrail to ui
* fix: don't show optional params if not set and don't show ui_friendly_name (internal param0
* fix(ui/add_guardrail_form.tsx): ensure dynamic provider value is used
* fix(ui/): just one-time update the provider map dictionary
* fix(ui/): show masked api base / api key on guardrail update
* refactor(aporia_ai/): refactor to show on UI
* feat(aporia_ai/): add aporia ai guardrail to UI
* refactor(guardrails_ai/): refactor to add via UI
* refactor(lasso.py): refactor to enable adding lasso guardrails via UI
* feat(pangea.py): add pangea guardrail on UI
* feat(panw): add panw prisma airs through UI
* test: update tests
* fix: fix ruff linting error
* test: update tests
* fix: add missing docs
* fix: fix guardrail init
* fix: suppress linting errors
* fix(proxy_server.py): fix linting error
* build(model_prices_and_context_window.json): remove 'supports_tool_choice' for specific mistral models
Closes https://github.com/BerriAI/litellm/issues/11750
* feat: initial commit adding cleaner ui for azure text moderation guardrails
* feat(guardrail_endpoints.py): add discoverable guardrail configs and improve converting base model to dict with types
* fix(guardrail_provider_fields.tsx): render from api endpoint correctly
* fix(guardrail_provider_fields.tsx): cleanup
* refactor(guardrail_endpoints.py): refactor to handle dictionaries with literal - allows multiselect
* feat(ui/): render dictionary with known keys correctly
* feat(ui/): render optional params on separate page
* style(ui/): style improvements to rendering optional params on the UI
* feat(azure/prompt_shield.py): add azure prompt shield back on UI
* fix(add_guardrail_form.tsx): fix form to handle updated api
* fix(guardrail_optional_params.tsx): ensure values are nested correctly for writing to api
* fix: fix linting error
* feat(text_moderation.py): handle str to int conversion
* fix(guardrail_info.tsx): only render pii settings if guardrail is presidio
* fix(guardrail_info.tsx): add guardrail specific fields to update settings
allows updating guardrail fields (e.g. severity threshold) post-create
* fix(guardrail_endpoints.py): set guardrail_id in guardrail object
ensures duplicate objects not created on guardrail update
* fix(guardrail_endpoints.py): allow provider specific fields to be updated on patch update
* refactor(guardrail_endpoints.py): remove duplicate info endpoint
* fix(guardrail_endpoints.py): mask sensitive keys on returning via guardrail `/info`
Prevent leaking keys
* fix(guardrail_optional_params.tsx): return numerical input when numerical component used
fixes issue where output was a str
* fix(guardrail_optional_params.tsx): render dict keys correctly
* fix(text_moderation.py): fix severity by category check
* fix(proxy/utils.py): check if guardrail should run for post call streaming hook
Prevents invalid guardrails from running if not requested
* test: fix import
* fix: fix linting error
* test: update test
* fix: fix tests
* fix: fix code qa errors
* fix(guardrail_endpoints.py): set max depth for function
* test: update recursive_detector.py
* test: update list
* build: merge main
* fix: fix ruff check errors
* build(model_prices_and_context_window.json): remove 'supports_tool_choice' for specific mistral models
Closes https://github.com/BerriAI/litellm/issues/11750
* feat: initial commit adding cleaner ui for azure text moderation guardrails
* feat(guardrail_endpoints.py): add discoverable guardrail configs and improve converting base model to dict with types
* fix(guardrail_provider_fields.tsx): render from api endpoint correctly
* fix(guardrail_provider_fields.tsx): cleanup
* refactor(guardrail_endpoints.py): refactor to handle dictionaries with literal - allows multiselect
* feat(ui/): render dictionary with known keys correctly
* feat(ui/): render optional params on separate page
* style(ui/): style improvements to rendering optional params on the UI
* feat(azure/prompt_shield.py): add azure prompt shield back on UI
* fix(add_guardrail_form.tsx): fix form to handle updated api
* fix(guardrail_optional_params.tsx): ensure values are nested correctly for writing to api
* fix: fix linting error
* test: update tests
* fix mapped tests (#12320)
* fix - use flush llm client cache
* faster mapped tests
* test_async_multiple_response_ids_routing
* fix tests
* test_ateam_member_update_admin_requires_premium
* regular mapped tests
* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206)"
This reverts commit 2c60c316ec.
* reset num workers
* check_for_litellm_module_deletion
* add test
* add code_qa_check_tests
* ci/cd new release
* docs(opentelemetry_integration.md): add otel debug flag to docs
help people debug their traces easily
* [Feat] Add failure logging support for s3 logger (#12299)
* add async_log_failure_event
* test_basic_s3_v2_logging_failure
* Comma separated spend and budget (#12317)
* all_keys_table, key_info_view, organizations
* teams
* the whole usage page
* teams page
* users table
* user info
* more commas
* some more commas
* Segregate MCP tools on connections using headers (#12296)
* Add get tools segregation
* add ui changes (#12302)
* resolve comments
* add mapped tests
* remove advanced settings (#12323)
* [Bug Fix] /generateContent API - Only pass supported params when using OpenAI models (#12297)
* fix - only pass GenericLiteLLMParams
* test_google_generate_content_with_openai
* Fix: Fix custom ca bundle support in aiohttp transport (#12281)
* Unify usage of get_ssl_configuration
* Fix doc
* [Feat] Add github co-pilot as a new LLM API provider (#12325)
* Litellm dev 03 05 2025 contributor prs (#9079)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
* Fix#7629 - Add tzdata package to Dockerfile (#8915)
* Add tzdata package to Dockerfile
* Move tzdata to python requirement.txt
* feat: add support for copilot provider (#8577)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat: add model information for copilot models
* fix: fix linting errors
* test: remove integration test for github_copilot + fix misisng mock
* fix: use print to make sure the logger message shown
* test: remove debug print
* fix lint (#11112)
* Add init files to make test directories Python packages and update import paths in test_token_counter.py (#11119)
* Update litellm/model_prices_and_context_window_backup.json
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
---------
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* refactor github copilot
* test_github_copilot_transformation.py
* test_github_copilot_authenticator.py
* add GitHub Copilot
* fix order
* doc fix
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
* add new env vars to docs
* ssl_cert_error (#12327)
* feat - add new banner on startup (#12328)
* [Security Bug Fix] Ensure only LLM API route fails get logged on Langfuse (and other loggers) (#12308)
* _is_proxy_only_llm_api_error
* test_proxy_only_error_true_for_llm_route
* add not on change
* Update tests/test_litellm/proxy/test_proxy_utils.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add test_post_call_failure_hook_auth_error_key_info_route
* test fix _is_proxy_only_llm_api_error
* test_chat_completion_request_with_redaction
* test_post_call_failure_hook_auth_error_llm_api_route
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Ensure message redaction works for responses API logging (#12291)
* add fixes to choice implementation redaction
* add isInstance check on responses API
* change datadog to revert back
* change datadog to revert back
* fix type errors
* Redaction test changes
* Redaction test changes
* Redaction test changes - remove changes
* [Feat] UI - Allow Viewing/Editing Team Based Callbacks (#12329)
* add logging settings view on UI
* fix change ordering
* fix type error
* test fix - import loc
* [Feat] QA - Arize Team based logging (#12331)
* add _get_tracer_with_dynamic_headers
* fix construct_dynamic_arize_headers
* [Feat] UI - Allow Viewing/Editing Team Based Callbacks (#12329)
* add logging settings view on UI
* fix change ordering
* add construct_dynamic_otel_headers for arize
* refactor common code
* test_construct_dynamic_arize_headers
* otel unit tests
* test_arize_dynamic_params
* test_arize_dynamic_headers_in_grpc_requests
* [Feat] UI - Allow setting Logging Callback Setting per Key (#12333)
* allow adding logging settings on a key
* add ability to edit key logging settings
* dont leak api keys on ui
* fix order
* add langsmith settings
* ui new build
* test_default_api_base
* fix new utils tests
* bump: version 1.73.7 → 1.74.0
* Add mcp server segregation comma separated support (#12326)
* add mcp comma separated support
* dont support legacy json array
* lint
* add live tail state to session storage (#12335)
* add aiml
* fix: fix unused imports
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
Co-authored-by: Jugal D. Bhatt <55304795+jugaldb@users.noreply.github.com>
Co-authored-by: Joost van Doorn <joost.van.doorn@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add JWTLiteLLMRoleMap
* test_sync_user_role_and_teams
* add sync_user_role_and_teams
* test_sync_user_role_and_teams
* fix types
* Sync User Roles and Teams with IDP
* Add test for JWT role mapping to LiteLLM roles
The test_list_callbacks_mixed_callback_types test was failing when run in parallel with other tests due to the proxy server's internal _PROXY_VirtualKeyModelMaxBudgetLimiter callback persisting across test runs. This fix filters out proxy internal callbacks from the test assertions to ensure consistent test behavior regardless of test execution order or parallelization.
* feat(azure/prompt_shield.py): initial commit adding prompt shield guardrail + auto discovery mechanism for guardrails
reduces amount of code needed outside of guardrail integration for instrumentation
* feat(azure/prompt_shield.py): working azure prompt shield guardrail integration
Addresses https://github.com/BerriAI/litellm/issues/12254
* test: unit tests for prompt_shield
* fix(prompt_shield.py): add event hook validation for prompt shield guardrail
ensures prompt shield guardrail raises error if asked to run post_call (only runs on user prompt)
* feat(azure/): working text_moderation integration
* fix(text_moderation.py): suppress linting error
* test(test_azure_text_moderation.py): add unit test
* test(test_azure_text_moderation.py): add unit test for responses
* fix(text_moderation.py): return streaming error correctly
ensures error returned to user
* fix: fix linting error
* fix: fix linting check
* test: change mistral model
service tier exceeded
* fix(exception_mapping_utils.py): cover mistral in exception mapping
* Move PANW Prisma AIRS test per feedback on PR #12116
- Move test to tests/test_litellm/proxy/guardrails/guardrail_hooks/
* Remove test file from old location
The test_keys_delete_error_handling test was failing with:
- ConnectionError when the mock wasn't properly applied
- The test was checking str(result.exception) without first verifying exception exists
This fix adds an explicit check that result.exception is not None before
attempting to convert it to string, preventing potential AttributeError
and making the test more robust.
* fix(proxy_server.py): handle empty config yaml
Fixes https://github.com/BerriAI/litellm/issues/12163
* fix(gemini/common_utils.py): replace models/ as expected, instead of using 'strip'
Fixes https://github.com/BerriAI/litellm/issues/12160
* fix(anthropic/experimental_pass_through/messages/transformation.py): check for env var when selecting api key
* docs(config_settings.md): add api key to docs
* fix(team_endpoints.py): prevent overwriting current list of team models on new model add
* fix(networking.tsx): fix default proxy base url
* fix(proxy_server.py): include team only models when retrieving all deployments on `/v2/model/info` helper util
ensures team only models are shown to user
* fix(router.py): check model name by team public model name when team id given
Fixes issue where team member could not see team only models when clicking into that team on `Models + Endpoints`
* fix(team_member_view.tsx): fix rendering team member budget, when budget is set
* test: update tests
* test: update unit test
* Fix user-team association issues in LiteLLM proxy
- Update list_team function to properly filter teams using user's teams array instead of only checking members_with_roles field
- Add Field descriptions and docstring to TeamMemberAddRequest and related models for better Swagger/OpenAPI documentation
- Maintain backward compatibility with fallback to members_with_roles if user lookup fails
This ensures users created with teams parameter appear correctly in team views and improves API documentation.
* Fix duplicate member checking in team_member_add endpoint
- Enhanced team_member_add_duplication_check to check both user_id and user_email
- Added additional duplicate prevention logic after user creation/lookup
- Fixed issue where users added by email could be duplicated in teams
- Added logging for debugging duplicate detection
This addresses the bug where adding the same user by email multiple times would create duplicate entries in the team's members_with_roles array.
* Improve duplicate member prevention in team_member_add
- Enhanced early duplicate check to handle both user_id and user_email
- Added late-stage duplicate prevention after user lookup/creation
- Fixed issue where users could be added multiple times by email
- Cleaned up debug logging
Note: There's still an edge case where the duplicate prevention may not work
correctly in all scenarios. This needs further investigation and testing.
* Refactor team_member_add endpoint for improved member management
- Split team_member_add functionality into smaller, dedicated functions for permission validation, member processing, and team member list updates.
- Enhanced permission checks to ensure only authorized users can add members.
- Streamlined member addition logic to reduce redundancy and improve readability.
- Maintained existing functionality while improving code structure and maintainability.
* Add tests for team_member_add helper functions
- Add test for _validate_team_member_add_permissions with admin user
- Add test for _validate_team_member_add_permissions with non-admin user
- Add test for _process_team_members with single member
- Add test for _process_team_members with multiple members
- Add test for _update_team_members_list with new member
- Add test for _update_team_members_list duplicate prevention
These tests ensure the refactored helper functions work correctly
after fixing the PLR0915 linting error.
* fix(llm_http_handler.py): raise clearer error on anthropic unified route
Fixes https://github.com/BerriAI/litellm/issues/12063
* fix(key_management_endpoints.py): add new param `new_key` for setting the regenerated key value
user request
* test: add unit tests
* fix(pass_through_endpoints.py): use data instead of json for passthrough requests
fixes bedrock latency issue
* Revert "fix(pass_through_endpoints.py): use data instead of json for passthrough requests"
This reverts commit 021dfd9165f837d37e6aad247ccbd0d2e8ca6043.
The test was failing because the managed_files hook was not configured.
This fix adds a DummyManagedFiles class and mocks the create_file
function as an AsyncMock to ensure proper test execution.
* test(test_router.py): initial unit test confirming router.afile_content uses dynamic api key / api base
* fix(managed_files.py): filter deployments for only those within file id mapping
ensure call works - only route to models where the file was written
* fix(proxy_server.py): fix loading in model ids from config, if config id is int
* fix(router.py): return all model file id mappings on create_file
if multiple deployments - this ensures all the file id mappings are bubbled up
Fixes issue when trying to use loadbalanced deployments - only 1 file id mapping was being stored
* fix(team_endpoints.py): support setting default key expiry
allows admin to set key expiry on all team member keys
makes it easier to setup default team for experimentation
* feat(key_management_endpoints.py): allows admin to set duration for keys created by team members
* feat(team_endpoints.py): support team_member_key_duration on `/team/update`
allows setting max time team member keys are valid for
* fix(team_info.tsx): ui component to update team member key duration
* fix(team_info.tsx): support updating team member key duration, if set
* feat(teams.tsx): add team member key duration param ui component
allow admin to set this on UI
* feat(ui_sso.py): support restricting ui access by sso group
allows controlling who can/can't access the UI
* feat(ssomodals.tsx): add initial commit adding sso group access to admin ui
* feat(proxy_server.py): support reading + writing ui_access_mode from db
allows admin to configure allowed sso groups from UI
* feat(ui_sso.py): support enforcing all teams on sso jwt handler
if ui access mode set via ui, support reading the value and enforcing it
* feat(ui/): ui component for controlling sso access group
allow admin to only allow users within specific sso group to log into UI
* fix(uiaccesscontrolform.tsx): fix field names
* feat(ui_sso.py): return received sso response in the clientside error message - enables easier debugging
* test: add unit tests
* fix: minor fixes
* SCIM fix new user roles
* test_create_user_defaults_to_viewer
* test_create_user_uses_default_internal_user_params_role
* fix default user for SCIM
* fix linting error
* fix(internal_user_endpoints.py): don't apply default internal user params if role is admin
prevent internal user restrictions from being applied to admin
* fix(proxy_server.py): fix model info v2 endpoint check - handle user_id being none
* fix(team_endpoints.py): ensure team doesn't lose all model access if set as empty string and new team model added
* fix(proxy_server.py): ensure model with team id is only added as valid for team which has that id
* fix(proxy_cli.py): check for module not found error on proxy import
Fixes https://github.com/BerriAI/litellm/issues/11836
* feat(proxy_server.py): utility function to get all models across all teams user is in
Allows user to see all team models on UI
* feat(proxy_server.py): return models accessible via team id's in `/v2/model/info`
Allows UI to tell user which team they can use to access model
* feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub
allows user to know what teams they can access a model through
* Revert "feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub"
This reverts commit f844c79383ec6739ed712f59e33a524a26b3d35a.
* fix(proxy_server.py): backend model info endpoint improvements
* UI Improvements for Default User access (#11952)
* feat(ui/): add a 'current team' and 'view' filters to the models page
allow user to see what all models they have access to within a specific team
* feat: working ui for seeing models in teams
* fix(model_dashboard.tsx): make current team filter more prominent
* style(model_dashboard.tsx): add a helpful note telling user how to create a model for the team they've selected
* style(model_dashboard.tsx): only show helpful note when current view is team, not for global
* fix(team_dropdown.tsx): allow searching by team id on create key modal
* feat(create_key_button.tsx): add helpful message when team selection is required
* fix: fix linting checks
* fix: fix ui linting error
* docs(team_endpoints.py): document new param
commit 440bc027251d8180174d762d83d271d0f7b68cc5
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 23:04:11 2025 -0700
fix: fix check
commit 89a7451cb9ee26ff9f642335714dcc6f449d1fc2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 22:42:30 2025 -0700
fix: fix test
commit 1322e3b3497e5d334fdcaa18f0cf7a98ea758df4
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 20:52:40 2025 -0700
style: add more tooltips
commit 172738b98b7864aabcacf3334a394098b300283f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 20:51:09 2025 -0700
feat(team_member_view.tsx): add a tooltip
commit 895eb28deb9127985e30b5e859e5bca8530951c9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:46:49 2025 -0700
fix(teams.tsx): support setting team member budget on create
commit 003cc54a6dd0f65030c4f39a8487adc771b62e11
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:40:49 2025 -0700
fix(team_member_view.tsx): style improvements
commit a627a044f21df788f80d92a4081212072be91632
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:40:01 2025 -0700
fix(team_member_view.tsx): handle scientific notation in string
commit c5a3b7bd8419f6394e1b490849555d02d473baed
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:34:25 2025 -0700
feat(team_membership_view.tsx): show team member spend + max budget on UI
commit e986d12ad5b07c676f4cac5e16745939d7473dee
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:28:06 2025 -0700
feat(team_member_view.tsx): show team member spend + budget on team info
commit 8e398607b25f8a8f0bab41964810b5dd27c5e3f2
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:18:16 2025 -0700
feat(team_info.tsx): show team member budget on team info
commit 1f56886b5913dafefc0c00fbe741c0c9c01144a6
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:15:30 2025 -0700
feat(team_endpoints.py): get team budget table on team info
allows user to see max budget set for team members
commit 0a4320bbfa406c24ad32a420f82152da7bdd7323
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 18:10:06 2025 -0700
feat(team_endpoints.py): return team member budget on team info
allows ui to display this to admin / team member
commit 6a4e29f87b333ae9977e8f878960e63becd89150
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 17:57:20 2025 -0700
fix(team_endpoints.py): support updating team budget on UI
commit 53f0fff34032977433dfe6935ce0a684a4141fd8
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 17:38:17 2025 -0700
feat(proxy/_types.py): return team member spend
update pydantic object to include spend
Allows showing spend of team member within team on UI
commit ef2a1a43ecf7fecfb904042cbf47b3d56246edcb
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 16:31:42 2025 -0700
feat(team_endpoints.py): support 'team_member_budget' param on `/team/update`
enables budget working across all team members
commit 512999f1249b00a02a30f049a0cfa36e829ff989
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 16:20:04 2025 -0700
test: add unit tests for default team member budget
commit 90fa3f61a2d63e12b9f3e1da9775f5c8b7294b5f
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 15:37:51 2025 -0700
feat(team_endpoints.py): support using default team member budget id, if set
allows all team members to use the same budget id
commit acef5324b1a0935a482c71060f610c3d8823e8c3
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 15:22:30 2025 -0700
feat(team_endpoints.py): support `team_member_budget` param on `/team/new`
Allow creating 1 budget for all users within team (makes it easier to increase/reduce budget if needed for all team members)
commit 2e867ac70fbd8768e7c27cf3b078e6dc10e566b9
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date: Fri Jun 20 13:45:06 2025 -0700
fix(ui_sso.py): ensure user is added to team, if set via default internal settings
allows users signed up via SSO to be added to default team
