* fix(route_checks.py): ensure disable llm api endpoints is correctly set
* fix(route_checks.py): raise httpexception
raise expected exceptions
* fix(router.py): handle team only wildcard models
fixes issue where team only wildcard models were not considered during auth checks
* fix(router.py): handle team only wildcard models
fixes issue where team only wildcard models were not considered during auth checks
* fix(enterprise/litellm_enterprise/proxy/auth/user_api_key_auth.py): bubble up exception if type is ProxyException
* docs(custom_auth.md): doc on bubbling up custom exceptions
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )
* fix(prometheus.py): working tag based metrics
* fix(prometheus.py): emit request tags on post call success hook
* fix(prometheus.py): add user agent tags on request failure
* fix(prometheus.py): add request tags to deployment failure metric
s
* fix(main.py): fix async retryer
Fixes https://github.com/BerriAI/litellm/issues/12830
* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers
clientside content-type != proxy content type, can cause requests to hang
* test(tests/): update tests
* feat(proxy_server.py): support batch polling interval
allows admin to control batch polling interval (default is 3600s)
easier debugging
* fix(proxy_settings_endpoint.py): ensure value is actually set before updating env var
* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError
* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError
* feat(route_checks.py): allow admin to disable proxy management endpoints on instance
useful for preventing multiple instances from doing admin actions
* docs(scaling_multiple_instances.md): add architecture doc on scaling multiple litellm instances
provide guidance on scaling proxy
* docs(scaling_multiple_instances.md): add doc on scaling across multiple regions for litellm
* fix(route_checks.py): allow disabling llm api endpoints on an instance
allows pure admin instance to exist
* refactor(enterprise/route_checks.py): refactor env var checks
* refactor: finish refactoring
* docs(control_plane_and_data_plane.md): refactor docs
* test: update tests
* feat(custom_guardrail.py): allow admin to run guardrails on specific modes based on request tags
Enables pre-call checks for openwebui but logging for claude code guardrails
* refactor(custom_guardrail.py): move tag based mode to enterprise
* test(test_custom_guardrail.py): update test
* feat(quick_start.md): add doc on tag-based guardrail mode
* docs(quick_start.md): cleanup doc
* docs: cleanup
* docs(openweb_ui.md): add tutorial to doc
* fix(custom_guardrail.py): ensure user is premium user
* test: update test
* use common helper create_invitation_for_user
* use common util in proxy
* fix create_invitation_for_user
* refactor base email
* test_get_invitation_link_creates_new_when_none_exist
* fix code QA checks
* feat(check_batch_cost.py): emit spend log on successful request
ensures cost tracked for batch requests
* feat(proxy_server.py): add background job to poll completed batch jobs
used for calculating cost for batch jobs
* fix(proxy_server.py): run batch cost tracking job every hour
batch jobs take time to complete, no need to run every few seconds
* feat(proxy_server.py): run batch cost tracking job every hour
* test(test_router.py): initial unit test confirming router.afile_content uses dynamic api key / api base
* fix(managed_files.py): filter deployments for only those within file id mapping
ensure call works - only route to models where the file was written
* fix(proxy_server.py): fix loading in model ids from config, if config id is int
* fix(router.py): return all model file id mappings on create_file
if multiple deployments - this ensures all the file id mappings are bubbled up
Fixes issue when trying to use loadbalanced deployments - only 1 file id mapping was being stored
* fix(team_endpoints.py): support setting default key expiry
allows admin to set key expiry on all team member keys
makes it easier to setup default team for experimentation
* feat(key_management_endpoints.py): allows admin to set duration for keys created by team members
* feat(team_endpoints.py): support team_member_key_duration on `/team/update`
allows setting max time team member keys are valid for
* fix(team_info.tsx): ui component to update team member key duration
* fix(team_info.tsx): support updating team member key duration, if set
* feat(teams.tsx): add team member key duration param ui component
allow admin to set this on UI
* feat(ui_sso.py): support restricting ui access by sso group
allows controlling who can/can't access the UI
* feat(ssomodals.tsx): add initial commit adding sso group access to admin ui
* feat(proxy_server.py): support reading + writing ui_access_mode from db
allows admin to configure allowed sso groups from UI
* feat(ui_sso.py): support enforcing all teams on sso jwt handler
if ui access mode set via ui, support reading the value and enforcing it
* feat(ui/): ui component for controlling sso access group
allow admin to only allow users within specific sso group to log into UI
* fix(uiaccesscontrolform.tsx): fix field names
* feat(ui_sso.py): return received sso response in the clientside error message - enables easier debugging
* test: add unit tests
* fix: minor fixes
* fix(prometheus.py): remove request increment from inside the log success event
it's only done on post-call success/failure
* fix(litellm_logging.py): add additional validation step for checking if 'stream' is true
prevent double counting on non-stream requests
* test: add unit testing to ensure stream is not incorrectly set to true
* feat(litellm_logging.py): emit request route in standard logging payload
used by prometheus streaming metrics for route
* fix: fix otel test
* fix: fix linting errors
* test: update test
* fix: fix linting error
* feat(enterprise/): fix remaining users check on license
* fix(usage_indicator.tsx): if no max user set, don't render remaining user info card
only for users with user limits on their license
* fix(leftnav.tsx): only show remaining users to admin
* feat(columns.tsx): don't allow sorting on model access groups
it's a list[str]
* feat(model_dashboard.tsx): add model access group filters
* docs(deploy.md): move docker recommendation to `main-stable`
* feat(enterprise/internal_user_endpoints.py): expose endpoint for checking available premium users
* feat(usage_indictor.tsx): add new element to help track remaining premium users
* feat(usage_indicator.tsx): show premium user remaining usage
allows users with user caps to know how much is left
* fix(vertex_and_google_ai_studio_gemini.py): bubble up stream is not finished, even if stop reason is given
prevents early completion of stream
Closes https://github.com/BerriAI/litellm/issues/11549
* fix(streaming_handler.py): respect is_finished = False in hidden params
internal logic for preventing ending stream early
* fix(litellm_license.py): add function to check if user is over limit
* fix(internal_user_endpoints.py): add function to check if user is over limit
* refactor: move test
* docs(customer_endpoints.py): document new param
Krrish Dholakia