* fix(enterprise/litellm_enterprise/proxy/auth/user_api_key_auth.py): bubble up exception if type is ProxyException
* docs(custom_auth.md): doc on bubbling up custom exceptions
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )
* fix(prometheus.py): working tag based metrics
* fix(prometheus.py): emit request tags on post call success hook
* fix(prometheus.py): add user agent tags on request failure
* fix(prometheus.py): add request tags to deployment failure metric
s
* fix(main.py): fix async retryer
Fixes https://github.com/BerriAI/litellm/issues/12830
* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers
clientside content-type != proxy content type, can cause requests to hang
* test(tests/): update tests
* feat(proxy_server.py): support batch polling interval
allows admin to control batch polling interval (default is 3600s)
easier debugging
* fix(proxy_settings_endpoint.py): ensure value is actually set before updating env var
* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError
* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError
* feat(route_checks.py): allow admin to disable proxy management endpoints on instance
useful for preventing multiple instances from doing admin actions
* docs(scaling_multiple_instances.md): add architecture doc on scaling multiple litellm instances
provide guidance on scaling proxy
* docs(scaling_multiple_instances.md): add doc on scaling across multiple regions for litellm
* fix(route_checks.py): allow disabling llm api endpoints on an instance
allows pure admin instance to exist
* refactor(enterprise/route_checks.py): refactor env var checks
* refactor: finish refactoring
* docs(control_plane_and_data_plane.md): refactor docs
* test: update tests
* feat(custom_guardrail.py): allow admin to run guardrails on specific modes based on request tags
Enables pre-call checks for openwebui but logging for claude code guardrails
* refactor(custom_guardrail.py): move tag based mode to enterprise
* test(test_custom_guardrail.py): update test
* feat(quick_start.md): add doc on tag-based guardrail mode
* docs(quick_start.md): cleanup doc
* docs: cleanup
* docs(openweb_ui.md): add tutorial to doc
* fix(custom_guardrail.py): ensure user is premium user
* test: update test
* use common helper create_invitation_for_user
* use common util in proxy
* fix create_invitation_for_user
* refactor base email
* test_get_invitation_link_creates_new_when_none_exist
* fix code QA checks
* feat(check_batch_cost.py): emit spend log on successful request
ensures cost tracked for batch requests
* feat(proxy_server.py): add background job to poll completed batch jobs
used for calculating cost for batch jobs
* fix(proxy_server.py): run batch cost tracking job every hour
batch jobs take time to complete, no need to run every few seconds
* feat(proxy_server.py): run batch cost tracking job every hour
* test(test_router.py): initial unit test confirming router.afile_content uses dynamic api key / api base
* fix(managed_files.py): filter deployments for only those within file id mapping
ensure call works - only route to models where the file was written
* fix(proxy_server.py): fix loading in model ids from config, if config id is int
* fix(router.py): return all model file id mappings on create_file
if multiple deployments - this ensures all the file id mappings are bubbled up
Fixes issue when trying to use loadbalanced deployments - only 1 file id mapping was being stored
* fix(team_endpoints.py): support setting default key expiry
allows admin to set key expiry on all team member keys
makes it easier to setup default team for experimentation
* feat(key_management_endpoints.py): allows admin to set duration for keys created by team members
* feat(team_endpoints.py): support team_member_key_duration on `/team/update`
allows setting max time team member keys are valid for
* fix(team_info.tsx): ui component to update team member key duration
* fix(team_info.tsx): support updating team member key duration, if set
* feat(teams.tsx): add team member key duration param ui component
allow admin to set this on UI
* feat(ui_sso.py): support restricting ui access by sso group
allows controlling who can/can't access the UI
* feat(ssomodals.tsx): add initial commit adding sso group access to admin ui
* feat(proxy_server.py): support reading + writing ui_access_mode from db
allows admin to configure allowed sso groups from UI
* feat(ui_sso.py): support enforcing all teams on sso jwt handler
if ui access mode set via ui, support reading the value and enforcing it
* feat(ui/): ui component for controlling sso access group
allow admin to only allow users within specific sso group to log into UI
* fix(uiaccesscontrolform.tsx): fix field names
* feat(ui_sso.py): return received sso response in the clientside error message - enables easier debugging
* test: add unit tests
* fix: minor fixes
* fix(prometheus.py): remove request increment from inside the log success event
it's only done on post-call success/failure
* fix(litellm_logging.py): add additional validation step for checking if 'stream' is true
prevent double counting on non-stream requests
* test: add unit testing to ensure stream is not incorrectly set to true
* feat(litellm_logging.py): emit request route in standard logging payload
used by prometheus streaming metrics for route
* fix: fix otel test
* fix: fix linting errors
* test: update test
* fix: fix linting error
* feat(enterprise/): fix remaining users check on license
* fix(usage_indicator.tsx): if no max user set, don't render remaining user info card
only for users with user limits on their license
* fix(leftnav.tsx): only show remaining users to admin
* feat(columns.tsx): don't allow sorting on model access groups
it's a list[str]
* feat(model_dashboard.tsx): add model access group filters
* docs(deploy.md): move docker recommendation to `main-stable`
* feat(enterprise/internal_user_endpoints.py): expose endpoint for checking available premium users
* feat(usage_indictor.tsx): add new element to help track remaining premium users
* feat(usage_indicator.tsx): show premium user remaining usage
allows users with user caps to know how much is left
* fix(vertex_and_google_ai_studio_gemini.py): bubble up stream is not finished, even if stop reason is given
prevents early completion of stream
Closes https://github.com/BerriAI/litellm/issues/11549
* fix(streaming_handler.py): respect is_finished = False in hidden params
internal logic for preventing ending stream early
* fix(litellm_license.py): add function to check if user is over limit
* fix(internal_user_endpoints.py): add function to check if user is over limit
* refactor: move test
* docs(customer_endpoints.py): document new param
* fix(main.py): use processed non-default-params as standard input params for langfuse
Fixes https://github.com/BerriAI/litellm/issues/11072
Fixes https://github.com/BerriAI/litellm/issues/11096
* fix(main.py): rename variable to be more accurate
* test(test_langfuse_e2e_test.py): add router unit test for langfuse e2e testing
Prevent https://github.com/BerriAI/litellm/issues/11072 from happening again
* build: update lock
* fix(utils.py): refactor optional params function
make it easier to get the standardized non default params
* fix(utils.py): improve process non default params function
* fix(main.py): include provider specific params in processed non default params used in logging
ensures user can see any provider specific params on langfuse
ensures user can see any provider specific params on langfus e
* refactor(litellm/): wrap enterprise package usage in safe try-excepts
prevents missing package error for OSS users
Fixes https://github.com/BerriAI/litellm/issues/10795
* ci(config.yml): add script to check for unsafe imports to ci/cd
Closes https://github.com/BerriAI/litellm/issues/10795
* fix: add missing docs
* test: test_openai_image_edit_litellm_sdk
* use n 4 for mapped tests (#11109)
* Fix/background health check (#10887)
* fix: improve health check logic by deep copying model list on each iteration
* test: add async test for background health check reflecting model list changes
* fix: validate health check interval before executing background health check
* fix: specify type for health check results dictionary
* fix(user_api_key_auth.py): handle user custom auth set with no custom settings
* bump: version 0.1.21 → 0.2.0
* ci(config.yml): run enterprise and litellm tests separately
* fix: fix linting error
* docs: add missing docs
* [Feat] Add content policy violation error mapping for image editd (#11113)
* feat: add image edit mapping for content policy violations
* test fix
* Expose `/list` and `/info` endpoints for Audit Log events (#11102)
* feat(audit_logging_endpoints.py): expose list endpoint to show all audit logs
make it easier for user to retrieve individual endpoints
* feat(enterprise/): add audit logging endpoint
* feat(audit_logging_endpoints.py): expose new GET `/audit/{id}` endpoint
make it easier to retrieve view individual audit logs
* feat(key_management_event_hooks.py): correctly show the key of the user who initiated the change
* fix(key_management_event_hooks.py): add key rotations as an audit log event
'
* test(test_audit_logging_endpoints.py): add simple unit testing for audit log endpoint
* fix: testing fixes
* fix: fix ruff check
* [Feat] Use aiohttp transport by default - 97% lower median latency (#11097)
* fix: add flag for disabling use_aiohttp_transport
* feat: add _create_async_transport
* feat: fixes for transport
* add httpx-aiohttp
* feat: fixes for transport
* refactor: fixes for transport
* build: fix deps
* fixes: test fixes
* fix: ensure aiohttp does not auto set content type
* test: test fixes
* feat: add LiteLLMAiohttpTransport
* fix: fixes for responses API handling
* test: fixes for responses API handling
* test: fixes for responses API handling
* feat: fixes for transport
* fix: base embedding handler
* test: test_async_http_handler_force_ipv4
* test: fix failing deepeval test
* fix: add YARL for bedrock urls
* fix: issues with transport
* fix: comment out linting issues
* test fix
* test: XAI is unstable
* test: fixes for using respx
* test: XAI fixes
* test: XAI fixes
* test: infinity testing fixes
* docs(config_settings.md): document param
* test: test_openai_image_edit_litellm_sdk
* test: remove deprecated test
* bump respx==0.22.0
* test: test_xai_message_name_filtering
* test: fix anthropic test after bumping httpx
* use n 4 for mapped tests (#11109)
* fix: use 1 session per event loop
* test: test_client_session_helper
* fix: linting error
* fix: resolving GET requests on httpx 0.28.1
* test fixes proxy unit tests
* fix: add ssl verify settings
* fix: proxy unit tests
* fix: refactor
* tests: basic unit tests for aiohttp transports
* tests: fixes xai
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* test: cleanup redundant test
* fix: fix import
* fix: fix linting errors
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: JuHyun Bae <jhyun0408@nate.com>
* feat(audit_logging_endpoints.py): expose list endpoint to show all audit logs
make it easier for user to retrieve individual endpoints
* feat(enterprise/): add audit logging endpoint
* feat(audit_logging_endpoints.py): expose new GET `/audit/{id}` endpoint
make it easier to retrieve view individual audit logs
* feat(key_management_event_hooks.py): correctly show the key of the user who initiated the change
* fix(key_management_event_hooks.py): add key rotations as an audit log event
'
* test(test_audit_logging_endpoints.py): add simple unit testing for audit log endpoint
* fix: testing fixes
* fix: fix ruff check
