* fix(create_user_button.tsx): allow admin to set models user has access to, on invite
Enables controlling model access on invite
* feat(auth_checks.py): enforce 'no-model-access' special model name on backend
prevent user from calling models if default key has no model access
* fix(chat_ui.tsx): allow user to input custom model
* fix(chat_ui.tsx): pull available models based on models key has access to
* style(create_user_button.tsx): move default model inside 'personal key creation' accordion
* fix(chat_ui.tsx): fix linting error
* test(test_auth_checks.py): add unit-test for special model name
* docs(internal_user_endpoints.py): update docstring
* fix test_moderations_bad_model
* Litellm dev 02 27 2025 p6 (#8891)
* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads
* fix(sagemaker/handler.py): support passing model id on async streaming
* fix(litellm_pre_call_utils.py): Fixes https://github.com/BerriAI/litellm/issues/7237
* Fix calling claude via invoke route + response_format support for claude on invoke route (#8908)
* fix(anthropic_claude3_transformation.py): fix amazon anthropic claude 3 tool calling transformation on invoke route
move to using anthropic config as base
* fix(utils.py): expose anthropic config via providerconfigmanager
* fix(llm_http_handler.py): support json mode on async completion calls
* fix(invoke_handler/make_call): support json mode for anthropic called via bedrock invoke
* fix(anthropic/): handle 'response_format: {"type": "text"}` + migrate amazon claude 3 invoke config to inherit from anthropic config
Prevents error when passing in 'response_format: {"type": "text"}
* test: fix test
* fix(utils.py): fix base invoke provider check
* fix(anthropic_claude3_transformation.py): don't pass 'stream' param
* fix: fix linting errors
* fix(converse_transformation.py): handle response_format type=text for converse
* converse_transformation: pass 'description' if set in response_format (#8907)
* test(test_bedrock_completion.py): e2e test ensuring tool description is passed in
* fix(converse_transformation.py): pass description, if set
* fix(transformation.py): Fixes https://github.com/BerriAI/litellm/issues/8767#issuecomment-2689887663
* Fix bedrock passing `response_format: {"type": "text"}` (#8900)
* fix(converse_transformation.py): ignore type: text, value in response_format
no-op for bedrock
* fix(converse_transformation.py): handle adding response format value to tools
* fix(base_invoke_transformation.py): fix 'get_bedrock_invoke_provider' to handle cross-region-inferencing models
* test(test_bedrock_completion.py): add unit testing for bedrock invoke provider logic
* test: update test
* fix(exception_mapping_utils.py): add context window exceeded error handling for databricks provider route
* fix(fireworks_ai/): support passing tools + response_format together
* fix: cleanup
* fix(base_invoke_transformation.py): fix imports
* (Feat) - Show Error Logs on LiteLLM UI (#8904)
* fix test_moderations_bad_model
* use async_post_call_failure_hook
* basic logging errors in DB
* show status on ui
* show status on ui
* ui show request / response side by side
* stash fixes
* working, track raw request
* track error info in metadata
* fix showing error / request / response logs
* show traceback on error viewer
* ui with traceback of error
* fix async_post_call_failure_hook
* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads
* test_get_error_information
* fix code quality
* rename proxy track cost callback test
* _should_store_errors_in_spend_logs
* feature flag error logs
* Revert "_should_store_errors_in_spend_logs"
This reverts commit 7f345df47762ff3be04e6fde2f13e70019ede4ee.
* Revert "feature flag error logs"
This reverts commit 0e90c022bbea3550f169118d81e60d711a4024fe.
* test_spend_logs_payload
* fix OTEL log_db_metrics
* fix import json
* fix ui linting error
* test_async_post_call_failure_hook
* test_chat_completion_bad_model_with_spend_logs
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* ui new build
* test_chat_completion_bad_model_with_spend_logs
* docs(release_cycle.md): document release cycle
* bump: version 1.62.0 → 1.62.1
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* fix test_moderations_bad_model
* use async_post_call_failure_hook
* basic logging errors in DB
* show status on ui
* show status on ui
* ui show request / response side by side
* stash fixes
* working, track raw request
* track error info in metadata
* fix showing error / request / response logs
* show traceback on error viewer
* ui with traceback of error
* fix async_post_call_failure_hook
* fix(http_parsing_utils.py): orjson can throw errors on some emoji's in text, default to json.loads
* test_get_error_information
* fix code quality
* rename proxy track cost callback test
* _should_store_errors_in_spend_logs
* feature flag error logs
* Revert "_should_store_errors_in_spend_logs"
This reverts commit 7f345df47762ff3be04e6fde2f13e70019ede4ee.
* Revert "feature flag error logs"
This reverts commit 0e90c022bbea3550f169118d81e60d711a4024fe.
* test_spend_logs_payload
* fix OTEL log_db_metrics
* fix import json
* fix ui linting error
* test_async_post_call_failure_hook
* test_chat_completion_bad_model_with_spend_logs
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* fix(user_api_key_auth.py): Fixes https://github.com/BerriAI/litellm/issues/8780
security fix - enforce model access checks on azure routes
* test(test_user_api_key_auth.py): add unit testing
* test(test_openai_endpoints.py): add e2e test to ensure azure routes also run through model validation checks
* fix(key_management_endpoints.py): initial commit with logic to get all keys for teams user is an admin for
* fix(key_managements_endpoints.py): return all keys for teams user is an admin for
* fix(key_management_endpoints.py): add query param to ensure user opts into seeing all team keys (not just their own)
* fix(regenerate_key_modal.tsx): fix key regenerate
* fix(proxy_server.py): fix model metrics check on none api base
* test(test_key_generate_prisma.py): remove redundant test
* test(test_proxy_utils.py): add unit test covering new management endpoint helper util
* fix: fix test
* test(test_proxy_server.py): fix test
* fix(model_checks.py): update returning known model from wildcard to filter based on given model prefix
ensures wildcard route - `vertex_ai/gemini-*` just returns known vertex_ai/gemini- models
* test(test_proxy_utils.py): add unit testing for new 'get_known_models_from_wildcard' helper
* test(test_models.py): add e2e testing for `/model_group/info` endpoint
* feat(prometheus.py): support tracking total requests by user_email on prometheus
adds initial support for tracking total requests by user_email
* test(test_prometheus.py): add testing to ensure user email is always tracked
* test: update testing for new prometheus metric
* test(test_prometheus_unit_tests.py): add user email to total proxy metric
* test: update tests
* test: fix spend tests
* test: fix test
* fix(pagerduty.py): fix linting error
* update team info endpoint
* clean up model alias
* fix model alias
* fix model alias card
* clean up naming on docs
* fix model alias card
* fix _model_in_team_aliases
* team alias - fix litellm.model_alias_map
* fix _update_model_if_team_alias_exists
* fix test_aview_spend_per_user
* Test model alias functionality with teams:
* complete e2e test
* test_update_model_if_team_alias_exists
* update team info endpoint
* clean up model alias
* fix model alias
* fix model alias card
* clean up naming on docs
* fix model alias card
* fix _model_in_team_aliases
* fix key_model_access_denied
* test_can_key_call_model_with_aliases
* fix test_aview_spend_per_user
* feat(handle_jwt.py): initial commit to allow scope based model access
* feat(handle_jwt.py): allow model access based on token scopes
allow admin to control model access from IDP
* test(test_jwt.py): add unit testing for scope based model access
* docs(token_auth.md): add scope based model access to docs
* docs(token_auth.md): update docs
* docs(token_auth.md): update docs
* build: add gemini commercial rate limits
* fix: fix linting error
* feat(proxy/_types.py): add new jwt field params
allows users + services to auth into proxy
* feat(handle_jwt.py): allow team role proxy access
allows proxy admin to set allowed team roles
* fix(proxy/_types.py): add 'routes' to role based permissions
allow proxy admin to restrict what routes a team can access easily
* feat(handle_jwt.py): support more flexible role based route access
v2 on role based 'allowed_routes'
* test(test_jwt.py): add unit test for rbac for proxy routes
* feat(handle_jwt.py): ensure cost tracking always works for any jwt request with `enforce_rbac=True`
* docs(token_auth.md): add documentation on controlling model access via OIDC Roles
* test: increase time delay before retrying
* test: handle model overloaded for test
* fix(key_management_endpoints.py): fix vulnerability where a user could update another user's keys
Resolves https://github.com/BerriAI/litellm/issues/8031
* test(key_management_endpoints.py): return consistent 403 forbidden error when modifying key that doesn't belong to user
* fix(internal_user_endpoints.py): return model max budget in internal user create response
Fixes https://github.com/BerriAI/litellm/issues/7047
* test: fix test
* test: update test to handle gemini token counter change
* fix(factory.py): fix bedrock http:// handling
* docs: fix typo in lm_studio.md (#8222)
* test: fix testing
* test: fix test
---------
Co-authored-by: foreign-sub <51928805+foreign-sub@users.noreply.github.com>
* test: add more unit testing for team member add
* fix(team_endpoints.py): add validation check to prevent same user from being added to team again
prevents duplicates
* fix(team_endpoints.py): raise error if `/team/member_delete` called on member that's not in team
prevent being able to call delete on same member multiple times
* test: update initial tests
* test: fix test
* test: update test to handle no member duplication
* build(schema.prisma): add new `sso_user_id` to LiteLLM_UserTable
easier way to store sso id for existing user
Allows existing user added to team, to login via SSO
* test(test_auth_checks.py): add unit testing for fuzzy user object get
* fix(handle_jwt.py): fix merge conflicts
* docs(token_auth.md): clarify title
* refactor(handle_jwt.py): add jwt auth manager + refactor to handle groups
allows user to call model if user belongs to group with model access
* refactor(handle_jwt.py): refactor to first check if service call then check user call
* feat(handle_jwt.py): new `enforce_team_access` param
only allows user to call model if a team they belong to has model access
allows controlling user model access by team
* fix(handle_jwt.py): fix error string, remove unecessary param
* docs(token_auth.md): add controlling model access for jwt tokens via teams to docs
* test: fix tests post refactor
* fix: fix linting errors
* fix: fix linting error
* test: fix import error
* refactor _add_callbacks_from_db_config
* fix check for _custom_logger_exists_in_litellm_callbacks
* move loc of test utils
* run ci/cd again
* test_add_custom_logger_callback_to_specific_event_with_duplicates_callbacks
* fix _custom_logger_class_exists_in_success_callbacks
* unit testing for test_add_callbacks_from_db_config
* test_custom_logger_exists_in_callbacks_individual_functions
* fix config.yml
* fix test test_stream_chunk_builder_openai_audio_output_usage - use direct dict comparison
* docs(reliability.md): add doc on disabling fallbacks per request
* feat(litellm_pre_call_utils.py): support reading request timeout from request headers - new `x-litellm-timeout` param
Allows setting dynamic model timeouts from vercel's AI sdk
* test(test_proxy_server.py): add simple unit test for reading request timeout
* test(test_fallbacks.py): add e2e test to confirm timeout passed in request headers is correctly read
* feat(main.py): support passing metadata to openai in preview
Resolves https://github.com/BerriAI/litellm/issues/6022#issuecomment-2616119371
* fix(main.py): fix passing openai metadata
* docs(request_headers.md): document new request headers
* build: Merge branch 'main' into litellm_dev_01_27_2025_p3
* test: loosen test
* feat(handle_jwt.py): initial commit adding custom RBAC support on jwt auth
allows admin to define user role field and allowed roles which map to 'internal_user' on litellm
* fix(auth_checks.py): ensure user allowed to access model, when calling via personal keys
Fixes https://github.com/BerriAI/litellm/issues/8029
* feat(handle_jwt.py): support role based access with model permission control on proxy
Allows admin to just grant users roles on IDP (e.g. Azure AD/Keycloak) and user can immediately start calling models
* docs(rbac): add docs on rbac for model access control
make it clear how admin can use roles to control model access on proxy
* fix: fix linting errors
* test(test_user_api_key_auth.py): add unit testing to ensure rbac role is correctly enforced
* test(test_user_api_key_auth.py): add more testing
* test(test_users.py): add unit testing to ensure user model access is always checked for new keys
Resolves https://github.com/BerriAI/litellm/issues/8029
* test: fix unit test
* fix(dot_notation_indexing.py): fix typing to work with python 3.8
* fix(http_handler.py): support passing ssl verify dynamically and using the correct httpx client based on passed ssl verify param
Fixes https://github.com/BerriAI/litellm/issues/6499
* feat(llm_http_handler.py): support passing `ssl_verify=False` dynamically in call args
Closes https://github.com/BerriAI/litellm/issues/6499
* fix(proxy/utils.py): prevent bad logs from breaking all cost tracking + reset list regardless of success/failure
prevents malformed logs from causing all spend tracking to break since they're constantly retried
* test(test_proxy_utils.py): add test to ensure bad log is dropped
* test(test_proxy_utils.py): ensure in-memory spend logs reset after bad log error
* test(test_user_api_key_auth.py): add unit test to ensure end user id as str works
* fix(auth_utils.py): ensure extracted end user id is always a str
prevents db cost tracking errors
* test(test_auth_utils.py): ensure get end user id from request body always returns a string
* test: update tests
* test: skip bedrock test- behaviour now supported
* test: fix testing
* refactor(spend_tracking_utils.py): reduce size of get_logging_payload
* test: fix test
* bump: version 1.59.4 → 1.59.5
* Revert "bump: version 1.59.4 → 1.59.5"
This reverts commit 1182b46b2ed814064f55f438c11b590cd7248596.
* fix(utils.py): fix spend logs retry logic
* fix(spend_tracking_utils.py): fix get tags
* fix(spend_tracking_utils.py): fix end user id spend tracking on pass-through endpoints
* feat(main.py): add new 'provider_specific_header' param
allows passing extra header for specific provider
* fix(litellm_pre_call_utils.py): add unit test for pre call utils
* test(test_bedrock_completion.py): skip test now that bedrock supports this
* fix(utils.py): don't pass 'anthropic-beta' header to vertex - will cause request to fail
* fix(utils.py): add flag to allow user to disable filtering invalid headers
ensure user can control behaviour
* style(utils.py): cleanup message
* test(test_utils.py): add unit test to cover invalid header filtering
* fix(proxy_server.py): fix custom openapi schema generation
* fix(utils.py): pass extra headers if set
* fix(main.py): fix image variation to use 'client' param
* fix(user_dashboard.tsx): fix spend calculation when team selected
sum all team keys, not user keys
* docs(admin_ui_sso.md): fix docs tabbing
* feat(user_api_key_auth.py): introduce new 'enforce_rbac' param on jwt auth
allows proxy admin to prevent any unmapped yet authenticated jwt tokens from calling proxy
Fixes https://github.com/BerriAI/litellm/issues/6793
* test: more unit testing + refactoring
* fix: fix returning id when obj not found in db
* fix(user_api_key_auth.py): add end user id tracking from jwt auth
* docs(token_auth.md): add doc on rbac with JWTs
* fix: fix unused params
* test: remove old test
* test: initial test to enforce all functions in user_api_key_auth.py have direct testing
* test(test_user_api_key_auth.py): add is_allowed_route unit test
* test(test_user_api_key_auth.py): add more tests
* test(test_user_api_key_auth.py): add complete testing coverage for all functions in `user_api_key_auth.py`
* test(test_db_schema_changes.py): add a unit test to ensure all db schema changes are backwards compatible
gives user an easy rollback path
* test: fix schema compatibility test filepath
* test: fix test
* fix(gemini/): support gemini 'frequency_penalty' and 'presence_penalty'
Closes https://github.com/BerriAI/litellm/issues/7748
* feat(proxy_server.py): new env var to disable prisma health check on startup
* test: fix test
* fix(gpt_transformation.py): fix response_format translation check for 4o models
Fixes https://github.com/BerriAI/litellm/issues/7616
* feat(key_management_endpoints.py): support 'temp_budget_increase' and 'temp_budget_expiry' fields
Allow proxy admin to grant temporary budget increases to keys
* fix(proxy/_types.py): enforce temp_budget_increase and temp_budget_expiry are always passed together
* feat(user_api_key_auth.py): initial working temp budget increase logic
ensures key budget exceeded error checks for temp budget in key metadata
* feat(proxy_server.py): return the key max budget and key spend in the response headers
Allows clientside user to know their remaining limits
* test: add unit testing for new proxy utils
Ensures new key budget is correctly handled
* docs(temporary_budget_increase.md): add doc on temporary budget increase
* fix(utils.py): remove 3.5 from response_format check for now
not all azure 3.5 models support response_format
* fix(user_api_key_auth.py): return valid user api key auth object on all paths
* build(model_prices_and_context_window.json): add azure o1 pricing
Closes https://github.com/BerriAI/litellm/issues/7712
* refactor: replace regex with string method for whitespace check in stop-sequences handling (#7713)
* Allows overriding keep_alive time in ollama (#7079)
* Allows overriding keep_alive time in ollama
* Also adds to ollama_chat
* Adds some info on the docs about this parameter
* fix: together ai warning (#7688)
Co-authored-by: Carl Senze <carl.senze@aleph-alpha.com>
* fix(proxy_server.py): handle config containing thread locked objects when using get_config_state
* fix(proxy_server.py): add exception to debug
* build(model_prices_and_context_window.json): update 'supports_vision' for azure o1
---------
Co-authored-by: Wolfram Ravenwolf <52386626+WolframRavenwolf@users.noreply.github.com>
Co-authored-by: Regis David Souza Mesquita <github@rdsm.dev>
Co-authored-by: Carl <45709281+capsenz@users.noreply.github.com>
Co-authored-by: Carl Senze <carl.senze@aleph-alpha.com>
* feat(ui_sso.py): support reading team ids from sso token
* feat(ui_sso.py): working upsert sso user teams membership in litellm - if team exists
Adds user to relevant teams, if user is part of teams and team exists on litellm
* fix(ui_sso.py): safely handle add team member task
* build(ui/): support setting team id when creating team on UI
* build(ui/): teams.tsx
allow setting team id on ui
* build(circle_ci/requirements.txt): add fastapi-sso to ci/cd testing
* fix: fix linting errors
Ishaan Jaff