* fix(key_management_endpoints.py): add new `/key/service_account/generate` endpoint
enables controlling team member permissions for creating service account keys
* refactor(ui/): use new /key/service-account/generate endpoint when creating keys on UI
enables team member permissions for service accounts
* fix(permission_definitions.tsx): add permission for creating service accounts on UI
* fix(key_management_endpoints.py): ignore user id for service account keys
* fix(key_management_endpoints.py): improve docstring
* docs(service_accounts.md): update docs
* refactor(aim.py): refactor to support adding aim guardrails on UI
* fix(base.py): add ui_friendly_name to config model
* feat(ui/): support loading new guardrails from backend api call
removes need to onboard each guardrail to ui
* fix: don't show optional params if not set and don't show ui_friendly_name (internal param0
* fix(ui/add_guardrail_form.tsx): ensure dynamic provider value is used
* fix(ui/): just one-time update the provider map dictionary
* fix(ui/): show masked api base / api key on guardrail update
* refactor(aporia_ai/): refactor to show on UI
* feat(aporia_ai/): add aporia ai guardrail to UI
* refactor(guardrails_ai/): refactor to add via UI
* refactor(lasso.py): refactor to enable adding lasso guardrails via UI
* feat(pangea.py): add pangea guardrail on UI
* feat(panw): add panw prisma airs through UI
* test: update tests
* fix: fix ruff linting error
* test: update tests
* fix: add missing docs
* fix: fix guardrail init
* fix: suppress linting errors
* fix(proxy_server.py): fix linting error
* fix(streaming_handler.py): don't return stream options in clientside response
* test: add unit test
* fix(custom_guardrail.py): fix start time
don't start until function actually called
leading to incorrect high latency numbers
* fix(custom_guardrail.py): fix start time
Fixes#9526
The embedding() function can return either an EmbeddingResponse or a
Coroutine[Any, Any, EmbeddingResponse] depending on whether aembedding=True
is passed. This adds @overload decorators to properly indicate the return
type based on the aembedding parameter.
This resolves type checking errors when using embedding() with aembedding=True
where the type checker couldn't determine that a coroutine was being returned.
* build(model_prices_and_context_window.json): remove 'supports_tool_choice' for specific mistral models
Closes https://github.com/BerriAI/litellm/issues/11750
* feat: initial commit adding cleaner ui for azure text moderation guardrails
* feat(guardrail_endpoints.py): add discoverable guardrail configs and improve converting base model to dict with types
* fix(guardrail_provider_fields.tsx): render from api endpoint correctly
* fix(guardrail_provider_fields.tsx): cleanup
* refactor(guardrail_endpoints.py): refactor to handle dictionaries with literal - allows multiselect
* feat(ui/): render dictionary with known keys correctly
* feat(ui/): render optional params on separate page
* style(ui/): style improvements to rendering optional params on the UI
* feat(azure/prompt_shield.py): add azure prompt shield back on UI
* fix(add_guardrail_form.tsx): fix form to handle updated api
* fix(guardrail_optional_params.tsx): ensure values are nested correctly for writing to api
* fix: fix linting error
* feat(text_moderation.py): handle str to int conversion
* fix(guardrail_info.tsx): only render pii settings if guardrail is presidio
* fix(guardrail_info.tsx): add guardrail specific fields to update settings
allows updating guardrail fields (e.g. severity threshold) post-create
* fix(guardrail_endpoints.py): set guardrail_id in guardrail object
ensures duplicate objects not created on guardrail update
* fix(guardrail_endpoints.py): allow provider specific fields to be updated on patch update
* refactor(guardrail_endpoints.py): remove duplicate info endpoint
* fix(guardrail_endpoints.py): mask sensitive keys on returning via guardrail `/info`
Prevent leaking keys
* fix(guardrail_optional_params.tsx): return numerical input when numerical component used
fixes issue where output was a str
* fix(guardrail_optional_params.tsx): render dict keys correctly
* fix(text_moderation.py): fix severity by category check
* fix(proxy/utils.py): check if guardrail should run for post call streaming hook
Prevents invalid guardrails from running if not requested
* test: fix import
* fix: fix linting error
* test: update test
* fix: fix tests
* fix: fix code qa errors
* fix(guardrail_endpoints.py): set max depth for function
* test: update recursive_detector.py
* test: update list
* build: merge main
* fix: fix ruff check errors
* build(model_prices_and_context_window.json): remove 'supports_tool_choice' for specific mistral models
Closes https://github.com/BerriAI/litellm/issues/11750
* feat: initial commit adding cleaner ui for azure text moderation guardrails
* feat(guardrail_endpoints.py): add discoverable guardrail configs and improve converting base model to dict with types
* fix(guardrail_provider_fields.tsx): render from api endpoint correctly
* fix(guardrail_provider_fields.tsx): cleanup
* refactor(guardrail_endpoints.py): refactor to handle dictionaries with literal - allows multiselect
* feat(ui/): render dictionary with known keys correctly
* feat(ui/): render optional params on separate page
* style(ui/): style improvements to rendering optional params on the UI
* feat(azure/prompt_shield.py): add azure prompt shield back on UI
* fix(add_guardrail_form.tsx): fix form to handle updated api
* fix(guardrail_optional_params.tsx): ensure values are nested correctly for writing to api
* fix: fix linting error
* test: update tests
* fix mapped tests (#12320)
* fix - use flush llm client cache
* faster mapped tests
* test_async_multiple_response_ids_routing
* fix tests
* test_ateam_member_update_admin_requires_premium
* regular mapped tests
* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206)"
This reverts commit 2c60c316ec.
* reset num workers
* check_for_litellm_module_deletion
* add test
* add code_qa_check_tests
* ci/cd new release
* docs(opentelemetry_integration.md): add otel debug flag to docs
help people debug their traces easily
* [Feat] Add failure logging support for s3 logger (#12299)
* add async_log_failure_event
* test_basic_s3_v2_logging_failure
* Comma separated spend and budget (#12317)
* all_keys_table, key_info_view, organizations
* teams
* the whole usage page
* teams page
* users table
* user info
* more commas
* some more commas
* Segregate MCP tools on connections using headers (#12296)
* Add get tools segregation
* add ui changes (#12302)
* resolve comments
* add mapped tests
* remove advanced settings (#12323)
* [Bug Fix] /generateContent API - Only pass supported params when using OpenAI models (#12297)
* fix - only pass GenericLiteLLMParams
* test_google_generate_content_with_openai
* Fix: Fix custom ca bundle support in aiohttp transport (#12281)
* Unify usage of get_ssl_configuration
* Fix doc
* [Feat] Add github co-pilot as a new LLM API provider (#12325)
* Litellm dev 03 05 2025 contributor prs (#9079)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
* Fix#7629 - Add tzdata package to Dockerfile (#8915)
* Add tzdata package to Dockerfile
* Move tzdata to python requirement.txt
* feat: add support for copilot provider (#8577)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat: add model information for copilot models
* fix: fix linting errors
* test: remove integration test for github_copilot + fix misisng mock
* fix: use print to make sure the logger message shown
* test: remove debug print
* fix lint (#11112)
* Add init files to make test directories Python packages and update import paths in test_token_counter.py (#11119)
* Update litellm/model_prices_and_context_window_backup.json
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
---------
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* refactor github copilot
* test_github_copilot_transformation.py
* test_github_copilot_authenticator.py
* add GitHub Copilot
* fix order
* doc fix
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
* add new env vars to docs
* ssl_cert_error (#12327)
* feat - add new banner on startup (#12328)
* [Security Bug Fix] Ensure only LLM API route fails get logged on Langfuse (and other loggers) (#12308)
* _is_proxy_only_llm_api_error
* test_proxy_only_error_true_for_llm_route
* add not on change
* Update tests/test_litellm/proxy/test_proxy_utils.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add test_post_call_failure_hook_auth_error_key_info_route
* test fix _is_proxy_only_llm_api_error
* test_chat_completion_request_with_redaction
* test_post_call_failure_hook_auth_error_llm_api_route
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Ensure message redaction works for responses API logging (#12291)
* add fixes to choice implementation redaction
* add isInstance check on responses API
* change datadog to revert back
* change datadog to revert back
* fix type errors
* Redaction test changes
* Redaction test changes
* Redaction test changes - remove changes
* [Feat] UI - Allow Viewing/Editing Team Based Callbacks (#12329)
* add logging settings view on UI
* fix change ordering
* fix type error
* test fix - import loc
* [Feat] QA - Arize Team based logging (#12331)
* add _get_tracer_with_dynamic_headers
* fix construct_dynamic_arize_headers
* [Feat] UI - Allow Viewing/Editing Team Based Callbacks (#12329)
* add logging settings view on UI
* fix change ordering
* add construct_dynamic_otel_headers for arize
* refactor common code
* test_construct_dynamic_arize_headers
* otel unit tests
* test_arize_dynamic_params
* test_arize_dynamic_headers_in_grpc_requests
* [Feat] UI - Allow setting Logging Callback Setting per Key (#12333)
* allow adding logging settings on a key
* add ability to edit key logging settings
* dont leak api keys on ui
* fix order
* add langsmith settings
* ui new build
* test_default_api_base
* fix new utils tests
* bump: version 1.73.7 → 1.74.0
* Add mcp server segregation comma separated support (#12326)
* add mcp comma separated support
* dont support legacy json array
* lint
* add live tail state to session storage (#12335)
* add aiml
* fix: fix unused imports
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
Co-authored-by: Jugal D. Bhatt <55304795+jugaldb@users.noreply.github.com>
Co-authored-by: Joost van Doorn <joost.van.doorn@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add watsonx/mistralai/mistral-large to model prices and context window json
- Set max_tokens and max_input_tokens to 131072 and max_output_tokens to 16384 to match IBM's official context window limit for Mistral Large models hosted on watsonx.
- Correct input_cost_per_token and output_cost_per_token to reflect IBM's pricing of $0.003 and $0.01 per 1000 tokens, respectively, converted to per-token rates (0.000003 and 0.00001).
- Confirmed function-calling support based on IBM documentation and Mistral's own API schema.
- Aligns with Mistral's own model entries that specify token windows of ~128K and matches LiteLLM's expected cost format.
**Sources:**
- IBM watsonx Foundation Models docs: https://www.ibm.com/docs/en/watsonx/w-and-w/2.1.x?topic=models-foundation-model-details#mistral-large
- IBM Pricing information for watsonx.ai models: https://www.ibm.com/products/watsonx-ai/pricing
- LiteLLM repo docs on token cost formatting and limits https://docs.litellm.ai/docs/proxy/custom_pricing.
- Imitating entries from litellm/model_prices_and_context_window.json for watsonx/ibm/granite-3-8b-instruct as well as Mistral's own mistral/mistral-large-latest and similar.
* Update model_prices_and_context_window.json
typo
* Update model_prices_and_context_window.json
* Update model_prices_and_context_window.json
Got ValueError on Python3.10 because datetime.fromisoformat could not handle 'Z' -> '+00:00' at the end of a timestamp. Instead, try with the previous behavior, except ValueError, then attempt removing Z, and adding +00:00 to support Python3.10
* add JWTLiteLLMRoleMap
* test_sync_user_role_and_teams
* add sync_user_role_and_teams
* test_sync_user_role_and_teams
* fix types
* Sync User Roles and Teams with IDP
* Add test for JWT role mapping to LiteLLM roles
* add fixes to choice implementation redaction
* add isInstance check on responses API
* change datadog to revert back
* change datadog to revert back
* fix type errors
* Redaction test changes
* Redaction test changes
* Redaction test changes - remove changes
* Litellm dev 03 05 2025 contributor prs (#9079)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
* Fix#7629 - Add tzdata package to Dockerfile (#8915)
* Add tzdata package to Dockerfile
* Move tzdata to python requirement.txt
* feat: add support for copilot provider (#8577)
* feat: add support for copilot provider
* test: add tests for github copilot
* chore: clean up github copilot authenticator
* test: add test for github copilot authenticator
* test: add test for github copilot for sonnet 3.7 thought model
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* feat: add model information for copilot models
* fix: fix linting errors
* test: remove integration test for github_copilot + fix misisng mock
* fix: use print to make sure the logger message shown
* test: remove debug print
* fix lint (#11112)
* Add init files to make test directories Python packages and update import paths in test_token_counter.py (#11119)
* Update litellm/model_prices_and_context_window_backup.json
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
---------
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
* refactor github copilot
* test_github_copilot_transformation.py
* test_github_copilot_authenticator.py
* add GitHub Copilot
* fix order
* doc fix
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com>
Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com>
Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com>
Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
