Commit Graph
22331 Commits
Author SHA1 Message Date
Ishaan Jaff 75dbb86d34 fix: use handle exception on proxy v1.72.0.dev1 2025-06-02 13:58:22 -07:00
Ishaan JaffandGitHub f247f1031b [Fix] Fix SCIM running patch operation case sensitivity (#11335)
* fix: fix SCIM patch op

* test: test SCIM patch op
2025-06-02 13:53:26 -07:00
Cole McIntoshandGitHub 702e399d94 Add presidio_language yaml configuration support for guardrails (#11331)
* feat: Add presidio_language parameter for PII analysis configuration via yaml config files

- Introduced presidio_language parameter in guardrail_initializers and guardrail_hooks to allow language specification for Presidio PII analysis via yaml config files
- Updated PresidioPresidioConfigModelUserInterface to include presidio_language with a default value of "en".
- Added tests to verify language configuration functionality, ensuring correct language usage in analyze requests.

* docs: Enhance PII masking documentation with language support details

- Added information on configuring the default language for PII analysis using the `presidio_language` parameter in both the UI and YAML configuration.
- Included supported language codes (English, Spanish, German) and their precedence in language settings.
- Provided examples for default language configuration and per-request language overrides to clarify usage.
2025-06-02 13:13:17 -07:00
Cole McIntoshandGitHub 94650c10fe feat: Add support for Cohere Embed v4.0 model (#11329)
- Updated model_prices_and_context_window.json to include embed-v4.0 with relevant pricing and metadata.
- Added embed-v4.0 to cohere_embedding_models in constants.py.
- Implemented comprehensive tests for Cohere Embed v4.0 in test_cohere.py, covering basic functionality, input types, error handling, and optional parameters.
2025-06-02 11:25:29 -07:00
Tyler WagnerandGitHub 63d7f99e27 fix: remove dupe server_id MCP config servers (#11327) 2025-06-02 09:04:55 -07:00
Matt NitzkenandGitHub dd17661b13 Add support for DataRobot as a provider in LiteLLM (#10385)
* Add support for DataRobot as a router in LiteLLM

* Updates to logic

* Changes to make things work better

* Capitalize bearer

* Revert change

* Undo and simplify things

* Add basic testing

* Add some extra handling

* More tests

* Lowercase

* Fix

* Comment

* Add local test_completion for datarobot

* Sync repo to main

* Update get_complete_url to accept deployments

* Migrate to OpenAILike

* Mock datarobot in test

* Migrate everything to OpenAI LIke

* Apply fixes and changes from review

* Update docs

* Update env vars

* Move tests
2025-06-02 08:37:21 -07:00
tanjiroandGitHub 7155e157da converted action buttons to sticky footer action buttons (#11293) 2025-06-02 07:50:12 -07:00
Krrish Dholakia 9630386f2b docs: add release candidate notice 2025-06-01 22:39:57 -07:00
Krish DholakiaandGitHub 83becdbc11 Litellm doc fixes 05 31 2025 (#11305)
* docs: cleanup

* docs: add anthropic file tutorial

* docs: add to sidebar
2025-06-01 00:53:56 -07:00
Ishaan Jaff bdfa24be23 update doc v1.72.0.rc 2025-05-31 20:57:48 -07:00
Krish DholakiaandGitHub a40b81cd6b Rate Limiting: Check all slots on redis, Reduce number of cache writes (#11299)
* fix(base_routing_strategy.py): compress increments to redis - reduces write ops

* fix(base_routing_strategy.py): make get and reset in memory keys atomic

* fix(base_routing_strategy.py): don't reset keys - causes discrepency on subsequent requests to instance

* fix(parallel_request_limiter.py): retrieve values of previous slots from cache

more accurate rate limiting with sliding window

* fix: fix test

* fix: fix linting error
v1.72.0.rc
2025-05-31 18:32:13 -07:00
Ishaan Jaff 10fa45d987 docs fix 2025-05-31 16:29:19 -07:00
AyrennCandGitHub 8ae79178ae feat: Add audio parameter support to gemini tts models (#11287)
* feat: Add Gemini TTS audio parameter support

- Add is_model_gemini_audio_model() method to detect TTS models
- Include 'audio' parameter in supported params for TTS models
- Map OpenAI audio parameter to Gemini speechConfig format
- Add _extract_audio_response_from_parts() method to transform audio
  output to openai format

* updated unit-test to use pcm16

* - created typedict for speechconfig
- simplified gemini tts model detection
- moved gemini_tts test to test_litellm

* simplified is_model_gemini_audio_model more
v1.72.0-nightly
2025-05-31 16:20:19 -07:00
Ishaan Jaff 13dc757873 bump: version 1.71.3 → 1.72.0 2025-05-31 15:54:01 -07:00
Ishaan Jaff 3f616423a4 docs fixes 2025-05-31 15:30:53 -07:00
Ishaan Jaff ab2f066df8 docs prometheus 2025-05-31 14:26:42 -07:00
06484f6e5a Xai, VertexAI, Google AI Studio - live web search support in OpenAI format (#11251)
* build(model_prices_and_context_window.json): fix 'supports_web_search' flag - openai only supports it on 2 models - gpt-4o-search-preview and gpt-4o-mini-search-preview

* feat(xai/chat): add xai web search options param support

* test: add max tokens to test

xai output very verbose

* build(xai/): add web search support for all xai models

* build(model_prices_and_cost.json): add gemini-2.0 supports web search

* feat(gemini/): map openai 'web_search_options' to google's 'googlesearch' tool

* build(model_prices_and_context_window.json): add supports_web_search for vertex_ai/gemini-2 models

* fix: fix circular reference error

* fix(convert_dict_to_response.py): handle scenario where xai returns finish reason as 'stop' for tool calls

* fix: reduce function size

* fix: import session handling

* Revert "fix: import session handling"

This reverts commit deb257dc10.

* fix: linting pin mypy

* [Feat]: Guardrails - Add streaming for bedrock post guard (#11247)

* feat: add streaming for bedrock post guard

* fix: bedrock guardrails

* fix: add clear comments

* Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: clean up bedrock guardrails

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* [Fix] Responses API - Session management  (#11254)

* fix: import session handling

* fix: imports for session handler

* tests: tests for session handler

* Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* bump: bump litellm enterprise

* fixes: test_create_user_default_budget

* fix(xai/): filter 'strict' on tool call

* test: update test for new error string

* fix(utils.py): default to None if not set in  model cost map

ensures consistent usage of 'supports_[x]' flags

* fix(fireworks_ai/): support fireworks ai document inlining on pdf's sent via openai 'file' message type

* test: update test

* test: name filter_value_from_dict

* fix(fireworks_ai/): handle cache control flag in messages

* fix(xai/chat): fix check

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-31 14:26:16 -07:00
Ishaan Jaff cdfb6b8c37 docs prometheus end user tracking 2025-05-31 14:21:37 -07:00
Ishaan JaffandGitHub 170af8f2c8 [Docs] 1.72.0-stable release note (#11295)
* draft 1.72.0 stable

* docs - note on aiohttp transport

* docs - section for guardrails

* clean up key highlights

* docs aiohttp transport

* docs cleanup

* docs organize logging/guardrail section

* docs logging+guardrails

* docs add prometheus note

* docs fixes release note

* docs 1.72.0-stable

* docs vector store permissions
2025-05-31 14:15:16 -07:00
Ishaan Jaff 16efb8db67 Revert "Make gemini stream thinking as reasoning_content (#11290)"
This reverts commit e0daa3da68.
2025-05-31 13:29:51 -07:00
Ishaan Jaff 6863073aa4 fix: tests 2025-05-31 13:14:37 -07:00
Ishaan Jaff 3b930f6736 fix: aiohttp handle transfer encoding errors gracefully 2025-05-31 13:02:07 -07:00
Ishaan Jaff 7d47417906 test: fixes 2025-05-31 12:42:56 -07:00
Ishaan Jaff 236975a742 fix: allow users to disable aiohttp transport 2025-05-31 12:32:24 -07:00
Ishaan Jaff e011167317 docs DISABLE_AIOHTTP_TRANSPORT 2025-05-31 12:30:52 -07:00
Ishaan Jaff f95754c67f (UI) new build 2025-05-31 12:25:29 -07:00
Ishaan Jaff ebf05c10a9 (ui) fix view 2025-05-31 12:21:40 -07:00
Ishaan Jaff 0dca4780c5 ui - fix permission checks 2025-05-31 12:19:37 -07:00
Ishaan Jaff b0f2d969e7 (ui) fix passing premium user 2025-05-31 12:15:56 -07:00
Ishaan Jaff 7b4fb48bd1 ui new build 2025-05-31 12:08:48 -07:00
Ishaan Jaff 3be42fd744 ui fixes 2025-05-31 12:08:09 -07:00
Ishaan Jaff 75f87724bd chore - vector store permissions enterprise 2025-05-31 12:01:51 -07:00
Ishaan Jaff 372de1476b (chore): mark object permissions as enterprise 2025-05-31 11:52:59 -07:00
Krish DholakiaandGitHub 39849627f7 feat(parallel_request_limiter_v2.py): add sliding window logic (#11283)
* feat(parallel_request_limiter_v2.py): add sliding window logic

allows rate limiting to work across minutes

* fix(parallel_request_limiter_v2.py): decrement usage on rate limit error

* fix(base_routing_strategy.py): fix merge from redis - preserve values in in-memory cache during gap b/w push to redis and read from redis

* fix(base_routing_strategy.py): catch the delta change during redis sync

ensures values are kept in sync

* fix(parallel_request_limiter_v2.py): update tpm tracking to use slot key logic

* fix: fix linting error

* test: update testing

* test: update tests

* test: skip on rate limit or internal server errors

* test: use pytest fixture instead

* test: bump mistral model
v1.71.3-rc
2025-05-31 10:06:42 -07:00
Ishaan Jaff 1a05f8d9e2 UI QA fixes 2025-05-31 09:41:22 -07:00
Ishaan JaffandGitHub 68fd17d15e [Fix] QA Fixes - Vector Store Object Permissions (#11291)
* fix: QA for key,team,org permissions

* fix: add_vector_store_to_registry

* fix: refactor bedrock guard

* fix: refactor using us east 1 with vector stores

* fix: code QA checks

* fix: testing for mgmt endpoints
2025-05-31 09:41:05 -07:00
Adam HolmbergandGitHub e0daa3da68 Make gemini stream thinking as reasoning_content (#11290)
When "Thought": True, return text as reasoning_content instead of
content.

fixes #10563
fixes #11000
2025-05-31 09:13:00 -07:00
Krrish Dholakia 51f716c762 build(VLLM-Passthrough-with-loadbalancing-support-(enables-using-model-list-for-VLLM-/classify-endpoint)): Closes #11205 2025-05-31 09:00:04 -07:00
மனோஜ்குமார் பழனிச்சாமிandGitHub 0fd4ee2f94 Increase timeout (#11288) 2025-05-31 07:31:14 -07:00
Bryan LowandGitHub d77b825814 Swap Cohere and Cohere Chat provider (#11173)
* fix cohere rerank provider

* swap cohere and cohere chat
2025-05-31 01:20:37 -07:00
Shuai ZhangandGitHub 712e042aa4 fix(secret-managers): Break AzureCredentialType restriction on AZURE_CREDENTIAL (#11272) 2025-05-31 01:03:08 -07:00
VigneshwarRajasekaranandGitHub 9df61ef08b Wrong parameter mapping for "frequency_penalty" to "repeat_penalty" in Ollama/completion/transformation.py (#11284) 2025-05-31 00:58:19 -07:00
Ishaan Jaff 15ea80d2cf ui new build v1.71.3-nightly 2025-05-30 23:08:07 -07:00
Ishaan Jaff c5873c6f1f UI QA fixes/cleanup 2025-05-30 23:06:45 -07:00
Ishaan Jaff 310d97c982 fix: fix linting error 2025-05-30 22:51:10 -07:00
Ishaan JaffandGitHub d7f19bbfe3 [Bug]: Performance Fix Max langfuse clients reached: 20 is greater than 20 (#11285)
* fix: initializing langfuse clients

* fix: initializing langfuse clients

* tests: tests for langfuse cache
2025-05-30 22:34:39 -07:00
7e49b4e2a0 [Feat] Enforce Vector Store Access Controls on LiteLLM Auth (#11281)
* fix LiteLLM_ObjectPermissionTable

* fix include object_permission for list key

* fix key list to inclue obj permissions

* fix object permissions for vector stores on key info

* add key edit view with vector stores

* allow editing vector stores permissions

* fixes obj permissions

* feat: add obj permission on UI

* fix: add object_permission:true

* ui show org vector stores on org info

* fix: show object permissions on /org/info

* feat: allow updating obj permissions for keys

* fixes: key object permissions

* fixes: team object permissions

* fixes: org object permissions

* fix vector store selector for Orgs

* feat: add auth checks for vector store permissions

* feat: working auth checks for vector store permissions

* test vector stores auth checks

* Update litellm/proxy/_types.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: linting

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-30 22:20:11 -07:00
Ishaan JaffandGitHub 2f8eb1dcc3 fix: dont require mcp pip for proxy (#11282) 2025-05-30 22:19:53 -07:00
Isaac WhitfieldandGitHub c698cc24aa Streamline session expiration in UI (#11279) 2025-05-30 17:26:23 -07:00
Ishaan JaffandGitHub a9d23f5d4a [Fix] Bedrock Guard Auth Param Persistence (#11270)
* fix: using bedrock guard

* fix: bedrock guard param persistence
2025-05-30 17:24:03 -07:00