Commit Graph
26757 Commits
Author SHA1 Message Date
Ishaan Jaffer 778e10119c test_azure_img_gen_health_check 2025-10-25 10:27:10 -07:00
Ishaan Jaffer caa7da98b7 TestAzureAIOCR 2025-10-25 10:26:41 -07:00
Ishaan Jaffer 0bedf1c0a7 fix tests 2025-10-25 10:19:24 -07:00
Ishaan Jaffer 9ef54ab686 ci/cd run again 2025-10-25 10:09:47 -07:00
Ishaan JaffandGitHub e4d5f00990 [Feat] New Guardrail - Dynamo AI Guardrail (#15920)
* add dynamo types

* fix Dynamo guard

* add dynamo guardrail

* add dynamo ai docs guard

* docs fix

* test dynamo

* test LASSO
2025-10-24 17:11:04 -07:00
Otavio BritoandGitHub 68b8b66b2f update vertex ai gemini costs (#15911) 2025-10-24 17:01:56 -07:00
Ishaan Jaffer a07ed76f54 fix linting error 2025-10-24 14:45:38 -07:00
Ishaan Jaffer 6dee76c580 UI fix linting errors 2025-10-24 14:36:55 -07:00
ShadiandGitHub 8a5ff84e49 fixed lasso import config, redis cluster hash tags for test keys (#15917) 2025-10-24 14:31:59 -07:00
superpoussin22andGitHub bd76d86b1c Add mistral medium 3 and Codestral 2 on vertex (#15887) 2025-10-24 12:58:47 -07:00
Sam ChouandGitHub a173232f8c Fix MLFlow tags - split request_tags into (key, val) if request_tag has colon (#15914)
* Fix mlflow tags - split request_tags into (key, val) if request_tag has colon

* Redundant name: tag_dict -> tags
2025-10-24 12:57:15 -07:00
8b1424166b attempt to avoid/minimize deadlocks (#15281)
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-10-24 12:22:38 -07:00
Sameer KankuteandGitHub 0f9996a4d0 Litellm sameer oct staging (#15806)
* Addd v2/chat support for cohere

* fix streaming

* Use v2_transformation for logging passthrough:

* Use v2_transformation for logging passthrough:

* Add test for checking if document and citation_options is getting passed

* Update the cohere model

* Add cost tracking for vertex ai passthrough batch jobs

* Add full passthrough support

* refactor code according to the comments

* Add passthrough handler

* remove invalid params

* Updated documentation

* Updated documentation

* Updated documentation

* Correct the import

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix documentation for get batch req (#15742)

* Add grounding info to responses API (#15737)

* Add grounding info to responses API

* fix lint errors

* Use typed objects for annotations

* Use typed objects for annotations

* fix mypy error

* Litellm fix json serialize alreting 2 (#15741)

* fix json serializable error for alerts

* Add test

* fix mypt errors

* fix mypt errors

* Add Qwen3 imported model support for AWS Bedrock (#15783)

* Add qwen imported model support

* fix mypy errors

* fix empty user message error (#15784)

* fix typed dict for list

* Add azure supported videos endpoint

* fix mapped tests

* add azure sora models to model map

* Add OpenAI video generation and content retrieval support (#15745)

* Add openai videos generation and retrieval support

* add retrieval endpoint

* Add docs

* Add imports

* remove orjson

* remove double import

* fix openai videos format

* remove mock code

* remove not required comments

* Add tests

* Add tests

* Add other video endpoints

* Fix cost calculation and transformation

* Fixed mypy tests

* remove not used imports

* fix typed dict for list

* fix mypy errors

* move directory

* make v2 chat default

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Fix mypy tests

* Revert "Add Azure Video Generation Support with Sora Integration"

* refactor videos repo

* add test

* Add azure openai videos support

* Add azure openai videos support

* Add router endpoint support for videos

* fix mypy error

* add azure models

* fix mapped test

* fix mypy error

* Add proxy router test

* Add proxy router test

* remove deprecated model name from tests

* fix import error

* fix import error

* Add gaurdrail integration in videos endpoint

* Add logging support for videos endpoint

* Add final documentation supporting videos integration

* fix model name and document input

* Update literals to avoid mypy errors

* Remove unused imports and print statements

* revert guardrail support for video generation and video remix

* revert guardrail support for video generation and video remix

* Fix failing mapped and llm translation tests
2025-10-24 12:17:22 -07:00
oroxenbergandGitHub c793bd5ba9 Lasso Security Guardrail: Add v3 API Support (#12452)
* 1. add v3 classify
2. add new classifix for masking
3. support same id for the conversation for pre and post
working with duplicates

* clean code, remove some debug and run tests

* update liter errors

* improvment for Code Organization, httpx Error Handling Specificity, Logging Improvements and Type

* transfer test test_lasso_guard_config to the new location

* Fix type hints and linting errors in lasso.py

- Add type: ignore for httpx module when None
- Fix return type issues in _handle_classification and _handle_masking
- Ensure masked_messages is not None before passing to _apply_masking_to_model_response
- Convert LassoResponse to dict for _log_masking_applied call
2025-10-24 11:03:58 -07:00
Sameer KankuteandGitHub c638f45213 Implement Bedrock Guardrail apply_guardrail endpoint support (#15892)
* Add bedrock support for apply gaurdrails

* Add bedrock support doc

* remove unused variable

* remove unused variable
2025-10-24 10:24:03 -07:00
Sameer KankuteandGitHub b9585b1db5 Update documentation for enable_caching_on_provider_specific_optional_params (#15885) 2025-10-24 10:22:27 -07:00
Alexsander HamirandGitHub 9338727960 feat(proxy): support absolute RPM/TPM in priority_reservation (#15813)
* feat(proxy): support absolute RPM/TPM in priority_reservation

Allow priority reservations as absolute values instead of percentages:
- Float: {'prod': 0.75} (75%, existing)
- RPM: {'prod': {'type': 'rpm', 'value': 750}}
- TPM: {'prod': {'type': 'tpm', 'value': 750000}}

Added _convert_to_percent() that converts absolute values to percentages
based on model capacity. Fully backward compatible.

* feat(types): convert priority_reservation Dict to TypedDict

Add PriorityReservationDict TypedDict to replace generic Dict type in priority_reservation configuration.

Changes:
- Add PriorityReservationDict to litellm/types/utils.py
- Update convert_priority_to_percent() signature in rate_limiter_utils.py
- Update litellm.priority_reservation type annotation in __init__.py

Improves IDE autocomplete and type checking for priority reservation configs.

* docs: update dynamic rate limiter priority reservation docs
2025-10-23 18:30:36 -07:00
c5fee97850 docs: add OpenAI responses api (#15868)
* docs: add tip openai page

* added responses api

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:59 -07:00
09c1ad190e docs: add tip openai page (#15866)
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
2025-10-23 18:25:25 -07:00
Ishaan Jaffer ea8a604188 fix to .debug 2025-10-23 18:08:56 -07:00
Ishaan Jaffer f4e98f715a fix linting 2025-10-23 18:07:28 -07:00
Ishaan Jaffer 76d658eee8 fix linting 2025-10-23 18:05:30 -07:00
Ishaan JaffandGitHub 5de912375c [Feat] UI - Add logos for search providers (#15872)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db

* add UI friendly name for search providers

* add ui friendly name for search providers

* add providers available

* working layout

* better layout

* clean add search tool

* update_router_search_tools

* fix remove in memory registry, since router is in mem store

* allow testing search tool connection

* clean create search tool

* add test_search_tool_connection

* fix: _init_search_tools_in_db

* add searchToolQueryCall

* fix icon

* clean tester

* add parallel ai logo

* add exa ai logo

* add google PSE logo

* add tavily logo

* add dataforseo + perplexity

* add parallel ai logo

* UI show logos for search
2025-10-23 18:00:40 -07:00
Ishaan JaffandGitHub fc9aba279e [Feat] UI - Search Tools, allow adding search tools on UI + testing search (#15871)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db

* add UI friendly name for search providers

* add ui friendly name for search providers

* add providers available

* working layout

* better layout

* clean add search tool

* update_router_search_tools

* fix remove in memory registry, since router is in mem store

* allow testing search tool connection

* clean create search tool

* add test_search_tool_connection

* fix: _init_search_tools_in_db

* add searchToolQueryCall

* fix icon

* clean tester
2025-10-23 17:59:29 -07:00
Ishaan JaffandGitHub d8ea1665c7 [Feat] - [Backend] Search APIs - Allow storing configured Search APIs in DB (#15862)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db
2025-10-23 17:57:49 -07:00
Byron GroganandGitHub bf47c25de0 [fix] Pass user-defined headers and extra_headers to image-edit calls (#15811) 2025-10-23 15:08:10 -07:00
Krish DholakiaandGitHub e0c4baf66f fix(ui/): fix routing for custom server root path (#15701)
* fix(ui/): fix routing for custom server root path

* fix: fix eslint errors
2025-10-23 13:59:29 -07:00
Jacob LinneyandGitHub 0644c20604 fix(vertex-ai): cost tracking for search spend (#15859) 2025-10-23 13:29:10 -07:00
Ishaan JaffandGitHub 6d947d7326 [Bug Fix] Exa Search API - ensure request params are sent to Exa AI (#15855)
* fix filter_out_litellm_params

* use filter_out_litellm_params

* test_filter_out_litellm_params
2025-10-23 11:56:30 -07:00
Ishaan JaffandGitHub 511d435f6f [Bug Fix]: Hooks broken on /bedrock passthrough due to missing metadata (#15849)
* refactor handle_bedrock_passthrough_router_model

* test_bedrock_router_passthrough_metadata_initialization
2025-10-23 11:52:37 -07:00
Alexsander HamirandGitHub 8c5118195d fix: replace deprecated gemini-1.5-pro-preview-0514 with gemini-2.5-flash-lite in function calling test (#15852) 2025-10-23 11:48:58 -07:00
Ishaan Jaffer d53a8b3e8d Revert "fix(responses-api): simplify reasoning item handling for gpt-5-codex (#15815)"
This reverts commit cc63cf28ff.
2025-10-23 11:17:36 -07:00
Ishaan Jaffer ae7b13550e test_models_by_provider 2025-10-23 09:10:41 -07:00
Ishaan Jaffer 8e65f99ebb test fix TTS 2025-10-23 08:50:54 -07:00
Ishaan Jaffer 5498a8b099 test_ensure_initialize_azure_sdk_client_always_used 2025-10-23 08:50:39 -07:00
Ishaan Jaffer 74b8a1dbdf test_aaamodel_prices_and_context_window_json_is_valid 2025-10-23 08:47:08 -07:00
Ishaan Jaffer bfe4167183 bump: version 1.78.7 → 1.78.8 2025-10-23 08:42:03 -07:00
Alexsander HamirandGitHub cc63cf28ff fix(responses-api): simplify reasoning item handling for gpt-5-codex (#15815)
Simplifies the _handle_reasoning_item method to only filter out None values
without attempting to construct ResponseReasoningItem objects. This approach:
- Avoids mutation of encrypted_content in reasoning items
- Fixes compatibility with gpt-5-codex and Codex CLI
- Removes dependency on ResponseReasoningItem import
- Simplifies the code by removing complex fallback logic

The previous approach was trying to construct ResponseReasoningItem objects
with required fields (id, summary), which caused issues with encrypted
payloads. The new approach simply filters None values while preserving
all other data intact.

Issue: https://github.com/BerriAI/litellm/issues/13484
Reported-by: rajpathak (OpenAI)
Tested-by: rajpathak (OpenAI Codex CLI)
2025-10-23 08:41:37 -07:00
Krish DholakiaandGitHub 573306f3cd (feat) Vector Stores: support Vertex AI Search API as vector store through LiteLLM (#15781)
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm

new vector store provider

* feat(vector_store/): use vector store id for vertex ai search api

* fix: transformation.py

cleanup

* fix: implement abstract function

* fix: fix linting error

* fix: main.py

fix check
2025-10-22 18:56:36 -07:00
Ishaan JaffandGitHub 3e4b5ef3a5 [Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI (#15821)
* add search cost tracking

* add cost tracking for tavily tiers

* add search to call types

* add search_provider_cost_per_query

* add cost tracking for search APIs

* add cost tracking search APIs

* docs cost tracking search

* docs search

* fix linting
2025-10-22 17:29:09 -07:00
Ishaan JaffandGitHub 143e314dda [Feat] Add Google PSE Search Provider (#15816)
* docs google PSE

* add SearchProviders

* add search providers

* add PSE search

* add SearchProviders

* get_provider_search_config

* add Search

* init Search

* add get_http_method on BaseSearch

* fixes for Google PSE

* TestGooglePSESearch
2025-10-22 16:01:28 -07:00
Ishaan JaffandGitHub ad62a6d3d1 [Feat] Add DataforSEO Search API (#15817)
* docs google PSE

* add SearchProviders

* add search providers

* add PSE search

* add SearchProviders

* get_provider_search_config

* add Search

* init Search

* add get_http_method on BaseSearch

* fixes for Google PSE

* TestGooglePSESearch

* add DATAFORSEO

* add DataForSEOSearchConfig

* TestDataForSEOSearch

* add DataForSEO

* fix base transform

* fix search

* fix dataforSEO

* docs fix

* fix linting

* fix linting
2025-10-22 16:00:40 -07:00
nuernberandGitHub 799a2b624a use proper bedrock model name in health check (#15808) 2025-10-22 15:24:57 -07:00
Ishaan JaffandGitHub ec6a5ffa2d [Fix] Azure AI Speech - Ensure voice is mapped from request body -> SSML body , allow sending role and style (#15810)
* update map_openai_params

* fix update voice transform

* fix text_to_speech_provider_config

* test_azure_ava_tts_with_custom_voice

* test Azure AVA style, role sent

* _build_express_as_element

* docs custom params

* build LANG

* fix transform

* fix transform

* fix speech

* docs update

* docs azure ai speech
2025-10-22 14:41:11 -07:00
Ishaan JaffandGitHub d91efa7a7b [Bug Fix]: ErrorEvent ValidationError when OpenAI Responses API returns nested error structure (#15804)
* add ErrorEventError nested field

* test_openai_responses_api_token_limit_error

* test_openai_responses_api_token_limit_error
2025-10-22 14:18:46 -07:00
Sameer KankuteandGitHub 2ab2d15efc Fix Token Spend is under budget for passthrough (#15805) v1.78.7-nightly 2025-10-22 10:55:22 -07:00
Ishaan Jaffer abe67df95e refactor large func 2025-10-22 10:04:52 -07:00
Ishaan Jaffer 57a2ec3beb fix: _extract_fields_recursive 2025-10-22 09:37:29 -07:00
Roman GandGitHub eac3cba44f Support for embeddings_by_type Response Format in Bedrock Cohere Embed v1 (#15707)
* feat(cohere): Enhance embedding transformation to support Bedrock's embeddings by type

* test(cohere): Add unit tests for embedding transformation responses
2025-10-22 09:32:11 -07:00
Ishaan Jaffer e80bba83e3 test fix 2025-10-22 09:29:04 -07:00