Commit Graph
3960 Commits
Author SHA1 Message Date
Sameer KankuteandGitHub 2337d150e4 Fix azure doesn't accept extra body param (#16116)
* Fix azure doesn't accept extra body param

* add test issue
2025-11-01 13:31:40 -07:00
Sameer KankuteandGitHub f804ab6de5 Add LLM provider response headers to Responses API (#16091)
* Add llm headers to responses api

* fix mock test
2025-11-01 13:25:56 -07:00
Sameer KankuteandGitHub 887b2e2d98 Add support for empty assitant message in between messages (#15850) 2025-11-01 13:25:21 -07:00
058b6a66eb Add custom_llm_provider support for video endpoints (non-generation) (#16121)
* fix model error for apis which don't need model

* fix print statments:

* fix mypy lint errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-11-01 12:09:11 -07:00
Krish DholakiaandGitHub 43aacf2dc0 (feat) Azure AI Vector Stores - support "virtual" indexes + create vector store on passthrough API (#16160)
* feat(vector_store_endpoints/endpoints.py): add new index_create endpoint

allows admin to create a virtual index, to do permission management for

* feat(key_management_endpoints.py): enable setting allowed_vector_store_indexes on keys

proxy admin can enable dev to create an index on a vector stor

* feat: initial commit adding vector store index passthrough logic to litellm

* feat: add vector store table

* fix(azure_ai/transformation.py): fix headers

* feat: track read/write endpoints by vector store integration

enables permissions by index to work

* fix: azure_ai/vector_stores/search

document the vector store endpoints correctly

 ensures permission management works as expected

* fix(proxy/utils.py): improve error message

* docs(azure_ai_vector_stores_passthrough.md): document azure ai passthrough vector store support

* docs(create.md): document azure ai support via passthrough for vector store create

* fix: fix code qa errors

* fix: document new allowed_vector_store_indexes endpoint
2025-11-01 12:01:32 -07:00
Krish DholakiaandGitHub b02be1ba70 (feat) Milvus - search vector store support + (fix) Passthrough Endpoints - support multi-part form data on passthrough (#16035)
* feat(milvus/): initial commit adding milvus vector store support to LiteLLM

allows querying milvus vector store through litellm

* feat(bedrock/vector_stores): support translating openai filters param to aws kb

adds filtering to aws kb

* feat(milvus/): add milvus vector store unified search support

allows calling milvus vector store in through chat completions

* docs(milvus_vector_stores.md): document new milvus vector search integration

* feat(pass_through_endpoints.py): support passing form data through to a passthrough endpoint

Closes LIT-1147

* fix: fix linting errors
2025-11-01 12:00:29 -07:00
Ishaan Jaffer 3c0d530197 test_async_vertexai_response_basic 2025-11-01 10:58:48 -07:00
Ishaan Jaffer a188e5f8e4 test_litellm_anthropic_prompt_caching_system 2025-11-01 10:51:15 -07:00
Ishaan Jaffer 2608d37e8e test prompt caching ant 2025-11-01 10:36:22 -07:00
Ishaan Jaffer b41ad66e38 ant test prompt caching 2025-11-01 10:09:01 -07:00
Ishaan Jaffer a45e2049f3 test_tool_call_with_empty_enum_property 2025-11-01 09:27:05 -07:00
Ishaan Jaffer 978c52291f test_rerank_custom_callbacks 2025-11-01 09:25:29 -07:00
Ishaan Jaffer bf934c0799 test_anthropic_api_prompt_caching_basic_with_cache_creation 2025-11-01 09:21:42 -07:00
Ishaan Jaffer 14543324af test_team_budget_metrics 2025-11-01 09:21:17 -07:00
Ishaan Jaffer f2b5a5f832 test google endpoints 2025-10-31 20:50:31 -07:00
Ishaan Jaffer cd379c1321 test_anthropic_websearch 2025-10-31 20:43:07 -07:00
Ishaan Jaffer b18cef300a undo test_anthropic_websearch change 2025-10-31 20:39:24 -07:00
Ishaan Jaffer 290e608818 get_computer_tool_beta_header 2025-10-31 20:38:37 -07:00
Ishaan Jaffer 014362cace fix ocr test 2025-10-31 20:32:03 -07:00
Ishaan Jaffer a964e7f24c test_stream_token_counting_anthropic_with_include_usage 2025-10-31 20:24:58 -07:00
Ishaan Jaffer 2f1e947c44 test_anthropic_api_prompt_caching_basic 2025-10-31 19:32:46 -07:00
Ishaan Jaffer 27a98de600 test_health_and_chat_completion 2025-10-31 19:28:59 -07:00
Ishaan Jaffer cd9cf2e6bd test fix 2025-10-31 19:23:08 -07:00
Ishaan Jaffer c918dafb32 test_router_fallbacks_with_custom_model_costs 2025-10-31 19:23:08 -07:00
Ishaan Jaffer 6be9b97d15 test_stream_token_counting_anthropic_with_include_usage 2025-10-31 19:15:07 -07:00
Ishaan Jaffer 6e46824939 test_streaming_response 2025-10-31 19:10:38 -07:00
Ishaan Jaffer 515ae9680e test_anthropic_tool_use 2025-10-31 19:08:32 -07:00
Ishaan Jaffer f173cb0f54 test_anthropic_text_editor 2025-10-31 19:07:46 -07:00
Ishaan Jaffer 928fddb3be test_anthropic_structured_output 2025-10-31 19:06:45 -07:00
Ishaan Jaffer 7f79abb552 test_aastreaming_tool_calls_valid_json_str 2025-10-31 19:05:31 -07:00
Ishaan Jaffer 2cd57540a4 fix test fixes 2025-10-31 18:31:00 -07:00
Ishaan Jaffer 22eb2f8033 Revert "Python entry-point for CustomLLM subclasses (#15881)"
This reverts commit 559ae96e38.
2025-10-31 18:24:39 -07:00
Ishaan Jaffer 94c2c28f3d claude-sonnet-4-5-20250929 fix 2025-10-31 18:20:52 -07:00
Ishaan Jaffer 159db27d5c fix test claude-sonnet-4-5-20250929 2025-10-31 18:13:29 -07:00
Ishaan Jaffer 16a1b57dd5 TestTavilySearch 2025-10-31 18:06:01 -07:00
Ishaan JaffandGitHub c59a0c9681 [Feat] UI - Allow setting cache settings on UI (#16143)
* add LiteLLM_CacheConfig

* add CacheSettingsField

* add UI cache saver

* feat add cache_settings_router

* fix schema

* fix ssl_check_hostname

* refactor into utils

* add groups for field names

* add test_connection in base cache

* add test_connection inredis and redis cluster

* feat _decrypt_db_variables

* add cache settings endpoints

* test_test_cache_connection_calls_cache_test_connection_with_params

* fix: add switch_on_llm_response_caching

* feat use CacheSettingsManager

* feat use CacheSettingsManager

* TestCacheSettingsManager

* fix update_config

* Cache Field test
2025-10-31 17:43:59 -07:00
yuneng-jiangandGitHub 725671395a [Feature] UI - Config Guardrails should not be editable and guardrail info fix (#16142)
* UI Config Guardrails should not be editable and guardrails info definition location fix

* Remove unused import

* Added literals for guardrail definition location
2025-10-31 16:29:24 -07:00
Sameer KankuteandGitHub 689b210c18 Add Xai websearch cost (#16001)
* Add xai websearch cost

* Add test for websearch cost in xai

* remove not required changes
2025-10-30 20:35:34 -07:00
Nicholas CoutureandGitHub f747a4a38a fix: Handle multiple rate limit types per descriptor and prevent IndexError (#16039)
* improve descriptor_key handling for multiple and missing rate limit descriptors in parallel request limiter v3

* Add tests for parallel request limiter v3 in proxy hooks
2025-10-30 20:12:54 -07:00
Sameer KankuteandGitHub c1369a07ba Add Add per model group header forwarding for Bedrock Invoke API (#16042) 2025-10-30 20:10:17 -07:00
John LathouwersandGitHub a6f740f28b Add OCI Signer Authentication. Closes #16048, Closes #15654 (#16064)
* Add OCI Signer Authentication. Closes #16048,  Closes #15654

* Fix linting error

* Remove Recommended, Catch None, Trim Whitespace

* Make method clear re Proxy vs SDK
2025-10-30 19:59:01 -07:00
pale-auraandGitHub 0d84c11bbe Fix vercel_ai_gateway entry for glm-4.6 (#16084)
* Fix vercel_ai_gateway entry for glm-4.6

* Fix: add unit test for vercel_ai_gateway/zai/glm-4.6

* Fix:

Add fix for unit test on vercel_ai_gateway/zai/glm-4.6

* Remove print statements in test_vercel_ai_gateway_glm46_cost_math
2025-10-30 19:40:35 -07:00
Geoffray ViossatandGitHub b29ba9e3be feat: handle Deepgram detected language when available (#16093) 2025-10-30 19:19:34 -07:00
Ishaan JaffandGitHub fd073ee682 Litellm docs readme fixes (#16107)
* add provider test

* docs readme.md

* docs providers

* order providers

* test_providers_alphabetically_ordered

* docs endpoint

* fix config
2025-10-30 17:05:32 -07:00
yuneng-jiangandGitHub 720ba865fb [Infra] Litellm Backend SSO Changes (#16029)
* SSO Backend changes

* Encrypt and Decrypt, load into os env

* Linting and addressing comments
2025-10-30 14:32:08 -07:00
Sameer KankuteandGitHub eed3ad0bdb Fix: Moderations endpoint now respects api_base configuration parameter (#16087)
* Update moderation to use api base

* Update moderation to use api base

* Fix mypy error
2025-10-30 11:01:26 -07:00
Sameer KankuteandGitHub 192935120d Add OpenAI-compatible annotations support for Cohere v2 citations 2025-10-29 19:12:17 -07:00
5e10ea4136 Improve(mcp): respect X-Forwarded- headers in OAuth endpoints (#16036)
* fix(mcp): respect X-Forwarded-Proto header in OAuth endpoints

When LiteLLM proxy is deployed behind a reverse proxy (like nginx or a load balancer) that terminates SSL/TLS, the proxy receives HTTP requests internally but should expose HTTPS URLs externally. This change detects the X-Forwarded-Proto header and uses it to construct correct redirect URIs and endpoint URLs.

Changes:
- Added X-Forwarded-Proto detection to authorize, token, oauth_protected_resource_mcp, oauth_authorization_server_mcp, and register_client endpoints
- Added comprehensive tests for X-Forwarded-Proto header support across all affected endpoints
- Fixed existing tests to properly mock request.headers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix formatting

* feat(mcp): support X-Forwarded-Host for proxy base URL reconstruction

Extended X-Forwarded-Proto support to also handle X-Forwarded-Host and X-Forwarded-Port headers. This allows LiteLLM to correctly construct redirect URIs and endpoint URLs when deployed behind a reverse proxy that changes the host/port.

Example scenario:
- Internal URL: http://localhost:8888/github/mcp
- External URL: https://proxy.abc.com/github/mcp
- Proxy sets: X-Forwarded-Proto: https, X-Forwarded-Host: proxy.abc.com

Changes:
- Added get_request_base_url() helper function to centralize X-Forwarded-* header handling
- Replaced all inline X-Forwarded-Proto checks with calls to the helper function
- Helper handles X-Forwarded-Proto, X-Forwarded-Host, and X-Forwarded-Port
- Added tests for X-Forwarded-Host scenarios in authorize and token endpoints

Fixes issue where protected resource URL mismatch occurred:
  Error: Protected resource http://proxy.abc.com:8888/github/mcp
  does not match expected https://proxy.abc.com/github/mcp

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: replace Yelp-specific hostnames with generic examples

Changed all references from chatproxy.yelpcorp.com to proxy.example.com in:
- test_proxy_forwarding.py (default host parameter)
- TEST_PROXY_FORWARDING.md (documentation examples)
- discoverable_endpoints.py (docstring example)
- test_discoverable_endpoints.py (test mock data)

This makes the code more generic and suitable for open source.
All 13 tests still passing.

* remove accidentally added files

* fix formatting

* add new test for get_base_url

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-29 19:11:32 -07:00
Jason RobertsandGitHub 6672250769 feat(guardrails): Add per-request profile overrides to PANW Prisma AIRS (#16069)
- Per-request metadata: profile_name, profile_id, user_ip, app_name
- Full litellm_call_id for traceability
- Optional profile_name (supports API keys with linked profiles)
- Backward compatible
2025-10-29 19:05:59 -07:00
Ishaan Jaffer 044e26081d test_get_request_body_nova_canvas_inference_profile_arn 2025-10-29 18:30:59 -07:00