Commit Graph
28857 Commits
Author SHA1 Message Date
Alexsander HamirandGitHub ea06fb70fb refactor: lazy load bedrock types (#18053) 2025-12-16 06:05:01 -08:00
Alexsander HamirandGitHub 6ca812130b [Refactor] litellm/init.py: lazy load LLMClientCache (#18008) 2025-12-16 05:44:06 -08:00
e31d8a1cf6 Router order parameter documentation (#18045)
* feat: Add deployment ordering for load balancing

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Refactor docs to simplify load balancing and deployment ordering explanations

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-16 15:57:05 +05:30
Krrish Dholakia 5e78ffd5fd fix: fix docker compose quick start 2025-12-16 11:26:38 +05:30
Krrish Dholakia 6041155de0 docs: cleanup 2025-12-16 11:21:41 +05:30
yuneng-jiangandGitHub 0b5317c12b Merge pull request #18030 from BerriAI/litellm_ui_milvus
[Feature] UI - Milvus Vector Store
2025-12-15 20:22:38 -08:00
Krish DholakiaandGitHub 23e70dd90b Litellm dev 12 15 2025 p1 (#18028)
* fix(openai/responses/guardrail_translation): fix basemodel import

ensures /v1/responses output is correctly processed by guardrail handler

* fix: fix linting error

* fix: fix linting errors
2025-12-16 09:48:49 +05:30
yuneng-jiang 0eba879179 Add ability to add embedding model for milvus 2025-12-15 20:12:44 -08:00
Ishaan JaffandGitHub 244d83ff47 [Docs] Litellm add docs vertex ai engine (#18027)
* new provider doc

* add to sidebar

* stash docs

* docs fix

* docs vertex agent engine
2025-12-15 20:11:40 -08:00
13c0ab8985 Cleanup PR template: remove redundant fields (#17956)
- Remove Title section (already in PR title)
- Remove screenshot requirement (CI validates tests)

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-12-16 09:39:22 +05:30
Cesar GarciaandGitHub cd9b71093a Improve issue labeling: add component dropdown and more provider keywords (#17957)
- Replace "ML Ops Team" dropdown with "What part of LiteLLM is this about?"
  with options: SDK, Proxy, UI Dashboard, Docs, Other
- Add component dropdown to feature_request.yml template
- Rename label-mlops.yml to label-component.yml and update to auto-label
  issues with sdk, proxy, ui-dashboard, or docs based on selection
- Add more provider keywords to issue-keyword-labeler: gemini, cohere,
  mistral, groq, ollama, deepseek
2025-12-16 09:38:39 +05:30
Ishaan JaffandGitHub ce113f4e4b [Docs] Add docs on using pydantic ai agents with LiteLLM A2a gateway (#18026)
* init A2AProviderConfigManager

* move file

* move file

* add pydnatic ai folder

* init providers

* test_pydantic_ai_non_streaming

* fix import

* INIT pydantic

* use_a2a_form_fields

* test_vertex_agent_engine_streaming

* add agent_engine

* init transform for agent engine

* init agent engine

* VertexAgentEngineSSEStreamIterator

* sample

* ui add new fields

* fix vertex_credentials

* working SSE iterator

* TestVertexAgentEngineTransformRequest

* fix code QA check

* stash docs

* docs fix

* fix logo

* docs fix

* doc pydantic ai

* docs pydantic ai

* new provider

* docs fix
2025-12-15 19:43:12 -08:00
Krrish Dholakia f90dea7315 fix(docker-compose.yml): move to docker.litellm.ai 2025-12-16 08:50:34 +05:30
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
32c07113cf [Feat] New Provider - VertexAI Agent Engine (#18014)
* init A2AProviderConfigManager

* move file

* move file

* add pydnatic ai folder

* init providers

* test_pydantic_ai_non_streaming

* fix import

* INIT pydantic

* use_a2a_form_fields

* test_vertex_agent_engine_streaming

* add agent_engine

* init transform for agent engine

* init agent engine

* VertexAgentEngineSSEStreamIterator

* sample

* ui add new fields

* fix vertex_credentials

* working SSE iterator

* TestVertexAgentEngineTransformRequest

* fix code QA check

* Potential fix for code scanning alert no. 3923: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-15 19:14:49 -08:00
Krish DholakiaandGitHub f58b76aee8 Revert "Revert "Litellm bedrock guardrails block precedence over masking (#17…" (#18023)
This reverts commit 0abe5cdce9.
2025-12-16 08:42:10 +05:30
Krish DholakiaandGitHub 0abe5cdce9 Revert "Litellm bedrock guardrails block precedence over masking (#17968)" (#18022)
This reverts commit a0754f1c88.
2025-12-16 08:41:36 +05:30
kothamahandGitHub a0754f1c88 Litellm bedrock guardrails block precedence over masking (#17968)
* prioritized bedrock guardrail blocking by removing early return based on masking flags

When mask_request_content: true or mask_response_content: true, the method immediately returning False.

The Result: Even when Bedrock Guardrails returned action: "BLOCKED" for dangerous content, LiteLLM would not raise an exception and allowing the content through the response. 

So removed that early condition which will return true for the blocked actions based on guardrails.

* Added test case for bedrock guardrail block content precedence
2025-12-16 08:41:13 +05:30
Krrish Dholakia 7c2478b70e docs: replace ghcr link with docker.litellm.ai 2025-12-16 08:35:45 +05:30
Doni CrosbyandGitHub e83f08c24d feat(venice.ai): add support for Venice.ai API via providers.json (#17962) 2025-12-16 08:24:16 +05:30
Dmitrii KomarovandGitHub df9d3abf49 Propagate token usage when generating images with Gemini (#17987) 2025-12-16 08:20:09 +05:30
edfb4148dc Add workflow to create daily staging branches (#18020)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-16 08:13:54 +05:30
yuneng-jiang be4f11eacc Fixing build 2025-12-15 18:38:41 -08:00
yuneng-jiang 8e3eb33165 Tests 2025-12-15 18:36:01 -08:00
yuneng-jiang 0ee924f91e Adding svg 2025-12-15 18:34:34 -08:00
yuneng-jiang fabbd28b45 Adding Milvus to Vector Store in UI 2025-12-15 18:31:35 -08:00
Shivam RawatandGitHub c754794bc3 [fix] add qwen3-embedding-8b input per token price (#18018)
* added embedding input token price

* added embedding input token price
2025-12-16 07:38:21 +05:30
Krrish Dholakia b0a9c85a52 fix: fix ruff linting errors 2025-12-16 07:38:07 +05:30
Cesar GarciaandGitHub 4fdbbdfe6d fix(anthropic): correct claude-3-7-sonnet max_tokens to 64K default (#17979)
Claude 3.7 Sonnet's default max_output_tokens is 64000, not 128000.
The 128K output limit requires the beta header 'output-128k-2025-02-19'.

This fixes the integration test failure where requests with max_tokens=128000
were being rejected by the Anthropic API.

Fixes test_multiturn_tool_calls in test_anthropic_responses_api.py
2025-12-16 07:27:40 +05:30
Ishaan JaffandGitHub a4fb0df028 [Feat] New provider - Agent Gateway, add pydantic ai agents (#18013)
* init A2AProviderConfigManager

* move file

* move file

* add pydnatic ai folder

* init providers

* test_pydantic_ai_non_streaming

* fix import

* INIT pydantic

* use_a2a_form_fields

* TestPydanticAITransformation
2025-12-15 17:40:58 -08:00
YutaSaitoandGitHub 867ac17677 Merge pull request #18010 from BerriAI/litellm_fix_mcp_deepcopy_error
fix: mcp deepcopy error
2025-12-16 07:37:25 +09:00
Cesar GarciaandGitHub df19a747a2 feat(custom_llm): add image_edit and aimage_edit support (#17999)
* feat(custom_llm): add image_edit and aimage_edit support

Add support for image_edit and aimage_edit methods in CustomLLM class,
allowing users to implement custom image editing providers.

Changes:
- Add image_edit() and aimage_edit() methods to CustomLLM base class
- Add custom provider detection in litellm.image_edit() function
- Add tests for sync and async image_edit with custom handlers

* docs: add image_edit to CustomLLM documentation

- Add /v1/images/edits to supported routes
- Add Image Edit section with example
- Update Custom Handler Spec with image_edit methods
2025-12-15 14:16:36 -08:00
Yuta Saito defea8b887 fix: mcp deepcopy error 2025-12-16 07:02:07 +09:00
Ishaan JaffandGitHub b57b1beb61 [Feat] Guardrails - litellm content filter (#18007)
* add br ssn

* better description
2025-12-15 13:26:20 -08:00
vasilisazaykaandGitHub d7e0044118 [docs] update SAP docs (#17974)
* docs(sap): update documentation

* docs(sap): update documentation

* docs(sap): update documentation

* docs(sap): update documentation
2025-12-15 13:25:26 -08:00
Alexsander HamirandGitHub 764a31f624 refactor: lazy load get_modified_max_tokens (#18002) 2025-12-15 12:33:01 -08:00
Alexsander HamirandGitHub 8f647dd25b [Refactor] litellm/init.py: lazy load caches (#18001) 2025-12-15 12:13:51 -08:00
Alexsander HamirandGitHub 0629dcfdd5 [Refactor] litellm/init.py: lazy load http handlers (#17997) 2025-12-15 11:50:22 -08:00
Alexsander HamirandGitHub 93b1da7911 [Refactor] lazy imports: Use per-attribute lazy imports and extract shared constants (#17994) 2025-12-15 10:38:54 -08:00
Sameer KankuteandGitHub 5a642c788a Merge pull request #17980 from BerriAI/litellm_add_provider_specific_tools_in_responses
Add provider specific tools support in responses api
2025-12-15 22:53:08 +05:30
Sameer KankuteandGitHub 174ab1fc66 Merge pull request #17967 from BerriAI/litellm_fireworks_ai_reasoning_param_support
Add support for reasoning param for fireworks AI models
2025-12-15 22:04:04 +05:30
Sameer KankuteandGitHub 8a6f6385ed Merge pull request #17966 from BerriAI/litellm_fix_gpt-1_image_cost_1
fix: cost calculation of gpt-image 1 model
2025-12-15 22:03:58 +05:30
Sameer Kankute 9866355315 Fix: Remove unused import 2025-12-15 21:59:46 +05:30
AlexsanderHamir 41b67b577b Fix: simplify PR template 2025-12-15 07:44:47 -08:00
Alexsander HamirandGitHub debf3598f8 Add: CI/CD rules to default PR template for LiteLLM team. (#17985) 2025-12-15 07:41:17 -08:00
Alexsander HamirandGitHub d743cde041 add: litellm team PR template (#17983) 2025-12-15 07:21:33 -08:00
Sameer KankuteandGitHub 57bf6f74e8 Merge pull request #17844 from SamAcctX/patch-2
OpenRouter GPT 5.2, Mistral 3, and Devstral 2
2025-12-15 18:32:50 +05:30
Sameer Kankute ccf3f7c5be Fix: use simpler implementation 2025-12-15 18:29:16 +05:30
Krish DholakiaandGitHub 26fd6d5362 Guardrails API - support LLM tool call response checks on /chat/completions, /v1/responses, /v1/messages on regular + streaming calls (#17619)
* fix(unified_guardrails.py): send all chunks on completion of final stream

* feat(generic_guardrail_api.py): handle tool call response on streaming LLM responses

* fix(anthropic/chat/guardrail_translation): initial commit adding anthropic tool response streaming guardrails

enables guardrail checks on tool response from llm's to work via `/v1/messages`

* feat(anthropic/): working guardrail checks on tool response from LLMs

ensures guardrail checks on anthropic /v1/messages works as expected

* feat(responses/guardrail_translation): support tool call response guardrails on streaming for /v1/responses

ensures complete coverage of tool call responses

* refactor(openai.py): refactor to use consistent pydantic model for responses api tool response on streaming

enables non-openai model tool call response to work correctly with guardrail checks on /v1/responses

* test: update tests

* fix: fix linting error

* fix: fix failing tests

* fix: fix import errors

* fix(openai/chat/guardrail_transformation): fix final chunk returned on streaming
2025-12-15 18:19:52 +05:30
Sameer Kankute 54b09cab28 Add test for handling provider specific tools 2025-12-15 17:41:38 +05:30
Sameer Kankute d3998d00dd Fix mypy errors 2025-12-15 17:41:05 +05:30