tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-14 16:25:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
26,310 Commits 2,072 Branches 1,425 Tags
0cb9ef5fb31c760cb4dd15e0b4e3c06004dbcc21
Commit Graph
3672 Commits
Author SHA1 Message Date
Krrish Dholakia 0cb9ef5fb3 fix(openai/responses): fallback to model construct, don't use it by default (causes downstream errors for nested values) 2025-10-10 18:07:18 -07:00
Ishaan JaffandGitHub bf209415da [Feat] VertexAI Gemma model family streaming support + Added MedGemma (#15427) 
* test_acompletion_filters_stream_and_stream_options

* fix: stream_options

* docs medgemma

* lint fix

* docs
 2025-10-10 14:22:27 -07:00
Ishaan JaffandGitHub ed62d6c943 [Fix] VertexAI - gemma model family support (custom endpoints) (#15419) 
* TestVertexGemmaiCompletion

* test vertex Gemma

* fix file name

* fix file naming

* add VertexAIGemmaModels

* add cost_router for vertexai

* fix main.py

* fix VertexGemmaConfig

* fix Vertex AI Gemma-AI Models Handler

* docs gemma

* fix ids

* test fix

* ruff check fixes

* docs fix

* docs fix

* test_acompletion_basic_request

* Revert "test_acompletion_basic_request"

This reverts commit fdaa5bc49e.

* test_acompletion_basic_request

* fix: async transform

* fix gemma: stream param

* test_acompletion_fake_streaming
 2025-10-10 13:30:43 -07:00
Sameer KankuteandGitHub 513925fd12 Add OCI Cohere support with tool calling and streaming capabilities (#15365) 
* Add cohere models for OCI integration

* fix mypy error
 2025-10-10 11:20:18 -07:00
Krish DholakiaandGitHub 9c3e29b9f0 Merge pull request #15330 from jlan-nl/litellm-fix-erroneous-gpt5-cooldown-trigger 
Minimal fix: gpt5 models should not go on cooldown when called with temperature!=1
 2025-10-09 22:34:30 -07:00
Krish DholakiaandGitHub 611a876889 Merge pull request #15380 from BerriAI/litellm_shared_heatlh_status 
Implement Shared Health Check State Across Pods
 2025-10-09 22:25:45 -07:00
Krish DholakiaandGitHub e1e1ea31dc Merge pull request #15379 from PostHog/feat/posthog-per-request-api-key 
feat: posthog per request api key
 2025-10-09 22:24:51 -07:00
Ishaan JaffandGitHub 52bbabd788 [Feat] Support for Vertex AI Gemma Models on Custom Endpoints (#15397) 
* TestVertexGemmaiCompletion

* test vertex Gemma

* fix file name

* fix file naming

* add VertexAIGemmaModels

* add cost_router for vertexai

* fix main.py

* fix VertexGemmaConfig

* fix Vertex AI Gemma-AI Models Handler

* docs gemma

* fix ids

* test fix

* ruff check fixes

* docs fix

* docs fix
 2025-10-09 19:20:02 -07:00
Ishaan JaffandGitHub 4827e62a21 Fix - OpenRouter cache_control to only apply to last content block (#15395) 
* fix: _move_cache_control_to_content

* test_openrouter_transform_request_with_cache_control_list_content

* fix code QA check

* fix: mypy lint

* fix handlers

* fix mypy

* fix - refactor code, fixes for QA checks
 2025-10-09 18:01:33 -07:00
Krish DholakiaandGitHub eb0b263532 Merge pull request #15340 from BerriAI/litellm_db_config_accidental_override_fix 
fix: Prevents DB from accidentally overriding config file values if they are empty in DB
 2025-10-09 17:31:59 -07:00
Alexsander HamirandGitHub c7234ba053 [Fix] - Sessions not being shared (#15388) 
* fix: share sessions

* fix: remove limit

* clean up

* fix: add missing consts

* add: E2E test

* fix: remove redefinition of unused verbose_logger

* fix: missing const import
 2025-10-09 16:57:40 -07:00
Ishaan JaffandGitHub 6cf57d85c3 [Feat] Add EnkryptAI Guardrails on LiteLLM (#15390) 
* feat: add EnkryptAIGuardrail

* fix types

* fix EnkryptAIGuardrails

* __all__ = ["EnkryptAIGuardrail"]
fix

* fix EnkryptAIGuardrail

* TestEnkryptAIExtraction

* fix mypy linting

* docs enkrypt ai

* add EnkryptAIGuardrailConfigModel

* fix EnkryptAIGuardrailConfigModel

* fix import struct

* fix EnkryptAIGuardrails
 2025-10-09 16:27:04 -07:00
Sameer Kankute dce6cd1051 Add shared healthcheck 2025-10-09 22:18:05 +05:30
Carlos Marchal dd560f792e feat: posthog per request api key 2025-10-09 18:32:11 +02:00
Tim Elfrink 02f0b1e83a Merge latest changes from origin/main into fix/redact-aws-credentials 
Resolved import conflict for CustomPricingLiteLLMParams which was moved
from litellm.types.router to litellm.types.utils in main branch.
 2025-10-09 08:12:22 +02:00
Krish DholakiaandGitHub e2e0cdde31 Merge pull request #15315 from lcfyi/lcfyi/add-support-for-parallel-tool-calls 
Fix parallel tool calls in the Anthropic passthrough adapter
 2025-10-08 23:01:55 -07:00
Krish DholakiaandGitHub 4615298ba1 Merge pull request #15336 from BerriAI/litellm_fix_litellm_Param_costing 
Fix litellm_param based costing
 2025-10-08 22:55:41 -07:00
Achintya Rajan 95e71b31a8 Update test_proxy_utils.py 2025-10-08 19:25:54 -07:00
Achintya Rajan 169c1f0f21 moved test to test_proxy_server 2025-10-08 19:24:16 -07:00
Krish DholakiaandGitHub d921df1baf Merge pull request #15343 from BerriAI/litellm_dev_10_08_2025_p1 
MCP - support converting OpenAPI specs to MCP servers
 2025-10-08 19:10:04 -07:00
Krish DholakiaandGitHub 12a1d081ee Merge branch 'main' into litellm_dev_09_11_2025_p1 2025-10-08 19:02:58 -07:00
Ishaan Jaffer c4022ade49 test mapped tests MCP 2025-10-08 18:34:30 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		4226314096 Add native Responses API support for litellm_proxy provider (#15347) 
* Initial plan

* Add native Responses API support for litellm_proxy provider

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-10-08 18:31:26 -07:00
Achintya Rajan 4bebb57611 moved test to test_litellm 2025-10-08 18:28:10 -07:00
Ishaan JaffandGitHub 2f42c806cb [Fix] x-litellm-cache-key header not being returned on cache hit (#15348) 
* fix: x-cache-key

* test_cache_key_in_hidden_params_acompletion

* fix: remove_cache_control_flag_from_messages_and_tools
 2025-10-08 18:10:43 -07:00
Ishaan JaffandGitHub 97031dc8ee Fix - (openrouter): move cache_control to content blocks for claude/gemini (#15345) 
* test_openrouter_transform_request_with_cache_control

* fix CacheControlSupportedModels

* test_openrouter_transform_request_with_cache_control_gemini
 2025-10-08 17:41:04 -07:00
Pierre-Emmanuel MERCIERandIshaan Jaffer 861750790b feat: add redis ssl and username support (#11319) 2025-10-08 16:48:07 -07:00
Krrish Dholakia cd13e533a2 feat(mcp_server_manager.py): expand allowed/disallowed list 2025-10-08 16:06:17 -07:00
Krrish Dholakia 28698933fc feat(mcp_server_manager.py): ensure list tools for openapi servers works as expected 2025-10-08 15:59:42 -07:00
Ishaan JaffandGitHub 1c56a0d856 [Fix] Watsonx - Apply correct prompt templates for openai/gpt-oss model family (#15341) 
* fix: apply_prompt_template

* Revert "fix: apply_prompt_template"

This reverts commit 3e0e40b497.

* add apply_prompt_template for WatsonX

* feat: add apply_prompt_template

* test_watsonx_gpt_oss_prompt_transformation

* Revert "add apply_prompt_template for WatsonX"

This reverts commit 3e80903796e39d4fe7206d63445e024c3ad8d0c4.

* add apply_prompt_template for WatsonX

* fix apply_prompt_template

* fix: add hf template handler

* fix hf_chat_template

* fix _get_tokenizer_config

* fix hf_chat_template

* add WatsonXModelPattern

* fix aapply_prompt_template
 2025-10-08 15:39:36 -07:00
Achintya Rajan ce8a520a86 new deep merge behavior when processing config 2025-10-08 14:41:41 -07:00
Sameer Kankute 85d4142845 Fix litellm_param based costing 2025-10-08 21:14:23 +05:30
IQHL (Hans Jacob Landelius) 6633b33085 formatting 2025-10-08 14:15:15 +02:00
IQHL (Hans Jacob Landelius) 5b9d516e43 unit test 2025-10-08 13:58:25 +02:00
Tim Elfrink f8af72bbf1 fix: redact AWS credentials when redact_user_api_key_info enabled 
Use SensitiveDataMasker in print_deployment() to mask all sensitive
credentials including AWS keys when redact_user_api_key_info is True.

Fixes #14839
 2025-10-08 10:31:35 +02:00
Leslie Cheng f0c9dfbaf8 Add some tests 2025-10-07 20:57:08 -07:00
Krish DholakiaandGitHub 13703f289b Merge pull request #15292 from timelfrink/fix/bedrock-prompt-caching-cost-calculation 
fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation
 2025-10-07 19:11:45 -07:00
Ishaan Jaffer e1ab3620ee fix: mapped tests 2025-10-07 17:55:52 -07:00
Ishaan Jaffer 8590646b84 fix code QA check 2025-10-07 17:49:57 -07:00
Ishaan JaffandGitHub 36c971a6fd [MCP Gateway] QA/Fixes - Ensure Team/Key level enforcement works for MCPs (#15305) 
* fix: _set_object_permission

* fix: _set_object_permission on teams

* fix: _set_object_permission

* fixes for team/key permissions

* statsh: object permission view

* fix: MCPServerPermissions

* fix: _get_team_object_permission

* test mcp checks for permissions

* fix server checks with prefix names

* test_list_tools_strips_prefix_when_matching_permissions

* ruff fix

* docs - refactor MCP

* docs update MCP docs

* docs allowed tools
 2025-10-07 17:34:48 -07:00
Ishaan JaffandGitHub 7b56ba240e [MCP Gateway] Litellm mcp fixes team control (#15304) 
* fix: _set_object_permission

* fix: _set_object_permission on teams

* fix: _set_object_permission

* fixes for team/key permissions

* statsh: object permission view

* fix: MCPServerPermissions
 2025-10-07 16:48:00 -07:00
Ishaan JaffandGitHub bc26eff98f Fix: Make PATCH /model/{model_id}/update handle team_id consistently with POST /model/new (#15297) 
* fix: _update_team_model_in_db

* test_patch_model_with_team_id_creates_proper_setup
 2025-10-07 14:04:08 -07:00
Ishaan JaffandGitHub 07a17d6d6b [Feat] Proxy CLI - dont store existing key in the URL, store it in the state param (#15290) 
* Feat: CLI Auth fixes for UI SSO

* fix auth.py

* fix test ui sso.py
 2025-10-07 12:36:17 -07:00
Tim Elfrink c5eb22381d fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation 
Fixes #15263

This PR fixes the cost calculation for Bedrock Anthropic models with prompt caching.

**Root Cause:**
PR #9838 incorrectly removed adding `cacheWriteInputTokens` to `prompt_tokens`
for Bedrock, based on the assumption that it would cause double counting (similar
to an Anthropic API issue). However, Bedrock's token structure is different:

- **Bedrock API**: `inputTokens`, `cacheReadInputTokens`, and `cacheWriteInputTokens`
  are ALL separate values that should be summed for total input tokens
- **Anthropic API**: Same structure - all three token types are separate

The fix in #9838 was later reverted for Anthropic (correctly re-adding
`cache_creation_input_tokens` to `prompt_tokens`), but Bedrock was never fixed.

**Changes:**
1. Re-add `cacheWriteInputTokens` to `input_tokens` in Bedrock transformation
2. Update test assertions to reflect correct behavior
3. Add regression test for prompt caching cost calculation
4. Fix typo in Anthropic transformation where `cache_creation_tokens` was
   incorrectly set to `cache_read_input_tokens`

**Testing:**
- All existing Bedrock transformation tests pass
- New test validates correct cost calculation with prompt caching
- Verified costs are non-negative and accurate
 2025-10-07 20:28:46 +02:00
Sameer KankuteandGitHub e4892735f0 fix gemini cli by actually streaming the response (#15264) 
* fix gemini cli by actually streaming the response

* fix cost tracking

* fix test
 2025-10-07 11:26:39 -07:00
Sameer KankuteandGitHub 73f96712f5 fix the reasoningresponse id (#15265) 2025-10-07 11:24:29 -07:00
Sameer KankuteandGitHub c0d0424eb8 Added streaming support for response api streaming image generation (#15269) 2025-10-07 08:15:57 -07:00
Krish DholakiaandGitHub 94a34dd53a Merge branch 'main' into litellm_dev_10_06_2025_p1 2025-10-06 20:06:30 -07:00
Krish DholakiaandGitHub 8b357c24c2 Merge pull request #15226 from otaviofbrito/chore/vertex-ai-context-caching 
Chore/vertex ai context caching
 2025-10-06 20:05:24 -07:00
Krish DholakiaandGitHub 6b4415684e Merge pull request #15253 from BerriAI/litellm_dev_10_06_2025_p2 
fix(azure/responses): remove invalid status param from azure call + MCP - support setting CA_BUNDLE_PATH
 2025-10-06 20:01:39 -07:00