tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-11 00:25:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
26,473 Commits 2,072 Branches 1,425 Tags
0212eb6f042c4bb40cc439dae20eeb476d44c18f
Commit Graph
3711 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub f13eb283e1 [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) 
* fix: add google_routes to llm_api_routes

* test: test_virtual_key_llm_api_routes_allows_google_routes
 2025-10-13 10:58:47 -07:00
Krish DholakiaandGitHub 2ea7005c40 Merge pull request #15448 from dhruvyad/main 
Get completion cost directly from OpenRouter
 2025-10-12 22:12:52 -07:00
Krrish Dholakia 1949436047 test: update test 2025-10-12 21:58:06 -07:00
Krish DholakiaandGitHub 04dc9d091c Merge branch 'main' into litellm_staging_branch_10_11_2025_p1 2025-10-12 21:55:44 -07:00
Dhruv Yadav 84a65440c5 add tests for openrouter cost tracking 2025-10-12 12:36:24 +05:30
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		f5359ba007 Fix apply_guardrail endpoint returning raw string instead of ApplyGuardrailResponse (#15436) 
* Initial plan

* Fix apply_guardrail endpoint to return ApplyGuardrailResponse

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-10-11 17:50:37 -07:00
Krrish Dholakia 6bd722bba4 fix: always retain config models 2025-10-11 16:09:33 -07:00
Krrish Dholakia 090a2c0cfe fix: remove unused uuid import 2025-10-11 15:38:28 -07:00
Krish DholakiaandGitHub 323389c244 (feat) Passthrough - allow admin to give access to specific passthrough endpoints 
(feat) Passthrough - allow admin to give access to specific passthrough endpoints
 2025-10-11 13:51:10 -07:00
Krish DholakiaandGitHub 5507d50acf Merge branch 'main' into litellm_dev_10_09_2025_p1 2025-10-11 13:06:34 -07:00
Krish DholakiaandGitHub ff930dc6df Merge branch 'main' into litellm_dev_10_10_2025_p2 2025-10-11 13:03:16 -07:00
Krish DholakiaandGitHub 07ba5c1ea9 Merge branch 'main' into litellm_dev_10_10_2025_p3 2025-10-11 12:58:53 -07:00
Ishaan Jaffer 97626f2d02 fix test 2025-10-11 11:30:19 -07:00
Ishaan Jaffer ce6102d54f test_azure_responses_api_status_error 2025-10-11 10:41:15 -07:00
Ishaan Jaffer 86881a8fc1 test_azure_responses_api_status_error 2025-10-11 10:21:07 -07:00
Ishaan Jaffer 33a4676184 test fix 2025-10-11 10:05:22 -07:00
Ishaan Jaffer 732618f55f test_together_ai_embedding 2025-10-11 09:33:19 -07:00
Ishaan Jaffer c467986b8b test_gemini_context_caching_with_ttl 2025-10-11 09:20:20 -07:00
Ishaan Jaffer a163bf9f51 test fix: MicrosoftSSOHandler 2025-10-11 08:55:23 -07:00
Ishaan Jaffer 3aa0f6ed72 test_generate_key_with_object_permission 2025-10-11 08:54:15 -07:00
Ishaan Jaffer 4d79280175 Gemma test fixes 2025-10-11 08:51:08 -07:00
Ishaan Jaffer 87719640eb test tag mgmt 2025-10-11 08:48:37 -07:00
Sameer KankuteandGitHub 138bdbb6d8 fix mapped tests 1 (#15445) 
* fix mapped tests

* fix mapped tests
 2025-10-11 08:33:08 -07:00
Ishaan Jaffer 9bde1c1a4c TestEnkryptAIGuardrailHooks 2025-10-10 19:59:41 -07:00
Ishaan Jaffer 8cb66168bc test fix 2025-10-10 19:57:17 -07:00
Krrish Dholakia 4ad1b9c9e1 fix(litellm_responses_transformation/transformation.py): several fixes for chat completion to responses api bridge 
Fix tool calling, prevent passing invalid stream options, prevent passing invalid user string (>64 char)

Allows gpt-5-codex to work on claude code
 2025-10-10 19:52:01 -07:00
Alexsander HamirandGitHub 5f5a7c917a fix: handle closed aiohttp sessions with detection and retry (#15442) 
Fixes RuntimeError "Session is closed" by:
- Checking session.closed before use and recreating if needed
- Catching RuntimeError during requests and retrying with new session
- Validating newly created sessions aren't already closed

Adds tests for both proactive detection and reactive retry scenarios.
 2025-10-10 19:42:58 -07:00
Krrish Dholakia fed1ba2891 feat(openai/): convert chat completion tool calls to responses api 
enables gpt-5-codex to work on claude code

 Closes LIT-1088
 2025-10-10 19:41:11 -07:00
Alexsander HamirandGitHub 2c9356c437 [Fix] - shared session parsing and usage issue (#15440) 
* Fix: Add shared_session to all_litellm_params to prevent JSON serialization error

The shared_session parameter (aiohttp.ClientSession) was being passed through
to provider API calls, causing "Object of type ClientSession is not JSON
serializable" errors during embedding requests.

Added shared_session to the all_litellm_params list so it's properly filtered
out as a LiteLLM-internal parameter and not passed to the provider's API.

* Fix: Add shared_session support for embedding calls with connection pooling

The shared_session parameter was not being properly handled in embedding calls,
causing it to be passed through to provider API requests where it's not needed.

Changes:
- Added shared_session to all_litellm_params to filter it from provider API request body
- Extract shared_session in main embedding() function and pass it explicitly
- Updated OpenAI embedding handlers (embedding() and aembedding()) to accept shared_session
- Pass shared_session to _get_openai_client for HTTP client creation

This enables proper connection pooling for embedding requests when shared_session
is provided, improving performance for high-throughput scenarios.

* test: add regression test for shared_session in embedding calls

Add comprehensive test to prevent JSON serialization errors when using
shared_session.

The test verifies two critical aspects:
1. shared_session is in all_litellm_params to prevent "Object of type
   ClientSession is not JSON serializable" errors
2. shared_session flows through the complete call chain across 6 layers:
   - litellm.embedding()
   - OpenAI.embedding/aembedding()
   - _get_openai_client()
   - AsyncHTTPHandler.create_client()
   - _create_async_transport()
   - _create_aiohttp_transport()

Similar to test_acompletion_session_reuse_e2e.py but focused on
embedding endpoints. Uses inspect.getsource() to verify the parameter
is not only accepted but actually passed through each layer.
 2025-10-10 19:26:53 -07:00
Ishaan JaffandGitHub 527c8f59fa [Feat] Tag Management - Add support for setting tag based budgets (#15433) 
* feat: add LiteLLM_TagTable

* fix: use new table for tag management

* fix - allow setting budgets for tags

* working tag creation

* fix schema.prisma

* add tag info

* ui fixes

* ui fix tag info

* TAG_CACHE_IN_MEMORY_TTL_SECONDS

* add Litellm_EntityType

* fix get_aggregated_db_spend_update_transactions

* fix: _update_entity_spend_in_db

* fix _tag_max_budget_check

* add tag budget check

* add tag_list_transactions

* test_get_tag_objects_batch

* test_update_tag_db_without_prisma_client

* fix get_tags_from_request_body

* get_tags_from_request_body

* fix get_tags_from_request_body

* fix spend tracking utils

* get_tags_from_request_body

* test_get_tags_from_request_body_with_metadata_tags

* feat: add _update_tag_cache spend tracking

* fix _PROXY_track_cost_callback

* test_tag_cache_update_multiple_tags

* fix tag info

* docs fix

* docs tag budgets

* doc fix

* docs fix

* fix tag budget

* docs tag budgets

* docs fix

* ruff fix
 2025-10-10 19:24:50 -07:00
Krrish Dholakia 144ea09549 docs(openai.md): document how to get reasoning content for gpt-5 models 2025-10-10 18:42:19 -07:00
Krrish Dholakia bff46da060 test(test_openai.py): add gpt-5 streaming with reasoning content test 2025-10-10 18:36:41 -07:00
Krrish Dholakia f49f540b6d feat(litellm_responses_transformation/transformation.py): parse thinking content in response<-> chat completion bridge 
allows gpt-5 to return thinking content when called via responses api
 2025-10-10 18:29:37 -07:00
Krrish Dholakia 0cb9ef5fb3 fix(openai/responses): fallback to model construct, don't use it by default (causes downstream errors for nested values) 2025-10-10 18:07:18 -07:00
Alexsander HamirandGitHub 9d7dea42d0 [Fix] - SensitiveDataMasker converts lists to string (#15420) 
* fix: preserve lists in SensitiveDataMasker to prevent string conversion

Added 'list' to allowed primitive types in mask_dict() to prevent lists like
tags from being converted to string representations in API responses.

Before: {"tags": "['East US 2', 'production', 'test']"}
After:  {"tags": ["East US 2", "production", "test"]}

* add: unit test
 2025-10-10 17:50:51 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		b9eb05ea63 [Feature]: Include server_name in /v1/mcp/server/health endpoint response (#15431) 
* Initial plan

* Add server_name to MCP health check response

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-10-10 17:37:53 -07:00
Krrish Dholakia 15b5e6f5d9 feat(responses_api/): fix missing streaming events on responses api <-> chat completion bridge 
ensure we are passing the required events when streaming non-openai models via responses api
 2025-10-10 14:53:15 -07:00
Ishaan JaffandGitHub bf209415da [Feat] VertexAI Gemma model family streaming support + Added MedGemma (#15427) 
* test_acompletion_filters_stream_and_stream_options

* fix: stream_options

* docs medgemma

* lint fix

* docs
 2025-10-10 14:22:27 -07:00
Ishaan JaffandGitHub ed62d6c943 [Fix] VertexAI - gemma model family support (custom endpoints) (#15419) 
* TestVertexGemmaiCompletion

* test vertex Gemma

* fix file name

* fix file naming

* add VertexAIGemmaModels

* add cost_router for vertexai

* fix main.py

* fix VertexGemmaConfig

* fix Vertex AI Gemma-AI Models Handler

* docs gemma

* fix ids

* test fix

* ruff check fixes

* docs fix

* docs fix

* test_acompletion_basic_request

* Revert "test_acompletion_basic_request"

This reverts commit fdaa5bc49e.

* test_acompletion_basic_request

* fix: async transform

* fix gemma: stream param

* test_acompletion_fake_streaming
 2025-10-10 13:30:43 -07:00
Sameer KankuteandGitHub 513925fd12 Add OCI Cohere support with tool calling and streaming capabilities (#15365) 
* Add cohere models for OCI integration

* fix mypy error
 2025-10-10 11:20:18 -07:00
Krish DholakiaandGitHub 9c3e29b9f0 Merge pull request #15330 from jlan-nl/litellm-fix-erroneous-gpt5-cooldown-trigger 
Minimal fix: gpt5 models should not go on cooldown when called with temperature!=1
 2025-10-09 22:34:30 -07:00
Krish DholakiaandGitHub 611a876889 Merge pull request #15380 from BerriAI/litellm_shared_heatlh_status 
Implement Shared Health Check State Across Pods
 2025-10-09 22:25:45 -07:00
Krish DholakiaandGitHub e1e1ea31dc Merge pull request #15379 from PostHog/feat/posthog-per-request-api-key 
feat: posthog per request api key
 2025-10-09 22:24:51 -07:00
Ishaan JaffandGitHub 52bbabd788 [Feat] Support for Vertex AI Gemma Models on Custom Endpoints (#15397) 
* TestVertexGemmaiCompletion

* test vertex Gemma

* fix file name

* fix file naming

* add VertexAIGemmaModels

* add cost_router for vertexai

* fix main.py

* fix VertexGemmaConfig

* fix Vertex AI Gemma-AI Models Handler

* docs gemma

* fix ids

* test fix

* ruff check fixes

* docs fix

* docs fix
 2025-10-09 19:20:02 -07:00
Ishaan JaffandGitHub 4827e62a21 Fix - OpenRouter cache_control to only apply to last content block (#15395) 
* fix: _move_cache_control_to_content

* test_openrouter_transform_request_with_cache_control_list_content

* fix code QA check

* fix: mypy lint

* fix handlers

* fix mypy

* fix - refactor code, fixes for QA checks
 2025-10-09 18:01:33 -07:00
Krrish Dholakia f2d4f03339 feat(pass_through_endpoints.py): only show available pass through endpoints based on what teams has access to 
makes it easier to control access
 2025-10-09 17:45:14 -07:00
Krish DholakiaandGitHub eb0b263532 Merge pull request #15340 from BerriAI/litellm_db_config_accidental_override_fix 
fix: Prevents DB from accidentally overriding config file values if they are empty in DB
 2025-10-09 17:31:59 -07:00
Krrish Dholakia 5aa5a3b425 fix(route_checks.py): support team metadata 2025-10-09 17:18:38 -07:00
Alexsander HamirandGitHub c7234ba053 [Fix] - Sessions not being shared (#15388) 
* fix: share sessions

* fix: remove limit

* clean up

* fix: add missing consts

* add: E2E test

* fix: remove redefinition of unused verbose_logger

* fix: missing const import
 2025-10-09 16:57:40 -07:00
Ishaan JaffandGitHub 6cf57d85c3 [Feat] Add EnkryptAI Guardrails on LiteLLM (#15390) 
* feat: add EnkryptAIGuardrail

* fix types

* fix EnkryptAIGuardrails

* __all__ = ["EnkryptAIGuardrail"]
fix

* fix EnkryptAIGuardrail

* TestEnkryptAIExtraction

* fix mypy linting

* docs enkrypt ai

* add EnkryptAIGuardrailConfigModel

* fix EnkryptAIGuardrailConfigModel

* fix import struct

* fix EnkryptAIGuardrails
 2025-10-09 16:27:04 -07:00