tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-16 18:16:34 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,916 Commits 2,072 Branches 1,425 Tags
6b3db2fd495d79d6cf802b4e2dee0b8d43f8db07
Commit Graph
3562 Commits
Author SHA1 Message Date
Georg WölfleinandGitHub dbfa8ec921 Fix end user cost tracking in the responses API (#15124) 
#13860
 2025-10-02 15:13:57 -07:00
Ishaan JaffandGitHub f8f4207994 [Security Fix] fix: don't log JWT SSO token on .info() log (#15145) 
* fix: get_redirect_response_from_openid

* fix info log check

* fix: forward_upstream_to_client
 2025-10-02 15:07:37 -07:00
Krish DholakiaandGitHub f6b67fd9bd Merge pull request #14799 from tyler-liner/chore/generation-name-opentelemetry 
fix (opentelemetry): use generation_name for span naming in logging method
 2025-10-01 21:39:38 -07:00
Krish DholakiaandGitHub 93e347b3d8 Merge pull request #15106 from plafleur/ISSUE-15105 
Guardrails - Don't run post_call guardrail if no text returned from Bedrock
 2025-10-01 21:29:14 -07:00
Ishaan JaffandGitHub d538cf489a [Feat] Fixes to dynamic rate limiter v3 - add saturatation detection (#15119) 
* test cases dynamic rate limits

* fix _handle_generous_mode

* docs add readme

* use configs for vars

* fix debug

* add comment

* test_dynamic_rate_limiter_v3.py

* test_concurrent_pre_call_hooks_stress
 2025-10-01 18:35:34 -07:00
Ishaan Jaffer 2bc5d93f23 use_callback_in_llm_call 2025-10-01 18:32:37 -07:00
Deepanshu LullaGitHubdeepanshu
 		68adca04c8 Gitlab based Prompt manager (#14988) 
* add prompt

* add prompt

* add prompt

* add prompt

* add prompt management via gitlab

* gitlab client

* gitlab client

* gitlab client

* fix lint issues

* fix lint issues

* remove router changes

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
 2025-10-01 18:13:11 -07:00
Ishaan JaffandGitHub 388761f52d [Fix] LiteLLM UI - Ensure OTEL settings are saved in DB after set on UI (#15118) 
* fix: fix _add_callback_from_db_to_in_memory_litellm_callbacks

* test_add_callback_from_db_to_in_memory_litellm_callbacks

* fix otel

* fix: fix _add_callback_from_db_to_in_memory_litellm_callbacks
 2025-10-01 15:33:22 -07:00
Ishaan JaffandGitHub e73d053de3 [Fix] Proxy Auth - Ensure LLM_API_KEYs can access pass through routes (#15115) 
* test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints

* fix: is_registered_pass_through_route

* docs fix
 2025-10-01 14:09:01 -07:00
Patrick Lafleur 8e5efd29df Fix comment 2025-10-01 16:39:26 -04:00
Patrick Lafleur d6be26dba6 Merge branch 'main' into ISSUE-15105 2025-10-01 14:44:40 -04:00
Patrick Lafleur 7ef71d4885 Fix text 2025-10-01 12:08:03 -04:00
Sameer KankuteandGitHub 7ec7e5332c Add generateContent cost tracking (#15014) 2025-10-01 09:03:45 -07:00
Patrick Lafleur 3e5d585f7d Don't run post_call guardrail if no text returned from bedrock 2025-10-01 11:56:50 -04:00
Krish DholakiaandGitHub 1503435d91 Merge pull request #15029 from henryhwang/gemini-adapter-fixes 
feat(gemini): Add full support for native Gemini API translation
 2025-09-30 21:19:18 -07:00
Ishaan JaffandGitHub 73bfef1a1f Revert "[Feature]: Replace HTTPException with ParallelRequestLimitError in pa…" (#15095) 
This reverts commit 71b9b58fa9.
 2025-09-30 21:17:04 -07:00
Ishaan Jaffer 395c32c38d test_azure_openai_assistants_e2e_operations_stream 2025-09-30 21:16:29 -07:00
Krish DholakiaandGitHub 64083111d3 (Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking 
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
 2025-09-30 21:14:16 -07:00
Henry Wang acc23b9757 fix issue from pr review 2025-10-01 11:57:10 +08:00
Henry H WangandGitHub 7c4439ba0a Merge branch 'BerriAI:main' into gemini-adapter-fixes 2025-10-01 09:53:51 +08:00
Ishaan Jaffer 8dd31a5fe8 test_azure_openai_assistants_e2e_operations_stream 2025-09-30 18:47:09 -07:00
Ishaan Jaffer 04b3ac89b8 test: QueryParams 2025-09-30 18:45:38 -07:00
Uzair AliandGitHub fcfe856e10 Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Ishaan JaffandGitHub 0ca11eefde [Feat] Guardrails - add logging for important status fields (#15090) 
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
 2025-09-30 18:38:07 -07:00
Henry Wang 4eee54b157 fix the test issue from the pr review 2025-10-01 09:08:25 +08:00
Ishaan Jaffer f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
malagsandGitHub 68189d1c04 [Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 
* Improved performance by reducing complexity

* Improved logic to prevent memory from increasing too much, added test

* Restore indent

* Restore indent

* Added type annotation

* Updated test to correctly initialize the expiration_heap
 2025-09-30 16:49:35 -07:00
Krish DholakiaandGitHub c475723c67 Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1 
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
 2025-09-30 15:32:53 -07:00
Krish DholakiaandGitHub 5619d350b6 Merge pull request #15074 from Jetemple/custom-logo-fix 
Make UI theme settings publicly accessible for custom branding
 2025-09-30 15:26:00 -07:00
Ishaan JaffandGitHub 0476a33d9f [Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087) 
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
 2025-09-30 15:01:38 -07:00
Ishaan JaffandGitHub 69a464fc97 [Fix Security] Ensure OCI secret fields not shared on /models and /v1/models endpoints (#15085) 
* fix: remove_sensitive_info_from_deployment

* fix: remove_sensitive_info_from_deployment

* test_model_info_v1_oci_secrets_not_leaked
 2025-09-30 14:55:54 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		71b9b58fa9 [Feature]: Replace HTTPException with ParallelRequestLimitError in parallel_request_limiter_v3 (#15033) 
* Initial plan

* Implement ParallelRequestLimitError custom exception to replace HTTPException

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

* Add ParallelRequestLimitError to litellm main module exports

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-30 13:57:14 -07:00
Krrish Dholakia 0cd61a6a6a fix: simplify testing 2025-09-30 12:37:25 -07:00
Eddie Richter 929510ef5d Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
Jack Temple 7212116d8c test: add UI theme settings retrieval and update tests 2025-09-30 10:01:44 -05:00
Henry Wang fcd539af33 fix the issue from the tests for pr review 2025-09-30 18:15:25 +08:00
Henry Wang cce05ac2b4 fix test issues from pr review 2025-09-30 16:44:15 +08:00
Henry Wang d838c96ffb fix test issues from pr review 2025-09-30 16:05:17 +08:00
Krish DholakiaandGitHub fd1a986596 Merge pull request #15013 from serializer/patch-1 
[bug]: Update request handling for original exceptions
 2025-09-29 21:59:26 -07:00
Krish DholakiaandGitHub 79d071c25c Merge pull request #15022 from BerriAI/litellm_gemini_tool_search 
Ignore type param for gemini tools
 2025-09-29 21:57:50 -07:00
Krish DholakiaandGitHub 9a44fcdc32 Merge pull request #15050 from uc4w6c/fix/regression-mcp-protocol-version 
fix: resolve regression with duplicate Mcp-Protocol-Version header
 2025-09-29 21:53:00 -07:00
Ishaan JaffandGitHub 708c0bd78d [Feat] Return Cost for Responses API Streaming requests (#15053) 
* test_basic_openai_responses_api_streaming

* _transform_chat_completion_usage_to_responses_usage

* ResponseAPIUsage.cost

* test fixes for anthropic cost with /responses

* fix mypy typng
 2025-09-29 19:47:04 -07:00
Ishaan Jaffer f1578b49e2 vertex_httpx_mock_post 2025-09-29 18:25:54 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		f22fd4cddd Fix: Add /v1/messages/count_tokens to Anthropic routes for non-admin user access (#15034) 
* Initial plan

* Fix: Add /v1/messages/count_tokens to Anthropic routes for user access

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-29 18:16:52 -07:00
Ishaan JaffGitHubCopilot
 		ebf72f5eb9 [Fix] Parallel Request Limiter v3 - use well known redis cluster hashing algorithm (#15052) 
* test_keyslot_for_redis_cluster

* fix _is_redis_cluster

* Update litellm/proxy/hooks/parallel_request_limiter_v3.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-09-29 18:12:44 -07:00
Ishaan JaffandGitHub f6d7683261 [Feat] LiteLLM Overhead metric tracking - Add support for tracking litellm overhead on cache hits (#15045) 
* test_litellm_overhead

* vertex track overhead

* fix config.yaml used for testing

* test_litellm_overhead_stream

* add update_response_metadata for caching handler

* add CachingDetails

* fix update_response_metadata import

* add CachingDetails metrics

* add CachingDetails

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit
 2025-09-29 17:33:27 -07:00
Ishaan Jaffer f303860881 Revert "test_keyslot_for_redis_cluster" 
This reverts commit 52de33787b.
 2025-09-29 16:42:15 -07:00
Ishaan Jaffer 52de33787b test_keyslot_for_redis_cluster 2025-09-29 16:41:32 -07:00
Alexsander HamirandGitHub d4830e34e5 fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 
* fix: remove redundant deep copy

set_model_list already does the deep copy at the beginning of the call.

* fix: remove unused model_list arguments

The `model_list` parameter was being passed to classes that did not use it.

* fix: reduce per-request memory and time from O(N×M) to O(N)

No need to create a whole array for a simple look up.

* add: missing test

* fix: remove unused parameter
 2025-09-29 15:49:46 -07:00
Ishaan Jaffer e0172b86e2 test_litellm_overhead_non_streaming 2025-09-29 15:48:32 -07:00