tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-15 12:21:11 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,896 Commits 2,072 Branches 1,425 Tags
d9664a3ee49e0c5f7ee3f5bb552a349994daa708
Commit Graph
3549 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub e73d053de3 [Fix] Proxy Auth - Ensure LLM_API_KEYs can access pass through routes (#15115) 
* test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints

* fix: is_registered_pass_through_route

* docs fix
 2025-10-01 14:09:01 -07:00
Sameer KankuteandGitHub 7ec7e5332c Add generateContent cost tracking (#15014) 2025-10-01 09:03:45 -07:00
Krish DholakiaandGitHub 1503435d91 Merge pull request #15029 from henryhwang/gemini-adapter-fixes 
feat(gemini): Add full support for native Gemini API translation
 2025-09-30 21:19:18 -07:00
Ishaan JaffandGitHub 73bfef1a1f Revert "[Feature]: Replace HTTPException with ParallelRequestLimitError in pa…" (#15095) 
This reverts commit 71b9b58fa9.
 2025-09-30 21:17:04 -07:00
Ishaan Jaffer 395c32c38d test_azure_openai_assistants_e2e_operations_stream 2025-09-30 21:16:29 -07:00
Krish DholakiaandGitHub 64083111d3 (Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking 
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
 2025-09-30 21:14:16 -07:00
Henry Wang acc23b9757 fix issue from pr review 2025-10-01 11:57:10 +08:00
Henry H WangandGitHub 7c4439ba0a Merge branch 'BerriAI:main' into gemini-adapter-fixes 2025-10-01 09:53:51 +08:00
Ishaan Jaffer 8dd31a5fe8 test_azure_openai_assistants_e2e_operations_stream 2025-09-30 18:47:09 -07:00
Ishaan Jaffer 04b3ac89b8 test: QueryParams 2025-09-30 18:45:38 -07:00
Uzair AliandGitHub fcfe856e10 Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Ishaan JaffandGitHub 0ca11eefde [Feat] Guardrails - add logging for important status fields (#15090) 
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
 2025-09-30 18:38:07 -07:00
Henry Wang 4eee54b157 fix the test issue from the pr review 2025-10-01 09:08:25 +08:00
Ishaan Jaffer f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
malagsandGitHub 68189d1c04 [Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 
* Improved performance by reducing complexity

* Improved logic to prevent memory from increasing too much, added test

* Restore indent

* Restore indent

* Added type annotation

* Updated test to correctly initialize the expiration_heap
 2025-09-30 16:49:35 -07:00
Krish DholakiaandGitHub c475723c67 Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1 
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
 2025-09-30 15:32:53 -07:00
Krish DholakiaandGitHub 5619d350b6 Merge pull request #15074 from Jetemple/custom-logo-fix 
Make UI theme settings publicly accessible for custom branding
 2025-09-30 15:26:00 -07:00
Ishaan JaffandGitHub 0476a33d9f [Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087) 
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
 2025-09-30 15:01:38 -07:00
Ishaan JaffandGitHub 69a464fc97 [Fix Security] Ensure OCI secret fields not shared on /models and /v1/models endpoints (#15085) 
* fix: remove_sensitive_info_from_deployment

* fix: remove_sensitive_info_from_deployment

* test_model_info_v1_oci_secrets_not_leaked
 2025-09-30 14:55:54 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		71b9b58fa9 [Feature]: Replace HTTPException with ParallelRequestLimitError in parallel_request_limiter_v3 (#15033) 
* Initial plan

* Implement ParallelRequestLimitError custom exception to replace HTTPException

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

* Add ParallelRequestLimitError to litellm main module exports

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-30 13:57:14 -07:00
Krrish Dholakia 0cd61a6a6a fix: simplify testing 2025-09-30 12:37:25 -07:00
Eddie Richter 929510ef5d Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
Jack Temple 7212116d8c test: add UI theme settings retrieval and update tests 2025-09-30 10:01:44 -05:00
Henry Wang fcd539af33 fix the issue from the tests for pr review 2025-09-30 18:15:25 +08:00
Henry Wang cce05ac2b4 fix test issues from pr review 2025-09-30 16:44:15 +08:00
Henry Wang d838c96ffb fix test issues from pr review 2025-09-30 16:05:17 +08:00
Krish DholakiaandGitHub fd1a986596 Merge pull request #15013 from serializer/patch-1 
[bug]: Update request handling for original exceptions
 2025-09-29 21:59:26 -07:00
Krish DholakiaandGitHub 79d071c25c Merge pull request #15022 from BerriAI/litellm_gemini_tool_search 
Ignore type param for gemini tools
 2025-09-29 21:57:50 -07:00
Krish DholakiaandGitHub 9a44fcdc32 Merge pull request #15050 from uc4w6c/fix/regression-mcp-protocol-version 
fix: resolve regression with duplicate Mcp-Protocol-Version header
 2025-09-29 21:53:00 -07:00
Ishaan JaffandGitHub 708c0bd78d [Feat] Return Cost for Responses API Streaming requests (#15053) 
* test_basic_openai_responses_api_streaming

* _transform_chat_completion_usage_to_responses_usage

* ResponseAPIUsage.cost

* test fixes for anthropic cost with /responses

* fix mypy typng
 2025-09-29 19:47:04 -07:00
Ishaan Jaffer f1578b49e2 vertex_httpx_mock_post 2025-09-29 18:25:54 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		f22fd4cddd Fix: Add /v1/messages/count_tokens to Anthropic routes for non-admin user access (#15034) 
* Initial plan

* Fix: Add /v1/messages/count_tokens to Anthropic routes for user access

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-29 18:16:52 -07:00
Ishaan JaffGitHubCopilot
 		ebf72f5eb9 [Fix] Parallel Request Limiter v3 - use well known redis cluster hashing algorithm (#15052) 
* test_keyslot_for_redis_cluster

* fix _is_redis_cluster

* Update litellm/proxy/hooks/parallel_request_limiter_v3.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-09-29 18:12:44 -07:00
Ishaan JaffandGitHub f6d7683261 [Feat] LiteLLM Overhead metric tracking - Add support for tracking litellm overhead on cache hits (#15045) 
* test_litellm_overhead

* vertex track overhead

* fix config.yaml used for testing

* test_litellm_overhead_stream

* add update_response_metadata for caching handler

* add CachingDetails

* fix update_response_metadata import

* add CachingDetails metrics

* add CachingDetails

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit

* test_litellm_overhead_cache_hit
 2025-09-29 17:33:27 -07:00
Ishaan Jaffer f303860881 Revert "test_keyslot_for_redis_cluster" 
This reverts commit 52de33787b.
 2025-09-29 16:42:15 -07:00
Ishaan Jaffer 52de33787b test_keyslot_for_redis_cluster 2025-09-29 16:41:32 -07:00
Alexsander HamirandGitHub d4830e34e5 fix: remove router inefficiencies (from O(M*N) to O(1)) - 62.5% faster P99 latency (#15046) 
* fix: remove redundant deep copy

set_model_list already does the deep copy at the beginning of the call.

* fix: remove unused model_list arguments

The `model_list` parameter was being passed to classes that did not use it.

* fix: reduce per-request memory and time from O(N×M) to O(N)

No need to create a whole array for a simple look up.

* add: missing test

* fix: remove unused parameter
 2025-09-29 15:49:46 -07:00
Ishaan Jaffer e0172b86e2 test_litellm_overhead_non_streaming 2025-09-29 15:48:32 -07:00
Yuta Saito 5359a0d6a6 fix: test 2025-09-30 07:25:32 +09:00
Ishaan JaffandGitHub 619577d4e8 [Feat] Add litellm overhead metric for VertexAI (#15040) 
* test_litellm_overhead

* vertex track overhead

* fix config.yaml used for testing

* test_litellm_overhead_stream

* add update_response_metadata for caching handler

* Revert "add update_response_metadata for caching handler"

This reverts commit f2a891f2b448b878a5dbf4b5b0a6166c807b3705.
 2025-09-29 15:15:25 -07:00
Krrish Dholakia 9ed83d44e3 test: remove unnecessary test 2025-09-29 13:35:34 -07:00
Sameer KankuteandGitHub 3ab1c31e4e (Feat) Add cost tracking for Vertex AI Passthrough /predict endpoint (#15019) 
* Add cost tracking for passthrough for predict endpoint

* restore file
 2025-09-29 13:19:04 -07:00
Krrish Dholakia bc6e6e7a28 fix(auth_checks.py): add auth checks to mcp server on call tools 2025-09-29 13:13:10 -07:00
Ishaan JaffandGitHub 4dc23e8079 Revert "feat: improve vertex AI/gemini api_base handling for proxy services (…" (#15042) 
This reverts commit ff2d19e4ca.
 2025-09-29 13:03:46 -07:00
Zero CloverandGitHub ff2d19e4ca feat: improve vertex AI/gemini api_base handling for proxy services (#15039) 2025-09-29 12:12:32 -07:00
Shubham Pathak 3f312d0527 Added test 2025-09-29 21:43:31 +05:30
Sameerlite 58955a0348 Ignore type param for gemini tools 2025-09-29 16:32:21 +05:30
Henry Wang 99a884019b test(gemini): Add unit tests for Google GenAI adapter 
This commit adds a comprehensive suite of unit tests for the Google GenAI adapter to ensure compliance with the project's contribution guidelines.

The new tests cover four main areas:
- Request parameter translation
- Streaming response handling
- Router methods for Google GenAI
- Proxy endpoints for Google GenAI

Additionally, this commit includes minor formatting and linting fixes identified during development.
 2025-09-29 18:51:35 +08:00
Krish DholakiaandGitHub 0d738b2899 Merge pull request #14986 from uc4w6c/fix/remove-servername-prefix-mcp_tools-tests 
Fix/remove servername prefix mcp tools tests
 2025-09-28 18:00:23 -07:00
Krish DholakiaandGitHub e7939b0521 Merge pull request #14983 from abhijitjavelin/main 
Feat: Add Javelin standalone guardrails integration for LiteLLM Proxy
 2025-09-28 17:59:07 -07:00