tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-07 10:21:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,876 Commits 2,072 Branches 1,425 Tags
73bfef1a1f900b3c8f97fedd2268a382604b671a
Commit Graph
25876 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ishaan JaffandGitHub 73bfef1a1f Revert "[Feature]: Replace HTTPException with ParallelRequestLimitError in pa…" (#15095) 
This reverts commit 71b9b58fa9.
 2025-09-30 21:17:04 -07:00
Ishaan Jaffer 395c32c38d test_azure_openai_assistants_e2e_operations_stream 2025-09-30 21:16:29 -07:00
Krish DholakiaandGitHub 64083111d3 (Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking 
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
 2025-09-30 21:14:16 -07:00
Ishaan Jaffer 8dd31a5fe8 test_azure_openai_assistants_e2e_operations_stream 2025-09-30 18:47:09 -07:00
Ishaan Jaffer 04b3ac89b8 test: QueryParams 2025-09-30 18:45:38 -07:00
Uzair AliandGitHub fcfe856e10 Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Alexsander HamirandGitHub 26145da3e7 perf(router): optimize _filter_cooldown_deployments to O(n) (#15091) 
Refactored to use set-based lookup and list comprehension instead
of two-pass approach with list.remove().

Old complexity: O(n×m + k×n)
- First loop: n deployments × m list lookups = O(n×m)
- Second loop: k removals × n list.remove() scans = O(k×n)

New complexity: O(m + n)
- Convert to set: O(m)
- Filter with O(1) set lookups: O(n)

Example with 100 deployments, 5 in cooldown:
- Old: ~1000 operations
- New: ~105 operations

Called on every request - high impact for production.
 2025-09-30 18:39:12 -07:00
Ishaan JaffandGitHub 0ca11eefde [Feat] Guardrails - add logging for important status fields (#15090) 
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
 2025-09-30 18:38:07 -07:00
Ishaan Jaffer f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
Ishaan Jaffer aac1129761 fix is_sensitive_key 2025-09-30 17:05:57 -07:00
malagsandGitHub 68189d1c04 [Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 
* Improved performance by reducing complexity

* Improved logic to prevent memory from increasing too much, added test

* Restore indent

* Restore indent

* Added type annotation

* Updated test to correctly initialize the expiration_heap
 2025-09-30 16:49:35 -07:00
Ishaan Jaffer f46f9d3fd9 docs azure passthrough api fixes 2025-09-30 15:54:09 -07:00
Krish DholakiaandGitHub c475723c67 Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1 
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
 2025-09-30 15:32:53 -07:00
Krish DholakiaandGitHub 5619d350b6 Merge pull request #15074 from Jetemple/custom-logo-fix 
Make UI theme settings publicly accessible for custom branding
 2025-09-30 15:26:00 -07:00
Ishaan JaffandGitHub 0476a33d9f [Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087) 
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
 2025-09-30 15:01:38 -07:00
Ishaan JaffandGitHub 69a464fc97 [Fix Security] Ensure OCI secret fields not shared on /models and /v1/models endpoints (#15085) 
* fix: remove_sensitive_info_from_deployment

* fix: remove_sensitive_info_from_deployment

* test_model_info_v1_oci_secrets_not_leaked
 2025-09-30 14:55:54 -07:00
Ishaan Jaffer 75d22d3d79 fix code qa check 2025-09-30 14:03:05 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		71b9b58fa9 [Feature]: Replace HTTPException with ParallelRequestLimitError in parallel_request_limiter_v3 (#15033) 
* Initial plan

* Implement ParallelRequestLimitError custom exception to replace HTTPException

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

* Add ParallelRequestLimitError to litellm main module exports

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-30 13:57:14 -07:00
Alexsander HamirandGitHub 05dd104ce6 perf(router): Cache nested dict lookups in hot path (#15084) 
Cache deployment["litellm_params"] and deployment["model_info"]
at loop start to avoid repeated dict hash lookups.

- _pre_call_checks: 3 fewer lookups per deployment per request
- deployment_callback_on_failure: 1 fewer lookup per failure
- _set_model_group_info: 4 fewer lookups per model

Saves CPU cycles on every routing decision and failure callback.
 2025-09-30 13:36:42 -07:00
Alexsander HamirandGitHub 927e15996e perf(router): Remove unnecessary hasattr checks in get_model_list() (#15082) 
Remove redundant hasattr() checks for model_list and model_group_alias
in get_model_list() method. Both attributes are always initialized in
__init__, making these runtime checks unnecessary overhead.

Changes:
- Remove hasattr(self, "model_list") check
- Remove hasattr(self, "model_group_alias") check
- Move model_group_alias initialization earlier in __init__ to ensure
  it's available when set_model_list() calls get_model_names()
- Simplify control flow by removing nested conditional blocks

Performance Impact:
- hasattr should not appear in the profile of a proxy server handling thousands of requests. This change ensures it no longer does.
 2025-09-30 13:35:27 -07:00
Ishaan JaffandGitHub 60230e5666 [Feat] UI - add snowflake on UI (#15083) 
* UI - add snowflake on UI

* fixes snowflake creds
 2025-09-30 13:16:04 -07:00
Ishaan JaffandGitHub 862736e74b feat: add groq/moonshotai/kimi-k2-instruct-0905 (#15079) 2025-09-30 12:51:21 -07:00
Krrish Dholakia 0cd61a6a6a fix: simplify testing 2025-09-30 12:37:25 -07:00
Krish DholakiaandGitHub 39d1d2a379 Merge pull request #14840 from eddierichter-amd/lemonade-integration 
Add AMD Lemonade provider support
 2025-09-30 12:34:30 -07:00
Ishaan Jaffer 45bfd2599c docs fix 2025-09-30 11:35:15 -07:00
Eddie Richter 7da05df534 Removing unecessary import 2025-09-30 12:12:24 -06:00
Eddie Richter 19e7070b73 Removing get_model_info from Lemonade provider. Implemented get_models which gets hooked into get_valid_models litellm utility. Also, added a simple cost calculator implementation for Lemonade so calling cost_calculator.completion_cost() doesn't return an error when a model is not found in the model_cost json. 2025-09-30 12:12:24 -06:00
Eddie Richter 1e1e4c36ac Fixing key name 2025-09-30 12:12:24 -06:00
Eddie Richter bbfa00c61b fixing mypy lint errors 2025-09-30 12:12:24 -06:00
Eddie Richter 3d62596daa fix lint-ruff 2025-09-30 12:12:24 -06:00
Eddie Richter abaf77da43 Small updates to documentation 2025-09-30 12:12:24 -06:00
Eddie Richter 929510ef5d Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
Eddie Richter 6916f43843 Adding functionality for Lemonade to check to see if it is aware of a model and if so use that model 2025-09-30 12:12:24 -06:00
Eddie Richter 351b63bc67 Adding max_tokens to constructor of LemonadeChatConfig 2025-09-30 12:12:24 -06:00
Eddie Richter f9e98f75a6 Adding max_input_tokens and max_output_tokens 2025-09-30 12:12:24 -06:00
Eddie Richter 0e045e0bb7 Setting the response model so the cost can be calculated 2025-09-30 12:12:24 -06:00
Eddie Richter eb71611a97 Adding lemonade transform 2025-09-30 12:12:24 -06:00
Eddie Richter ae92404d05 Initial addition of Lemonade provider. 2025-09-30 12:12:24 -06:00
Jack Temple 7212116d8c test: add UI theme settings retrieval and update tests 2025-09-30 10:01:44 -05:00
Jack Temple 382614911f fix: make /get/ui_theme_settings public for all users to access custom branding 2025-09-30 09:23:23 -05:00
Krish DholakiaandGitHub fd1a986596 Merge pull request #15013 from serializer/patch-1 
[bug]: Update request handling for original exceptions
 2025-09-29 21:59:26 -07:00
Krish DholakiaandGitHub 79d071c25c Merge pull request #15022 from BerriAI/litellm_gemini_tool_search 
Ignore type param for gemini tools
 2025-09-29 21:57:50 -07:00
Krish DholakiaandGitHub db62e33a4f Merge pull request #15025 from Isydmr/main 
fix: use extra_query for download results (Batch API)
 2025-09-29 21:57:15 -07:00
Krish DholakiaandGitHub 551a171243 Merge pull request #15043 from cedarm/fix-remove-vertex-latest 
fix: remove invalid vertex -latest models
 2025-09-29 21:54:53 -07:00
Krish DholakiaandGitHub 9a44fcdc32 Merge pull request #15050 from uc4w6c/fix/regression-mcp-protocol-version 
fix: resolve regression with duplicate Mcp-Protocol-Version header
 2025-09-29 21:53:00 -07:00
Krish DholakiaandGitHub 56fccee731 Merge pull request #15058 from uc4w6c/doc/add_api_key_to_bedrock 
doc: add missing api_key parameter
 2025-09-29 21:52:31 -07:00
Yuta Saito def4afedd7 doc: add missing api_key parameter 2025-09-30 12:52:17 +09:00
Ishaan JaffandGitHub 708c0bd78d [Feat] Return Cost for Responses API Streaming requests (#15053) 
* test_basic_openai_responses_api_streaming

* _transform_chat_completion_usage_to_responses_usage

* ResponseAPIUsage.cost

* test fixes for anthropic cost with /responses

* fix mypy typng
 2025-09-29 19:47:04 -07:00
Ishaan Jaffer 3e474b9e81 fix claude-sonnet-4-5 model cost map 2025-09-29 18:26:29 -07:00
Ishaan Jaffer f1578b49e2 vertex_httpx_mock_post 2025-09-29 18:25:54 -07:00