tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-15 18:21:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,916 Commits 2,072 Branches 1,425 Tags
6b3db2fd495d79d6cf802b4e2dee0b8d43f8db07
Commit Graph
25916 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Krish DholakiaandGitHub 6b3db2fd49 Merge pull request #15140 from niharm/fix-sonnet-4-5-200k-pricing 
Price Fix: Add 200K prices for Sonnet 4.5
 2025-10-02 17:28:47 -07:00
Georg WölfleinandGitHub dbfa8ec921 Fix end user cost tracking in the responses API (#15124) 
#13860
 2025-10-02 15:13:57 -07:00
Ishaan JaffandGitHub f8f4207994 [Security Fix] fix: don't log JWT SSO token on .info() log (#15145) 
* fix: get_redirect_response_from_openid

* fix info log check

* fix: forward_upstream_to_client
 2025-10-02 15:07:37 -07:00
Amir RefaeeandGitHub 8991657d67 added railtracks to projects using litellm (#15144) 2025-10-02 14:21:38 -07:00
Mubashir OsmaniandGitHub f2107a189d add azure_ai grok-4 model family (#15137) 
* added oauth mcp to docs

* added azure ai/grok-4 model family

* Revert "added oauth mcp to docs"

This reverts commit 950b7cef44f14b2db1429f6fbd32548a7c95d325.
 2025-10-02 14:21:12 -07:00
nihar cd7acd2eb2 Add 200K prices for Sonnet 4.5 2025-10-02 12:39:44 -07:00
Krrish Dholakia ebba9e0b2a docs: cleanup docs 2025-10-02 09:51:16 -07:00
Krish DholakiaandGitHub f6b67fd9bd Merge pull request #14799 from tyler-liner/chore/generation-name-opentelemetry 
fix (opentelemetry): use generation_name for span naming in logging method
 2025-10-01 21:39:38 -07:00
Krish DholakiaandGitHub cb39a1bb60 Merge pull request #15024 from kowyo/main 
docs: use docker compose instead of docker-compose
 2025-10-01 21:34:42 -07:00
Krish DholakiaandGitHub 93e347b3d8 Merge pull request #15106 from plafleur/ISSUE-15105 
Guardrails - Don't run post_call guardrail if no text returned from Bedrock
 2025-10-01 21:29:14 -07:00
Ishaan JaffandGitHub d538cf489a [Feat] Fixes to dynamic rate limiter v3 - add saturatation detection (#15119) 
* test cases dynamic rate limits

* fix _handle_generous_mode

* docs add readme

* use configs for vars

* fix debug

* add comment

* test_dynamic_rate_limiter_v3.py

* test_concurrent_pre_call_hooks_stress
 2025-10-01 18:35:34 -07:00
Ishaan Jaffer 2bc5d93f23 use_callback_in_llm_call 2025-10-01 18:32:37 -07:00
Deepanshu LullaGitHubdeepanshu
 		68adca04c8 Gitlab based Prompt manager (#14988) 
* add prompt

* add prompt

* add prompt

* add prompt

* add prompt management via gitlab

* gitlab client

* gitlab client

* gitlab client

* fix lint issues

* fix lint issues

* remove router changes

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
 2025-10-01 18:13:11 -07:00
Ishaan JaffandGitHub 388761f52d [Fix] LiteLLM UI - Ensure OTEL settings are saved in DB after set on UI (#15118) 
* fix: fix _add_callback_from_db_to_in_memory_litellm_callbacks

* test_add_callback_from_db_to_in_memory_litellm_callbacks

* fix otel

* fix: fix _add_callback_from_db_to_in_memory_litellm_callbacks
 2025-10-01 15:33:22 -07:00
Ishaan JaffandGitHub d9664a3ee4 fix gpt-5-chat-latest on model cost map (#15116) 2025-10-01 14:35:57 -07:00
Ishaan JaffandGitHub e73d053de3 [Fix] Proxy Auth - Ensure LLM_API_KEYs can access pass through routes (#15115) 
* test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints

* fix: is_registered_pass_through_route

* docs fix
 2025-10-01 14:09:01 -07:00
Patrick Lafleur 8e5efd29df Fix comment 2025-10-01 16:39:26 -04:00
Patrick Lafleur d6be26dba6 Merge branch 'main' into ISSUE-15105 2025-10-01 14:44:40 -04:00
Luiz Rennó CostaGitHubLuiz Rennó Costa
 		7e56600896 fix: model_group not always present in litellm_params, and metadata reference location (#15108) 
Co-authored-by: Luiz Rennó Costa <luiz.renno@ifood.com.br>
 2025-10-01 11:39:49 -07:00
Patrick LafleurandGitHub e7fd1fb96b Fix missing HTTPException import (#15111) 2025-10-01 11:39:30 -07:00
Patrick Lafleur 7ef71d4885 Fix text 2025-10-01 12:08:03 -04:00
Sameer KankuteandGitHub 7ec7e5332c Add generateContent cost tracking (#15014) 2025-10-01 09:03:45 -07:00
Patrick Lafleur 3e5d585f7d Don't run post_call guardrail if no text returned from bedrock 2025-10-01 11:56:50 -04:00
Ishaan Jaffer ab00ca2de9 bump: version 1.77.6 → 1.77.7 2025-10-01 08:54:03 -07:00
Krrish Dholakia 7fd24a2632 bump: version 1.77.6 → 1.77.7 2025-09-30 21:23:42 -07:00
Krish DholakiaandGitHub 1503435d91 Merge pull request #15029 from henryhwang/gemini-adapter-fixes 
feat(gemini): Add full support for native Gemini API translation
 2025-09-30 21:19:18 -07:00
Ishaan JaffandGitHub 73bfef1a1f Revert "[Feature]: Replace HTTPException with ParallelRequestLimitError in pa…" (#15095) 
This reverts commit 71b9b58fa9.
 2025-09-30 21:17:04 -07:00
Ishaan Jaffer 395c32c38d test_azure_openai_assistants_e2e_operations_stream 2025-09-30 21:16:29 -07:00
Krish DholakiaandGitHub 64083111d3 (Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking 
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
 2025-09-30 21:14:16 -07:00
Henry Wang acc23b9757 fix issue from pr review 2025-10-01 11:57:10 +08:00
Henry H WangandGitHub 7c4439ba0a Merge branch 'BerriAI:main' into gemini-adapter-fixes 2025-10-01 09:53:51 +08:00
Ishaan Jaffer 8dd31a5fe8 test_azure_openai_assistants_e2e_operations_stream 2025-09-30 18:47:09 -07:00
Ishaan Jaffer 04b3ac89b8 test: QueryParams 2025-09-30 18:45:38 -07:00
Uzair AliandGitHub fcfe856e10 Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Alexsander HamirandGitHub 26145da3e7 perf(router): optimize _filter_cooldown_deployments to O(n) (#15091) 
Refactored to use set-based lookup and list comprehension instead
of two-pass approach with list.remove().

Old complexity: O(n×m + k×n)
- First loop: n deployments × m list lookups = O(n×m)
- Second loop: k removals × n list.remove() scans = O(k×n)

New complexity: O(m + n)
- Convert to set: O(m)
- Filter with O(1) set lookups: O(n)

Example with 100 deployments, 5 in cooldown:
- Old: ~1000 operations
- New: ~105 operations

Called on every request - high impact for production.
 2025-09-30 18:39:12 -07:00
Ishaan JaffandGitHub 0ca11eefde [Feat] Guardrails - add logging for important status fields (#15090) 
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
 2025-09-30 18:38:07 -07:00
Henry Wang 4eee54b157 fix the test issue from the pr review 2025-10-01 09:08:25 +08:00
Ishaan Jaffer f205b2c0a5 test fixes 2025-09-30 17:08:49 -07:00
Ishaan Jaffer aac1129761 fix is_sensitive_key 2025-09-30 17:05:57 -07:00
malagsandGitHub 68189d1c04 [Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000) 
* Improved performance by reducing complexity

* Improved logic to prevent memory from increasing too much, added test

* Restore indent

* Restore indent

* Added type annotation

* Updated test to correctly initialize the expiration_heap
 2025-09-30 16:49:35 -07:00
Ishaan Jaffer f46f9d3fd9 docs azure passthrough api fixes 2025-09-30 15:54:09 -07:00
Krish DholakiaandGitHub c475723c67 Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1 
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
 2025-09-30 15:32:53 -07:00
Krish DholakiaandGitHub 5619d350b6 Merge pull request #15074 from Jetemple/custom-logo-fix 
Make UI theme settings publicly accessible for custom branding
 2025-09-30 15:26:00 -07:00
Ishaan JaffandGitHub 0476a33d9f [Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087) 
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
 2025-09-30 15:01:38 -07:00
Ishaan JaffandGitHub 69a464fc97 [Fix Security] Ensure OCI secret fields not shared on /models and /v1/models endpoints (#15085) 
* fix: remove_sensitive_info_from_deployment

* fix: remove_sensitive_info_from_deployment

* test_model_info_v1_oci_secrets_not_leaked
 2025-09-30 14:55:54 -07:00
Ishaan Jaffer 75d22d3d79 fix code qa check 2025-09-30 14:03:05 -07:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>ishaan-jaff
 		71b9b58fa9 [Feature]: Replace HTTPException with ParallelRequestLimitError in parallel_request_limiter_v3 (#15033) 
* Initial plan

* Implement ParallelRequestLimitError custom exception to replace HTTPException

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

* Add ParallelRequestLimitError to litellm main module exports

Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ishaan-jaff <29436595+ishaan-jaff@users.noreply.github.com>
 2025-09-30 13:57:14 -07:00
Alexsander HamirandGitHub 05dd104ce6 perf(router): Cache nested dict lookups in hot path (#15084) 
Cache deployment["litellm_params"] and deployment["model_info"]
at loop start to avoid repeated dict hash lookups.

- _pre_call_checks: 3 fewer lookups per deployment per request
- deployment_callback_on_failure: 1 fewer lookup per failure
- _set_model_group_info: 4 fewer lookups per model

Saves CPU cycles on every routing decision and failure callback.
 2025-09-30 13:36:42 -07:00
Alexsander HamirandGitHub 927e15996e perf(router): Remove unnecessary hasattr checks in get_model_list() (#15082) 
Remove redundant hasattr() checks for model_list and model_group_alias
in get_model_list() method. Both attributes are always initialized in
__init__, making these runtime checks unnecessary overhead.

Changes:
- Remove hasattr(self, "model_list") check
- Remove hasattr(self, "model_group_alias") check
- Move model_group_alias initialization earlier in __init__ to ensure
  it's available when set_model_list() calls get_model_names()
- Simplify control flow by removing nested conditional blocks

Performance Impact:
- hasattr should not appear in the profile of a proxy server handling thousands of requests. This change ensures it no longer does.
 2025-09-30 13:35:27 -07:00
Ishaan JaffandGitHub 60230e5666 [Feat] UI - add snowflake on UI (#15083) 
* UI - add snowflake on UI

* fixes snowflake creds
 2025-09-30 13:16:04 -07:00