tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-23 20:26:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
33,391 Commits 2,072 Branches 1,425 Tags
73fd5a41e48acb76952bb76bccafb166bd30dca2
Commit Graph
6757 Commits
Author SHA1 Message Date
Ron ZhongandGitHub 73fd5a41e4 feat: Singapore guardrail policies (PDPA + MAS AI Risk Management) (#21948) 
* feat: Singapore PDPA PII protection guardrail policy template

Add Singapore Personal Data Protection Act (PDPA) guardrail support:

Regex patterns (patterns.json):
- sg_nric: NRIC/FIN detection ([STFGM] + 7 digits + checksum letter)
- sg_phone: Singapore phone numbers (+65/0065/65 prefix)
- sg_postal_code: 6-digit postal codes (contextual)
- passport_singapore: Passport numbers (E/K + 7 digits, contextual)
- sg_uen: Unique Entity Numbers (3 formats)
- sg_bank_account: Bank account numbers (dash format, contextual)

YAML policy templates (5 sub-guardrails):
- sg_pdpa_personal_identifiers: s.13 Consent
- sg_pdpa_sensitive_data: Advisory Guidelines
- sg_pdpa_do_not_call: Part IX DNC Registry
- sg_pdpa_data_transfer: s.26 overseas transfers
- sg_pdpa_profiling_automated_decisions: Model AI Governance Framework

Policy template entry in policy_templates.json with 9 guardrail definitions
(4 regex-based + 5 YAML conditional keyword matching).

Tests:
- test_sg_patterns.py: regex pattern unit tests
- test_sg_pdpa_guardrails.py: conditional keyword matching tests (100+ cases)

* feat: MAS AI Risk Management Guidelines guardrail policy template

Add Monetary Authority of Singapore (MAS) AI Risk Management Guidelines
guardrail support for financial institutions:

YAML policy templates (5 sub-guardrails):
- sg_mas_fairness_bias: Blocks discriminatory financial AI (credit/loans/insurance by protected attributes)
- sg_mas_transparency_explainability: Blocks opaque/unexplainable AI for consequential financial decisions
- sg_mas_human_oversight: Blocks fully automated financial decisions without human-in-the-loop
- sg_mas_data_governance: Blocks unauthorized sharing/mishandling of financial customer data
- sg_mas_model_security: Blocks adversarial attacks, model poisoning, inversion on financial AI

Policy template entry in policy_templates.json with 5 guardrail definitions.
Aligned with MAS FEAT Principles, Project MindForge, and NIST AI RMF.

Tests:
- test_sg_mas_ai_guardrails.py: conditional keyword matching tests (100+ cases)

* fix: address SG pattern review feedback

- Update NRIC lowercase test for IGNORECASE runtime behavior
- Add keyword context guard to sg_uen pattern to reduce false positives

* docs: clarify MAS AIRM timeline references

- Explicitly mark MAS AIRM as Nov 2025 consultation draft
- Add 2018 qualifier for FEAT principles in MAS policy descriptions
- Update MAS guardrail wording to avoid release-year ambiguity

* chore: commit resolved MAS policy conflicts

* test:

* chore:
 2026-02-23 12:08:22 -08:00
Julio Quinteros ProandGitHub 36813199b6 Merge pull request #21943 from jquinter/fix/interactions-incomplete-status 
fix(tests): add INCOMPLETE to interactions status enum expected values
 2026-02-23 16:31:41 -03:00
Julio Quinteros ProandClaude Opus 4.6 f94d0fe0b6 fix: add INCOMPLETE status to Interactions API enum and test 
Google added INCOMPLETE to the Interactions API OpenAPI spec status enum.
Update both the Status3 enum in the SDK types and the test's expected
values to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 15:07:41 -03:00
ryan-crabbeandGitHub c4c48fe977 Merge pull request #21942 from BerriAI/litellm_network_mock 
feat: Litellm network mock
 2026-02-23 10:07:11 -08:00
Ryan Crabbe d99d87f614 clean up mock transport: remove streaming, add defensive parsing 2026-02-23 09:16:47 -08:00
Julio Quinteros ProandClaude Opus 4.6 bf8c219860 fix(tests): use os.path instead of Path to avoid NameError 
Path is not imported at module level. Use os.path.join which is already
available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 13:56:11 -03:00
Julio Quinteros Proandgreptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> a74b6eee23 Update tests/test_litellm/test_utils.py 
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
 2026-02-23 13:55:49 -03:00
Julio Quinteros ProandClaude Opus 4.6 11a774e110 fix(tests): use absolute path for model_prices JSON in validation test 
The test used a relative path 'litellm/model_prices_and_context_window.json'
which only works when pytest runs from a specific working directory.
Use os.path based on __file__ to resolve the path reliably.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 13:55:49 -03:00
Julio Quinteros ProandClaude Opus 4.6 fb8b11cc0a fix(tests): use counter-based mock for time.time in prisma self-heal test 
The test used a fixed side_effect list for time.time(), but the number
of calls varies by Python version, causing StopIteration on 3.12 and
AssertionError on 3.14. Replace with an infinite counter-based callable
and assert the timestamp was updated rather than checking for an exact
value.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-23 13:25:02 -03:00
Sameer KankuteandGitHub f97ee62fb0 Merge pull request #21909 from BerriAI/litellm_cost_tracking_gemini 
Add  Priority PayGo cost tracking gemini/vertex ai
 2026-02-23 18:58:57 +05:30
Sameer KankuteandGitHub 61e63b6553 Merge pull request #21904 from BerriAI/litellm_fix_model_cost_map 
fix model cost map for anthropic fast and inference_geo
 2026-02-23 18:57:15 +05:30
Sameer Kankute 2f8d36be1b Fix test_aaamodel_prices_and_context_window_json_is_valid 2026-02-23 18:56:12 +05:30
Sameer KankuteandGitHub 9b5bbee906 Merge pull request #21786 from BerriAI/litellm_oss_staging_02_21_2026 
Litellm oss staging 02 21 2026
 2026-02-23 18:51:55 +05:30
Sameer KankuteandGitHub 8decf04d8a Merge pull request #21877 from BerriAI/litellm_oss_staging_02_22_2026 
Litellm oss staging 02 22 2026
 2026-02-23 18:50:47 +05:30
Sameer KankuteandGitHub 37d45139f2 Merge pull request #21917 from BerriAI/litellm_fix_model_cost_map_wildcard 
Fix: Anthropic model wildcard access issue
 2026-02-23 18:45:49 +05:30
TomAlonandGitHub 99184c48d9 Add Noma guardrails v2 based on custom guardrails (#21400) 2026-02-23 05:05:27 -08:00
Sameer KankuteandGitHub c7aafdf794 Merge pull request #21926 from BerriAI/main 
merge main in oss 21 02
 2026-02-23 18:17:30 +05:30
Sameer KankuteandGitHub 57af8e6a93 Merge pull request #21924 from BerriAI/main 
merge main in oss 22 02
 2026-02-23 18:11:36 +05:30
Sameer Kankute 4ff1651699 Fix: Anthropic model wildcard access issue 2026-02-23 17:12:55 +05:30
Harshit Jain 9fc3c77c42 fix: ensure arrival_time is set before calculating queue time 2026-02-23 17:04:47 +05:30
Sameer KankuteandGitHub 3561bfb96c Merge pull request #21640 from ta-stripe/feat/regional-sts-endpoint-for-auth_with_role_name 
feat(bedrock): support optional regional STS endpoint in role assumption
 2026-02-23 13:28:54 +05:30
Sameer KankuteandGitHub 55ee8cdd56 Merge pull request #21701 from ta-stripe/fix/bedrock-openai-imported-model-name-encoding 
fix(bedrock): encode model arns for OpenAI compatible bedrock imported models
 2026-02-23 13:25:02 +05:30
Sameer Kankute f54fb9aeb1 Add tests for fast and us 2026-02-23 11:25:47 +05:30
Sameer Kankute 22bccc4f61 Fix entries with fast and us/ 2026-02-23 11:23:24 +05:30
Harshit JainandGitHub 304862175e Merge branch 'main' into litellm_fix_langfuse_otel_trace_v2 2026-02-22 19:54:05 +05:30
Harshit28j 1cc185eb16 fix: update opentelemetry with greptile review 2026-02-22 19:53:18 +05:30
Krish DholakiaandGitHub 76ccc9e844 Guardrail Policy Versioning (#21862) 
* feat: initial commit, adding support for policy versioning on litellm

* fix(policy_registry): support policy versioning

* fix: multiple QA fixes for policy flow builder with guardrail versioning on litellm

* feat: ui improvements

* feat: add prisma migration

* fix: address greptile fixes
 2026-02-21 20:14:31 -08:00
Krish DholakiaandGitHub 52585eb2d7 Revert "fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)" (#21876) 
This reverts commit bce078a796.
 2026-02-21 20:12:01 -08:00
Edwin IsacGitHubyuneng-jiangmilan-berriIshaan Jaff
 		bce078a796 fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870) 
* server root path regression doc

* fixing syntax

* fix: replace Zapier webhook with Google Form for survey submission (#21621)

* Replace Zapier webhook with Google Form for survey submission

* Add back error logging for survey submission debugging

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth"

This reverts commit 0e1db3f7e4, reversing
changes made to 7e2d6f2355.

* test_vertex_ai_gemini_2_5_pro_streaming

* UI new build

* fix rendering

* ui new build

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* release note docs

* docs

* adding image

* fix(vertex_ai): enable context-1m-2025-08-07 beta header

The `context-1m-2025-08-07` Anthropic beta header was set to `null` for vertex_ai,
causing it to be filtered out when users set `extra_headers: {anthropic-beta: context-1m-2025-08-07}`.

This prevented using Claude's 1M context window feature via Vertex AI, resulting in
`prompt is too long: 460500 tokens > 200000 maximum` errors.

Fixes #21861

---------

Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: milan-berri <milan@berri.ai>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2026-02-21 20:11:13 -08:00
LeeJuOhandGitHub 50f36d9ca6 fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo (#21754) 
* fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo

* fix(budget): update stale docstring on get_budget_reset_time
 2026-02-21 19:35:06 -08:00
Ryan Crabbe 94b76ea9ad feat: add network_mock transport for benchmarking proxy overhead without real API calls 
Intercepts at httpx transport layer so the full proxy path (auth, routing,
OpenAI SDK, response transformation) is exercised with zero-latency responses.
Activated via `litellm_settings: { network_mock: true }` in proxy config.
 2026-02-21 17:52:39 -08:00
Ishaan Jaffer 52294029a0 test_vertex_ai_gemini_2_5_pro_streaming 2026-02-21 16:59:22 -08:00
Ishaan Jaffer 2270a3aaf3 Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth" 
This reverts commit 0e1db3f7e4, reversing
changes made to 7e2d6f2355.
 2026-02-21 16:57:42 -08:00
Ryan Crabbe 643c9b6c04 Merge remote-tracking branch 'origin/main' into litellm_perf_user_api_key_auth 2026-02-21 16:03:31 -08:00
Ishaan Jaffer d31d5b8486 fix failing tests 2026-02-21 15:48:26 -08:00
Ishaan Jaffer 26ea29afd3 test_get_usage_as_dict 2026-02-21 15:39:06 -08:00
Krish DholakiaandGitHub 1f7eeb274c Agent Builder - improve rejected response detection based on agent response (#21850) 
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent

* feat: new response rejection detector

* fix: multiple fixes

* feat: add mcp tools support to agent builder

create an agent with access to llm's + mcp servers
 2026-02-21 15:34:42 -08:00
Krish DholakiaandGitHub 9fc6fd647c Agent Builder - support new experimental agent builder, to ensure agents pass compliance checks (#21817) 
* fix: feat: add litellm_system_prompt support

* feat: support new 'litellm_agent' model provider

* feat: ui/ - new agent builder ui

* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal

* feat(agentbuilderview.tsx): run compliance datasets against litellm agent
 2026-02-21 15:32:47 -08:00
Ishaan JaffandGitHub bab4127cae fix(tests): fix flaky test_use_prisma_db_push_flag_behavior (#21849) 
Replace Click CliRunner with standalone_mode=False to avoid
"I/O operation on closed file" errors caused by Click's stream
isolation in CI environments.
 2026-02-21 15:23:55 -08:00
Ishaan JaffandGitHub f74a1c94df test(router): add coverage tests for _is_complexity_router_deployment and init_complexity_router_deployment (#21848) 2026-02-21 15:21:10 -08:00
Darien KindlundGitHubClaude Opus 4.6
 		ca5c109a92 feat: add optional digest mode for Slack alert types (#21683) 
Adds per-alert-type digest mode that aggregates duplicate alerts
within a configurable time window and emits a single summary message
with count, start/end timestamps.

Configuration via general_settings.alert_type_config:
  alert_type_config:
    llm_requests_hanging:
      digest: true
      digest_interval: 86400

Digest key: (alert_type, request_model, api_base)
Default interval: 24 hours
Window type: fixed interval

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-02-21 15:19:17 -08:00
Ryan Crabbe c7ad8053b1 Merge origin/main into litellm_perf_user_api_key_auth 
Resolve conflicts:
- pass_through_endpoints.py: take main's version, re-apply
  MAPPED_PASS_THROUGH_PREFIXES startswith(tuple) optimization
- test_user_api_key_auth.py: keep both auth optimization regression
  tests and JWT admin identity field tests
 2026-02-21 15:14:20 -08:00
Ishaan JaffandGitHub 9f459c5c57 fix(logging): preserve pass-through endpoint response_cost (#21844) 
* fix(logging): preserve pass-through endpoint response_cost in async_success_handler

Two places in the logging pipeline were overwriting response_cost that
pass-through handlers (Gemini/Vertex) had already calculated:

1. _process_hidden_params_and_response_cost fell through to
   _response_cost_calculator which returns None for pass-through calls
2. async_success_handler pass-through branch unconditionally set
   response_cost = None (introduced in PR #19887)

Now both places check if response_cost is already set before overwriting.

* test: add regression test for pass-through endpoint response_cost preservation
 2026-02-21 15:09:45 -08:00
Ryan Crabbe bb24ebebd9 Merge origin/main into litellm_perf_convert_model_response_frozensets 
Resolve conflict: keep main's provider_specific_fields passthrough
preservation while using frozenset set-difference optimization.
 2026-02-21 14:52:43 -08:00
Ishaan JaffandGitHub 8afeaf8da4 fix(tests): fix flaky test_create_vertex_fine_tune_jobs_mocked - handle background Datadog flush (#21838) 2026-02-21 14:44:01 -08:00
Ishaan JaffandGitHub d7b22d340b fix(tests): move test_router_azure_acompletion to llm_translation testing (#21837) 2026-02-21 14:41:53 -08:00
Ishaan JaffandGitHub 59e5b7e8c6 fix(tests): use monkeypatch.setenv for Redis pool max_connections tests (#21834) 
Replace patch('litellm._redis._get_redis_client_logic') with monkeypatch.setenv
in test_max_connections_url_config and test_max_connections_url_config_string_value.

The mock was unreliable in CI (REDIS_URL is set to the real Redis Cloud server),
causing the pool to silently use the real config instead of the test config.
Using monkeypatch.setenv tests the full env-var→pool chain more robustly and
matches the actual production code path.
 2026-02-21 14:38:28 -08:00
Ishaan JaffandGitHub 235a47c576 fix(tests): mock test_claude_tool_use_with_gemini to fix flaky CI (#21832) 
* ui fixes

* fix(tests): mock test_claude_tool_use_with_gemini to avoid MALFORMED_FUNCTION_CALL flakiness
 2026-02-21 14:34:54 -08:00
Ishaan JaffandGitHub fb4249005e fix(tests): add atexit.register mock to prevent Click isolation stream closure in test_use_prisma_db_push_flag_behavior (#21829) 2026-02-21 14:28:02 -08:00
yuneng-jiangandGitHub 8c5be4cb62 Revert "fix(proxy): recover from prisma-query-engine zombie process (#21707)" 
This reverts commit 977ad015ca.
 2026-02-21 14:20:06 -08:00