Commit Graph

408 Commits

Author SHA1 Message Date
Julio Quinteros Pro bc9c28eb80 Merge pull request #22397 from BerriAI/fix/codeql-custom-workflow
fix(ci): replace default CodeQL with custom workflow to unblock CI
2026-02-28 17:19:42 -03:00
Ishaan Jaff b5f5b42035 bump: litellm-enterprise 0.1.32 → 0.1.33 + manual publish workflow (#22421)
* bump: litellm-enterprise 0.1.32 → 0.1.33

* ci: add manual workflow to publish litellm-enterprise to PyPI

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Apply suggestion from @greptile-apps[bot]

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* ci: add manual workflow to publish litellm-proxy-extras to PyPI

* fix(ci): commit before publish, add poetry.lock update to enterprise + proxy-extras workflows

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-28 10:56:15 -08:00
Julio Quinteros Pro ce0753243b Merge pull request #22193 from BerriAI/test/secret-scan-ci
test(ci): add secret scan test and CI job
2026-02-28 14:05:55 -03:00
Chesars 10a91c5199 fix(ci): remove duplicate env key in scan_duplicate_issues workflow
The greptile suggestion in #22034 was applied without removing the
original env block, leaving a duplicate env key that makes the YAML
invalid. GitHub fails to parse the workflow on every push to main,
creating failed run entries ("No jobs were run").
2026-02-28 13:27:59 -03:00
Cesar Garcia 7f5c8653f0 Merge pull request #18478 from Chesars/fix/prevent-scheduled-workflow-in-forks
fix: update_price_and_context_window workflow from running in forks
2026-02-28 13:10:15 -03:00
Julio Quinteros Pro d7340b595b Update .github/workflows/codeql.yml
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-28 12:16:42 -03:00
Julio Quinteros Pro 53f3123030 fix(ci): add custom CodeQL workflow to replace expensive default setup
The default CodeQL setup runs all 45 Python security queries against the
entire codebase. Two queries (CleartextLogging, PolynomialReDoS) produce
result sets > 2 GiB, causing 49+ minute runs that fail and block CI.

- Add custom workflow with 30-minute timeout and concurrency limits
- Exclude py/clear-text-logging-sensitive-data (CWE-312)
- Exclude py/polynomial-redos (CWE-730)
- Skip scanning tests/, docs/, and UI build output

NOTE: The Default Setup must be disabled in repo Settings > Code security
before merging, otherwise both will run simultaneously.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 11:40:22 -03:00
Julio Quinteros Pro 5a28ca985c Update .github/workflows/scan_duplicate_issues.yml
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-28 00:17:31 -03:00
Julio Quinteros Pro 94b7342da8 Update .github/workflows/check_duplicate_issues.yml
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-28 00:17:22 -03:00
Julio Quinteros Pro 1c376afc85 fix(ci): use secrets context in ggshield step condition
Step-level env is not visible to the if condition — reference
secrets directly so ggshield actually runs when the key is configured.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:51:28 -03:00
Julio Quinteros Pro 05c3a95da8 fix(ci): add permissions block to secret-scan job
Address github-advanced-security bot review comment by setting explicit
minimal permissions (contents: read) for the GITHUB_TOKEN.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:48:43 -03:00
Julio Quinteros Pro 2fce35a162 test(ci): add secret scan test and CI job to prevent hardcoded credentials
- Add unit test that scans Python source for Base64 Basic Auth patterns
  that would be flagged by secret scanners like GitGuardian/ggshield
- Add secret-scan job to the linting CI workflow that runs the test on
  every PR and optionally runs ggshield if GITGUARDIAN_API_KEY is set

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:46:42 -03:00
Julio Quinteros Pro db3d61f433 feat(ci): add duplicate issue detection and auto-close bot
Add a Python script that detects duplicate issues using title similarity
(difflib.SequenceMatcher) and closes them via the gh CLI. Two-tier system:
- 0.6 threshold: informational comment via existing wow-actions step
- 0.85 threshold: auto-close with comment, label, and not_planned reason

Includes a workflow_dispatch workflow for one-time batch scans and
integrates auto-close into the existing check_duplicate_issues workflow
for newly opened issues.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 16:49:06 -03:00
Ryan Crabbe 079ff24d78 Revert duplicate issue checker to text-based matching, remove duplicate PR workflow
Remove the Claude Code-powered duplicate PR detection workflow and revert
the duplicate issue checker back to wow-actions/potential-duplicates with
text similarity matching.
2026-02-23 15:28:13 -08:00
Krrish Dholakia a26f83fd3c fix: update calendly on repo 2026-02-23 06:13:59 -08:00
Ryan Crabbe c7d3198d9a fix: pass prompt as env var in duplicate detection workflows
Fixes "Input must be provided either through stdin or as a prompt
argument" error by moving the prompt to a PROMPT env variable
instead of inline multiline shell string.
2026-02-21 14:29:08 -08:00
Ryan Crabbe 1d0f91010b feat: switch duplicate detection workflows from opencode to Claude Code
Route through LiteLLM proxy using LITELLM_VIRTUAL_KEY and LITELLM_BASE_URL
secrets. Also adds --repo flag to all gh commands to fix missing repo context.
2026-02-20 17:51:12 -08:00
yuneng-jiang deeaae7e10 Merge pull request #21606 from BerriAI/litellm_ai-duplicate-issue-detection
feat: upgrade duplicate issue detection to be AI-powered instead of title text
2026-02-20 09:48:32 -08:00
Julio Quinteros Pro b551b98b26 ci: further split b2/b3 to isolate single heavy files
Isolate the two dominant files so they no longer block smaller tests:
- proxy-unit-b2: test_proxy_server.py alone (2750 lines)
- proxy-unit-b3: test_proxy_server_*.py + test_proxy_setting_guardrails.py (618 lines)
- proxy-unit-b4: test_proxy_utils.py alone (2339 lines)
- proxy-unit-b5: test_proxy_token_counter.py (1279 lines)
- proxy-unit-b6: test_[r-t]*.py (renamed from b4, 1988 lines)
- proxy-unit-b7: test_[u-z]*.py (renamed from b5, 2394 lines)

Matrix grows from 18 → 20 jobs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 14:08:25 -03:00
Julio Quinteros Pro c8ddbd90d1 ci: rebalance matrix groups based on actual timings
Split the two slowest groups based on measured wall-clock times:
- proxy-unit-b2 (was 7m15s, test_proxy_[s-z]*):
  → proxy-unit-b2: test_proxy_s*.py  (server + setting_guardrails, ~3368 lines)
  → proxy-unit-b3: test_proxy_[t-z]*.py (utils + token_counter, ~3618 lines)
- proxy-unit-b3 (was 4m30s, test_[r-z]*):
  → proxy-unit-b4: test_[r-t]*.py (response_polling + search + skills + realtime, ~1988 lines)
  → proxy-unit-b5: test_[u-z]*.py (user_api_key_auth + zero_cost + update_spend + unit_tests, ~2394 lines)

proxy-unit-a2 (6m15s) will self-resolve once PR #21679 merges
(55 skip markers added to test_key_generate_prisma.py).

Matrix grows from 16 → 18 jobs; all groups expected ≤ 3-4m.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 14:08:25 -03:00
Julio Quinteros Pro 1572162fdc ci: split slow test matrix groups to reduce wall-clock time
Three groups were bottlenecking CI (proxy-unit-b: 15min, other: 20+min,
proxy-unit-a: 6min). Split each into smaller parallel jobs based on
actual line counts of the test files.

proxy-unit-a (6min) → proxy-unit-a1 + proxy-unit-a2
  - a1: test_[a-j]*.py  (jwt 1564, auth_checks 978, google_gemini 478, ...)
  - a2: test_[k-o]*.py  (key_generate_prisma 4346, ...)

proxy-unit-b (15min) → proxy-unit-b1 + proxy-unit-b2 + proxy-unit-b3
  - b1: prisma/project/prompt + test_proxy_[c-r]*.py  (config, custom, routes, ...)
  - b2: test_proxy_[s-z]*.py  (proxy_server 2745, proxy_utils 2339, proxy_token_counter 1276)
  - b3: test_[r-z]*.py  (response_polling 1399, user_api_key_auth 1136, ...)

other (20+min) → other-1 + other-2 + other-3
  - other-1: responses (5942) + caching (1723) + types (819) ≈ 8.5k lines
  - other-2: enterprise (3062) + google_genai (2511) + router_utils (1982) ≈ 7.6k lines
  - other-3: remaining 11 dirs ≈ 8.0k lines

Total matrix jobs: 11 → 16. No test files are added, removed, or skipped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 14:08:25 -03:00
Ryan Crabbe dc13378505 feat: add AI-powered duplicate PR detection via opencode
Same approach as the duplicate issue detector — uses opencode run
with gh pr commands to find potentially duplicate open PRs when
external contributors open new PRs. Skips core team and bots.
2026-02-19 17:51:44 -08:00
Julio Quinteros Pro 11d0fca0de fix(ci): drop PAT_TOKEN_2 approval step, use github.token for auto-merge
PAT_TOKEN_2 does not have the scope for addPullRequestReview.
github.token cannot approve its own PR either, so drop the approval
step entirely. Auto-merge with github.token is enough: the PR will
merge automatically once required CI checks pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 21:17:40 -03:00
Julio Quinteros Pro 41776b0382 feat(ci): auto-approve and auto-merge the regenerated poetry.lock PR
Now that "Allow GitHub Actions to create and approve pull requests" is
enabled in repo settings:
- PR creation uses github.token (no secret needed)
- Approval uses PAT_TOKEN_2 (GitHub requires a different identity from
  the PR creator to approve)
- Auto-merge is enabled with --squash so the PR merges as soon as
  required checks pass

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 20:23:07 -03:00
Julio Quinteros Pro 9c70cd615a fix(ci): use PAT_TOKEN_2 for gh pr create
github.token cannot open PRs when "Allow GitHub Actions to create and
approve pull requests" is disabled in repo settings. PAT_TOKEN_2
bypasses that restriction.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 20:13:58 -03:00
Julio Quinteros Pro 08b5907c9c fix(ci): remove --no-update flag removed in Poetry 2.x
The workflow fails with:
  The option "--no-update" does not exist

--no-update was removed in Poetry 2.x. Plain `poetry lock` is the
correct equivalent — it re-solves only what pyproject.toml requires
without upgrading already-locked packages.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 20:07:14 -03:00
Julio Quinteros Pro 54470ec1d9 fix(ci): use github.token with explicit permissions instead of PAT secret
Drop the PAT_TOKEN_2 secret (whose scope is unknown) in favour of the
built-in github.token, which is always available. Grant it exactly the
two permissions it needs:
  - contents: write      → push the auto/regenerate-* branch
  - pull-requests: write → open the PR via gh cli

No external secret needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:56:59 -03:00
Julio Quinteros Pro 8cc50d6736 chore: fix stale GH_TOKEN comment 2026-02-19 19:55:44 -03:00
Julio Quinteros Pro e590674083 fix(ci): use PAT_TOKEN_2 instead of non-existent GH_TOKEN secret
GH_TOKEN is not configured in this repository. The correct PAT secret
is PAT_TOKEN_2, which has the permissions needed to push branches and
open PRs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:55:33 -03:00
Julio Quinteros Pro 755dd46d45 fix(ci): fall back to github.token when GH_TOKEN secret is not set
When secrets.GH_TOKEN is not configured, the workflow fails immediately with:
  "Input required and not supplied: token"

Using || github.token ensures a valid token is always available.
GH_TOKEN (PAT) is preferred when set; github.token is used as fallback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:52:01 -03:00
Julio Quinteros Pro 8edc6effb1 fix(ci): fix YAML syntax error in regenerate-poetry-lock workflow
A heredoc inside \$() inside a double-quoted string inside a YAML multiline
run block breaks the YAML parser at line 60. Replaced with: write the PR
body to /tmp/pr-body.md using a standalone heredoc, then pass it via
gh pr create --body-file.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:45:14 -03:00
Julio Quinteros Pro b9e79cc07b fix(ci): restrict GITHUB_TOKEN to contents:read via explicit permissions block
GitHub Advanced Security flagged that the workflow had no permissions block,
leaving GITHUB_TOKEN with its default broad scope. All write operations
(git push, gh pr create) already use GH_TOKEN (PAT), so the implicit
GITHUB_TOKEN only needs read access.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:24:57 -03:00
Julio Quinteros Pro 5681229e7c fix(ci): force-push bot branch to handle pre-existing branch from prior run
A re-run within the same second (or a leftover branch) would cause
`git push` to fail. Adding -f is safe since this is a bot-owned branch
that is immediately turned into a PR and never used for anything else.

Fixes inline suggestion from Greptile review.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:20:01 -03:00
Julio Quinteros Pro dfbf14e626 fix(ci): pass GH_TOKEN to checkout so git push can create the branch
Without the token in the checkout step the subsequent `git push` uses the
default GITHUB_TOKEN which lacks permission to push new branches, causing
the workflow to fail silently. Fixes issue flagged by Greptile review.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:18:18 -03:00
Julio Quinteros Pro d7d651dfeb ci: auto-regenerate poetry.lock when pyproject.toml changes on main
Adds a workflow that triggers whenever pyproject.toml is merged into main
and opens a PR with the refreshed lock file, fixing the recurring CI failure:
"pyproject.toml changed significantly since poetry.lock was last generated."

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 19:13:54 -03:00
Ryan Crabbe 6f0011ee2a perf: upgrade duplicate issue detection to AI-powered search via opencode
Replace the text-similarity based wow-actions/potential-duplicates with
opencode CLI + Claude Sonnet for semantic duplicate detection. The LLM
reads the full issue body and searches existing issues by keywords,
error messages, providers, and components — not just title similarity.

Requires ANTHROPIC_API_KEY repo secret.
2026-02-19 13:49:09 -08:00
Julio Quinteros Pro 24ead9b4e1 fix(ci): install enterprise package into main project venv, not enterprise's own venv
Running `cd enterprise && poetry run pip install -e .` causes poetry to
create a separate venv in `enterprise/.venv` (since enterprise/ has its
own pyproject.toml). The main project's tests run with `.venv/bin/python`,
so the enterprise package installed in `enterprise/.venv` is never seen.

Fix: run `poetry run pip install -e enterprise/` from the repo root so
poetry uses the main project's venv. This ensures litellm_enterprise is
importable when tests run.

This explains why enterprise tests kept failing with:
  AttributeError: '_PROXY_LiteLLMManagedFiles' object has no attribute
  '_check_file_deletion_allowed'
even after --force-reinstall was added — the reinstall was going to the
wrong virtual environment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 21:24:06 -03:00
jquinter b29ee632d8 Merge pull request #21481 from BerriAI/fix/enterprise-editable-install-ci
fix(ci): force-reinstall enterprise package to override PyPI version
2026-02-18 19:05:50 -03:00
yuneng-jiang 5625fc1537 Merge pull request #21495 from BerriAI/litellm_server_root_path_non_root
[Infra] Change Server Root Path GitHub action test to non root image
2026-02-18 13:56:08 -08:00
Julio Quinteros Pro bbbac1ae0e fix(ci): apply --force-reinstall --no-deps to enterprise install in all CI configs
The same PyPI-override issue existed in test-litellm.yml, test-mcp.yml,
and .circleci/config.yml. Also adds --no-deps (enterprise has no runtime
deps) to avoid redundant dependency resolution on every forced reinstall.

Addresses greptile review comments on PR #21481.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 18:53:45 -03:00
yuneng-jiang d1961072e8 adjusting the server root path test to non root image 2026-02-18 11:56:10 -08:00
Julio Quinteros Pro 262c16adf5 fix(ci): force-reinstall enterprise package to override PyPI version
poetry install includes litellm-enterprise from PyPI, then the editable
install step runs. When the same version is already installed, pip may
skip the editable install leaving the PyPI build in place - which may
lack methods added after the latest PyPI release. Adding
--force-reinstall ensures the local editable version always wins.

Fixes enterprise tests failing with AttributeError on methods that exist
locally but not in the cached PyPI-installed package.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-18 12:02:11 -03:00
Julio Quinteros Pro 58f23cbb80 fix(ci): add prisma generate step to matrix CI workflow
tests/proxy_unit_tests/test_key_generate_prisma.py imports PrismaClient
at module level, which triggers a Prisma binary check. Without running
prisma generate first, all tests in that file ERROR at collection time
with "Unable to find Prisma binaries. Please run 'prisma generate' first."

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-17 22:24:20 -03:00
Julio Quinteros Pro 44feb55840 improve(ci): enhance test stability with better isolation and distribution
Implements three key improvements to reduce test flakiness from parallel execution:

1. **Split Vertex AI tests into separate group** (workers: 1)
   - Vertex AI tests often have environment variable pollution issues
   - Running serially prevents cross-test interference with GOOGLE_APPLICATION_CREDENTIALS
   - Isolates authentication-related test failures

2. **Reduce workers for other LLM tests** (4 -> 2)
   - Decreases chance of race conditions and state conflicts
   - Still parallel but with less contention

3. **Add --dist=loadscope to pytest-xdist**
   - Keeps tests from the same file together on one worker
   - Reduces interference between unrelated test modules
   - Data shows 70% pass rate WITH loadscope vs 40% WITHOUT
   - Better test isolation while maintaining parallelism

Note: loadscope exposes one tokenizer cache issue in core-utils which will be
fixed in a separate PR. The tradeoff is worth it (7/10 pass vs 4/10 without).

These changes address the root causes of intermittent test failures in:
PRs #21268, #21271, #21272, #21273, #21275, #21276:
- Environment variable pollution (GOOGLE_APPLICATION_CREDENTIALS, VERTEXAI_PROJECT)
- Global state conflicts (litellm.known_tokenizer_config)
- Async mock timing issues with parallel execution

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 21:49:20 -03:00
Julio Quinteros Pro 48105e650b fix: remove pytest-retry to avoid conflicting retry plugins
- Remove pytest-retry from dev dependencies in pyproject.toml
- Add pytest-xdist as proper dev dependency (was only in pip install)
- Update CI workflow to reflect proper dependency management
- Prevents conflict between pytest-retry and pytest-rerunfailures

Having both pytest-retry and pytest-rerunfailures installed simultaneously
causes unpredictable behavior and excessive retries.
2026-02-17 16:55:37 -03:00
Julio Quinteros Pro bf157acccc fix: restore pytest-xdist to CI workflow
- Add pytest-xdist back to pip install line (required for -n flag)
- Was accidentally removed when removing pytest-retry
- Without pytest-xdist, all CI jobs fail with 'unrecognized option -n'
2026-02-17 16:08:24 -03:00
Julio Quinteros Pro bd9b239ed7 fix(ci): address Greptile review feedback
- Remove pytest-retry to avoid duplicate retry mechanisms (only use pytest-rerunfailures)
- Remove --dist loadgroup flag (no tests use xdist_group marker)
- Remove unused LITELLM_CI environment variable
- Remove sequential test step with error masking
- Simplify workflow for clarity

This fixes the issue where tests could be retried 60+ times due to
duplicate retry plugins (pytest-retry with retries=20 + pytest-rerunfailures
with --reruns 2-3).

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 15:58:33 -03:00
Julio Quinteros Pro 7a097ae97f fix(ci): reduce parallelism and add retry logic
- Reduce workers from 4 to 2 to avoid race conditions
  - Add --reruns with 2-3 retries per test group
  - Increase timeout from 15 to 20 minutes
  - Add better test isolation
2026-02-17 15:43:43 -03:00
yuneng-jiang 5ac3430866 only tests for /ui 2026-02-16 20:08:38 -08:00
yuneng-jiang a3ff9030ed passing in masster key for api calls 2026-02-16 19:15:12 -08:00