Commit Graph

36366 Commits

Author SHA1 Message Date
yuneng-jiang 5cec43cbb6 Merge pull request #24802 from BerriAI/litellm_/modest-easley
[Refactor] Extract helper methods in guardrail handlers to fix PLR0915
2026-03-30 11:42:12 -07:00
yuneng-jiang 0ccaff8ed9 Merge pull request #24805 from BerriAI/litellm_/compassionate-kirch
[Test] Mock DeepInfra completion tests to avoid real API calls
2026-03-30 11:42:00 -07:00
Yuneng Jiang 868468db14 [Test] Mock DeepInfra completion tests to avoid real API calls
The DeepInfra tests were making real API calls and failing with AuthenticationError.
Mock the HTTP layer to verify request shape instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 11:07:34 -07:00
Yuneng Jiang 92e7aaa001 [Refactor] Extract helper methods in guardrail handlers to fix PLR0915
Ruff flagged `process_output_streaming_response` (A2A) and
`process_output_response` (Anthropic) for exceeding 50 statements.
Extract inline logic into private helpers to bring both under the limit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 10:41:45 -07:00
Krrish Dholakia a278933f18 docs: cleanup 2026-03-30 09:41:09 -07:00
Krrish Dholakia c154d5668b Merge pull request #24792 from BerriAI/ossf-scorecord
adopt OpenSSF Scorecard (https://scorecard.dev/)
2026-03-30 09:37:34 -07:00
Krrish Dholakia 1eaf8ed2d5 Merge pull request #24800 from BerriAI/litellm_vanta_announcement
Litellm vanta announcement
2026-03-30 09:27:55 -07:00
Krrish Dholakia 0128925281 docs: cleanup image 2026-03-30 09:26:46 -07:00
Krrish Dholakia bc89ad28ab docs: cleanup 2026-03-30 09:19:48 -07:00
Krrish Dholakia 5671baaadf fix: fix setup wizard 2026-03-30 09:06:31 -07:00
Krrish Dholakia 05134fc70b Create scorecard.yml 2026-03-30 07:47:06 -07:00
Krrish Dholakia 58120537af Merge pull request #24756 from BerriAI/litellm_/pensive-sinoussi
[Fix] Remove NLP_CLOUD_API_KEY requirement from test_exceptions
2026-03-28 22:04:49 -07:00
Krrish Dholakia a671083596 Merge pull request #24755 from BerriAI/litellm_test_cleanup
Litellm test cleanup
2026-03-28 22:04:26 -07:00
Krrish Dholakia 85cc7bc433 refactor: make unit test 2026-03-28 21:44:46 -07:00
Krrish Dholakia 1ba109079d test: update tests 2026-03-28 21:22:04 -07:00
Krrish Dholakia 9e070143fb test: update key names 2026-03-28 21:13:16 -07:00
Krrish Dholakia 25f2baad71 test: cleanup dead tests 2026-03-28 20:49:02 -07:00
Yuneng Jiang 14095ee144 [Fix] Remove NLP_CLOUD_API_KEY requirement from test_exceptions.py
The test only needs a bad key to verify AuthenticationError handling.
No other tests use this env var, and it's not provisioned in CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 20:38:54 -07:00
Krrish Dholakia a92b31a636 test: remove dead tests 2026-03-28 20:36:08 -07:00
Krrish Dholakia fe379fd738 test: rename env var 2026-03-28 20:27:39 -07:00
Krrish Dholakia 0fef88d67c test: remove dead tests 2026-03-28 20:23:44 -07:00
yuneng-jiang c27114470e Merge pull request #24754 from BerriAI/litellm_gha_p3b
[Infra] Remove CircleCI jobs now covered by GitHub Actions
2026-03-28 20:23:27 -07:00
Krrish Dholakia 1fb677702d test: update to new vertex ai keys 2026-03-28 20:19:05 -07:00
Krrish Dholakia 92db2df2f6 Merge pull request #23794 from ndgigliotti/feat/bedrock-structured-output-cost-json
Bedrock: move native structured output model list to cost JSON, add Sonnet 4.6
2026-03-28 20:04:47 -07:00
Krrish Dholakia ce26dd4101 test: update tests 2026-03-28 19:42:14 -07:00
Krrish Dholakia cdcab8a243 refactor: cleanup deprecated models 2026-03-28 19:39:11 -07:00
Krrish Dholakia bc829d51f2 test: test 2026-03-28 19:17:38 -07:00
Yuneng Jiang 7aec9101f5 [Infra] Remove CircleCI jobs now covered by GitHub Actions
Removes 10 CircleCI jobs that are fully duplicated by GHA workflows:
- caching_unit_tests → test-unit-caching-redis.yml
- litellm_security_tests → test-unit-security.yml
- litellm_proxy_unit_testing_{key_generation,part1,part2} → test-unit-proxy-db.yml + test-unit-proxy-legacy.yml
- litellm_mapped_tests_{llms,core,litellm_core_utils,mcps,integrations} → test-unit-*.yml workflows

Also cleans up upload-coverage requires and workflow entries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 19:06:48 -07:00
Krrish Dholakia a41ba7bb6a test: update test 2026-03-28 19:01:55 -07:00
Krrish Dholakia bee1607248 test: update test apis 2026-03-28 18:57:27 -07:00
Krrish Dholakia 564ad3195b test: update testing 2026-03-28 18:42:55 -07:00
Krrish Dholakia 5cd8ca2365 refactor: refactor testing 2026-03-28 18:39:32 -07:00
Krrish Dholakia 6df995d0d1 refactor: dead test cleanup 2026-03-28 18:27:25 -07:00
Krrish Dholakia 44b03e6138 fix: fix azure tests 2026-03-28 18:19:41 -07:00
Krrish Dholakia c75fb3650a fix: fix azure audio test 2026-03-28 18:14:01 -07:00
Krrish Dholakia b63b43fe09 docs: add faq to docs 2026-03-28 18:14:01 -07:00
Krrish Dholakia bc33cf662f Merge pull request #24449 from J-Byron/feat/prometheus-org-budget-metrics
Feat/prometheus org budget metrics
2026-03-28 17:32:17 -07:00
yuneng-jiang 40d4e79a00 Merge pull request #24742 from BerriAI/litellm_gha_p3
[Infra] Add unit test workflows for Postgres, Redis, and security suites
2026-03-28 14:54:20 -07:00
Yuneng Jiang 3b5b98327e [Fix] Use integration-redis-postgres env for Redis workflows since Postgres always starts
GHA doesn't support conditional service containers, so the Postgres container
always starts even for Redis-only jobs. Use integration-redis-postgres
environment for any workflow with enable-redis so the Postgres container gets
valid credentials.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 14:25:29 -07:00
Yuneng Jiang 3ae80407dd [Fix] Move Postgres username and password to environment secrets
Move POSTGRES_USER and POSTGRES_PASSWORD from hardcoded values to
environment secrets so no credentials appear in workflow files at all.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 13:31:58 -07:00
Yuneng Jiang d42e2f6429 [Fix] Move Postgres DATABASE_URL to environment secret to avoid credential leak warnings
The hardcoded postgresql://postgres:postgres@localhost connection string was
being flagged by secret scanners. Move DATABASE_URL to a GHA environment
secret (integration-postgres) so the password is never in the workflow file.

Changes:
- _test-unit-services-base.yml: DATABASE_URL now comes from secrets, environment
  is derived from enable-* flags (integration-postgres, integration-redis, or
  integration-redis-postgres)
- test-unit-proxy-db.yml: switched to push-only trigger (uses secrets now)
- test-unit-security.yml: switched to push-only trigger (uses secrets now)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 13:28:41 -07:00
Yuneng Jiang 6549f3eb1a [Infra] Add unit test workflows for Postgres, Redis, and security test suites
Add three new GHA workflows for tests requiring service containers, plus a
reusable base workflow that provides Postgres and cloud Redis support.

New workflows:
- test-unit-proxy-db.yml: proxy DB tests (key generation, auth checks,
  remaining) using a local Postgres container with a 3-way descriptive matrix
- test-unit-caching-redis.yml: caching tests that need Redis but no provider
  API keys, using cloud Redis via the integration-redis environment
- test-unit-security.yml: proxy security tests using a local Postgres container

Reusable base (_test-unit-services-base.yml):
- Local Postgres pinned by digest (postgres@sha256:705a5d5b...)
- Cloud Redis credentials scoped to the integration-redis GHA environment
- Environment binding is derived from enable-redis flag inside the base
  (not caller-controllable) to prevent secret scope bypass
- Supports workers=0 for tests that cannot run in parallel

Security hardening:
- All actions pinned to commit SHAs
- persist-credentials: false on all checkouts
- permissions: contents: read only
- Postgres-only workflows (proxy-db, security) use zero secrets and trigger on
  both pull_request and push to main/litellm_*
- Redis workflow triggers on push only (not pull_request) to prevent external
  PRs from accessing Redis Cloud credentials
- Added ${TEST_PATH:?} guard to both _test-unit-base.yml and
  _test-unit-services-base.yml to fail fast on empty test paths
- All files pass zizmor --pedantic with zero findings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 12:06:45 -07:00
yuneng-jiang 666a31d47a Merge pull request #24741 from BerriAI/litellm_gha_p2
[Fix] Test Isolation and Path Resolution for GHA Unit Tests
2026-03-28 11:32:48 -07:00
Yuneng Jiang 7851567091 [Fix] Scope documentation workflow to match CircleCI and add missing router settings
Revert path fixes for documentation tests that CircleCI never ran
(test_exception_types, test_general_setting_keys, test_readme_providers,
test_standard_logging_payload). Update the GHA workflow to run only the
4 tests CircleCI actually executed: test_env_keys, test_router_settings,
test_api_docs, test_circular_imports.

Add 2 missing router_settings keys (enable_health_check_routing,
health_check_staleness_threshold) and 27 missing general_settings keys
to config_settings.md so test_router_settings passes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 11:23:53 -07:00
Yuneng Jiang 7100ed5d0a [Fix] Test isolation for agent health checks and documentation test path resolution
Fix agent health check tests failing with 500 errors in parallel CI by
mocking prisma_client to None. Fix documentation validation tests using
CWD-relative paths that break depending on the working directory.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 11:00:22 -07:00
yuneng-jiang 428d837704 Merge pull request #24740 from BerriAI/litellm_unit_test_workflow_isolation
[Infra] Isolate unit test workflows with hardened security posture
2026-03-28 10:30:13 -07:00
Yuneng Jiang c717189ed2 [Infra] Remove workflows that require API keys or external services
These test suites are not pure unit tests and don't belong in Phase 1:
- litellm_utils_tests: health check tests need OPENAI_API_KEY
- pass_through_unit_tests: tests hit real Anthropic API
- router_unit_tests: tests call real OpenAI moderation endpoints
- proxy_security_tests: requires DATABASE_URL (Postgres)
- documentation_tests: requires docs directory at specific relative path

These will be re-added in later phases with proper secret scoping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 10:16:19 -07:00
Yuneng Jiang a34ed20901 [Infra] Fix job naming in reusable workflow callers
Rename job keys from generic 'test' to descriptive names (e.g.,
'core-utils', 'proxy-auth', 'router') so GitHub checks display as
'core-utils / run' instead of 'test / test'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 10:07:32 -07:00
Yuneng Jiang 3d527b722d [Infra] Add isolated unit test workflows with hardened security posture
Replace monolithic matrix workflow with individual, descriptively-named
workflow files. Each workflow uses a shared reusable base and follows
least-privilege security: zero secrets, read-only permissions, SHA-pinned
actions, persist-credentials: false, and env-var indirection to prevent
template injection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 09:56:58 -07:00
ryan-crabbe-berri 2eb3c20e76 Merge pull request #24718 from BerriAI/litellm_ryan-march-26
litellm ryan march 26
2026-03-28 09:01:11 -07:00