Commit Graph
16 Commits
Author SHA1 Message Date
Ishaan Jaffer e8461b5b97 style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
a6c30b30bf build: migrate packaging, CI, and Docker from Poetry to uv (#25007)
* build: migrate packaging metadata to uv

* ci: move automation and local tooling to uv

* docker: migrate image builds and runtime setup to uv

* docs: update install and deployment guidance for uv

* chore: align auxiliary scripts and tests with uv

* test: harden test_litellm isolation

* fix: keep release and health check images self-contained

* build: pin uv tooling and health check deps

* test: isolate bedrock image request formatting from suite state

* test: cover sandbox executor requirements flow

* ci: fix circleci no-op command steps

* ci: fix circleci publish workflow parsing

* fix: stabilize remaining uv migration CI checks

* ci: increase matrix test timeout headroom

* fix: restore published docker and license coverage

* fix: restore proxy runtime build parity

* fix: restore proxy extras parity and venv migrations

* ci: persist uv path across circleci steps

* fix: keep psycopg binary in default test env

* docker: preserve prisma cache across stages

* test: run local proxy checks through uv python

* build: restore runtime deps moved into ci

* build: refresh uv lock after upstream merge

* fix: restore module import in test_check_migration after merge

The conflict resolution imported only the function but the test body
references check_migration as a module throughout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching

- Move google-generativeai, Pillow, tenacity back to ci group (they are
  lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
  in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
  from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
  environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
  deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate uv.lock after removing nodejs-wheel-binaries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): use cache/restore instead of cache to prevent cache poisoning

The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert

The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv cache in publish workflow

Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): remove duplicate verbose_logger mock in test_check_migration

The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): free disk space before Docker build in test-server-root-path

The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 11:46:23 -07:00
yuneng-jiangandClaude Opus 4.6 a81a1968ed Fix test isolation: clear litellm.callbacks and model_fallbacks between tests
The isolate_litellm_state conftest fixture saved/restored litellm.callbacks
but never cleared it before each test, unlike the other callback lists. It
also didn't handle litellm.model_fallbacks. Leaked callbacks and fallback
config caused mocked tests to route through Router/fallback paths, hitting
real APIs with mock keys.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 16:43:15 -07:00
Julio Quinteros ProandClaude Sonnet 4.6 d7b0a8c559 fix(tests): restore disable_aiohttp_transport and force_ipv4 in isolate_litellm_state
Many tests across the llms group (sap, compactifai, vercel_ai_gateway, mistral,
zai, heroku) set litellm.disable_aiohttp_transport = True without restoring it.
When these tests run before test_ssl_context_transport or test_session_reuse_chain
in the same xdist worker, _create_async_transport() returns None (because aiohttp
is disabled AND force_ipv4 is False), causing both tests to fail with
'assert None is not None'.

Fix: extend isolate_litellm_state in conftest.py to also save and restore
disable_aiohttp_transport and force_ipv4, following the same pattern already
used for callbacks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-17 21:18:49 -03:00
Julio Quinteros Pro 890cc08a3a fix: resolve merge conflict and Greptile feedback
- Remove pytest-retry config from pyproject.toml (fixes merge conflict with main)
- Fix asymmetric callback restoration in isolate_litellm_state fixture
  - Now properly saves and restores all callback lists
  - Prevents test pollution from callback state leakage
- Add cache flush after module reload in setup_and_teardown
  - Prevents stale client instances after importlib.reload
- Remove unused imports (curr_dir, Router)

This addresses:
1. Merge conflict in pyproject.toml (CONFLICTING status)
2. Greptile's feedback about asymmetric callback handling
3. Missing cache flush after module reload
4. Code cleanliness (unused variables)
2026-02-17 18:20:44 -03:00
Julio Quinteros Pro e37befd5b4 fix: address Greptile feedback - remove duplicates and fix deprecations
- Remove duplicate @pytest.fixture decorator on setup_and_teardown
- Delete conftest_improved.py (duplicate file, pytest only loads conftest.py)
- Remove deprecated event_loop fixture override
- Add asyncio_default_fixture_loop_scope config in pyproject.toml (modern approach)

This fixes pytest-asyncio >=0.22 deprecation warnings while maintaining
session-scoped event loop behavior.
2026-02-17 18:20:43 -03:00
Julio Quinteros ProandClaude Sonnet 4.5 53d9cad8f5 fix(tests): restore autouse=True to setup_and_teardown fixture
Critical fix for Greptile feedback: The setup_and_teardown fixture was
missing the autouse=True parameter, causing the module reload logic to
never execute. This would result in test pollution as callbacks would
chain across modules.

Changes:
- Add autouse=True to setup_and_teardown fixture in conftest.py
- Add autouse=True to setup_and_teardown fixture in conftest_improved.py

Note: conftest_improved.py is intentionally kept as a reference
implementation showing the recommended improvements. It demonstrates
better patterns for test isolation that can be adopted later.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 18:20:23 -03:00
Julio Quinteros ProandClaude Sonnet 4.5 3b176a0970 fix(tests): remove asyncio.get_event_loop_policy deprecation warning
- Replace asyncio.get_event_loop_policy() with asyncio.new_event_loop()
- Use asyncio.set_event_loop() to set the event loop
- Fixes deprecation warning in Python 3.16
- Updated both conftest.py and conftest_improved.py

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 18:20:23 -03:00
Julio Quinteros Pro f9ea565b3b fix(tests): improve test isolation in conftest.py
- Move cache flushing to function scope
  - Disable module reload in parallel mode
  - Remove manual event loop creation
2026-02-17 18:20:23 -03:00
YutaSaitoandGitHub 8b33328cc1 Perf speed up pytest (#15951)
* perf: Skip sleep delays in base_mail.py during tests to improve test speed

* perf: Mock datetime.now in parallel_request_limiter_v3.py to improve test speed

* pref: Mock urllib system calls in test_aiohttp_transport.py to improve test speed

* chore: add --durations=50 to visualize slowest tests

* pref: reduce setup phase overhead by widening fixture scope in conftest.py

* test: stabilize flaky tests

* fix: minor issue
2025-10-27 19:43:40 -07:00
Ishaan JaffandGitHub 04dc1a5351 [Feat] Add support for returning images with gemini/gemini-2.5-flash-image-preview with /chat/completions (#13983)
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview

* add image in ChatCompletionResponseMessage

* test_gemini_image_generation_async

* Revert "Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers"

This reverts commit 539b94ad4e, reversing
changes made to 71af7bcf9c.

* include `image` in Delta

* fix _process_candidates should show the image response

* fix: _handle_special_delta_attributes

* test_gemini_image_generation_async_stream

* image_generation_chat

* UI - allow looking at generated images from /chat/completions

* _create_streaming_choice

* fix import StreamingChoices

* fix ChatCompletionResponseMessage

* test_gemini_image_generation

* add gemini img migration

* fix _extract_candidate_metadata

* ui fix

* fix batch endpoint test
2025-08-27 16:16:19 -07:00
Krish DholakiaandGitHub 900bd10905 Merge branch 'main' into feature/enhance_logging_for_containers 2025-08-26 23:21:15 -07:00
Ishaan JaffandGitHub b9132968b2 [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905)
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f48935652f054777652938ad71976cce.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
2025-08-23 13:13:23 -07:00
Davide Pugliese 81f2563338 Enhance logging for containers 2025-08-08 18:28:35 +02:00
Ishaan JaffandGitHub 39955129f5 fix mapped tests (#12320)
* fix - use flush llm client cache

* faster mapped tests

* test_async_multiple_response_ids_routing

* fix tests

* test_ateam_member_update_admin_requires_premium

* regular mapped tests

* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206)"

This reverts commit 2c60c316ec.

* reset num workers
2025-07-04 10:04:43 -07:00
Krish DholakiaandGitHub ef42461c1e Litellm fix GitHub action testing (#11163)
* test: add __init__.py files

* refactor: rename test folder to avoid naming conflict

* test: update workflows

* test: update tests

* test: update imports

* test: update tests

* test: remove unused import

* ci(test-litellm.yml): add pytest retry to github workflow

* test: fix test
2025-05-26 14:41:42 -07:00