Commit Graph
14 Commits
Author SHA1 Message Date
yuneng-jiangandClaude Opus 4.6 a81a1968ed Fix test isolation: clear litellm.callbacks and model_fallbacks between tests
The isolate_litellm_state conftest fixture saved/restored litellm.callbacks
but never cleared it before each test, unlike the other callback lists. It
also didn't handle litellm.model_fallbacks. Leaked callbacks and fallback
config caused mocked tests to route through Router/fallback paths, hitting
real APIs with mock keys.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 16:43:15 -07:00
Julio Quinteros ProandClaude Sonnet 4.6 d7b0a8c559 fix(tests): restore disable_aiohttp_transport and force_ipv4 in isolate_litellm_state
Many tests across the llms group (sap, compactifai, vercel_ai_gateway, mistral,
zai, heroku) set litellm.disable_aiohttp_transport = True without restoring it.
When these tests run before test_ssl_context_transport or test_session_reuse_chain
in the same xdist worker, _create_async_transport() returns None (because aiohttp
is disabled AND force_ipv4 is False), causing both tests to fail with
'assert None is not None'.

Fix: extend isolate_litellm_state in conftest.py to also save and restore
disable_aiohttp_transport and force_ipv4, following the same pattern already
used for callbacks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-17 21:18:49 -03:00
Julio Quinteros Pro 890cc08a3a fix: resolve merge conflict and Greptile feedback
- Remove pytest-retry config from pyproject.toml (fixes merge conflict with main)
- Fix asymmetric callback restoration in isolate_litellm_state fixture
  - Now properly saves and restores all callback lists
  - Prevents test pollution from callback state leakage
- Add cache flush after module reload in setup_and_teardown
  - Prevents stale client instances after importlib.reload
- Remove unused imports (curr_dir, Router)

This addresses:
1. Merge conflict in pyproject.toml (CONFLICTING status)
2. Greptile's feedback about asymmetric callback handling
3. Missing cache flush after module reload
4. Code cleanliness (unused variables)
2026-02-17 18:20:44 -03:00
Julio Quinteros Pro e37befd5b4 fix: address Greptile feedback - remove duplicates and fix deprecations
- Remove duplicate @pytest.fixture decorator on setup_and_teardown
- Delete conftest_improved.py (duplicate file, pytest only loads conftest.py)
- Remove deprecated event_loop fixture override
- Add asyncio_default_fixture_loop_scope config in pyproject.toml (modern approach)

This fixes pytest-asyncio >=0.22 deprecation warnings while maintaining
session-scoped event loop behavior.
2026-02-17 18:20:43 -03:00
Julio Quinteros ProandClaude Sonnet 4.5 53d9cad8f5 fix(tests): restore autouse=True to setup_and_teardown fixture
Critical fix for Greptile feedback: The setup_and_teardown fixture was
missing the autouse=True parameter, causing the module reload logic to
never execute. This would result in test pollution as callbacks would
chain across modules.

Changes:
- Add autouse=True to setup_and_teardown fixture in conftest.py
- Add autouse=True to setup_and_teardown fixture in conftest_improved.py

Note: conftest_improved.py is intentionally kept as a reference
implementation showing the recommended improvements. It demonstrates
better patterns for test isolation that can be adopted later.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 18:20:23 -03:00
Julio Quinteros ProandClaude Sonnet 4.5 3b176a0970 fix(tests): remove asyncio.get_event_loop_policy deprecation warning
- Replace asyncio.get_event_loop_policy() with asyncio.new_event_loop()
- Use asyncio.set_event_loop() to set the event loop
- Fixes deprecation warning in Python 3.16
- Updated both conftest.py and conftest_improved.py

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-17 18:20:23 -03:00
Julio Quinteros Pro f9ea565b3b fix(tests): improve test isolation in conftest.py
- Move cache flushing to function scope
  - Disable module reload in parallel mode
  - Remove manual event loop creation
2026-02-17 18:20:23 -03:00
YutaSaitoandGitHub 8b33328cc1 Perf speed up pytest (#15951)
* perf: Skip sleep delays in base_mail.py during tests to improve test speed

* perf: Mock datetime.now in parallel_request_limiter_v3.py to improve test speed

* pref: Mock urllib system calls in test_aiohttp_transport.py to improve test speed

* chore: add --durations=50 to visualize slowest tests

* pref: reduce setup phase overhead by widening fixture scope in conftest.py

* test: stabilize flaky tests

* fix: minor issue
2025-10-27 19:43:40 -07:00
Ishaan JaffandGitHub 04dc1a5351 [Feat] Add support for returning images with gemini/gemini-2.5-flash-image-preview with /chat/completions (#13983)
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview

* add image in ChatCompletionResponseMessage

* test_gemini_image_generation_async

* Revert "Merge pull request #13394 from Deviad/feature/enhance_logging_for_containers"

This reverts commit 539b94ad4e, reversing
changes made to 71af7bcf9c.

* include `image` in Delta

* fix _process_candidates should show the image response

* fix: _handle_special_delta_attributes

* test_gemini_image_generation_async_stream

* image_generation_chat

* UI - allow looking at generated images from /chat/completions

* _create_streaming_choice

* fix import StreamingChoices

* fix ChatCompletionResponseMessage

* test_gemini_image_generation

* add gemini img migration

* fix _extract_candidate_metadata

* ui fix

* fix batch endpoint test
2025-08-27 16:16:19 -07:00
Krish DholakiaandGitHub 900bd10905 Merge branch 'main' into feature/enhance_logging_for_containers 2025-08-26 23:21:15 -07:00
Ishaan JaffandGitHub b9132968b2 [Perf] Improvements for Async Success Handler (Logging Callbacks) - Approx +130 RPS (#13905)
* [Performance] Reduce Significant CPU overhead from litellm_logging.py (#13895)

* fix: litellm.configured_cold_storage_logger

* fix Session Management - Non-OpenAI Models docs

* ruff fix

* test fix

* create LoggingWorker

* add GLOBAL_LOGGING_WORKER for async task handling

* fix logging tests

* add conftest

* fix conftest

* test fix location of encode bedrock runtime modelid arn

* fix conftest.py

* tuning LoggingWorker

* conftest.py

* fix conftest batches/

* test_async_chat_azure

* event_loop

* test_bedrock_streaming_passthrough_test2

* fix GLOBAL_LOGGING_WORKER

* logging worker

* add flush for global logging worker

* Revert "fix GLOBAL_LOGGING_WORKER"

This reverts commit d254f508f48935652f054777652938ad71976cce.

* fix conftest clear_queue

* fix conftest clear_queue

* setup_and_teardown for llm translation

* docs AWS_REGION

* test_async_chat_azure

* change test DIR

* run ci/cd again

* use 1 job for litellm_router_unit_testing

* fix space

* fix litellm_router_unit_testing

* test_aaarouter_dynamic_cooldown_message_retry_time

* litellm_router_unit_testing

* conftest.py clearing qu

* fixes litellm_router_unit_testing

* fixes clear_queue

* fix router_unit_tests

* remove conftest

* add back conftest for router

* fix event loop test

* test fix

* fixes for LoggingWorker

* ruff fix
2025-08-23 13:13:23 -07:00
Davide Pugliese 81f2563338 Enhance logging for containers 2025-08-08 18:28:35 +02:00
Ishaan JaffandGitHub 39955129f5 fix mapped tests (#12320)
* fix - use flush llm client cache

* faster mapped tests

* test_async_multiple_response_ids_routing

* fix tests

* test_ateam_member_update_admin_requires_premium

* regular mapped tests

* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206)"

This reverts commit 2c60c316ec.

* reset num workers
2025-07-04 10:04:43 -07:00
Krish DholakiaandGitHub ef42461c1e Litellm fix GitHub action testing (#11163)
* test: add __init__.py files

* refactor: rename test folder to avoid naming conflict

* test: update workflows

* test: update tests

* test: update imports

* test: update tests

* test: remove unused import

* ci(test-litellm.yml): add pytest retry to github workflow

* test: fix test
2025-05-26 14:41:42 -07:00