09271a4dc5
Mark test_redis_cache_completion_stream as flaky with retries
...
The test intermittently fails in CI due to Redis cache write propagation
delays, causing the second call to miss the cache and hit OpenAI directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-15 20:44:18 -07:00
ff869e91b0
Fix flaky caching tests: use mock_response, add parallelism, remove fail-fast
...
- Replace real OpenAI/Anthropic/Bedrock API calls with mock_response in
~20 cache tests to eliminate network-dependent flakiness
- Remove -x (fail-fast) from caching_unit_tests so all failures are reported
- Add parallelism: 2 with circleci tests run --split-by=timings
- Improve pip dependency cache key (v2-caching-deps) with fallback key
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-03-15 15:23:19 -07:00
cc3f9cd65b
fix(ci): stabilize CI tests - conditional import, mock fixes, timing adjustments
...
Fix 1.1: Make ResponseApplyPatchToolCall import conditional with try/except
for compatibility with openai==1.100.1 (CI environment)
Fix 1.2: Move Router creation inside mock context in vector store tests
so mocks are applied before Router captures function references
Fix 1.3: Update test_model_group_info_e2e to check for 'anthropic/*'
wildcard group instead of specific model names not in proxy config
Fix 2.1: Increase redis cache test sleep from 1s to 5s
Fix 2.2: Increase spend accuracy test sleep from 25s to 45s
Fix 2.3: Add 0.5s sleep between budget test calls
Fix 2.4: Increase vertex AI spend test sleep from 20s to 40s
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com >
2026-03-13 00:01:25 +00:00
3a96c700b4
Adjusted based on comments
2025-11-07 15:02:41 -08:00
0bedf1c0a7
fix tests
2025-10-25 10:19:24 -07:00
9a35d760cf
test_async_custom_handler_stream
2025-09-27 12:37:56 -07:00
f7213bacfe
test_async_custom_handler_embedding_optional_param
2025-09-27 12:18:17 -07:00
1ea046cc61
test: update tests to new deployment model ( #10142 )
...
* test: update tests to new deployment model
* test: update model name
* test: skip cohere rbac issue test
* test: update test - replace gpt-4o model
2025-04-18 14:22:12 -07:00
8a235e7d38
(Refactor / QA) - Use LoggingCallbackManager to append callbacks and ensure no duplicate callbacks are added ( #8112 )
...
* LoggingCallbackManager
* add logging_callback_manager
* use logging_callback_manager
* add add_litellm_failure_callback
* use add_litellm_callback
* use add_litellm_async_success_callback
* add_litellm_async_failure_callback
* linting fix
* fix logging callback manager
* test_duplicate_multiple_loggers_test
* use _reset_all_callbacks
* fix testing with dup callbacks
* test_basic_image_generation
* reset callbacks for tests
* fix check for _add_custom_logger_to_list
* fix test_amazing_sync_embedding
* fix _get_custom_logger_key
* fix batches testing
* fix _reset_all_callbacks
* fix _check_callback_list_size
* add callback_manager_test
* fix test gemini-2.0-flash-thinking-exp-01-21
2025-01-30 19:35:50 -08:00
3671829e39
Complete 'requests' library removal ( #7350 )
...
* refactor: initial commit moving watsonx_text to base_llm_http_handler + clarifying new provider directory structure
* refactor(watsonx/completion/handler.py): move to using base llm http handler
removes 'requests' library usage
* fix(watsonx_text/transformation.py): fix result transformation
migrates to transformation.py, for usage with base llm http handler
* fix(streaming_handler.py): migrate watsonx streaming to transformation.py
ensures streaming works with base llm http handler
* fix(streaming_handler.py): fix streaming linting errors and remove watsonx conditional logic
* fix(watsonx/): fix chat route post completion route refactor
* refactor(watsonx/embed): refactor watsonx to use base llm http handler for embedding calls as well
* refactor(base.py): remove requests library usage from litellm
* build(pyproject.toml): remove requests library usage
* fix: fix linting errors
* fix: fix linting errors
* fix(types/utils.py): fix validation errors for modelresponsestream
* fix(replicate/handler.py): fix linting errors
* fix(litellm_logging.py): handle modelresponsestream object
* fix(streaming_handler.py): fix modelresponsestream args
* fix: remove unused imports
* test: fix test
* fix: fix test
* test: fix test
* test: fix tests
* test: fix test
* test: fix patch target
* test: fix test
2024-12-22 07:21:25 -08:00
fc13c023b7
build(config.yml): add codecov to repo ( #6172 )
...
* build(config.yml): add codecov to repo
ensures all commits have testing coverage
* build(config.yml): fix ci config
* build: fix .yml
* build(config.yml): fix ci/cd
* ci(config.yml): specify module to measure code coverage for
* ci(config.yml): update config.yml version
* ci: trigger new run
* ci(config.yml): store combine
* build(config.yml): check files before combine
* ci(config.yml): fix check
* ci(config.yml): add codecov coverage to ci/cd
* ci(config.yml): add codecov to router tests
* ci(config.yml): wait for router testing to complete before running codecov upload
* ci(config.yml): handle multiple coverage.xml's
* fix(router.py): cleanup print stack
* ci(config.yml): fix config
* ci(config.yml): fix config
2024-10-12 14:48:17 -07:00
d57be47b0f
Litellm ruff linting enforcement ( #5992 )
...
* ci(config.yml): add a 'check_code_quality' step
Addresses https://github.com/BerriAI/litellm/issues/5991
* ci(config.yml): check why circle ci doesn't pick up this test
* ci(config.yml): fix to run 'check_code_quality' tests
* fix(__init__.py): fix unprotected import
* fix(__init__.py): don't remove unused imports
* build(ruff.toml): update ruff.toml to ignore unused imports
* fix: fix: ruff + pyright - fix linting + type-checking errors
* fix: fix linting errors
* fix(lago.py): fix module init error
* fix: fix linting errors
* ci(config.yml): cd into correct dir for checks
* fix(proxy_server.py): fix linting error
* fix(utils.py): fix bare except
causes ruff linting errors
* fix: ruff - fix remaining linting errors
* fix(clickhouse.py): use standard logging object
* fix(__init__.py): fix unprotected import
* fix: ruff - fix linting errors
* fix: fix linting errors
* ci(config.yml): cleanup code qa step (formatting handled in local_testing)
* fix(_health_endpoints.py): fix ruff linting errors
* ci(config.yml): just use ruff in check_code_quality pipeline for now
* build(custom_guardrail.py): include missing file
* style(embedding_handler.py): fix ruff check
2024-10-01 19:44:20 -04:00
3560f0ef2c
refactor: move all testing to top-level of repo
...
Closes https://github.com/BerriAI/litellm/issues/486
2024-09-28 21:08:14 -07:00
yuneng-jiang