10 Commits

Author	SHA1	Message	Date
stuxf	a6c30b30bf	build: migrate packaging, CI, and Docker from Poetry to uv (#25007) ... * build: migrate packaging metadata to uv * ci: move automation and local tooling to uv * docker: migrate image builds and runtime setup to uv * docs: update install and deployment guidance for uv * chore: align auxiliary scripts and tests with uv * test: harden test_litellm isolation * fix: keep release and health check images self-contained * build: pin uv tooling and health check deps * test: isolate bedrock image request formatting from suite state * test: cover sandbox executor requirements flow * ci: fix circleci no-op command steps * ci: fix circleci publish workflow parsing * fix: stabilize remaining uv migration CI checks * ci: increase matrix test timeout headroom * fix: restore published docker and license coverage * fix: restore proxy runtime build parity * fix: restore proxy extras parity and venv migrations * ci: persist uv path across circleci steps * fix: keep psycopg binary in default test env * docker: preserve prisma cache across stages * test: run local proxy checks through uv python * build: restore runtime deps moved into ci * build: refresh uv lock after upstream merge * fix: restore module import in test_check_migration after merge The conflict resolution imported only the function but the test body references check_migration as a module throughout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching - Move google-generativeai, Pillow, tenacity back to ci group (they are lazily imported and bloat the base SDK install needlessly) - Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant in Docker where system Node.js is already installed via apk) - Remove all nodejs-wheel node replacement and venv npm patching blocks from Dockerfiles since the wheel is no longer installed - Add --no-default-groups to CodSpeed benchmark workflow so the benchmark environment matches the old minimal pip install footprint - Apply standard uv two-phase Docker pattern: copy metadata first, install deps (cached layer), then copy source and install project - Replace CircleCI enterprise no-op with proper uv sync command Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate uv.lock after removing nodejs-wheel-binaries Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): use cache/restore instead of cache to prevent cache poisoning The old workflow used actions/cache/restore (read-only). The uv migration changed it to actions/cache (read-write), which zizmor flags as a cache poisoning risk. Restore the safer read-only variant. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert The setup-uv action enables caching by default, which zizmor flags as a cache poisoning risk. Disable it since we already use a read-only cache/restore step. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): disable setup-uv cache in publish workflow Silences zizmor cache-poisoning alert. Publishing workflow runs infrequently on protected branches so caching adds no real benefit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(test): remove duplicate verbose_logger mock in test_check_migration The logger was patched twice — first via mocker.patch() then via mocker.patch.object(autospec=True). The second call fails because autospec cannot inspect an already-mocked attribute. Remove the redundant first patch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): free disk space before Docker build in test-server-root-path The Dockerfile.non_root build ran out of disk on the CI runner. Remove Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2026-04-09 11:46:23 -07:00
Ishaan Jaff	28c33f53a3	CircleCI test stability (#23055) ... * fix: resolve ruff lint errors and mypy type error - Remove unused import get_user_credential (F401) - Add noqa: PLR0915 for 3 large functions exceeding 50 statements - Cast result_data['q'] to str for _append_domain_filters (mypy arg-type) Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: add /vertex_ai/live to supported endpoints and azure gpt-5.1 reasoning flags - Add /vertex_ai/live to JSON schema validation enum in test_utils.py - Add supports_none_reasoning_effort=true to 10 azure/gpt-5.1 model entries (matching the OpenAI gpt-5.1 behavior) Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: handle non-string team_alias/key_alias in PolicyMatchContext Prevent Pydantic validation errors when team_alias or key_alias are not proper strings (e.g. MagicMock in tests). Only pass values that are actually strings; default to None otherwise. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: initialize jwt_handler.litellm_jwtauth in JWT test The test_jwt_non_admin_team_route_access test was failing because user_api_key_auth now accesses jwt_handler.litellm_jwtauth.virtual_key_claim_field before reaching the mocked JWTAuthManager.auth_builder. Initialize the jwt_handler with a default LiteLLM_JWTAuth object. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: add missing mock attributes to MCP server test The test_add_update_server_fallback_to_server_id test was failing because MagicMock auto-creates attributes when accessed. build_mcp_server_from_table accesses many fields via getattr(), which on a MagicMock returns another MagicMock instead of None, causing Pydantic validation errors in MCPServer. Explicitly set all required mock attributes. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: update UI tests for leftnav, navbar, and KeyLifecycleSettings - leftnav: Add mock for useTeams hook, add isUserTeamAdminForAnyTeam to roles mock, update topLevelLabels to match current component menu items - navbar: Add mocks for useDisableBouncingIcon, BlogDropdown, UserDropdown, and serverRootPath. Update test to work with the new component structure. - KeyLifecycleSettings: Fix placeholder and tooltip assertions to match actual component behavior Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: update health check test assertion from 'connected' to 'healthy' The /health/readiness endpoint now returns {"status": "healthy"} with the DB status in a separate field, instead of the previous {"status": "connected"}. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: clear litellm.api_key in OpenRouter validate_environment test The test_validate_environment_raises_without_key test was failing because litellm.api_key may be set globally in the test environment. Clear it along with OPENROUTER_API_KEY and OR_API_KEY env vars using monkeypatch. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: patch HTTPHandler class-level in VLLM embedding test The test_encoding_format_not_sent_in_actual_request test was patching client.post on an instance, but the handler uses the class method. Patch HTTPHandler.post at class level, add caching=False to prevent cache hits, and remove broad try/except that hid errors. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: make test_redaction_responses_api_stream resilient to async callback timing Replace fixed 1s sleep with polling wait for async_log_success_event. Streaming success handler runs via asyncio.create_task; 1s was insufficient in CI. Add 0.5s initial sleep for event loop to schedule the task, then poll up to 10s for the callback to fire. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: update dompurify and svgo to fix security CVEs - CVE-2026-0540: dompurify XSS vulnerability - fix by upgrading to 3.3.2+ - CVE-2026-29074: svgo DoS via entity expansion - fix by upgrading to 3.3.3+ Added npm overrides in docs/my-website/package.json and regenerated package-lock.json. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: remove unused json import in config_override_endpoints.py Ruff F401: json is imported but unused (safe_json_loads/safe_dumps are used instead) Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: add missing MCP mock attributes and provider documentation entries - Add missing mock attributes to test_add_update_server_with_alias and test_add_update_server_without_alias (same fix as fallback test) - Add bedrock_mantle and searchapi to provider_endpoints_support.json - Remove unused json import from config_override_endpoints.py Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: override _supports_reasoning_effort_level for Azure gpt5_series prefix The Azure GPT-5 config uses 'gpt5_series/' as a routing prefix, but _supports_factory(model='gpt5_series/gpt-5.1') fails to resolve because 'gpt5_series' is not a recognized provider. Override the method to strip the prefix and prepend 'azure/' for correct model info lookup. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: accept both 'healthy' and 'connected' in health check test The test_health_and_chat_completion test runs against both source builds (which return 'healthy') and pip-installed versions (which may return 'connected'). Accept both values. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: mock extract_mcp_auth_context in streamable HTTP MCP handler test The handle_streamable_http_mcp function now calls extract_mcp_auth_context before session_manager.handle_request, but the test didn't mock it. The auth extraction fails with the minimal mock scope, preventing handle_request from being called. Also relax assertion to not check exact args since the send wrapper may be modified by debug injection. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: add test for _combine_fallback_usage to satisfy router code coverage The router_code_coverage.py check requires all functions in router.py to be called in test files. Add a basic test for _combine_fallback_usage. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: add @log_guardrail_information decorator to CrowdStrike AIDR guardrail The check_guardrail_apply_decorator.py CI check requires all guardrail apply_guardrail methods to have the @log_guardrail_information decorator. The CrowdStrike AIDR handler was missing it. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: document PRISMA_RECONNECT_ESCALATION_THRESHOLD and REDIS_CLUSTER_NODES env keys Add missing environment variable documentation to config_settings.md to satisfy the test_env_keys.py CI check. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: document enforced_file_expires_after and enforced_batch_output_expires_after in new_team docstring The test_api_docs.py CI check validates that all Pydantic model fields are documented in the function docstring. Add missing parameter docs for enforced_file_expires_after and enforced_batch_output_expires_after. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: regenerate poetry.lock to match pyproject.toml The poetry.lock file was out of sync with pyproject.toml, causing proxy_e2e_azure_batches_tests to fail during dependency installation. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: set master_key=None in test_create_file_with_deep_nested_litellm_metadata The test was missing the master_key monkeypatch that other tests in the same file set. In CI with parallel execution (-n 4), another test may set master_key to a non-None value, causing auth failures (500) when the test sends 'Bearer test-key'. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: document enforced_*_expires_after in update_team docstring too Same missing params as new_team - also needed in update_team docstring for the test_api_docs.py CI check to pass. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: use get_async_httpx_client in a2a_protocol and add master_key monkeypatch to files tests - Replace httpx.AsyncClient() with get_async_httpx_client() in a2a_protocol/main.py to satisfy the ensure_async_clients_test CI check - Add httpxSpecialProvider.A2AProvider enum value - Add master_key=None monkeypatch to test_managed_files_with_loadbalancing Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: remove unused httpx import from a2a_protocol/main.py Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: use cache-key-only param for A2A extra_headers to avoid AsyncHTTPHandler init error The 'extra_headers' key in params was being passed to AsyncHTTPHandler.__init__() which doesn't accept it. Use 'disable_aiohttp_transport' as the cache-key-only param since it's explicitly filtered out before reaching the constructor. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: add additionalProperties:false and resolve $defs/$ref in Anthropic output_format schemas Anthropic API now requires additionalProperties=false for all object-type schemas in output_format. Also resolve $defs/$ref references by inlining them using unpack_defs before sending to Anthropic, since Anthropic doesn't support external schema references. Fixes: llm_translation_testing Anthropic JSON schema failures Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: allowlist CVE-2026-2297 and GHSA-qffp-2rhf-9h96 in security scans - CVE-2026-2297: Python 3.13 SourcelessFileLoader audit hook bypass, no fix available in base image - GHSA-qffp-2rhf-9h96: tar hardlink path traversal, from nodejs_wheel bundled npm, not used in application runtime code Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: isolate files endpoint tests from shared proxy state in CI parallel execution Override user_api_key_auth dependency to return a fixed UserAPIKeyAuth with PROXY_ADMIN role, avoiding auth lookups via prisma_client, user_api_key_cache, or master_key. Set prisma_client=None to prevent DB state contamination. Use try/finally to clean up dependency overrides. Fixes persistent test_create_file_with_deep_nested_litellm_metadata and test_managed_files_with_loadbalancing 500 errors in CI with -n 4. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: apply same auth override to test_managed_files_with_loadbalancing Same CI parallel execution fix as test_create_file_with_deep_nested - override user_api_key_auth dependency and set prisma_client=None. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>	2026-03-07 15:19:39 -08:00
Alexsander Hamir	415a8ab9a6	Fix: remove merge markdown (#17586)	2025-12-06 05:38:16 -08:00
Ishaan Jaffer	8b499adba6	Revert "Add license metadata to health/readiness endpoint. (#15997)" ... This reverts commit d89990e0c5.	2025-12-05 19:31:30 -08:00
Ishaan Jaffer	27a98de600	test_health_and_chat_completion	2025-10-31 19:28:59 -07:00
Ishaan Jaffer	cd9cf2e6bd	test fix	2025-10-31 19:23:08 -07:00
Ishaan Jaffer	2cd57540a4	fix test fixes	2025-10-31 18:31:00 -07:00
Javier de la Torre	e6a7cae7e1	fix(apscheduler): prevent memory leaks from jitter and frequent job intervals (#15846) ... * fix(apscheduler): prevent memory leaks from jitter and frequent job intervals Fixes critical memory leak in APScheduler that causes 35GB+ memory allocations during proxy startup and operation. The leak was identified through Memray analysis showing massive allocations in normalize() and _apply_jitter() functions. Key changes: 1. Remove jitter parameters from all scheduled jobs - jitter was causing expensive normalize() calculations leading to memory explosion 2. Configure AsyncIOScheduler with optimized job_defaults: - misfire_grace_time: 3600s (increased from 120s) to prevent backlog calculations that trigger memory leaks - coalesce: true to collapse missed runs - max_instances: 1 to prevent concurrent job execution - replace_existing: true to avoid duplicate jobs on restart 3. Increase minimum job intervals: - PROXY_BATCH_WRITE_AT: 30s (was 10s) - add_deployment/get_credentials jobs: 30s (was 10s) 4. Use fixed intervals with small random offsets instead of jitter for job distribution across workers 5. Explicitly configure jobstores and executors to minimize overhead 6. Disable timezone awareness to reduce computation Memory impact: - Before: 35GB with 483M allocations during startup - After: <1GB with normal allocation patterns Performance notes: - Minimum job intervals increased from 10s to 30s (configurable via env vars) - Jobs can still be distributed across workers using random start offsets - No functional changes to job behavior, only timing and memory optimization Testing: - Added comprehensive test suite for scheduler configuration - Verified no job execution backlog on startup - Tested duplicate job prevention with replace_existing Related issue: Memory leak in production proxy servers with APScheduler \ud83e\udd16 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: update PROXY_BATCH_WRITE_AT default value from 10s to 30s Update documentation to reflect the new default value for PROXY_BATCH_WRITE_AT changed in PR #15846. The default was increased from 10 seconds to 30 seconds to prevent memory leaks in APScheduler. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: Move APScheduler config to constants.py Address code review feedback from ishaan-jaff: - Move scheduler configuration variables (coalesce, misfire_grace_time, max_instances, replace_existing) to litellm/constants.py - Update all references in proxy_server.py to use the constants - Improves maintainability and makes configuration values centralized Requested-by: @ishaan-jaff Related: #15846 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>	2025-10-28 19:30:17 -07:00
Andrew Bernat	d89990e0c5	Add license metadata to health/readiness endpoint. (#15997) ... * health: expose license metadata (available & expiration) in /health/readiness endpoint * test: add health readiness license metadata coverage * test: ensure /health/readiness response includes license metadata * chore: remove standalone license metadata test as requested; existing test covers codepath --------- Co-authored-by: Plan42.ai <robot@plan42.ai>	2025-10-28 19:21:54 -07:00
Ishaan Jaff	ddfe687b13	(fix) don't block proxy startup if license check fails & using prometheus (#6839) ... * fix - don't block proxy startup if not a premium user * test_litellm_proxy_server_config_with_prometheus * add test for proxy startup * fix remove unused test * fix startup test * add comment on bad-license	2024-11-20 17:55:39 -08:00