cb95b1cf92
fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support ( #20492 )
...
Fixes #19578
---
When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:
- Hardcoded paths:
- `/var/lib/litellm/ui`
- `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI
---
Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.
---
- **`LITELLM_UI_PATH`** — Custom UI directory location
- Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
- Default: packaged UI path (otherwise)
- Example: `/app/var/litellm/ui` for `emptyDir` volumes
- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
- Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
- Default: current working directory (otherwise)
- Example: `/app/var/litellm/assets`
---
UI is detected as **pre-restructured and ready** if any of the following apply:
1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
(resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations
---
**`litellm/proxy/proxy_server.py`**
- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths
**`docker/Dockerfile.non_root`**
- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images
**`tests/proxy_unit_tests/test_ui_path_detection.py`**
- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks
---
**`docs/my-website/docs/proxy/config_settings.md`**
- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases
**`docs/my-website/docs/proxy/prod.md`**
- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`
**`docker/README.md`**
- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths
---
- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout
---
```yaml
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
initContainers:
- name: setup-ui
image: ghcr.io/berriai/litellm:main-stable
command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
volumeMounts:
- name: ui-volume
mountPath: /app/var/litellm/ui
containers:
- name: litellm
env:
- name: LITELLM_UI_PATH
value: "/app/var/litellm/ui"
- name: LITELLM_ASSETS_PATH
value: "/app/var/litellm/assets"
securityContext:
readOnlyRootFilesystem: true
volumeMounts:
- name: ui-volume
mountPath: /app/var/litellm/ui
volumes:
- name: ui-volume
emptyDir:
sizeLimit: 100Mi
2026-02-12 19:39:04 +05:30
9083b06ba7
Fix test_provider_specific_header_in_request
2026-02-11 16:56:26 +05:30
51d565f619
fix conflicts with main- (this PR is from upstream/main)
2026-02-07 03:10:53 +05:30
0cb79ace97
Fixing tests
2026-02-05 21:44:00 -08:00
3504f05a5c
Adding tests + update pyproject
2026-02-05 21:00:05 -08:00
1f4222e6b2
Add support for 0 cost models
2026-02-02 13:29:37 +05:30
2e6659d9cb
test_embedding fix
2026-01-31 13:43:53 -08:00
586b041837
fix(test): update test_chat_completion to handle metadata in body
...
The proxy now adds metadata to the request body during processing.
Updated test to compare fields individually and strip metadata from
body comparison.
Fixes litellm_proxy_unit_testing_part2 CI failure.
2026-01-31 21:25:39 +00:00
f9eea06a37
Add tests for native_background_mode feature
...
Added 8 new unit tests for the native_background_mode feature:
- test_polling_disabled_when_model_in_native_background_mode
- test_polling_disabled_for_native_background_mode_with_provider_list
- test_polling_enabled_when_model_not_in_native_background_mode
- test_polling_enabled_when_native_background_mode_is_none
- test_polling_enabled_when_native_background_mode_is_empty_list
- test_native_background_mode_exact_match_required
- test_native_background_mode_with_provider_prefix_in_request
- test_native_background_mode_with_router_lookup
Committed-By-Agent: cursor
2026-01-27 16:48:22 -08:00
45954155d7
Merge pull request #19799 from BerriAI/litellm_sso_email_casing
...
[Fix] SSO Email Case Sensitivity
2026-01-27 09:52:03 -08:00
50612715a5
Merge pull request #19814 from BerriAI/litellm_team_member_add_fix
...
[Fix] /team/member_add User Email and ID Verifications
2026-01-27 09:49:01 -08:00
0214cb04cd
Merge branch 'main' into litellm_oss_staging_01_26_2026
2026-01-27 17:00:58 +05:30
7307992cea
fixing tests
2026-01-26 20:17:53 -08:00
ef7261d0eb
Fixing tests
2026-01-26 20:07:30 -08:00
b920be3ee7
fix: server rooth path ( #19790 )
2026-01-26 09:48:06 -08:00
79603b9c3a
fix: optimize logo fetching and resolve mcp import blockers ( #19719 )
2026-01-25 23:08:16 -08:00
0d75e506c5
fix mock_handler
2026-01-20 18:01:22 -08:00
b9de10bd27
test token ctr
2026-01-20 17:53:53 -08:00
ddebdd47bc
[Feat] Add Support for Claude Code Max/OAuth 2 on LiteLLM AI Gateway ( #19453 )
...
* fix count_tokens_with_anthropic_api
* remove outdated file
* fix ANTHROPIC_TOKEN_COUNTING_BETA_VERSION
* refactor: get_token_counter
* init test suite for token counter
* init token counters
* fix: fix pyrightI
* fix Code QA issues
* feat: add OAUTH handling ant
* feat: Oauth handling Ant
* test anthopic common utils
* fix code QA
* docs
2026-01-20 17:21:17 -08:00
6e8dd06d18
fixing test
2026-01-16 19:06:23 -08:00
809b4cb310
Revert "Revert "[Feature] Deleted Keys and Deleted Teams Table""
2026-01-16 14:25:23 -08:00
034e3a6d44
Revert "[Feature] Deleted Keys and Deleted Teams Table"
2026-01-17 06:46:41 +09:00
bd4a893daf
fixing tests
2026-01-16 12:42:35 -08:00
3a55f68ea4
test: handle wildcard routes in route validation test
2026-01-16 12:53:27 +09:00
960bdbbd8e
Revert "[Feat] Add support for 0 cost models"
2026-01-16 09:33:38 +09:00
11cc8dc9d2
Merge pull request #19037 from Jetemple/json-log-fix
...
fix: enable JSON logging via configuration and add regression test
2026-01-14 13:51:36 +05:30
c5ced033c9
fix: anthropic during call guardrail error
2026-01-14 13:37:01 +09:00
272a48d880
[bug fix] do not fallback to token counter if disable_token_counter is enabled ( #19041 )
...
* do not fallback to token counter if disable_token_counter is enabled, and return errors instead
* add exceptions and exception utils to map the same as /v1/chat/completions
* use safe_json_loads
2026-01-13 16:53:38 -08:00
9e08c2207f
fix: enable JSON logging via configuration and add regression test
2026-01-13 09:38:19 -07:00
762a3ef090
Add support for 0 cost models
2026-01-13 16:39:57 +05:30
f1d516f0ae
test fixes
2026-01-10 15:15:41 -08:00
35c636ba97
test_health_check_not_called_when_disabled
2026-01-10 13:55:11 -08:00
bab43ee9dd
fix list keys
2026-01-07 14:34:37 +05:30
5534038e93
Fix CI: Revert security scan changes and add GitGuardian ignore rules ( #18358 )
2025-12-22 17:03:53 -08:00
81dc70673a
Merge remote-tracking branch 'origin' into litellm_ui_unset_values
2025-12-22 11:44:41 -08:00
6112160a16
Revert "[Fix] Security - Remove example API keys with high entropy ( #18255 )"
...
This reverts commit 24edbccf5c
2025-12-20 20:48:11 +05:30
eec05c036a
test_create_skill_sdk
2025-12-20 15:08:31 +05:30
ffcac2eebc
Allow deleting key expiry
2025-12-19 18:04:04 -08:00
f05e600969
Merge pull request #18236 from BerriAI/litellm_add_background_responses_cost_tracking
...
Add cost tracking for responses api in background mode
2025-12-20 00:14:09 +05:30
24edbccf5c
[Fix] Security - Remove example API keys with high entropy ( #18255 )
2025-12-19 10:09:50 -08:00
09b5ee9428
[Feat] Unified Skills API - works across Anthropic, Vertex, Azure, Bedrock ( #18232 )
...
* init LiteLLM_SkillsTable
* init LiteLLMSkillsTransformationHandler
* init LiteLLMSkillsTransformationHandler
* init skills
* init SkillsInjectionHook
* init litellm skills handler
* _fetch_skill_from_db
* LiteLLMSkillsTransformationHandler
* add _prisma_skill_to_litellm
* use SkillPromptInjectionHandler
* refactor skills handler
* add slack GIF
* test_skill_code_execution_via_deployment_hook
* init readme
* test_get_skill_sdk
* test skills in DB
* fix code qa
* init with containers param
* SkillsInjectionHook
* fix type
* convert_skill_to_anthropic_tool, get_litellm_code_execution_tool_anthropic
* fix messages skills handler
* test_slack_gif_skill_creates_gif
* init SkillsInjectionHook
* clean async_pre_call_hook
* fixes
* working unified skills API! yeet
* fix mypy linting
* LiteLLM_SkillsTable
* llm-sandbox==0.3.31
* fix
2025-12-19 18:55:59 +05:30
7d0f41f437
Add cost tracking for responses api in background mode
2025-12-19 13:35:48 +05:30
48a3a741e5
Allow base_model for non Azure providers in proxy ( #18038 )
...
* Allow base_model for non Azure providers in proxy
* Add tests
2025-12-17 02:24:12 +04:00
32fdb9e60e
fix: Add headers to Request scope in JWT tests to fix KeyError ( #17927 )
...
- Add 'headers': [] to all Request(scope={'type': 'http'}) instances in test_jwt.py
- Fixes KeyError: 'headers' when accessing request.headers in user_api_key_auth
- All 7 previously failing tests now pass:
- test_allow_access_by_email (2 variants)
- test_allowed_routes_admin (4 variants)
- test_team_token_output (2 variants)
The Starlette Request object requires 'headers' key in scope dictionary
when accessing request.headers property.
2025-12-13 10:36:13 -08:00
5b6b613561
[Fix] CI/CD - Fix failing proxy unit test and langfuse trace_id test ( #17924 )
...
* fix: correct Request headers format in JWT auth test
Fix test_jwt_non_admin_team_route_access by converting headers to bytes
format as required by Starlette's ASGI specification. Headers must be
bytes tuples with lowercase header names.
This allows dict(request.headers) to work correctly and enables the
authorization check to run, producing the expected error message.
* fix: ignore UUID trace_id from standard_logging_object, use litellm_call_id
The issue was that standard_logging_object.trace_id contains a UUID
(from litellm_trace_id default), which was being used instead of
falling back to litellm_call_id. This caused the test to fail because
it expected 'my-unique-call-id' but got a UUID.
Now we properly detect UUIDs (36 chars with 4 hyphens in specific positions)
and ignore them, allowing the fallback to litellm_call_id to work correctly.
This ensures we use litellm_call_id when no explicit trace_id is provided,
which gets stored in the cache and returned by _get_trace_id().
* fix: use existing_trace_id when provided instead of litellm_call_id
When existing_trace_id is provided in metadata, it should be used as the
trace_id to return (and store in cache), not litellm_call_id. This fixes
the test case where existing_trace_id is set and should be returned by
_get_trace_id().
2025-12-13 09:32:43 -08:00
e9baa83a0f
[Fix] CI/CD – Clean Up Performance PR Changes & others ( #17838 )
2025-12-11 12:50:03 -08:00
82f0c3c887
Support model names with slashes on Gemini endpoints ( #17743 )
...
* Support model names with slashes on Gemini endpoints
* Fix test
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update tests/proxy_unit_tests/test_google_endpoint_routing.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-12-09 18:40:51 -08:00
958c190134
Fix flanky tests ( #17665 )
...
* Fix test_delete_polling_removes_from_cache mock setup
- Mock async_delete_cache to properly execute the real implementation path
- Ensures init_async_client() is called and delete() is invoked on the returned client
- Fixes AssertionError: Expected 'delete' to be called once. Called 0 times.
* fix: resolve timeout in add_model_tab test by mocking useProviderFields hook
- Mock useProviderFields hook to prevent network calls and React Query delays
- Use waitFor to properly handle async operations
- Test now passes reliably without 10s timeout
* fix: add test timeout to prevent CI timeout failure
- Add 15 second timeout to 'should display Test Connect and Add Model buttons' test
- Test takes ~6 seconds locally, but CI was timing out at default 5 second limit
- Ensures test has sufficient time to complete in CI environment
* test: quarantine flaky test_oidc_circleci_with_azure
Quarantine test that fails with 401 Unauthorized from Azure OAuth.
The test is flaky and blocks CI builds. Marked with @pytest.mark.skip
until Azure authentication can be fixed or migrated to our own account.
2025-12-08 12:21:26 -08:00
b83bc10562
Merge pull request #16862 from xianzongxie-stripe/add_polling_via_cache_feature
...
Add polling via cache feature
2025-12-08 08:41:25 +05:30
d3d005f9bf
fixing tests
2025-12-06 21:23:49 -08:00
Achilleas Athanasiou Fragkoulis