mirror of https://github.com/tiennm99/litellm.git synced 2026-06-17 22:48:35 +00:00

Files

T

Achilleas Athanasiou Fragkoulis cb95b1cf92 fix: Add LITELLM_UI_PATH and LITELLM_ASSETS_PATH for read-only filesystem support (#20492 )

Fixes #19578

---

When deploying the LiteLLM proxy with `readOnlyRootFilesystem: true` in Kubernetes, UI routes returned `404` because:

- Hardcoded paths:
  - `/var/lib/litellm/ui`
  - `/var/lib/litellm/assets`
- Runtime copy/restructure operations failed on read-only filesystems
- No detection mechanism for pre-restructured UI

---

Add configurable environment variables with intelligent detection, graceful fallbacks, and code quality improvements.

---

- **`LITELLM_UI_PATH`** — Custom UI directory location
  - Default: `/var/lib/litellm/ui` (when `LITELLM_NON_ROOT=true`)
  - Default: packaged UI path (otherwise)
  - Example: `/app/var/litellm/ui` for `emptyDir` volumes

- **`LITELLM_ASSETS_PATH`** — Custom assets directory location
  - Default: `/var/lib/litellm/assets` (when `LITELLM_NON_ROOT=true`)
  - Default: current working directory (otherwise)
  - Example: `/app/var/litellm/assets`

---

UI is detected as **pre-restructured and ready** if any of the following apply:

1. **Primary**: `.litellm_ui_ready` marker file exists (created by Dockerfile)
2. **Fallback**: Pattern-based detection — finds *any* subdirectory containing `index.html`
   (resilient to UI structure changes; no hardcoded route names)
3. **Safety**: Filesystem writability check before operations

---

**`litellm/proxy/proxy_server.py`**

- `_validate_ui_directory()` — Verifies UI has required structure (`index.html`, `_next/`)
- `_is_ui_pre_restructured()` — Pattern-based detection (not hardcoded routes)
- `_try_populate_ui_directory()` — Helper for clean error handling
- Refactored UI path decision tree with numbered cases (1, 2, 3, 4a, 4b)
- Updated UI path logic to use `LITELLM_UI_PATH`
- Added writability checks before copy/restructure operations
- Graceful fallback to packaged UI if operations fail
- Updated `server_root_path` replacement with read-only check
- Simplified assets directory creation (try/except instead of complex parent checks)
- Updated `get_image()` endpoint to use `LITELLM_ASSETS_PATH`
- Added validation for packaged and final UI paths

**`docker/Dockerfile.non_root`**

- Added `touch .litellm_ui_ready` marker after UI restructuring
- Enables automatic detection of pre-built UI in Docker images

**`tests/proxy_unit_tests/test_ui_path_detection.py`**

- Added comprehensive unit tests for new functionality
- Tests env var handling, detection logic, and writability checks

---

**`docs/my-website/docs/proxy/config_settings.md`**

- Added `LITELLM_UI_PATH` and `LITELLM_ASSETS_PATH` to env vars table
- Documented defaults and use cases

**`docs/my-website/docs/proxy/prod.md`**

- Added comprehensive "Read-Only Root Filesystem" section
- Quick fixes for permission errors
- Full Kubernetes setup with `initContainer` + `emptyDir` volumes
- API-only deployment option
- Environment variables reference table
- Notes on migrations, caching, and `server_root_path`

**`docker/README.md`**

- Updated hardened setup notes to mention pre-built UI
- Added details about UI serving from read-only paths

---

- No breaking changes
- Existing deployments continue working without modifications
- New env vars are optional with sensible defaults
- Detection logic supports both old and new builds
- Graceful fallbacks throughout

---

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      initContainers:
        - name: setup-ui
          image: ghcr.io/berriai/litellm:main-stable
          command: ["sh", "-c", "cp -r /var/lib/litellm/ui/* /app/var/litellm/ui/"]
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      containers:
        - name: litellm
          env:
            - name: LITELLM_UI_PATH
              value: "/app/var/litellm/ui"
            - name: LITELLM_ASSETS_PATH
              value: "/app/var/litellm/assets"
          securityContext:
            readOnlyRootFilesystem: true
          volumeMounts:
            - name: ui-volume
              mountPath: /app/var/litellm/ui
      volumes:
        - name: ui-volume
          emptyDir:
            sizeLimit: 100Mi

2026-02-12 19:39:04 +05:30

3.4 KiB

Raw Blame History

Docker Development Guide

This guide provides instructions for building and running the LiteLLM application using Docker and Docker Compose.

Prerequisites

Docker
Docker Compose

Building and Running the Application

To build and run the application, you will use the docker-compose.yml file located in the root of the project. This file is configured to use the Dockerfile.non_root for a secure, non-root container environment.

1. Set the Master Key

The application requires a MASTER_KEY for signing and validating tokens. You must set this key as an environment variable before running the application.

Create a .env file in the root of the project and add the following line:

MASTER_KEY=your-secret-key

Replace your-secret-key with a strong, randomly generated secret.

2. Build and Run the Containers

Once you have set the MASTER_KEY, you can build and run the containers using the following command:

docker compose up -d --build

This command will:

Build the Docker image using Dockerfile.non_root.
Start the litellm, litellm_db, and prometheus services in detached mode (-d).
The --build flag ensures that the image is rebuilt if there are any changes to the Dockerfile or the application code.

3. Verifying the Application is Running

You can check the status of the running containers with the following command:

docker compose ps

To view the logs of the litellm container, run:

docker compose logs -f litellm

4. Stopping the Application

To stop the running containers, use the following command:

docker compose down

Hardened / Offline Testing

To ensure changes are safe for non-root, read-only root filesystems and restricted egress, always validate with the hardened compose file:

docker compose -f docker-compose.yml -f docker-compose.hardened.yml build --no-cache
docker compose -f docker-compose.yml -f docker-compose.hardened.yml up -d

This setup:

Builds from docker/Dockerfile.non_root with Prisma engines and Node toolchain baked into the image.
Runs the proxy as a non-root user with a read-only rootfs and only writable tmpfs mounts:
- /app/cache (Prisma/NPM cache; backing PRISMA_BINARY_CACHE_DIR, NPM_CONFIG_CACHE, XDG_CACHE_HOME)
- /app/migrations (Prisma migration workspace; backing LITELLM_MIGRATION_DIR)
Pre-builds and serves the admin UI from read-only paths:
- /var/lib/litellm/ui (pre-restructured Next.js UI with .litellm_ui_ready marker)
- /var/lib/litellm/assets (UI logos and assets)
Routes all outbound traffic through a local Squid proxy that denies egress, so Prisma migrations must use the cached CLI and engines.

You should also verify offline Prisma behaviour with:

docker run --rm --network none --entrypoint prisma ghcr.io/berriai/litellm:main-stable --version

This command should succeed (showing engine versions) even with --network none, confirming that Prisma binaries are available without network access.

Troubleshooting

build_admin_ui.sh: not found: This error can occur if the Docker build context is not set correctly. Ensure that you are running the docker-compose command from the root of the project.
Master key is not initialized: This error means the MASTER_key environment variable is not set. Make sure you have created a .env file in the project root with the MASTER_KEY defined.

3.4 KiB Raw Blame History