Files
DocsGPT/docs/content/Deploying/Development-Environment.mdx
T
Alex 7da46c2bea feat: air-gapped deployment guide, no implicit downloads
- Ship tiktoken's cl100k_base inside the package and build the encoding
  from it, so token counting never downloads anything.
- Default EMBEDDINGS_CACHE_DIR to <data home>/models instead of FastEmbed's
  temp dir, and read tokenizer.json and repo metadata from that cache, so
  a model downloads once and survives reboots.
- TTS_PROVIDER=none and STT_PROVIDER=none switch the speech features off:
  the endpoints return 404, audio files fail to ingest with a clear
  message, /api/config reports tts_available/stt_available, and the UI
  hides the Speak and microphone buttons.
- Drop the Google Fonts Roboto import from the web UI.
- prefetch-models fills the cache the app reads; verify-offline checks the
  packaged encoding.
- Docs: new Air-Gapped Deployment guide, settings and cache notes.
2026-09-15 17:54:24 +01:00

210 lines
9.6 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Setting Up a Development Environment
description: Guide to setting up a development environment for DocsGPT, including backend and frontend setup.
---
import { Callout } from 'nextra/components'
# Setting Up a Development Environment
This guide will walk you through setting up a development environment for DocsGPT. This setup allows you to modify and test the application's backend and frontend components.
## 1. Spin Up Postgres and Redis
For development purposes, you can quickly start Postgres and Redis containers. Postgres is the user-data store for DocsGPT (conversations, agents, prompts, sources, attachments, workflows, logs, and token usage), and Redis is used as the cache and Celery broker. We provide a dedicated Docker Compose file, `docker-compose-dev.yaml`, located in the `deployment` directory, that includes only these essential services. The backend applies the Alembic schema automatically on first boot (`AUTO_MIGRATE=true` / `AUTO_CREATE_DB=true` ship enabled), so no separate migration step is required. You can still run `python scripts/db/init_postgres.py` explicitly if you prefer.
You can find the `docker-compose-dev.yaml` file [here](https://github.com/arc53/DocsGPT/blob/main/deployment/docker-compose-dev.yaml).
**Steps to start Postgres and Redis:**
1. Navigate to the root directory of your DocsGPT repository in your terminal.
2. Run the following commands to build and start the containers defined in `docker-compose-dev.yaml`:
```bash
docker compose -f deployment/docker-compose-dev.yaml build
docker compose -f deployment/docker-compose-dev.yaml up -d
```
These commands will start Postgres and Redis in detached mode, running in the background. When the Flask backend boots against the fresh Postgres instance, it will automatically create the database (if missing) and apply the current Alembic schema.
<Callout type="info" emoji="ℹ️">
MongoDB is no longer required for a default DocsGPT install. If you
specifically want to use MongoDB Atlas as the vector store
(`VECTOR_STORE=mongodb`), start it on the side via
`deployment/docker-compose.mongo.yaml`. For migrating an existing
Mongo-based install to Postgres, see
[PostgreSQL for User Data](/Deploying/Postgres-Migration).
</Callout>
## 2. Run the Backend
To run the DocsGPT backend locally, you'll need to set up a Python environment and install the necessary dependencies.
**Prerequisites:**
* **Python 3.12:** Ensure you have Python 3.12 installed on your system. You can check your Python version by running `python --version` or `python3 --version` in your terminal.
**Steps to run the backend:**
1. **Configure Environment Variables:**
DocsGPT backend settings are configured using environment variables. You can set these either in a `.env` file or directly in the `settings.py` file. For a comprehensive overview of all settings, please refer to the [DocsGPT Settings Guide](/Deploying/DocsGPT-Settings).
* **Option 1: Using a `.env` file (Recommended):**
* If you haven't already, create a file named `.env` in the **root directory** of your DocsGPT project.
* Modify the `.env` file to adjust settings as needed. You can find a comprehensive list of configurable options in [`docsgpt/core/settings.py`](https://github.com/arc53/DocsGPT/blob/main/docsgpt/core/settings.py).
* **Option 2: Exporting Environment Variables:**
* Alternatively, you can export environment variables directly in your terminal. However, using a `.env` file is generally more organized for development.
2. **Create a Python Virtual Environment (Optional but Recommended):**
Using a virtual environment isolates project dependencies and avoids conflicts with system-wide Python packages.
* **macOS and Linux:**
```bash
python -m venv venv
. venv/bin/activate
```
* **Windows:**
```bash
python -m venv venv
venv/Scripts/activate
```
3. **Embedding Model (no action needed):**
The embedding model is downloaded automatically the first time you ingest a document, and cached under `models/` in the repository root for subsequent runs. Set `EMBEDDINGS_CACHE_DIR` to use another directory.
For an offline or air-gapped machine, fetch it ahead of time instead:
```bash
python -m docsgpt.scripts.prefetch_models
```
4. **Install Backend Dependencies:**
Navigate to the root of your DocsGPT repository and install the required Python packages:
```bash
pip install -r docsgpt/requirements.txt
```
Dependencies are declared in `pyproject.toml` and locked in `uv.lock`; the
`requirements*.txt` files are exported from that lock, so with
[uv](https://docs.astral.sh/uv/) installed `uv sync` sets up the same
environment (plus the test tools) in one step.
Optional extras are not installed by default. Add them when you need the
feature (each file is the core set plus the extra):
```bash
pip install -r docsgpt/requirements-docling.txt # docling parser engine: OCR backend, read_document structured output
pip install -r docsgpt/requirements-milvus.txt # VECTOR_STORE=milvus
# or with uv: uv sync --extra docling --extra milvus
```
The docling file adds the PyTorch CPU index; pip handles that as-is, while
`uv pip install -r` needs `UV_INDEX_STRATEGY=unsafe-best-match` (or use
`uv sync --extra docling`, which reads the lock).
A feature whose extra is missing fails with the exact command to run.
After changing `pyproject.toml`, run `uv lock` and
`bash scripts/export_requirements.sh` so the exported files stay in sync
(CI checks this).
5. **Run the Backend:**
For local development, run the ASGI composition under uvicorn. It serves the **whole** application, hot-reloads on source changes, and matches the production runtime:
```bash
uvicorn docsgpt.asgi:asgi_app --host 0.0.0.0 --port 7091 --reload
```
This makes the backend accessible on `http://localhost:7091`. Production uses `gunicorn -k uvicorn_worker.UvicornWorker` against the same `docsgpt.asgi:asgi_app` target.
A plain Flask run is a faster inner loop (quick startup, the Werkzeug interactive debugger):
```bash
flask --app docsgpt/app.py run --host=0.0.0.0 --port=7091
```
But it serves **only** the WSGI Flask app and omits the native-async routes mounted on the ASGI shell in `docsgpt/asgi.py`: the `/mcp` FastMCP endpoint, the chat reconnect reader `GET /api/messages/<id>/events`, the notification stream `GET /api/events`, the remote-device command stream `GET /api/devices/sessions/<id>/events`, and artifact downloads `GET /api/artifacts/<id>/download`. Under `flask run` those paths return 404 — chat still works (`POST /stream` is a Flask route), but live notifications, stream auto-resume, paired devices and artifact downloads don't. Use `flask run` only when you don't need them.
6. **Start the Celery Worker:**
Open a new terminal window (and activate your virtual environment if you used one). Start the Celery worker to handle background tasks:
```bash
celery -A docsgpt.app.celery worker -l INFO
```
This command will start the Celery worker, which processes tasks such as document parsing and vector embedding.
**macOS note:** Due to a threading issue, start Celery with the solo pool:
```bash
python -m celery -A docsgpt.app.celery worker -l INFO --pool=solo
```
**Running in Debugger (VSCode):**
For easier debugging, you can launch the Flask app and Celery worker directly from VSCode's debugger.
* Press <kbd>Shift</kbd> + <kbd>Cmd</kbd> + <kbd>D</kbd> (macOS) or <kbd>Shift</kbd> + <kbd>Windows</kbd> + <kbd>D</kbd> (Windows) to open the Run and Debug view.
* You should see configurations named "Flask" and "Celery". Select the desired configuration and click the "Start Debugging" button (green play icon).
## 3. Start the Frontend
To run the DocsGPT frontend locally, you'll need Node.js and npm (Node Package Manager).
**Prerequisites:**
* **Node.js version 16 or higher:** Ensure you have Node.js version 16 or greater installed. You can check your Node.js version by running `node -v` in your terminal. npm is usually bundled with Node.js.
**Steps to start the frontend:**
1. **Navigate to the Frontend Directory:**
In your terminal, change the current directory to the `frontend` folder within your DocsGPT repository:
```bash
cd frontend
```
2. **Install Global Packages (If Needed):**
If you don't have `husky` and `vite` installed globally, you can install them:
```bash
npm install husky -g
npm install vite -g
```
You can skip this step if you already have these packages installed or prefer to use local installations (though global installation simplifies running the commands in this guide).
3. **Install Frontend Dependencies:**
Install the project's frontend dependencies using npm:
```bash
npm install --include=dev
```
This command reads the `package.json` file in the `frontend` directory and installs all listed dependencies, including development dependencies.
4. **Run the Frontend App:**
Start the frontend development server:
```bash
npm run dev
```
This command will start the Vite development server. The frontend application will typically be accessible at [http://localhost:5173/](http://localhost:5173/). The terminal will display the exact URL where the frontend is running.
With both the backend and frontend running, you should now have a fully functional DocsGPT development environment. You can access the application in your browser at [http://localhost:5173/](http://localhost:5173/) and start developing!