mirror of
https://github.com/tiennm99/DocsGPT.git
synced 2026-10-03 11:11:58 +00:00
feat: air-gapped deployment guide, no implicit downloads
- Ship tiktoken's cl100k_base inside the package and build the encoding from it, so token counting never downloads anything. - Default EMBEDDINGS_CACHE_DIR to <data home>/models instead of FastEmbed's temp dir, and read tokenizer.json and repo metadata from that cache, so a model downloads once and survives reboots. - TTS_PROVIDER=none and STT_PROVIDER=none switch the speech features off: the endpoints return 404, audio files fail to ingest with a clear message, /api/config reports tts_available/stt_available, and the UI hides the Speak and microphone buttons. - Drop the Google Fonts Roboto import from the web UI. - prefetch-models fills the cache the app reads; verify-offline checks the packaged encoding. - Docs: new Air-Gapped Deployment guide, settings and cache notes.
This commit is contained in:
Alex committed
1 parent
ca69d1ea29
commit
7da46c2bea
38 files changed
+100892
-95
No files matched your search
@@ -0,0 +1,197 @@
|
||||
---
|
||||
title: Air-Gapped Deployment
|
||||
description: Run DocsGPT on a network with no internet access.
|
||||
---
|
||||
|
||||
import { Callout } from 'nextra/components'
|
||||
|
||||
# Air-Gapped Deployment
|
||||
|
||||
DocsGPT runs without internet access once three things are inside your network: the software, the embedding models, and an LLM you host yourself. This guide covers bringing them in, the settings that switch off features that need the internet, blocking outbound traffic, and checking that nothing gets out.
|
||||
|
||||
## What needs the internet
|
||||
|
||||
With a self-hosted LLM, chat, document ingestion and retrieval work offline. These parts of DocsGPT contact external services by default and need a setting:
|
||||
|
||||
| Feature | What it contacts | Air-gapped setting |
|
||||
| --- | --- | --- |
|
||||
| Default LLM (`LLM_PROVIDER=docsgpt`) | DocsGPT's hosted model API | Point DocsGPT at your own model server |
|
||||
| Version check | `gptcloud.arc53.com`, when the worker starts and periodically after | `VERSION_CHECK=0` |
|
||||
| Text-to-speech (Speak button) | Google (`google_tts`) or ElevenLabs | `TTS_PROVIDER=none` |
|
||||
| Speech-to-text (microphone, audio files) | OpenAI (`openai`) | `STT_PROVIDER=none` |
|
||||
| `read_webpage` chat tool | Any URL the model chooses | Leave it out of `DEFAULT_CHAT_TOOLS` |
|
||||
| Embedding models | Hugging Face, the first time a model is used | Bundled in the Docker image; `docsgpt prefetch-models` for pip |
|
||||
|
||||
Features whose purpose is to fetch outside content cannot work offline: URL, crawler, sitemap, GitHub and Reddit sources, the Google Drive, SharePoint and Confluence connectors, web search and other internet-facing tools, and hosted LLM or embedding providers.
|
||||
|
||||
<Callout type="info">
|
||||
Keep the default document parser. `DOC_PARSER_ENGINE=anydoc` needs no models, and OCR is off unless you turn it on. Use the default backend image rather than the `-docling` variant, and don't install the `docling` extra.
|
||||
</Callout>
|
||||
|
||||
## 1. Bring the software in
|
||||
|
||||
### Docker
|
||||
|
||||
The backend image already contains everything the default configuration loads at run time: both default embedding models (granite for new installs, mpnet for upgrades), their tokenizers and the token-counting encoding. A container downloads nothing on first use.
|
||||
|
||||
On a machine with internet access, pull the images and save them to one file:
|
||||
|
||||
```bash
|
||||
TAG=latest # or a release, e.g. 0.19.0
|
||||
docker pull arc53/docsgpt:$TAG
|
||||
docker pull arc53/docsgpt-fe:$TAG
|
||||
docker pull redis:6-alpine
|
||||
docker pull postgres:16-alpine
|
||||
docker save -o docsgpt-images.tar \
|
||||
arc53/docsgpt:$TAG arc53/docsgpt-fe:$TAG redis:6-alpine postgres:16-alpine
|
||||
```
|
||||
|
||||
Copy `docsgpt-images.tar` and the [standalone Compose file](/Deploying/Docker-Deploying#quickest-setup-pre-built-images-no-checkout) into the air-gapped network, then load the images (or push them to your internal registry):
|
||||
|
||||
```bash
|
||||
docker load -i docsgpt-images.tar
|
||||
```
|
||||
|
||||
Set `DOCSGPT_IMAGE_TAG` to the same tag when you start the stack. Bring your model server's image and weights in the same way, following its own documentation.
|
||||
|
||||
To use an embedding model other than the two defaults, build your own backend image with it baked in: `docker build -f docsgpt/Dockerfile --build-arg EMBEDDINGS_PREFETCH=<model> .`
|
||||
|
||||
### pip
|
||||
|
||||
On a connected machine with the same operating system, CPU architecture and Python version as the target, download the packages and the models:
|
||||
|
||||
```bash
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
pip download docsgpt -d wheelhouse
|
||||
pip install --no-index --find-links wheelhouse docsgpt
|
||||
DOCSGPT_HOME=./docsgpt-home docsgpt prefetch-models
|
||||
```
|
||||
|
||||
`prefetch-models` stores both default embedding models and their tokenizers under `models/` in the [data home](/Deploying/Pip-Install). Pass model names to fetch a different set.
|
||||
|
||||
Copy `wheelhouse/` and `docsgpt-home/models/` to the target, install from the wheelhouse, and put `models/` in the target's data home (or point `EMBEDDINGS_CACHE_DIR` at it):
|
||||
|
||||
```bash
|
||||
pip install --no-index --find-links wheelhouse docsgpt
|
||||
```
|
||||
|
||||
## 2. Point DocsGPT at your own LLM
|
||||
|
||||
Run an OpenAI-compatible model server inside the network, such as vLLM, Ollama, a llama.cpp server or Text Generation Inference, and set:
|
||||
|
||||
```env
|
||||
LLM_PROVIDER=openai
|
||||
OPENAI_BASE_URL=http://llm.internal:8000/v1
|
||||
LLM_NAME=your-model-name
|
||||
API_KEY=None
|
||||
```
|
||||
|
||||
Setting `OPENAI_BASE_URL` also removes DocsGPT's hosted model from the model list. See [Local inference engines](/Models/local-inference) for the base URL of each server.
|
||||
|
||||
## 3. Switch off the features that call out
|
||||
|
||||
Add these to `.env`:
|
||||
|
||||
```env
|
||||
VERSION_CHECK=0
|
||||
TTS_PROVIDER=none
|
||||
STT_PROVIDER=none
|
||||
DEFAULT_CHAT_TOOLS=["memory","scheduler"]
|
||||
HF_HUB_OFFLINE=1
|
||||
```
|
||||
|
||||
- `VERSION_CHECK=0` stops the worker's anonymous version check.
|
||||
- `TTS_PROVIDER=none` hides the Speak button on answers.
|
||||
- `STT_PROVIDER=none` hides the microphone button, and audio files fail to ingest with a message naming the setting. If your model server offers an OpenAI-compatible `/v1/audio/transcriptions` endpoint, keep `STT_PROVIDER=openai` instead: speech-to-text uses `OPENAI_BASE_URL` too.
|
||||
- `DEFAULT_CHAT_TOOLS` defaults to `memory`, `read_webpage` and `scheduler`; this list drops `read_webpage`.
|
||||
- `HF_HUB_OFFLINE=1` makes the Hugging Face libraries read models from the local cache only. A model that was never fetched fails at once with an error, instead of waiting on a connection that never opens. The libraries read it from the process environment: Docker Compose's `env_file` provides that, and for pip installs export it in the environment that starts `docsgpt api` and `docsgpt worker`.
|
||||
|
||||
Leave the document parsing and OCR settings at their defaults.
|
||||
|
||||
## 4. Block outbound traffic in the network
|
||||
|
||||
DocsGPT doesn't block outbound connections itself. Enforce that in the network, where the same rule also covers Postgres, Redis, your model server and anything you add later.
|
||||
|
||||
**Hosts with no route out.** The simplest setup: servers whose firewall allows only internal address ranges and your internal DNS.
|
||||
|
||||
**Docker.** Attach the services to a network created with `internal: true`, which has no route out of the Docker host. A container on an internal network can't publish ports, so give only a reverse proxy (nginx, Caddy, Traefik) a second, regular network and serve the UI and API through it:
|
||||
|
||||
```yaml
|
||||
networks:
|
||||
airgap:
|
||||
internal: true
|
||||
edge: {}
|
||||
|
||||
services:
|
||||
backend:
|
||||
networks: [airgap]
|
||||
worker:
|
||||
networks: [airgap]
|
||||
frontend:
|
||||
networks: [airgap]
|
||||
postgres:
|
||||
networks: [airgap]
|
||||
redis:
|
||||
networks: [airgap]
|
||||
proxy:
|
||||
image: nginx:stable
|
||||
networks: [airgap, edge]
|
||||
ports: ["443:443"]
|
||||
```
|
||||
|
||||
Docker's published ports bypass host firewalls such as `ufw`. If you rely on host rules instead of internal networks, filter container traffic in the `DOCKER-USER` iptables chain.
|
||||
|
||||
**Kubernetes.** With a network plugin that enforces NetworkPolicy, deny egress for the namespace except to its own pods and cluster DNS:
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: deny-external-egress
|
||||
namespace: docsgpt
|
||||
spec:
|
||||
podSelector: {}
|
||||
policyTypes: [Egress]
|
||||
egress:
|
||||
- to:
|
||||
- podSelector: {}
|
||||
- to:
|
||||
- namespaceSelector: {}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
k8s-app: kube-dns
|
||||
ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
- protocol: TCP
|
||||
port: 53
|
||||
```
|
||||
|
||||
Add a rule for your model server if it runs in another namespace.
|
||||
|
||||
**Egress proxy.** When some traffic must leave the network, for example to a hosted LLM, send it through a forward proxy that allows only those hosts. The Python HTTP libraries DocsGPT uses honour the standard variables; exclude internal services in `NO_PROXY`:
|
||||
|
||||
```env
|
||||
HTTPS_PROXY=http://proxy.internal:3128
|
||||
HTTP_PROXY=http://proxy.internal:3128
|
||||
NO_PROXY=localhost,127.0.0.1,backend,llm.internal
|
||||
```
|
||||
|
||||
- Features that fetch user-supplied URLs connect to the resolved IP address, so a hostname allowlist can't match them. Keep those features off.
|
||||
- If the proxy inspects TLS, add its CA certificate with `SSL_CERT_FILE` and `REQUESTS_CA_BUNDLE`. For model downloads through such a proxy, also set `HF_HUB_DISABLE_XET=1`: Hugging Face's transfer client ignores `SSL_CERT_FILE`.
|
||||
|
||||
## 5. Verify
|
||||
|
||||
Check that the models load with networking disabled:
|
||||
|
||||
```bash
|
||||
docker run --rm --network none arc53/docsgpt:$TAG python -m docsgpt.scripts.verify_offline
|
||||
```
|
||||
|
||||
On a pip install, run `docsgpt verify-offline` on the air-gapped host. Both check token counting, each default embedding model's tokenizer, and an embedding; pass a model name to check a different model.
|
||||
|
||||
Then use the app: upload a document and ask a question about it. Your firewall or proxy logs should show no connection attempts from DocsGPT to outside hosts. The web UI loads its fonts and scripts from the DocsGPT server only.
|
||||
|
||||
<Callout type="info">
|
||||
The [chat widget](/Extensions/chat-widget) loads its font from Google Fonts and, unless you pass `buttonIcon`, its launcher icon from a CDN. Set `apiHost` to your own DocsGPT API; the default is DocsGPT's cloud.
|
||||
</Callout>
|
||||
@@ -78,7 +78,7 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
|
||||
|
||||
3. **Embedding Model (no action needed):**
|
||||
|
||||
The embedding model is downloaded automatically the first time you ingest a document, and cached for subsequent runs. Set `EMBEDDINGS_CACHE_DIR` to control where.
|
||||
The embedding model is downloaded automatically the first time you ingest a document, and cached under `models/` in the repository root for subsequent runs. Set `EMBEDDINGS_CACHE_DIR` to use another directory.
|
||||
|
||||
For an offline or air-gapped machine, fetch it ahead of time instead:
|
||||
|
||||
|
||||
@@ -67,7 +67,9 @@ Here are some of the most fundamental settings you'll likely want to configure:
|
||||
|
||||
- **`OPENAI_BASE_URL`**: Specifically used when `LLM_PROVIDER` is set to `openai` but you are connecting to a local inference engine (like Ollama, Llama.cpp, etc.) that exposes an OpenAI-compatible API. This setting tells DocsGPT where to find your local LLM server.
|
||||
|
||||
- **`STT_PROVIDER`**: Selects the speech-to-text provider used for microphone transcription in chat and for audio file ingestion through the parser pipeline.
|
||||
- **`STT_PROVIDER`**: Selects the speech-to-text provider used for microphone transcription in chat and for audio file ingestion through the parser pipeline. Set it to `none` to switch speech-to-text off: the microphone button is hidden, and audio files fail to ingest with a message naming the setting.
|
||||
|
||||
- **`TTS_PROVIDER`**: Selects the text-to-speech provider behind the Speak button on answers: `google_tts` (default) or `elevenlabs`. Both are hosted services. Set it to `none` to switch text-to-speech off and hide the button.
|
||||
|
||||
- **`SERVE_UI`**: Whether the API process serves the web UI shipped in the Python package (`pip install docsgpt`). Defaults to `true`; set `false` when a separate frontend, such as the Docker frontend image, serves the UI. Has no effect in the backend Docker image, which carries no UI build.
|
||||
|
||||
@@ -238,7 +240,7 @@ The settings below control speech-to-text behaviour for both voice input and aud
|
||||
|
||||
| Setting | Purpose | Typical values |
|
||||
| --- | --- | --- |
|
||||
| `STT_PROVIDER` | Speech-to-text backend provider. | `openai`, `faster_whisper` |
|
||||
| `STT_PROVIDER` | Speech-to-text backend provider, or `none` to switch speech-to-text off. | `openai`, `faster_whisper`, `none` |
|
||||
| `OPENAI_STT_MODEL` | OpenAI transcription model used when `STT_PROVIDER=openai`. | `gpt-4o-mini-transcribe` |
|
||||
| `STT_LANGUAGE` | Optional language hint passed to the provider. Leave unset for auto-detection when supported. | `en`, `es`, unset |
|
||||
| `STT_MAX_FILE_SIZE_MB` | Maximum file size accepted by the synchronous `/api/stt` endpoint. | `50` |
|
||||
|
||||
@@ -49,7 +49,7 @@ pipx runpip docsgpt install --force-reinstall --no-deps --index-url https://down
|
||||
|
||||
## Configure
|
||||
|
||||
DocsGPT keeps its runtime files in a **data home**: the `.env` file it reads settings from, uploaded files under `inputs/` and vector indexes under `indexes/`. The data home is the directory you run the commands from, or the directory `DOCSGPT_HOME` points to. `DOCSGPT_ENV_FILE` points at a `.env` kept somewhere else. Both variables must be set in the process environment, not in `.env`: they decide where `.env` is read from.
|
||||
DocsGPT keeps its runtime files in a **data home**: the `.env` file it reads settings from, uploaded files under `inputs/`, vector indexes under `indexes/` and downloaded embedding models under `models/`. The data home is the directory you run the commands from, or the directory `DOCSGPT_HOME` points to. `DOCSGPT_ENV_FILE` points at a `.env` kept somewhere else. Both variables must be set in the process environment, not in `.env`: they decide where `.env` is read from.
|
||||
|
||||
Create a `.env` in the data home. The minimum for a hosted LLM:
|
||||
|
||||
@@ -87,7 +87,7 @@ The worker is not optional: query embedding runs on it, so search fails without
|
||||
Other commands:
|
||||
|
||||
- `docsgpt api --reload`: a development server with auto-reload.
|
||||
- `docsgpt prefetch-models`: download the embedding, tokenizer and parser models ahead of time, for machines that go offline.
|
||||
- `docsgpt prefetch-models`: download the embedding models and their tokenizers ahead of time, for machines that go offline (see [Air-Gapped Deployment](/Deploying/Air-Gapped)).
|
||||
- `docsgpt verify-offline`: check that a prepared install starts with networking off.
|
||||
- `docsgpt reembed`: re-embed every index after changing `EMBEDDINGS_NAME` (see [Upgrading](/upgrading)).
|
||||
|
||||
|
||||
@@ -27,6 +27,10 @@ export default {
|
||||
"title": "☸️ Deploying on Kubernetes",
|
||||
"href": "/Deploying/Kubernetes-Deploying"
|
||||
},
|
||||
"Air-Gapped": {
|
||||
"title": "🔒 Air-Gapped Deployment",
|
||||
"href": "/Deploying/Air-Gapped"
|
||||
},
|
||||
"Hosting-the-app": {
|
||||
"title": "☁️ Hosting DocsGPT",
|
||||
"href": "/Deploying/Hosting-the-app"
|
||||
|
||||
@@ -37,7 +37,7 @@ Set `EMBEDDINGS_NAME` in your `.env` to a registry name or a Hugging Face reposi
|
||||
EMBEDDINGS_NAME=ibm-granite/granite-embedding-311m-multilingual-r2
|
||||
```
|
||||
|
||||
The model is downloaded on first use and cached; set `EMBEDDINGS_CACHE_DIR` to control where. There is no `model/` folder to populate by hand, and a filesystem path is not accepted as a model name.
|
||||
The model is downloaded on first use and cached, together with its tokenizer, under `models/` in the data home; set `EMBEDDINGS_CACHE_DIR` to use another directory. There is no folder to populate by hand, and a filesystem path is not accepted as a model name.
|
||||
|
||||
DocsGPT knows the pooling, vector width and context window of the models in its registry (`all-mpnet-base-v2`, `granite-embedding-311m-multilingual-r2`, `granite-embedding-97m-multilingual-r2`). For any other repository it reads those from the repository's own `1_Pooling/config.json` and `modules.json`. If a repository declares neither, mean pooling with L2 normalization is assumed and a warning is logged — pin the real values with `EMBEDDINGS_POOLING` (`cls` or `mean`) and `EMBEDDINGS_NORMALIZE`.
|
||||
|
||||
|
||||
+1
-2
@@ -108,11 +108,10 @@ COPY --from=builder /venv /venv
|
||||
# EMBEDDINGS_CACHE_DIR / HF_HUB_CACHE FastEmbed models and their tokenizers
|
||||
# (chunking reads tokenizer.json from
|
||||
# the same hub-layout snapshot)
|
||||
# TIKTOKEN_CACHE_DIR cl100k_base for token accounting
|
||||
# DOCLING_ARTIFACTS_PATH docling's models (docling extra only)
|
||||
# tiktoken's cl100k_base encoding ships inside the package.
|
||||
ENV EMBEDDINGS_CACHE_DIR=/app/models \
|
||||
HF_HUB_CACHE=/app/models \
|
||||
TIKTOKEN_CACHE_DIR=/app/models/tiktoken \
|
||||
DOCLING_ARTIFACTS_PATH=/app/models/docling \
|
||||
HF_HUB_DISABLE_TELEMETRY=1 \
|
||||
PATH="/venv/bin:$PATH"
|
||||
|
||||
@@ -345,6 +345,15 @@ class StoreAttachment(Resource):
|
||||
return make_response(jsonify({"success": False, "error": "Failed to store attachment"}), 400)
|
||||
|
||||
|
||||
_STT_DISABLED_MESSAGE = "Speech-to-text is disabled on this server."
|
||||
_TTS_DISABLED_MESSAGE = "Text-to-speech is disabled on this server."
|
||||
|
||||
|
||||
def _feature_disabled(message: str):
|
||||
"""404 for a speech feature whose provider is set to ``none``."""
|
||||
return make_response(jsonify({"success": False, "message": message}), 404)
|
||||
|
||||
|
||||
@attachments_ns.route("/stt")
|
||||
class SpeechToText(Resource):
|
||||
@api.expect(
|
||||
@@ -368,6 +377,8 @@ class SpeechToText(Resource):
|
||||
jsonify({"success": False, "message": "Authentication required"}),
|
||||
401,
|
||||
)
|
||||
if not STTCreator.is_enabled(settings.STT_PROVIDER):
|
||||
return _feature_disabled(_STT_DISABLED_MESSAGE)
|
||||
|
||||
file = request.files.get("file")
|
||||
if not file or file.filename == "":
|
||||
@@ -440,6 +451,8 @@ class LiveSpeechToTextStart(Resource):
|
||||
jsonify({"success": False, "message": "Authentication required"}),
|
||||
401,
|
||||
)
|
||||
if not STTCreator.is_enabled(settings.STT_PROVIDER):
|
||||
return _feature_disabled(_STT_DISABLED_MESSAGE)
|
||||
|
||||
redis_client = _require_live_stt_redis()
|
||||
if hasattr(redis_client, "status_code"):
|
||||
@@ -501,6 +514,8 @@ class LiveSpeechToTextChunk(Resource):
|
||||
jsonify({"success": False, "message": "Authentication required"}),
|
||||
401,
|
||||
)
|
||||
if not STTCreator.is_enabled(settings.STT_PROVIDER):
|
||||
return _feature_disabled(_STT_DISABLED_MESSAGE)
|
||||
|
||||
redis_client = _require_live_stt_redis()
|
||||
if hasattr(redis_client, "status_code"):
|
||||
@@ -817,6 +832,8 @@ class TextToSpeech(Resource):
|
||||
@api.expect(tts_model)
|
||||
@api.doc(description="Synthesize audio speech from text")
|
||||
def post(self):
|
||||
if not TTSCreator.is_enabled(settings.TTS_PROVIDER):
|
||||
return _feature_disabled(_TTS_DISABLED_MESSAGE)
|
||||
data = request.get_json()
|
||||
text = data["text"]
|
||||
try:
|
||||
|
||||
@@ -203,12 +203,16 @@ def health():
|
||||
@app.route("/api/config")
|
||||
def get_config():
|
||||
from docsgpt.graphrag import graphrag_available
|
||||
from docsgpt.stt.stt_creator import STTCreator
|
||||
from docsgpt.tts.tts_creator import TTSCreator
|
||||
|
||||
response = {
|
||||
"auth_type": settings.AUTH_TYPE,
|
||||
"requires_auth": settings.AUTH_TYPE in ["simple_jwt", "session_jwt", "oidc"],
|
||||
"graphrag_available": graphrag_available(),
|
||||
"hybrid_available": settings.VECTOR_STORE == "pgvector",
|
||||
"tts_available": TTSCreator.is_enabled(settings.TTS_PROVIDER),
|
||||
"stt_available": STTCreator.is_enabled(settings.STT_PROVIDER),
|
||||
}
|
||||
if settings.AUTH_TYPE == "oidc":
|
||||
response["oidc"] = {
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -57,7 +57,8 @@ class Settings(BaseSettings):
|
||||
# Intra-op threads for the local ONNX runner; None = every core. It scales sub-linearly,
|
||||
# so several single-threaded workers beat one many-threaded process on the same cores.
|
||||
EMBEDDINGS_THREADS: Optional[int] = None
|
||||
EMBEDDINGS_CACHE_DIR: Optional[str] = None # where FastEmbed caches model artifacts
|
||||
# Embedding models and their tokenizers. Persistent by default: FastEmbed's own default is the temp dir.
|
||||
EMBEDDINGS_CACHE_DIR: Optional[str] = Field(default_factory=lambda: str(home_dir() / "models"))
|
||||
# Pooling ("cls"/"mean") and L2 normalisation. Read from the model's own repository;
|
||||
# set these only for a repository that declares neither, or to override what it declares.
|
||||
EMBEDDINGS_POOLING: Optional[str] = None
|
||||
@@ -360,9 +361,9 @@ class Settings(BaseSettings):
|
||||
# Encryption settings
|
||||
ENCRYPTION_SECRET_KEY: str = "default-docsgpt-encryption-key"
|
||||
|
||||
TTS_PROVIDER: str = "google_tts" # google_tts or elevenlabs
|
||||
TTS_PROVIDER: str = "google_tts" # google_tts, elevenlabs, or none to switch text-to-speech off
|
||||
ELEVENLABS_API_KEY: Optional[str] = None
|
||||
STT_PROVIDER: str = "openai" # openai or faster_whisper
|
||||
STT_PROVIDER: str = "openai" # openai, faster_whisper, or none to switch speech-to-text off
|
||||
OPENAI_STT_MODEL: str = "gpt-4o-mini-transcribe"
|
||||
STT_LANGUAGE: Optional[str] = None
|
||||
STT_MAX_FILE_SIZE_MB: int = 50
|
||||
|
||||
@@ -2,7 +2,7 @@ from pathlib import Path
|
||||
from typing import Dict, Union
|
||||
|
||||
from docsgpt.core.settings import settings
|
||||
from docsgpt.parser.file.base_parser import BaseParser
|
||||
from docsgpt.parser.file.base_parser import BaseParser, DocumentParseError
|
||||
from docsgpt.stt.stt_creator import STTCreator
|
||||
from docsgpt.stt.upload_limits import enforce_audio_file_size_limit
|
||||
|
||||
@@ -17,6 +17,10 @@ class AudioParser(BaseParser):
|
||||
|
||||
def parse_file(self, file: Path, errors: str = "ignore") -> Union[str, list[str]]:
|
||||
_ = errors
|
||||
if not STTCreator.is_enabled(settings.STT_PROVIDER):
|
||||
raise DocumentParseError(
|
||||
f"{file.name}: audio files need speech-to-text, which is disabled (STT_PROVIDER=none)."
|
||||
)
|
||||
try:
|
||||
enforce_audio_file_size_limit(file.stat().st_size)
|
||||
except OSError:
|
||||
|
||||
@@ -217,19 +217,21 @@ class HuggingFaceCounter(TokenCounter):
|
||||
|
||||
|
||||
def _tokenizer_file(repo: str) -> str:
|
||||
"""Path to ``repo``'s ``tokenizer.json``, from the hub cache when present.
|
||||
"""Path to ``repo``'s ``tokenizer.json``, from the embedding model cache when present.
|
||||
|
||||
A warmed cache (the Docker image bakes the default models) answers without
|
||||
touching the network. ``hf_hub_download`` would otherwise revalidate the
|
||||
revision with a HEAD request on every process start, and stall for the
|
||||
etag timeout on a host that cannot reach huggingface.co.
|
||||
FastEmbed's snapshot of the repository in ``EMBEDDINGS_CACHE_DIR`` already
|
||||
holds the file, so a warmed cache answers without touching the network.
|
||||
``hf_hub_download`` would otherwise revalidate the revision with a HEAD
|
||||
request on every process start, and stall for the etag timeout on a host
|
||||
that cannot reach huggingface.co.
|
||||
"""
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
cache_dir = settings.EMBEDDINGS_CACHE_DIR or None
|
||||
try:
|
||||
return hf_hub_download(repo, "tokenizer.json", local_files_only=True)
|
||||
return hf_hub_download(repo, "tokenizer.json", local_files_only=True, cache_dir=cache_dir)
|
||||
except Exception: # noqa: BLE001 -- not cached: fetch it
|
||||
return hf_hub_download(repo, "tokenizer.json")
|
||||
return hf_hub_download(repo, "tokenizer.json", cache_dir=cache_dir)
|
||||
|
||||
|
||||
def _load_hf_counter(repo: str) -> Optional[HuggingFaceCounter]:
|
||||
|
||||
@@ -1,14 +1,13 @@
|
||||
"""Download the model artifacts a fresh container would otherwise fetch.
|
||||
"""Download the embedding models a fresh install would otherwise fetch.
|
||||
|
||||
Run at image build time so a fresh container does not download on its first
|
||||
request, and an air-gapped install works at all. Two things are warmed:
|
||||
|
||||
* Embedding models, into FastEmbed's cache. Both the legacy and the current
|
||||
default are baked: an upgraded deployment keeps using mpnet until it runs
|
||||
``reembed``, while a new one starts on granite.
|
||||
* tiktoken's ``cl100k_base`` encoding, which token accounting uses on every
|
||||
chat. tiktoken caches it under ``TIKTOKEN_CACHE_DIR`` (a temp dir when
|
||||
unset), so the image sets that variable and this warms it.
|
||||
request, and before moving an install onto a host without internet access.
|
||||
Both the legacy and the current default embedding model are fetched: an
|
||||
upgraded deployment keeps using mpnet until it runs ``reembed``, while a new
|
||||
one starts on granite. Each model lands in ``EMBEDDINGS_CACHE_DIR`` together
|
||||
with its tokenizer, which chunking reads from the same snapshot. tiktoken's
|
||||
``cl100k_base`` encoding ships inside the package, so there is nothing to warm
|
||||
for it.
|
||||
|
||||
Usage::
|
||||
|
||||
@@ -34,23 +33,6 @@ logger = logging.getLogger("prefetch_models")
|
||||
#: Fetched when no names are given.
|
||||
DEFAULT_MODELS = (DEFAULT_LEGACY, DEFAULT_NEW_INSTALL)
|
||||
|
||||
#: tiktoken encodings the application loads (``docsgpt.utils.get_encoding``).
|
||||
TIKTOKEN_ENCODINGS = ("cl100k_base",)
|
||||
|
||||
|
||||
def prefetch_tiktoken(names: Sequence[str] = TIKTOKEN_ENCODINGS) -> List[str]:
|
||||
"""Warm tiktoken's cache for each encoding in ``names``.
|
||||
|
||||
Returns:
|
||||
The encodings fetched.
|
||||
"""
|
||||
import tiktoken
|
||||
|
||||
for name in names:
|
||||
logger.info("Fetching tiktoken encoding %s", name)
|
||||
tiktoken.get_encoding(name)
|
||||
return list(names)
|
||||
|
||||
|
||||
def prefetch(names: Sequence[str], cache_dir: Optional[str] = None) -> List[str]:
|
||||
"""Fetch each named model's artifacts.
|
||||
@@ -97,6 +79,22 @@ def prefetch(names: Sequence[str], cache_dir: Optional[str] = None) -> List[str]
|
||||
return fetched
|
||||
|
||||
|
||||
def _cache_dir() -> Optional[str]:
|
||||
"""``EMBEDDINGS_CACHE_DIR`` from the environment, else the directory the app reads.
|
||||
|
||||
The image build sets the variable and copies in only this module's imports,
|
||||
so settings are loaded only when the variable is absent.
|
||||
"""
|
||||
import os
|
||||
|
||||
configured = os.environ.get("EMBEDDINGS_CACHE_DIR")
|
||||
if configured:
|
||||
return configured
|
||||
from docsgpt.core.settings import settings
|
||||
|
||||
return settings.EMBEDDINGS_CACHE_DIR or None
|
||||
|
||||
|
||||
def _parse(argv: Optional[Sequence[str]], prog: str, description: str) -> list[str]:
|
||||
import argparse
|
||||
|
||||
@@ -109,13 +107,11 @@ def _parse(argv: Optional[Sequence[str]], prog: str, description: str) -> list[s
|
||||
|
||||
def main(argv: Optional[Sequence[str]] = None) -> int:
|
||||
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
|
||||
import os
|
||||
|
||||
names = _parse(argv, "prefetch-models", "Download the embedding models and the tiktoken encodings into the local caches.")
|
||||
fetched = prefetch(names, os.environ.get("EMBEDDINGS_CACHE_DIR"))
|
||||
logger.info("Cached %d model(s): %s", len(fetched), ", ".join(fetched))
|
||||
encodings = prefetch_tiktoken()
|
||||
logger.info("Cached tiktoken encoding(s): %s", ", ".join(encodings))
|
||||
names = _parse(argv, "prefetch-models", "Download the embedding models and their tokenizers into the model cache.")
|
||||
cache_dir = _cache_dir()
|
||||
fetched = prefetch(names, cache_dir)
|
||||
logger.info("Cached %d model(s) in %s: %s", len(fetched), cache_dir or "FastEmbed's default cache", ", ".join(fetched))
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ import time
|
||||
from typing import Callable, List, Optional, Sequence
|
||||
|
||||
from docsgpt.core.optional_deps import is_available
|
||||
from docsgpt.scripts.prefetch_models import DEFAULT_MODELS, TIKTOKEN_ENCODINGS
|
||||
from docsgpt.scripts.prefetch_models import DEFAULT_MODELS
|
||||
from docsgpt.vectorstore.model_registry import resolve
|
||||
|
||||
logger = logging.getLogger("verify_offline")
|
||||
@@ -50,16 +50,12 @@ def verify(models: Sequence[str]) -> bool:
|
||||
"""Run every check; return whether all passed."""
|
||||
ok = True
|
||||
|
||||
def tiktoken_check(encoding: str) -> Callable[[], object]:
|
||||
def run() -> object:
|
||||
import tiktoken
|
||||
def tiktoken_check() -> object:
|
||||
from docsgpt.utils import get_encoding
|
||||
|
||||
return f"{len(tiktoken.get_encoding(encoding).encode('hello world'))} tokens"
|
||||
return f"{len(get_encoding().encode('hello world'))} tokens"
|
||||
|
||||
return run
|
||||
|
||||
for encoding in TIKTOKEN_ENCODINGS:
|
||||
ok &= _check(f"tiktoken {encoding}", tiktoken_check(encoding))
|
||||
ok &= _check("tiktoken cl100k_base", tiktoken_check)
|
||||
|
||||
for name in models:
|
||||
spec = resolve(name)
|
||||
|
||||
@@ -1,7 +1,12 @@
|
||||
from typing import Optional
|
||||
|
||||
from docsgpt.stt.base import BaseSTT
|
||||
from docsgpt.stt.faster_whisper_stt import FasterWhisperSTT
|
||||
from docsgpt.stt.openai_stt import OpenAISTT
|
||||
|
||||
#: STT_PROVIDER value that switches speech-to-text off.
|
||||
DISABLED = "none"
|
||||
|
||||
|
||||
class STTCreator:
|
||||
stt_providers = {
|
||||
@@ -9,6 +14,11 @@ class STTCreator:
|
||||
"faster_whisper": FasterWhisperSTT,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def is_enabled(stt_type: Optional[str]) -> bool:
|
||||
"""False when the provider is ``none`` or empty: speech-to-text is switched off."""
|
||||
return (stt_type or "").strip().lower() not in ("", DISABLED)
|
||||
|
||||
@classmethod
|
||||
def create_stt(cls, stt_type, *args, **kwargs) -> BaseSTT:
|
||||
stt_class = cls.stt_providers.get(stt_type.lower())
|
||||
|
||||
@@ -1,7 +1,11 @@
|
||||
from typing import Optional
|
||||
|
||||
from docsgpt.tts.google_tts import GoogleTTS
|
||||
from docsgpt.tts.elevenlabs import ElevenlabsTTS
|
||||
from docsgpt.tts.base import BaseTTS
|
||||
|
||||
#: TTS_PROVIDER value that switches text-to-speech off.
|
||||
DISABLED = "none"
|
||||
|
||||
|
||||
class TTSCreator:
|
||||
@@ -10,9 +14,14 @@ class TTSCreator:
|
||||
"elevenlabs": ElevenlabsTTS,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def is_enabled(tts_type: Optional[str]) -> bool:
|
||||
"""False when the provider is ``none`` or empty: text-to-speech is switched off."""
|
||||
return (tts_type or "").strip().lower() not in ("", DISABLED)
|
||||
|
||||
@classmethod
|
||||
def create_tts(cls, tts_type, *args, **kwargs)-> BaseTTS:
|
||||
tts_class = cls.tts_providers.get(tts_type.lower())
|
||||
if not tts_class:
|
||||
raise ValueError(f"No tts class found for type {tts_type}")
|
||||
return tts_class(*args, **kwargs)
|
||||
return tts_class(*args, **kwargs)
|
||||
+36
-3
@@ -6,7 +6,7 @@ import logging
|
||||
import os
|
||||
import re
|
||||
import uuid
|
||||
from pathlib import PurePosixPath
|
||||
from pathlib import Path, PurePosixPath
|
||||
from typing import List
|
||||
|
||||
import tiktoken
|
||||
@@ -22,11 +22,44 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
_encoding = None
|
||||
|
||||
# cl100k_base ships with the package; tiktoken would download it on first use.
|
||||
_CL100K_BASE_FILE = Path(__file__).resolve().parent / "core" / "encodings" / "cl100k_base.tiktoken"
|
||||
_CL100K_BASE_SHA256 = "223921b76ee99bde995b7ff738513eef100fb51d18c93597a113bcffe865b2a7"
|
||||
# Pattern and special tokens exactly as tiktoken_ext.openai_public defines cl100k_base.
|
||||
_CL100K_BASE_PAT_STR = (
|
||||
r"""'(?i:[sdmt]|ll|ve|re)|[^\r\n\p{L}\p{N}]?+\p{L}++|\p{N}{1,3}+| ?[^\s\p{L}\p{N}]++[\r\n]*+|\s++$|\s*[\r\n]|\s+(?!\S)|\s"""
|
||||
)
|
||||
_CL100K_BASE_SPECIAL_TOKENS = {
|
||||
"<|endoftext|>": 100257,
|
||||
"<|fim_prefix|>": 100258,
|
||||
"<|fim_middle|>": 100259,
|
||||
"<|fim_suffix|>": 100260,
|
||||
"<|endofprompt|>": 100276,
|
||||
}
|
||||
|
||||
def get_encoding():
|
||||
|
||||
def _load_cl100k_base() -> tiktoken.Encoding:
|
||||
"""Build cl100k_base from the packaged ranks file."""
|
||||
data = _CL100K_BASE_FILE.read_bytes()
|
||||
if hashlib.sha256(data).hexdigest() != _CL100K_BASE_SHA256:
|
||||
raise ValueError(f"{_CL100K_BASE_FILE} does not match the cl100k_base checksum")
|
||||
ranks = {
|
||||
base64.b64decode(token): int(rank)
|
||||
for token, rank in (line.split() for line in data.splitlines() if line)
|
||||
}
|
||||
return tiktoken.Encoding(
|
||||
name="cl100k_base",
|
||||
pat_str=_CL100K_BASE_PAT_STR,
|
||||
mergeable_ranks=ranks,
|
||||
special_tokens=_CL100K_BASE_SPECIAL_TOKENS,
|
||||
)
|
||||
|
||||
|
||||
def get_encoding() -> tiktoken.Encoding:
|
||||
"""The cl100k_base encoding, loaded once per process."""
|
||||
global _encoding
|
||||
if _encoding is None:
|
||||
_encoding = tiktoken.get_encoding("cl100k_base")
|
||||
_encoding = _load_cl100k_base()
|
||||
return _encoding
|
||||
|
||||
|
||||
|
||||
@@ -97,17 +97,18 @@ def _register(model: EmbeddingModel) -> None:
|
||||
def _read_repo_json(repo: str, filename: str) -> Optional[dict]:
|
||||
"""Fetch one small JSON from ``repo``, or ``None`` when it is not there.
|
||||
|
||||
Reads through the Hugging Face hub cache, so a warmed image finds it
|
||||
Reads through the embedding model cache, so a warmed install finds it
|
||||
offline. Every failure -- absent file, no network, malformed JSON -- is the
|
||||
same answer to the caller: this repository does not tell us.
|
||||
"""
|
||||
try:
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
cache_dir = settings.EMBEDDINGS_CACHE_DIR or None
|
||||
try:
|
||||
path = hf_hub_download(repo_id=repo, filename=filename, local_files_only=True)
|
||||
path = hf_hub_download(repo_id=repo, filename=filename, local_files_only=True, cache_dir=cache_dir)
|
||||
except Exception: # noqa: BLE001 -- not cached: fetch it
|
||||
path = hf_hub_download(repo_id=repo, filename=filename)
|
||||
path = hf_hub_download(repo_id=repo, filename=filename, cache_dir=cache_dir)
|
||||
with open(path, encoding="utf-8") as handle:
|
||||
return json.load(handle)
|
||||
except Exception as exc:
|
||||
|
||||
@@ -30,6 +30,7 @@ import {
|
||||
selectSelectedDocs,
|
||||
selectSelectedModel,
|
||||
selectSourceDocs,
|
||||
selectSttAvailable,
|
||||
selectToken,
|
||||
setSelectedDocs,
|
||||
} from '../preferences/preferenceSlice';
|
||||
@@ -348,6 +349,7 @@ export default function MessageInput({
|
||||
const token = useSelector(selectToken);
|
||||
const attachments = useSelector(selectAttachments);
|
||||
const selectedModel = useSelector(selectSelectedModel);
|
||||
const sttAvailable = useSelector(selectSttAvailable);
|
||||
const unreadableAttachmentIds = useMemo(
|
||||
() =>
|
||||
new Set(
|
||||
@@ -1771,7 +1773,7 @@ export default function MessageInput({
|
||||
loading={toolsLoading}
|
||||
/>
|
||||
)}
|
||||
{ENABLE_VOICE_INPUT && (
|
||||
{ENABLE_VOICE_INPUT && sttAvailable && (
|
||||
<MicButton
|
||||
recordingState={recordingState}
|
||||
loading={loading}
|
||||
|
||||
@@ -27,6 +27,7 @@ import {
|
||||
selectChunks,
|
||||
selectSelectedDocs,
|
||||
selectToken,
|
||||
selectTtsAvailable,
|
||||
} from '../preferences/preferenceSlice';
|
||||
import { isToolCallRunning } from '../utils/streamingStatusUtils';
|
||||
import AnswerFlow from './AnswerFlow';
|
||||
@@ -110,6 +111,7 @@ const ConversationBubble = forwardRef<
|
||||
// const bubbleRef = useRef<HTMLDivElement | null>(null);
|
||||
const chunks = useSelector(selectChunks);
|
||||
const selectedDocs = useSelector(selectSelectedDocs);
|
||||
const ttsAvailable = useSelector(selectTtsAvailable);
|
||||
const [isEditClicked, setIsEditClicked] = useState(false);
|
||||
const [editInputBox, setEditInputBox] = useState<string>('');
|
||||
const messageRef = useRef<HTMLDivElement>(null);
|
||||
@@ -554,9 +556,11 @@ const ConversationBubble = forwardRef<
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
<div className="relative block items-center justify-center">
|
||||
<SpeakButton text={message} />
|
||||
</div>
|
||||
{ttsAvailable && (
|
||||
<div className="relative block items-center justify-center">
|
||||
<SpeakButton text={message} />
|
||||
</div>
|
||||
)}
|
||||
{handleFeedback && (
|
||||
<>
|
||||
<div className="relative flex items-center justify-center">
|
||||
|
||||
@@ -81,6 +81,8 @@ const makeStore = (
|
||||
agentFolders: null,
|
||||
roles: [],
|
||||
rolesResolved: false,
|
||||
ttsAvailable: true,
|
||||
sttAvailable: true,
|
||||
};
|
||||
const conversation: ConversationState = {
|
||||
queries: [],
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { useEffect } from 'react';
|
||||
import { useDispatch, useSelector } from 'react-redux';
|
||||
|
||||
import userService from '../api/services/userService';
|
||||
import {
|
||||
getDocs,
|
||||
getConversations,
|
||||
@@ -12,6 +13,7 @@ import {
|
||||
setConversations,
|
||||
setPrompts,
|
||||
setSourceDocs,
|
||||
setSpeechAvailability,
|
||||
} from '../preferences/preferenceSlice';
|
||||
|
||||
/**
|
||||
@@ -22,6 +24,7 @@ import {
|
||||
* - Fetching and setting up source documents
|
||||
* - Fetching and setting up prompts
|
||||
* - Fetching and setting up conversations
|
||||
* - Reading which speech features the server has switched on
|
||||
*
|
||||
* @param isAuthLoading -
|
||||
*/
|
||||
@@ -30,6 +33,22 @@ export default function useDataInitializer(isAuthLoading: boolean) {
|
||||
const token = useSelector(selectToken);
|
||||
const conversations = useSelector(selectConversations);
|
||||
|
||||
// Speech features; /api/config needs no auth.
|
||||
useEffect(() => {
|
||||
userService
|
||||
.getConfig()
|
||||
.then((response) => response.json())
|
||||
.then((config) => {
|
||||
dispatch(
|
||||
setSpeechAvailability({
|
||||
tts: config?.tts_available !== false,
|
||||
stt: config?.stt_available !== false,
|
||||
}),
|
||||
);
|
||||
})
|
||||
.catch(() => undefined);
|
||||
}, [dispatch]);
|
||||
|
||||
// Initialize documents
|
||||
useEffect(() => {
|
||||
// Skip if auth is still loading
|
||||
|
||||
@@ -1,6 +1,3 @@
|
||||
@import url('https://fonts.googleapis.com/css2?family=Roboto:wght@300;400;500;700&display=swap')
|
||||
layer(base);
|
||||
|
||||
@import 'tailwindcss';
|
||||
|
||||
@import 'tw-animate-css';
|
||||
@@ -8,7 +5,6 @@ layer(base);
|
||||
@custom-variant dark (&:is(.dark *));
|
||||
|
||||
@theme {
|
||||
--font-roboto: Roboto, sans-serif;
|
||||
--font-mono:
|
||||
'IBMPlexMono-Medium', ui-monospace, SFMono-Regular, Menlo, Consolas,
|
||||
monospace;
|
||||
|
||||
@@ -120,7 +120,7 @@ export default function AgentDetailsModal({
|
||||
</div>
|
||||
{sharedToken ? (
|
||||
<div className="flex flex-col gap-2">
|
||||
<p className="font-roboto dark:text-foreground inline text-sm leading-normal font-medium break-all text-gray-700">
|
||||
<p className="dark:text-foreground inline text-sm leading-normal font-medium break-all text-gray-700">
|
||||
<a
|
||||
href={`${baseURL}/shared/agent/${sharedToken}`}
|
||||
target="_blank"
|
||||
@@ -172,7 +172,7 @@ export default function AgentDetailsModal({
|
||||
{apiKey ? (
|
||||
<div className="flex flex-col gap-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<div className="font-roboto dark:text-foreground text-sm leading-normal font-medium break-all text-gray-700">
|
||||
<div className="dark:text-foreground text-sm leading-normal font-medium break-all text-gray-700">
|
||||
{apiKey}
|
||||
{!apiKey.includes('...') && (
|
||||
<CopyButton
|
||||
@@ -231,7 +231,7 @@ export default function AgentDetailsModal({
|
||||
</div>
|
||||
{webhookUrl ? (
|
||||
<div className="flex flex-col gap-2">
|
||||
<p className="font-roboto dark:text-foreground text-sm leading-normal font-medium break-all text-gray-700">
|
||||
<p className="dark:text-foreground text-sm leading-normal font-medium break-all text-gray-700">
|
||||
<a href={webhookUrl} target="_blank" rel="noreferrer">
|
||||
{webhookUrl}
|
||||
</a>
|
||||
|
||||
@@ -44,6 +44,10 @@ export interface Preference {
|
||||
// not act before the first /me response (avoids a first-paint admin bounce).
|
||||
roles: string[];
|
||||
rolesResolved: boolean;
|
||||
// Speech features the server has switched on (/api/config). True until the
|
||||
// config loads, so a backend without these flags keeps showing the controls.
|
||||
ttsAvailable: boolean;
|
||||
sttAvailable: boolean;
|
||||
}
|
||||
|
||||
const initialState: Preference = {
|
||||
@@ -74,6 +78,8 @@ const initialState: Preference = {
|
||||
agentFolders: null,
|
||||
roles: [],
|
||||
rolesResolved: false,
|
||||
ttsAvailable: true,
|
||||
sttAvailable: true,
|
||||
};
|
||||
|
||||
export const prefSlice = createSlice({
|
||||
@@ -142,6 +148,13 @@ export const prefSlice = createSlice({
|
||||
state.roles = [];
|
||||
state.rolesResolved = false;
|
||||
},
|
||||
setSpeechAvailability: (
|
||||
state,
|
||||
action: PayloadAction<{ tts: boolean; stt: boolean }>,
|
||||
) => {
|
||||
state.ttsAvailable = action.payload.tts;
|
||||
state.sttAvailable = action.payload.stt;
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
@@ -166,6 +179,7 @@ export const {
|
||||
setAgentFolders,
|
||||
setRoles,
|
||||
clearRoles,
|
||||
setSpeechAvailability,
|
||||
} = prefSlice.actions;
|
||||
export default prefSlice.reducer;
|
||||
|
||||
@@ -314,3 +328,7 @@ export const selectRolesResolved = (state: RootState) =>
|
||||
state.preference.rolesResolved;
|
||||
export const selectIsAdmin = (state: RootState) =>
|
||||
state.preference.roles.includes('admin');
|
||||
export const selectTtsAvailable = (state: RootState) =>
|
||||
state.preference.ttsAvailable;
|
||||
export const selectSttAvailable = (state: RootState) =>
|
||||
state.preference.sttAvailable;
|
||||
@@ -56,6 +56,8 @@ const preloadedState: { preference: Preference } = {
|
||||
agentFolders: null,
|
||||
roles: [],
|
||||
rolesResolved: false,
|
||||
ttsAvailable: true,
|
||||
sttAvailable: true,
|
||||
},
|
||||
};
|
||||
const store = configureStore({
|
||||
|
||||
@@ -1854,6 +1854,81 @@ class TestTextToSpeech:
|
||||
assert _get_response_status(response) == 400
|
||||
assert _get_response_json(response)["success"] is False
|
||||
|
||||
@patch("docsgpt.api.user.attachments.routes.TTSCreator.create_tts")
|
||||
def test_tts_disabled_returns_404_without_a_provider(self, mock_create_tts, flask_app):
|
||||
from docsgpt.api.user.attachments import routes
|
||||
|
||||
app = Flask(__name__)
|
||||
with patch.object(routes.settings, "TTS_PROVIDER", "none"), app.test_request_context(
|
||||
"/api/tts",
|
||||
method="POST",
|
||||
json={"text": "Hello world"},
|
||||
):
|
||||
response = routes.TextToSpeech().post()
|
||||
assert _get_response_status(response) == 404
|
||||
assert _get_response_json(response) == {
|
||||
"success": False,
|
||||
"message": "Text-to-speech is disabled on this server.",
|
||||
}
|
||||
mock_create_tts.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestSpeechToTextDisabled:
|
||||
"""STT_PROVIDER=none turns every transcription endpoint off before any provider is built."""
|
||||
|
||||
DISABLED = {"success": False, "message": "Speech-to-text is disabled on this server."}
|
||||
|
||||
@patch("docsgpt.api.user.attachments.routes.STTCreator.create_stt")
|
||||
def test_stt_returns_404(self, mock_create_stt, flask_app):
|
||||
from docsgpt.api.user.attachments import routes
|
||||
|
||||
app = Flask(__name__)
|
||||
with patch.object(routes.settings, "STT_PROVIDER", "none"), app.test_request_context(
|
||||
"/api/stt",
|
||||
method="POST",
|
||||
data={"file": (io.BytesIO(b"audio-bytes"), "clip.wav")},
|
||||
content_type="multipart/form-data",
|
||||
):
|
||||
request.decoded_token = {"sub": "test_user"}
|
||||
response = routes.SpeechToText().post()
|
||||
assert _get_response_status(response) == 404
|
||||
assert _get_response_json(response) == self.DISABLED
|
||||
mock_create_stt.assert_not_called()
|
||||
|
||||
def test_live_stt_start_returns_404(self, flask_app):
|
||||
from docsgpt.api.user.attachments import routes
|
||||
|
||||
app = Flask(__name__)
|
||||
with patch.object(routes.settings, "STT_PROVIDER", "none"), app.test_request_context(
|
||||
"/api/stt/live/start", method="POST", json={}
|
||||
):
|
||||
request.decoded_token = {"sub": "test_user"}
|
||||
response = routes.LiveSpeechToTextStart().post()
|
||||
assert _get_response_status(response) == 404
|
||||
assert _get_response_json(response) == self.DISABLED
|
||||
|
||||
@patch("docsgpt.api.user.attachments.routes.STTCreator.create_stt")
|
||||
def test_live_stt_chunk_returns_404(self, mock_create_stt, flask_app):
|
||||
from docsgpt.api.user.attachments import routes
|
||||
|
||||
app = Flask(__name__)
|
||||
with patch.object(routes.settings, "STT_PROVIDER", "none"), app.test_request_context(
|
||||
"/api/stt/live/chunk",
|
||||
method="POST",
|
||||
data={
|
||||
"session_id": "abc",
|
||||
"chunk_index": "0",
|
||||
"file": (io.BytesIO(b"audio-bytes"), "chunk.wav"),
|
||||
},
|
||||
content_type="multipart/form-data",
|
||||
):
|
||||
request.decoded_token = {"sub": "test_user"}
|
||||
response = routes.LiveSpeechToTextChunk().post()
|
||||
assert _get_response_status(response) == 404
|
||||
assert _get_response_json(response) == self.DISABLED
|
||||
mock_create_stt.assert_not_called()
|
||||
|
||||
|
||||
# =====================================================================
|
||||
# Coverage gap tests (lines 136, 256, 330, 337, 443, 457, 560, 590)
|
||||
|
||||
@@ -57,12 +57,23 @@ class TestSettingsFollowTheHome:
|
||||
assert fresh.MILVUS_URI == str(tmp_path.resolve() / "milvus_local.db")
|
||||
assert fresh.LANCEDB_PATH == str(tmp_path.resolve() / "data" / "lancedb")
|
||||
|
||||
def test_model_cache_defaults_under_the_home(self, monkeypatch, tmp_path):
|
||||
"""FastEmbed's own default is the temp dir, which a reboot wipes."""
|
||||
from docsgpt.core.settings import Settings
|
||||
|
||||
monkeypatch.setenv(paths.HOME_ENV, str(tmp_path))
|
||||
monkeypatch.delenv("EMBEDDINGS_CACHE_DIR", raising=False)
|
||||
assert Settings(_env_file=None).EMBEDDINGS_CACHE_DIR == str(tmp_path.resolve() / "models")
|
||||
|
||||
def test_env_overrides_still_win(self, monkeypatch, tmp_path):
|
||||
from docsgpt.core.settings import Settings
|
||||
|
||||
monkeypatch.setenv(paths.HOME_ENV, str(tmp_path))
|
||||
monkeypatch.setenv("LANCEDB_PATH", "/srv/lancedb")
|
||||
assert Settings(_env_file=None).LANCEDB_PATH == "/srv/lancedb"
|
||||
monkeypatch.setenv("EMBEDDINGS_CACHE_DIR", "/srv/models")
|
||||
fresh = Settings(_env_file=None)
|
||||
assert fresh.LANCEDB_PATH == "/srv/lancedb"
|
||||
assert fresh.EMBEDDINGS_CACHE_DIR == "/srv/models"
|
||||
|
||||
|
||||
class TestPackageDir:
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from docsgpt.parser.file.audio_parser import AudioParser
|
||||
from docsgpt.parser.file.base_parser import DocumentParseError
|
||||
from docsgpt.parser.file.bulk import get_default_file_extractor
|
||||
from docsgpt.stt.upload_limits import AudioFileTooLargeError
|
||||
|
||||
@@ -44,6 +47,20 @@ def test_audio_parser_transcribes_file(
|
||||
)
|
||||
|
||||
|
||||
@patch("docsgpt.parser.file.audio_parser.STTCreator.create_stt")
|
||||
@patch("docsgpt.parser.file.audio_parser.settings")
|
||||
def test_audio_parser_refuses_when_speech_to_text_is_disabled(
|
||||
mock_settings, mock_create_stt, tmp_path
|
||||
):
|
||||
mock_settings.STT_PROVIDER = "none"
|
||||
audio_file = tmp_path / "meeting.wav"
|
||||
audio_file.write_bytes(b"audio-bytes")
|
||||
|
||||
with pytest.raises(DocumentParseError, match="STT_PROVIDER"):
|
||||
AudioParser().parse_file(audio_file)
|
||||
mock_create_stt.assert_not_called()
|
||||
|
||||
|
||||
@patch("docsgpt.stt.upload_limits.settings")
|
||||
def test_audio_parser_rejects_oversized_files(mock_limit_settings, tmp_path):
|
||||
mock_limit_settings.STT_MAX_FILE_SIZE_MB = 1
|
||||
|
||||
@@ -309,7 +309,7 @@ class TestTokenizerFile:
|
||||
def test_cache_hit_makes_no_online_call(self, monkeypatch):
|
||||
calls = []
|
||||
|
||||
def fake_download(repo, filename, local_files_only=False):
|
||||
def fake_download(repo, filename, local_files_only=False, cache_dir=None):
|
||||
calls.append(local_files_only)
|
||||
return "/cache/tokenizer.json"
|
||||
|
||||
@@ -322,7 +322,7 @@ class TestTokenizerFile:
|
||||
def test_cache_miss_falls_back_to_online(self, monkeypatch):
|
||||
calls = []
|
||||
|
||||
def fake_download(repo, filename, local_files_only=False):
|
||||
def fake_download(repo, filename, local_files_only=False, cache_dir=None):
|
||||
calls.append(local_files_only)
|
||||
if local_files_only:
|
||||
raise FileNotFoundError("not cached")
|
||||
@@ -333,3 +333,20 @@ class TestTokenizerFile:
|
||||
monkeypatch.setitem(sys.modules, "huggingface_hub", fake_hub)
|
||||
assert tokenization._tokenizer_file("org/model") == "/downloaded/tokenizer.json"
|
||||
assert calls == [True, False]
|
||||
|
||||
def test_reads_the_embedding_model_cache(self, monkeypatch):
|
||||
"""FastEmbed's snapshot already holds tokenizer.json; a second cache downloads it again."""
|
||||
cache_dirs = []
|
||||
|
||||
def fake_download(repo, filename, local_files_only=False, cache_dir=None):
|
||||
cache_dirs.append(cache_dir)
|
||||
if local_files_only:
|
||||
raise FileNotFoundError("not cached")
|
||||
return "/models/tokenizer.json"
|
||||
|
||||
fake_hub = types.ModuleType("huggingface_hub")
|
||||
fake_hub.hf_hub_download = fake_download
|
||||
monkeypatch.setitem(sys.modules, "huggingface_hub", fake_hub)
|
||||
monkeypatch.setattr(tokenization.settings, "EMBEDDINGS_CACHE_DIR", "/models")
|
||||
tokenization._tokenizer_file("org/model")
|
||||
assert cache_dirs == ["/models", "/models"]
|
||||
@@ -95,17 +95,12 @@ class TestMain:
|
||||
prefetch_models.main(["granite-97m"])
|
||||
assert spy.call_args.args[1] == "/app/models"
|
||||
|
||||
def test_cache_dir_defaults_to_the_one_the_app_reads(self, fake_fastembed, monkeypatch):
|
||||
"""Without the variable, models must land where the running app looks for them."""
|
||||
from docsgpt.core.settings import settings
|
||||
|
||||
class TestPrefetchTiktoken:
|
||||
def test_warms_every_listed_encoding(self):
|
||||
"""The image sets TIKTOKEN_CACHE_DIR; warming fills it at build time."""
|
||||
fake = MagicMock()
|
||||
module = types.ModuleType("tiktoken")
|
||||
module.get_encoding = fake
|
||||
with patch.dict(sys.modules, {"tiktoken": module}):
|
||||
fetched = prefetch_models.prefetch_tiktoken()
|
||||
assert fetched == list(prefetch_models.TIKTOKEN_ENCODINGS)
|
||||
assert [c.args[0] for c in fake.call_args_list] == list(prefetch_models.TIKTOKEN_ENCODINGS)
|
||||
|
||||
def test_cl100k_is_the_encoding_token_counting_uses(self):
|
||||
assert "cl100k_base" in prefetch_models.TIKTOKEN_ENCODINGS
|
||||
monkeypatch.delenv("EMBEDDINGS_CACHE_DIR", raising=False)
|
||||
monkeypatch.setattr(settings, "EMBEDDINGS_CACHE_DIR", "/home/docsgpt/models")
|
||||
with patch.object(prefetch_models, "prefetch", return_value=[]) as spy:
|
||||
prefetch_models.main(["granite-97m"])
|
||||
assert spy.call_args.args[1] == "/home/docsgpt/models"
|
||||
@@ -8,8 +8,11 @@ from docsgpt.scripts import verify_offline
|
||||
|
||||
|
||||
def _fake_tiktoken(monkeypatch):
|
||||
"""The check must exercise the app's own loader, which reads the packaged encoding."""
|
||||
encoding = types.SimpleNamespace(name="cl100k_base", encode=lambda text: [1, 2])
|
||||
monkeypatch.setattr("docsgpt.utils.get_encoding", lambda: encoding)
|
||||
module = types.ModuleType("tiktoken")
|
||||
module.get_encoding = lambda name: types.SimpleNamespace(encode=lambda text: [1, 2])
|
||||
module.get_encoding = lambda name: (_ for _ in ()).throw(AssertionError("downloaded via tiktoken"))
|
||||
monkeypatch.setitem(sys.modules, "tiktoken", module)
|
||||
|
||||
|
||||
|
||||
@@ -51,6 +51,16 @@ def test_stt_type_case_insensitivity(stt_creator):
|
||||
assert result == instance
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["none", "NONE", " none ", "", None])
|
||||
def test_is_enabled_false_when_switched_off(value):
|
||||
assert STTCreator.is_enabled(value) is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["openai", "faster_whisper"])
|
||||
def test_is_enabled_for_a_provider(value):
|
||||
assert STTCreator.is_enabled(value) is True
|
||||
|
||||
|
||||
def test_stt_providers_integrity(stt_creator):
|
||||
providers = stt_creator.stt_providers
|
||||
assert "openai" in providers
|
||||
|
||||
@@ -95,6 +95,28 @@ class TestConfigRoute:
|
||||
data = json.loads(response.data)
|
||||
assert data["hybrid_available"] is False
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_speech_features_available_by_default(self, client):
|
||||
with patch("docsgpt.app.settings") as mock_settings:
|
||||
mock_settings.AUTH_TYPE = None
|
||||
mock_settings.TTS_PROVIDER = "google_tts"
|
||||
mock_settings.STT_PROVIDER = "openai"
|
||||
response = client.get("/api/config")
|
||||
data = json.loads(response.data)
|
||||
assert data["tts_available"] is True
|
||||
assert data["stt_available"] is True
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_speech_features_unavailable_when_disabled(self, client):
|
||||
with patch("docsgpt.app.settings") as mock_settings:
|
||||
mock_settings.AUTH_TYPE = None
|
||||
mock_settings.TTS_PROVIDER = "none"
|
||||
mock_settings.STT_PROVIDER = "none"
|
||||
response = client.get("/api/config")
|
||||
data = json.loads(response.data)
|
||||
assert data["tts_available"] is False
|
||||
assert data["stt_available"] is False
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_oidc_config_exposes_login_paths(self, client):
|
||||
with patch("docsgpt.app.settings") as mock_settings:
|
||||
|
||||
@@ -40,6 +40,47 @@ class TestGetEncoding:
|
||||
enc2 = get_encoding()
|
||||
assert enc1 is enc2
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_loads_the_packaged_encoding_without_network(self, monkeypatch):
|
||||
"""Token counting runs on every chat; it must never download the encoding."""
|
||||
import tiktoken
|
||||
import tiktoken.load
|
||||
|
||||
from docsgpt import utils
|
||||
|
||||
def no_download(*args, **kwargs):
|
||||
raise AssertionError("tiktoken tried to fetch cl100k_base")
|
||||
|
||||
monkeypatch.setattr(utils, "_encoding", None)
|
||||
monkeypatch.setattr(tiktoken.load, "read_file", no_download)
|
||||
monkeypatch.setattr(tiktoken, "get_encoding", no_download)
|
||||
assert utils.get_encoding().encode("hello world") == [15339, 1917]
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_packaged_encoding_matches_the_tiktoken_definition(self, monkeypatch):
|
||||
from tiktoken_ext import openai_public
|
||||
|
||||
from docsgpt import utils
|
||||
|
||||
monkeypatch.setattr(utils, "_encoding", None)
|
||||
monkeypatch.setattr(openai_public, "load_tiktoken_bpe", lambda *args, **kwargs: {})
|
||||
reference = openai_public.cl100k_base()
|
||||
encoding = utils.get_encoding()
|
||||
assert encoding.name == reference["name"]
|
||||
assert encoding._pat_str == reference["pat_str"]
|
||||
assert encoding._special_tokens == reference["special_tokens"]
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_corrupt_packaged_encoding_is_rejected(self, monkeypatch, tmp_path):
|
||||
from docsgpt import utils
|
||||
|
||||
corrupt = tmp_path / "cl100k_base.tiktoken"
|
||||
corrupt.write_bytes(b"IQ== 0\n")
|
||||
monkeypatch.setattr(utils, "_encoding", None)
|
||||
monkeypatch.setattr(utils, "_CL100K_BASE_FILE", corrupt)
|
||||
with pytest.raises(ValueError, match="cl100k_base"):
|
||||
utils.get_encoding()
|
||||
|
||||
|
||||
class TestGetGptModel:
|
||||
|
||||
|
||||
@@ -53,6 +53,16 @@ def test_tts_type_case_insensitivity(tts_creator):
|
||||
assert result == instance
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["none", "NONE", " none ", "", None])
|
||||
def test_is_enabled_false_when_switched_off(value):
|
||||
assert TTSCreator.is_enabled(value) is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["google_tts", "elevenlabs"])
|
||||
def test_is_enabled_for_a_provider(value):
|
||||
assert TTSCreator.is_enabled(value) is True
|
||||
|
||||
|
||||
def test_tts_providers_integrity(tts_creator):
|
||||
providers = tts_creator.tts_providers
|
||||
assert "google_tts" in providers
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
"""Local embeddings run through FastEmbed, configured from the model registry."""
|
||||
|
||||
import sys
|
||||
import types
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import numpy as np
|
||||
@@ -9,6 +11,9 @@ from docsgpt.vectorstore import embeddings_local
|
||||
from docsgpt.vectorstore.embeddings_local import EmbeddingsWrapper
|
||||
from docsgpt.vectorstore.model_registry import GRANITE_97M, MPNET
|
||||
|
||||
# The autouse fixture below replaces this for every test; keep the real one.
|
||||
_READ_REPO_JSON = embeddings_local._read_repo_json
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clear_registration():
|
||||
@@ -142,6 +147,23 @@ class TestSettingsPassthrough:
|
||||
EmbeddingsWrapper(MPNET.name)
|
||||
assert text_embedding.call_args.kwargs["cache_dir"] == "/models"
|
||||
|
||||
def test_repo_metadata_reads_the_embedding_model_cache(self, monkeypatch, tmp_path):
|
||||
"""Pooling metadata lives beside the model, not in a second hub cache."""
|
||||
config = tmp_path / "config.json"
|
||||
config.write_text('{"pooling_mode_cls_token": true}')
|
||||
calls = []
|
||||
|
||||
def fake_download(repo_id, filename, local_files_only=False, cache_dir=None):
|
||||
calls.append((local_files_only, cache_dir))
|
||||
return str(config)
|
||||
|
||||
fake_hub = types.ModuleType("huggingface_hub")
|
||||
fake_hub.hf_hub_download = fake_download
|
||||
monkeypatch.setitem(sys.modules, "huggingface_hub", fake_hub)
|
||||
monkeypatch.setattr(embeddings_local.settings, "EMBEDDINGS_CACHE_DIR", "/models")
|
||||
assert _READ_REPO_JSON("org/model", "1_Pooling/config.json") == {"pooling_mode_cls_token": True}
|
||||
assert calls == [(True, "/models")]
|
||||
|
||||
|
||||
class TestEmbedding:
|
||||
def test_embed_documents_returns_plain_lists(self, fake_fastembed):
|
||||
|
||||
Reference in new issue
Block a user