Files
goclaw/Dockerfile
T
ace07509b7 feat(skills): system skills integration — toggle, dep checking, per-item install (#161)
* feat(infra): add runtime package support for skills

Install nodejs, npm, pandoc, github-cli + pre-install Python packages
(openpyxl, pandas, python-pptx, markitdown) and Node packages
(docx, pptxgenjs). Configure runtime dirs for agent pip/npm installs
with PIP_TARGET, NPM_CONFIG_PREFIX, NODE_PATH to enable dynamic
package installation in read-only container environment.

* feat(infra): add bundled skills with runtime package support

- Add 5 bundled skills: docx, pdf, pptx, xlsx, skill-creator from container skills-store
- Wire GOCLAW_BUILTIN_SKILLS_DIR env var in gateway and CLI
- Support optional runtime packages alongside dynamic skill loading
- Update Dockerfile to COPY bundled-skills at /app/bundled-skills/
- Add PIP_CACHE_DIR in docker-entrypoint.sh for clean pip installs
- Document bundled skills in 14-skills-runtime.md section 6

* feat(infra): remove ai-multimodal skill directory from bundled skills

Remove the ai-multimodal skill package as part of consolidating runtime
package support for bundled skills. This directory is no longer needed
in the bundled skills structure.

* feat(ci): add semantic release and Docker Hub publishing

Add go-semantic-release workflow to auto-create semver tags on merge to
main. Extend docker-publish to push all variants to both GHCR and
Docker Hub (digitop/goclaw).

* feat(skills): add system skills infrastructure with is_system column, dep scanning, and seeder

- Migration 000017: add is_system boolean column with partial index
- Store layer: UpsertSystemSkill, delete protection, IsSystemSkill
- ListAccessible auto-includes system skills (no grants needed)
- ListWithGrantStatus returns is_system field
- Dependency scanner: auto-detect deps from scripts/ or skill-manifest.json
- Dependency checker: verify system binaries, Python/Node packages
- Seeder: seed bundled skills into DB on startup (idempotent via hash)
- Gateway wiring: GOCLAW_BUNDLED_SKILLS_DIR env for bundled skills
- HTTP: delete guard (403), slug conflict check (409), rescan-deps endpoint
- UI: System badge, hide delete for system skills, rescan deps button
- Agent skills tab: "Always available" for system skills
- i18n: en/vi/zh keys for system skills, deps scanning

* feat(skills): conditional system prompt, skill manifests, and Zip Slip fix

- System prompt: only show package list when python3/node are available
- Add skill-manifest.json for pdf, docx, xlsx, pptx bundled skills
- Fix Zip Slip vulnerability in office/unpack.py (all 3 copies)

* refactor(skills): extract shared office code to _shared/ and deduplicate

Move office scripts (pack, unpack, validate, schemas, validators) from
duplicated copies in docx/xlsx/pptx to skills/_shared/office/ with
symlinks. Remove soffice.py (non-functional in containers) and update
SKILL.md references to use soffice binary directly. Update seeder
copyDir to follow symlinks.

Removes ~45K lines of duplicate code across 3 skills.

* fix(skills): address code review findings for system skills integration

- H1: Remove dead symlink branch in copyDir (filepath.Walk follows symlinks)
- H3: Fix rescan-deps to query ALL skills (including archived) and re-activate
  when deps become available; add ListAllSkills() + Status field to SkillInfo
- H4: Add Status field to SkillCreateParams, stop overloading Visibility
- M1: Batch Python/Node dep checks into single subprocess per runtime
- M4: Add rows.Err() check in ListSkills to prevent caching partial results

* feat(skills): async dep checking with realtime WS events

Split Seed() into sync DB upsert + async CheckDepsAsync() goroutine.
Gateway startup no longer blocks on Python/Node subprocess dep checks.

- Seed() returns seeded skills list, all initially status="active"
- CheckDepsAsync() runs in background, emits skill.deps.checked per-skill
- skill.deps.complete event emitted when all checks finish
- Each failed dep check: archives skill + BumpVersion() for immediate
  cache invalidation so next agent turn picks up the change
- UI: use-query-invalidation listens to skill.deps.* events → auto-refresh
  skills list in realtime

* feat(skills): system skills integration with toggle, dep checking, and per-item install

- Add is_system, deps, enabled columns to skills table (migration 017)
- Seed bundled core skills (pdf, docx, pptx, xlsx, skill-creator) on startup
- PYTHONPATH-based dep detection — eliminates false positives from local modules
- Per-item dep install UI with individual status (installing/success/error)
- Enable/disable toggle for core and custom skills (independent of dep status)
- Re-run dep check when skill is toggled back on
- Inline skill thresholds: 40 skills / 5000 tokens before switching to search mode
- Fix UpsertSystemSkill: backfill null file_hash without bumping DB version
- Remove redundant skill-manifest.json files (replaced by deps JSONB column)
- Show author from frontmatter in custom skills tab
- Runtime checker for python3/pip3/node/npm availability
- WS events for dep checking/installing progress
- docs: add 15-core-skills-system.md, 16-skill-publishing.md

---------

Co-authored-by: Goon <duy@wearetopgroup.com>
2026-03-12 09:20:41 +07:00

103 lines
3.3 KiB
Docker

# syntax=docker/dockerfile:1
# ── Stage 1: Build ──
FROM golang:1.26-bookworm AS builder
WORKDIR /src
# Cache dependencies
COPY go.mod go.sum ./
RUN go mod download
# Copy source
COPY . .
# Build args
ARG ENABLE_OTEL=false
ARG ENABLE_TSNET=false
ARG ENABLE_REDIS=false
ARG VERSION=dev
# Build static binary (CGO disabled for scratch/alpine compatibility)
RUN set -eux; \
TAGS=""; \
if [ "$ENABLE_OTEL" = "true" ]; then TAGS="otel"; fi; \
if [ "$ENABLE_TSNET" = "true" ]; then \
if [ -n "$TAGS" ]; then TAGS="$TAGS,tsnet"; else TAGS="tsnet"; fi; \
fi; \
if [ "$ENABLE_REDIS" = "true" ]; then \
if [ -n "$TAGS" ]; then TAGS="$TAGS,redis"; else TAGS="redis"; fi; \
fi; \
if [ -n "$TAGS" ]; then TAGS="-tags $TAGS"; fi; \
CGO_ENABLED=0 GOOS=linux \
go build -ldflags="-s -w -X github.com/nextlevelbuilder/goclaw/cmd.Version=${VERSION}" \
${TAGS} -o /out/goclaw .
# ── Stage 2: Runtime ──
FROM alpine:3.22
ARG ENABLE_SANDBOX=false
ARG ENABLE_PYTHON=false
ARG ENABLE_NODE=false
ARG ENABLE_FULL_SKILLS=false
# Install ca-certificates + wget (healthcheck) + optional runtimes.
# ENABLE_FULL_SKILLS=true pre-installs all skill deps (larger image, no on-demand install needed).
# Otherwise, skill packages are installed on-demand via the admin UI.
RUN set -eux; \
apk add --no-cache ca-certificates wget; \
if [ "$ENABLE_SANDBOX" = "true" ]; then \
apk add --no-cache docker-cli; \
fi; \
if [ "$ENABLE_FULL_SKILLS" = "true" ]; then \
apk add --no-cache python3 py3-pip nodejs npm pandoc github-cli doas; \
echo "permit nopass goclaw as root cmd apk" > /etc/doas.d/goclaw.conf; \
pip3 install --no-cache-dir --break-system-packages \
pypdf openpyxl pandas python-pptx markitdown defusedxml lxml; \
npm install -g --cache /tmp/npm-cache docx pptxgenjs; \
rm -rf /tmp/npm-cache /root/.cache /var/cache/apk/*; \
else \
if [ "$ENABLE_PYTHON" = "true" ]; then \
apk add --no-cache python3 py3-pip doas; \
echo "permit nopass goclaw as root cmd apk" > /etc/doas.d/goclaw.conf; \
fi; \
if [ "$ENABLE_NODE" = "true" ]; then \
apk add --no-cache nodejs npm; \
fi; \
fi
# Non-root user
RUN adduser -D -u 1000 -h /app goclaw
WORKDIR /app
# Copy binary, migrations, and bundled skills
COPY --from=builder /out/goclaw /app/goclaw
COPY --from=builder /src/migrations/ /app/migrations/
COPY --from=builder /src/skills/ /app/bundled-skills/
COPY docker-entrypoint.sh /app/docker-entrypoint.sh
RUN chmod +x /app/docker-entrypoint.sh
# Create data directories (owned by goclaw user)
RUN mkdir -p /app/workspace /app/data /app/sessions /app/skills /app/tsnet-state /app/.goclaw \
&& chown -R goclaw:goclaw /app
# Default environment
ENV GOCLAW_CONFIG=/app/config.json \
GOCLAW_WORKSPACE=/app/workspace \
GOCLAW_DATA_DIR=/app/data \
GOCLAW_SESSIONS_STORAGE=/app/sessions \
GOCLAW_SKILLS_DIR=/app/skills \
GOCLAW_MIGRATIONS_DIR=/app/migrations \
GOCLAW_HOST=0.0.0.0 \
GOCLAW_PORT=18790
USER goclaw
EXPOSE 18790
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD wget -qO- http://localhost:18790/health || exit 1
ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD ["serve"]