mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-07-31 08:25:03 +00:00
* feat(infra): add runtime package support for skills Install nodejs, npm, pandoc, github-cli + pre-install Python packages (openpyxl, pandas, python-pptx, markitdown) and Node packages (docx, pptxgenjs). Configure runtime dirs for agent pip/npm installs with PIP_TARGET, NPM_CONFIG_PREFIX, NODE_PATH to enable dynamic package installation in read-only container environment. * feat(infra): add bundled skills with runtime package support - Add 5 bundled skills: docx, pdf, pptx, xlsx, skill-creator from container skills-store - Wire GOCLAW_BUILTIN_SKILLS_DIR env var in gateway and CLI - Support optional runtime packages alongside dynamic skill loading - Update Dockerfile to COPY bundled-skills at /app/bundled-skills/ - Add PIP_CACHE_DIR in docker-entrypoint.sh for clean pip installs - Document bundled skills in 14-skills-runtime.md section 6 * feat(infra): remove ai-multimodal skill directory from bundled skills Remove the ai-multimodal skill package as part of consolidating runtime package support for bundled skills. This directory is no longer needed in the bundled skills structure. * feat(ci): add semantic release and Docker Hub publishing Add go-semantic-release workflow to auto-create semver tags on merge to main. Extend docker-publish to push all variants to both GHCR and Docker Hub (digitop/goclaw). * feat(skills): add system skills infrastructure with is_system column, dep scanning, and seeder - Migration 000017: add is_system boolean column with partial index - Store layer: UpsertSystemSkill, delete protection, IsSystemSkill - ListAccessible auto-includes system skills (no grants needed) - ListWithGrantStatus returns is_system field - Dependency scanner: auto-detect deps from scripts/ or skill-manifest.json - Dependency checker: verify system binaries, Python/Node packages - Seeder: seed bundled skills into DB on startup (idempotent via hash) - Gateway wiring: GOCLAW_BUNDLED_SKILLS_DIR env for bundled skills - HTTP: delete guard (403), slug conflict check (409), rescan-deps endpoint - UI: System badge, hide delete for system skills, rescan deps button - Agent skills tab: "Always available" for system skills - i18n: en/vi/zh keys for system skills, deps scanning * feat(skills): conditional system prompt, skill manifests, and Zip Slip fix - System prompt: only show package list when python3/node are available - Add skill-manifest.json for pdf, docx, xlsx, pptx bundled skills - Fix Zip Slip vulnerability in office/unpack.py (all 3 copies) * refactor(skills): extract shared office code to _shared/ and deduplicate Move office scripts (pack, unpack, validate, schemas, validators) from duplicated copies in docx/xlsx/pptx to skills/_shared/office/ with symlinks. Remove soffice.py (non-functional in containers) and update SKILL.md references to use soffice binary directly. Update seeder copyDir to follow symlinks. Removes ~45K lines of duplicate code across 3 skills. * fix(skills): address code review findings for system skills integration - H1: Remove dead symlink branch in copyDir (filepath.Walk follows symlinks) - H3: Fix rescan-deps to query ALL skills (including archived) and re-activate when deps become available; add ListAllSkills() + Status field to SkillInfo - H4: Add Status field to SkillCreateParams, stop overloading Visibility - M1: Batch Python/Node dep checks into single subprocess per runtime - M4: Add rows.Err() check in ListSkills to prevent caching partial results * feat(skills): async dep checking with realtime WS events Split Seed() into sync DB upsert + async CheckDepsAsync() goroutine. Gateway startup no longer blocks on Python/Node subprocess dep checks. - Seed() returns seeded skills list, all initially status="active" - CheckDepsAsync() runs in background, emits skill.deps.checked per-skill - skill.deps.complete event emitted when all checks finish - Each failed dep check: archives skill + BumpVersion() for immediate cache invalidation so next agent turn picks up the change - UI: use-query-invalidation listens to skill.deps.* events → auto-refresh skills list in realtime * feat(skills): system skills integration with toggle, dep checking, and per-item install - Add is_system, deps, enabled columns to skills table (migration 017) - Seed bundled core skills (pdf, docx, pptx, xlsx, skill-creator) on startup - PYTHONPATH-based dep detection — eliminates false positives from local modules - Per-item dep install UI with individual status (installing/success/error) - Enable/disable toggle for core and custom skills (independent of dep status) - Re-run dep check when skill is toggled back on - Inline skill thresholds: 40 skills / 5000 tokens before switching to search mode - Fix UpsertSystemSkill: backfill null file_hash without bumping DB version - Remove redundant skill-manifest.json files (replaced by deps JSONB column) - Show author from frontmatter in custom skills tab - Runtime checker for python3/pip3/node/npm availability - WS events for dep checking/installing progress - docs: add 15-core-skills-system.md, 16-skill-publishing.md --------- Co-authored-by: Goon <duy@wearetopgroup.com>
103 lines
3.3 KiB
Docker
103 lines
3.3 KiB
Docker
# syntax=docker/dockerfile:1
|
|
|
|
# ── Stage 1: Build ──
|
|
FROM golang:1.26-bookworm AS builder
|
|
|
|
WORKDIR /src
|
|
|
|
# Cache dependencies
|
|
COPY go.mod go.sum ./
|
|
RUN go mod download
|
|
|
|
# Copy source
|
|
COPY . .
|
|
|
|
# Build args
|
|
ARG ENABLE_OTEL=false
|
|
ARG ENABLE_TSNET=false
|
|
ARG ENABLE_REDIS=false
|
|
ARG VERSION=dev
|
|
|
|
# Build static binary (CGO disabled for scratch/alpine compatibility)
|
|
RUN set -eux; \
|
|
TAGS=""; \
|
|
if [ "$ENABLE_OTEL" = "true" ]; then TAGS="otel"; fi; \
|
|
if [ "$ENABLE_TSNET" = "true" ]; then \
|
|
if [ -n "$TAGS" ]; then TAGS="$TAGS,tsnet"; else TAGS="tsnet"; fi; \
|
|
fi; \
|
|
if [ "$ENABLE_REDIS" = "true" ]; then \
|
|
if [ -n "$TAGS" ]; then TAGS="$TAGS,redis"; else TAGS="redis"; fi; \
|
|
fi; \
|
|
if [ -n "$TAGS" ]; then TAGS="-tags $TAGS"; fi; \
|
|
CGO_ENABLED=0 GOOS=linux \
|
|
go build -ldflags="-s -w -X github.com/nextlevelbuilder/goclaw/cmd.Version=${VERSION}" \
|
|
${TAGS} -o /out/goclaw .
|
|
|
|
# ── Stage 2: Runtime ──
|
|
FROM alpine:3.22
|
|
|
|
ARG ENABLE_SANDBOX=false
|
|
ARG ENABLE_PYTHON=false
|
|
ARG ENABLE_NODE=false
|
|
ARG ENABLE_FULL_SKILLS=false
|
|
|
|
# Install ca-certificates + wget (healthcheck) + optional runtimes.
|
|
# ENABLE_FULL_SKILLS=true pre-installs all skill deps (larger image, no on-demand install needed).
|
|
# Otherwise, skill packages are installed on-demand via the admin UI.
|
|
RUN set -eux; \
|
|
apk add --no-cache ca-certificates wget; \
|
|
if [ "$ENABLE_SANDBOX" = "true" ]; then \
|
|
apk add --no-cache docker-cli; \
|
|
fi; \
|
|
if [ "$ENABLE_FULL_SKILLS" = "true" ]; then \
|
|
apk add --no-cache python3 py3-pip nodejs npm pandoc github-cli doas; \
|
|
echo "permit nopass goclaw as root cmd apk" > /etc/doas.d/goclaw.conf; \
|
|
pip3 install --no-cache-dir --break-system-packages \
|
|
pypdf openpyxl pandas python-pptx markitdown defusedxml lxml; \
|
|
npm install -g --cache /tmp/npm-cache docx pptxgenjs; \
|
|
rm -rf /tmp/npm-cache /root/.cache /var/cache/apk/*; \
|
|
else \
|
|
if [ "$ENABLE_PYTHON" = "true" ]; then \
|
|
apk add --no-cache python3 py3-pip doas; \
|
|
echo "permit nopass goclaw as root cmd apk" > /etc/doas.d/goclaw.conf; \
|
|
fi; \
|
|
if [ "$ENABLE_NODE" = "true" ]; then \
|
|
apk add --no-cache nodejs npm; \
|
|
fi; \
|
|
fi
|
|
|
|
# Non-root user
|
|
RUN adduser -D -u 1000 -h /app goclaw
|
|
WORKDIR /app
|
|
|
|
# Copy binary, migrations, and bundled skills
|
|
COPY --from=builder /out/goclaw /app/goclaw
|
|
COPY --from=builder /src/migrations/ /app/migrations/
|
|
COPY --from=builder /src/skills/ /app/bundled-skills/
|
|
COPY docker-entrypoint.sh /app/docker-entrypoint.sh
|
|
RUN chmod +x /app/docker-entrypoint.sh
|
|
|
|
# Create data directories (owned by goclaw user)
|
|
RUN mkdir -p /app/workspace /app/data /app/sessions /app/skills /app/tsnet-state /app/.goclaw \
|
|
&& chown -R goclaw:goclaw /app
|
|
|
|
# Default environment
|
|
ENV GOCLAW_CONFIG=/app/config.json \
|
|
GOCLAW_WORKSPACE=/app/workspace \
|
|
GOCLAW_DATA_DIR=/app/data \
|
|
GOCLAW_SESSIONS_STORAGE=/app/sessions \
|
|
GOCLAW_SKILLS_DIR=/app/skills \
|
|
GOCLAW_MIGRATIONS_DIR=/app/migrations \
|
|
GOCLAW_HOST=0.0.0.0 \
|
|
GOCLAW_PORT=18790
|
|
|
|
USER goclaw
|
|
|
|
EXPOSE 18790
|
|
|
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
|
CMD wget -qO- http://localhost:18790/health || exit 1
|
|
|
|
ENTRYPOINT ["/app/docker-entrypoint.sh"]
|
|
CMD ["serve"]
|