Commit Graph

12 Commits

Author SHA1 Message Date
Goon 391a263709 fix(ci): make discord notify resilient to docker failures + DRY env vars
- notify-discord uses always() so it fires even if docker jobs fail
- Move GHCR_IMAGE/DOCKERHUB_IMAGE to workflow-level env block
2026-03-19 12:29:22 +07:00
Goon a3a4d18e9a fix(ci): inline Docker build into release workflow
go-semantic-release creates tags via GITHUB_TOKEN, which doesn't
trigger other workflows (GitHub anti-loop policy). This caused
docker-publish.yaml to stop firing since v1.2.0.

Move Docker image builds (7 variants + web) directly into release.yaml
as parallel jobs alongside build-binaries.
2026-03-19 12:23:54 +07:00
Thieu Nguyen 7f19db14f5 fix(install): bundle migrations in release tarball and install script (#249)
The install script only copied the binary, so `goclaw migrate up` and
`goclaw onboard` failed when run from the installed location because the
migrations directory was missing.

- Include migrations/ in the release tarball
- Install migrations to /usr/local/share/goclaw/migrations
- Guide users to export GOCLAW_MIGRATIONS_DIR before onboard
- Align next-steps output with onboard's own instructions
2026-03-18 07:32:53 +07:00
viettranx 843b550651 feat: runtime packages UI, pkg-helper, configurable shell deny groups (#244)
Runtime package management with security hardening:

- pkg-helper: root-privileged daemon for apk install/uninstall via Unix socket
- HTTP API: /v1/packages (list/install/uninstall/runtimes), admin role required for writes
- Shell deny groups: 15 configurable groups (per-agent overrides via context)
- Packages UI: Web page for managing system/pip/npm packages with confirmation dialogs
- Docker: privilege separation (root entrypoint → su-exec drop), init for zombie reaping
- Security: umask socket creation, persist file validation, deny pattern hardening
  (Node.js fetch/http, Python from/import, curl localhost, sensitive env vars)
- Auth: empty gateway token → admin role (dev/single-user mode)
2026-03-17 19:50:26 +07:00
Goon 9429a7c844 ci: publish Docker images to GHCR and Docker Hub (#237)
* feat(ci): add node/python/full runtime variants to Docker publish

Add runtime image variants alongside existing build-tag variants:
- :node (pre-installed Node.js)
- :python (pre-installed Python)
- :full (Node.js + Python + all skill deps)

* feat(ci): add install scripts, release binaries, and Discord notifications

- scripts/install.sh: one-liner binary installer from GitHub Releases
- scripts/setup-docker.sh: interactive Docker setup with variant selection
  (alpine/node/python/full), auto-generates .env + docker-compose.yaml
  with persistent volumes for data, skills, workspace, storage
- release.yaml: build cross-platform binaries (linux/darwin × amd64/arm64)
  and attach to GitHub Release, notify Discord on new releases
- .gitignore: ignore Apple Double (._*) files from external volumes

* docs(docker): add pre-built image references and update docker-compose workflows

- Add `image:` directives to docker-compose.yml, docker-compose.selfservice.yml, and docker-compose.upgrade.yml pointing to ghcr.io/nextlevelbuilder/goclaw pre-built images
- Add Docker Hub mirror references (digitop/goclaw) for public access
- Document available image tags (latest, node, python, full, otel, tsnet, redis) with descriptions
- Update README with pre-built image pull instructions and semver tag examples
- Clarify deployment workflows: use pre-built images by default (no --build), add --build only when building from source
- Update upgrade workflow to pull pre-built images instead of rebuild
- Add note about build args requirement for otel/tsnet/redis overlays
- Update .dockerignore to exclude macOS temp files (._*)

* fix(ci): use claude_code_oauth_token instead of anthropic_api_key
2026-03-17 12:44:18 +07:00
Goon 75c570e951 feat(security): credentialed exec + HTTP RBAC + API key cache (#197)
- Secure CLI credential injection via AES-256-GCM encrypted env vars
- API key management with fine-grained RBAC scopes
- resolveAuth/requireAuth middleware across all 25+ HTTP handlers
- In-memory API key cache with TTL, negative caching, pubsub invalidation
- Sandbox-first execution (fails if unavailable, no silent fallback)
- Credential scrubbing, constant-time token comparison, Admin-only CLI creds
- SQL migration 000020: secure_cli_binaries + api_keys tables
- 14 unit tests for cache and RBAC with race detector

Closes #197
2026-03-15 20:13:18 +07:00
Goon 51677b8ff7 "Claude Code Review workflow" 2026-03-14 14:53:30 +07:00
Goon de03bc79b2 "Claude PR Assistant workflow" 2026-03-14 14:53:25 +07:00
Viet Tran ace07509b7 feat(skills): system skills integration — toggle, dep checking, per-item install (#161)
* feat(infra): add runtime package support for skills

Install nodejs, npm, pandoc, github-cli + pre-install Python packages
(openpyxl, pandas, python-pptx, markitdown) and Node packages
(docx, pptxgenjs). Configure runtime dirs for agent pip/npm installs
with PIP_TARGET, NPM_CONFIG_PREFIX, NODE_PATH to enable dynamic
package installation in read-only container environment.

* feat(infra): add bundled skills with runtime package support

- Add 5 bundled skills: docx, pdf, pptx, xlsx, skill-creator from container skills-store
- Wire GOCLAW_BUILTIN_SKILLS_DIR env var in gateway and CLI
- Support optional runtime packages alongside dynamic skill loading
- Update Dockerfile to COPY bundled-skills at /app/bundled-skills/
- Add PIP_CACHE_DIR in docker-entrypoint.sh for clean pip installs
- Document bundled skills in 14-skills-runtime.md section 6

* feat(infra): remove ai-multimodal skill directory from bundled skills

Remove the ai-multimodal skill package as part of consolidating runtime
package support for bundled skills. This directory is no longer needed
in the bundled skills structure.

* feat(ci): add semantic release and Docker Hub publishing

Add go-semantic-release workflow to auto-create semver tags on merge to
main. Extend docker-publish to push all variants to both GHCR and
Docker Hub (digitop/goclaw).

* feat(skills): add system skills infrastructure with is_system column, dep scanning, and seeder

- Migration 000017: add is_system boolean column with partial index
- Store layer: UpsertSystemSkill, delete protection, IsSystemSkill
- ListAccessible auto-includes system skills (no grants needed)
- ListWithGrantStatus returns is_system field
- Dependency scanner: auto-detect deps from scripts/ or skill-manifest.json
- Dependency checker: verify system binaries, Python/Node packages
- Seeder: seed bundled skills into DB on startup (idempotent via hash)
- Gateway wiring: GOCLAW_BUNDLED_SKILLS_DIR env for bundled skills
- HTTP: delete guard (403), slug conflict check (409), rescan-deps endpoint
- UI: System badge, hide delete for system skills, rescan deps button
- Agent skills tab: "Always available" for system skills
- i18n: en/vi/zh keys for system skills, deps scanning

* feat(skills): conditional system prompt, skill manifests, and Zip Slip fix

- System prompt: only show package list when python3/node are available
- Add skill-manifest.json for pdf, docx, xlsx, pptx bundled skills
- Fix Zip Slip vulnerability in office/unpack.py (all 3 copies)

* refactor(skills): extract shared office code to _shared/ and deduplicate

Move office scripts (pack, unpack, validate, schemas, validators) from
duplicated copies in docx/xlsx/pptx to skills/_shared/office/ with
symlinks. Remove soffice.py (non-functional in containers) and update
SKILL.md references to use soffice binary directly. Update seeder
copyDir to follow symlinks.

Removes ~45K lines of duplicate code across 3 skills.

* fix(skills): address code review findings for system skills integration

- H1: Remove dead symlink branch in copyDir (filepath.Walk follows symlinks)
- H3: Fix rescan-deps to query ALL skills (including archived) and re-activate
  when deps become available; add ListAllSkills() + Status field to SkillInfo
- H4: Add Status field to SkillCreateParams, stop overloading Visibility
- M1: Batch Python/Node dep checks into single subprocess per runtime
- M4: Add rows.Err() check in ListSkills to prevent caching partial results

* feat(skills): async dep checking with realtime WS events

Split Seed() into sync DB upsert + async CheckDepsAsync() goroutine.
Gateway startup no longer blocks on Python/Node subprocess dep checks.

- Seed() returns seeded skills list, all initially status="active"
- CheckDepsAsync() runs in background, emits skill.deps.checked per-skill
- skill.deps.complete event emitted when all checks finish
- Each failed dep check: archives skill + BumpVersion() for immediate
  cache invalidation so next agent turn picks up the change
- UI: use-query-invalidation listens to skill.deps.* events → auto-refresh
  skills list in realtime

* feat(skills): system skills integration with toggle, dep checking, and per-item install

- Add is_system, deps, enabled columns to skills table (migration 017)
- Seed bundled core skills (pdf, docx, pptx, xlsx, skill-creator) on startup
- PYTHONPATH-based dep detection — eliminates false positives from local modules
- Per-item dep install UI with individual status (installing/success/error)
- Enable/disable toggle for core and custom skills (independent of dep status)
- Re-run dep check when skill is toggled back on
- Inline skill thresholds: 40 skills / 5000 tokens before switching to search mode
- Fix UpsertSystemSkill: backfill null file_hash without bumping DB version
- Remove redundant skill-manifest.json files (replaced by deps JSONB column)
- Show author from frontmatter in custom skills tab
- Runtime checker for python3/pip3/node/npm availability
- WS events for dep checking/installing progress
- docs: add 15-core-skills-system.md, 16-skill-publishing.md

---------

Co-authored-by: Goon <duy@wearetopgroup.com>
2026-03-12 09:20:41 +07:00
viettranx 0d3230b2bf feat(cache): add build-tag-gated Redis cache backend
Add optional Redis cache support via `go build -tags redis`, following
the same paired-stub pattern as OTel and Tailscale. The Cache[V] interface
is unchanged; Redis and in-memory implementations are injected at startup
without altering usage logic.

- Add RedisCache[V] implementation with JSON serialization, fail-open on errors
- Add gateway_redis.go / gateway_redis_noop.go paired wiring files
- Refactor GroupWriterCache and ContextFileInterceptor to accept injected caches
- Add GOCLAW_REDIS_DSN env var, docker-compose.redis.yml overlay
- Update Dockerfile and GitHub Actions with ENABLE_REDIS build arg
- Add Redis variant to CI matrix (5 variants: latest, otel, tsnet, redis, full)
2026-03-07 19:27:24 +07:00
viettranx 043149bd43 ci: add CI workflow, Makefile targets, and fix typing keepalive race (#41)
- Add GitHub Actions CI with parallel Go (build/test -race/vet) and Web UI (pnpm build) jobs
- Add Makefile targets: test, vet, check-web, setup, ci
- Fix data race in typing keepalive: remove nil assignment after close(keepaliveDone)
  so the goroutine can safely read the channel without holding the mutex

Co-Authored-By: Duc Nguyen <me@vanducng.dev>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 10:22:30 +07:00
Bruno Clermont 51f8b895a3 feat: Publish pre-built Docker images to GHCR (#23)
* feat: add GitHub Actions workflow to publish Docker images to GHCR

Build and push all 4 variants (latest, otel, tsnet, full) on version
tags with multi-platform support (linux/amd64, linux/arm64).

Closes #19

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add web UI image build to GHCR publish workflow

Adds a separate job to build and push the ui/web Dockerfile as
ghcr.io/<repo>-web with the same multi-platform and tagging strategy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Bruno Clermont <bruno.clermont@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 15:06:53 +07:00