Files
goclaw/cmd/gateway_setup.go
T
Viet TranandGitHub e183b459c9 feat: SQLite desktop edition — full desktop app with team tasks (#505)
* feat(store): add SQLite backend foundation with build-tag injection

Add sqlitestore package (//go:build sqlite) as alternative to PostgreSQL:
- pool.go: WAL mode, busy_timeout, 4 concurrent read connections
- helpers.go: ? param helpers, JSON array storage, nullable/update utils
- scope.go: tenant scope with ? placeholders (vs PG's $N)
- schema.sql: 1296-line flattened DDL from 29 PG migrations (51 tables)
- schema.go: embedded schema with transactional apply + version tracking
- factory.go: NewSQLiteStores() stub (stores wired in Phase 2)

Build-tag split for store initialization:
- cmd/gateway_stores_pg.go (//go:build !sqlite) — default PG-only
- cmd/gateway_stores_sqlite.go (//go:build sqlite) — runtime PG/SQLite switch
- cmd/gateway_setup.go: extracted wireTracingAndCron() shared helper

Config: GOCLAW_STORAGE_BACKEND + GOCLAW_SQLITE_PATH env vars.
Security: goclaw.db added to DenyPaths (exec, read_file, filesystem tools).

* feat(edition): add centralized edition package for feature tier limits

New internal/edition/ package — single source of truth for all edition limits:
- Edition struct with MaxAgents, MaxTeams, MaxChannels, KGEnabled, TeamFullMode, etc.
- Standard (default, all features) and Lite (desktop, 5 agents, 1 team) presets
- atomic.Pointer for thread-safe Current()/SetCurrent()

Wiring:
- cmd/gateway.go: GOCLAW_EDITION env override (lite/standard) at startup
- cmd/gateway_stores_sqlite.go: auto-set Lite when backend=sqlite
- /v1/edition HTTP endpoint for UI comparison modal (public, no auth)

* feat(sqlitestore): implement Phase 2A core stores + sqliteonly build tag

Implement 12 SQLite store backends (4200+ LOC) mirroring existing PG stores:
- SessionStore, AgentStore, ProviderStore, TracingStore, SnapshotStore
- ConfigSecretsStore, SystemConfigStore, TenantStore, HeartbeatStore
- BuiltinToolStore, BuiltinToolTenantConfigStore, SkillTenantConfigStore

All stores wired in factory.go. Remaining stores (Memory, Cron, Skills,
Teams, etc.) left nil — gateway handles gracefully.

Add sqliteonly build tag for PG-free desktop builds:
- go build .                  → PG only
- go build -tags sqlite .     → PG + SQLite (runtime switch)
- go build -tags sqliteonly .  → SQLite only (no pgx dependency)

Key SQLite adaptations:
- ? placeholders (not $N), json_extract/json_each/json_array_length
- DISTINCT ON → GROUP BY + Go dedup, ANY($1) → IN (?,?...)
- rows.Err() checks on all scan loops, execMapUpdateWhereTenant in helpers
- GetOrCreateUserProfile uses RowsAffected() instead of PG's xmax trick

* feat(sqlitestore): implement Phase 2B feature stores

Port 8 additional store backends to SQLite (19 new files, ~6000 LOC):

Cron: cron.go, cron_crud.go, cron_exec.go, cron_scheduler.go
  - Job scheduling with cache, ListDue, MarkRunning, MarkComplete

Skills: skills.go, skills_crud.go, skills_content.go, skills_grants.go
  - CRUD, grants, content management. LIKE search (no FTS/vector in Lite)

MCP: mcp_servers.go, mcp_servers_access.go, mcp_user_credentials.go
  - Server CRUD, agent/user grants, encrypted credentials

Channels: channel_instances.go, pairing.go, pending_messages.go, contacts.go
  - Channel management, device pairing, message queue, contact store

Teams: teams.go, teams_tasks.go, teams_tasks_lifecycle.go, teams_tasks_activity.go
  - Team/task CRUD, lifecycle transitions, activity log, progress tracking
  - JSON array for members/blocked_by (replaces PG text[])

Remaining nil stores: Memory, AgentLinks, KnowledgeGraph, Activity,
SecureCLI, APIKeys, ConfigPermissions — gateway handles gracefully.

* fix(sqlitestore): fix critical arg ordering + data races in Phase 2B stores

C1: ListTasks arg mismatch — limit+1 in userID slot, wrong results
C2: SearchTasks missing duplicate userID arg for (? = '' OR t.user_id = ?)
H3: cron_crud.go cacheLoaded written without mutex → use InvalidateCache()
H4: cron_exec.go discarded ExecContext error → log warning
H1: Add rows.Err() checks in cron_crud, cron_scheduler, cron_exec, pairing

* feat(sqlitestore): implement Phase 2C — Memory, Activity, APIKeys, ConfigPermissions

Complete remaining essential stores (6 new files, ~900 LOC):

Memory: memory.go, memory_docs.go, memory_search.go
  - Document/chunk CRUD, LIKE-based search (no vector in Lite edition)
  - Embedding methods return empty results gracefully

Activity: activity.go — simple activity logging
APIKeys: api_keys.go — API key CRUD with SHA-256 hash lookup
ConfigPermissions: config_permissions.go — permission rules with TTL cache

24/27 stores now wired. Only AgentLinks, KnowledgeGraph, SecureCLI
remain nil (disabled in Lite edition by design).

Total: 49 files, 11,435 LOC in internal/store/sqlitestore/

* fix(sqlitestore): fix variable shadow in GetDocument + handle chunk delete error

C1: GetDocument scopeClause used := inside if/else blocks, shadowing
outer err variable — query errors silently swallowed. Fixed by renaming
to tcErr matching PG pattern.

H1: IndexDocument chunk deletion ExecContext error was discarded, could
cause duplicate chunks. Now returns error on failure.

* feat(desktop): Phase 3 — Wails v2 desktop app shell with chat UI

Desktop app (ui/desktop/) using Wails v2 + React 19 + Tailwind CSS 4:

Go backend:
- main.go: Wails entry point with embedded frontend assets
- app.go: gateway embedding via goroutine, health check, Wails bindings
- keyring.go: OS keyring secrets with file fallback
- cmd/gateway_export.go: exports RunGateway() for desktop embedding

React frontend:
- WS v3 client: protocol handshake, exponential backoff, call queue
- Chat system: Zustand stores, RAF-batched streaming, 10 event handlers
- Components: MessageBubble, MarkdownRenderer (rehype-sanitize),
  CodeBlock, ToolCallBlock, ThinkingBlock, ActivityIndicator, InputBar
- Layout: AppShell (2-column), Sidebar with agent/session list, TopBar
- Onboarding wizard (5 steps): welcome, gateway, provider, agent, ready
- Magic Blue theme (dark/light), Inter + JetBrains Mono typography

Build: all 3 variants pass (PG, sqlite, sqliteonly)

* chore: ignore Wails build artifacts (wailsjs, build, package.json.md5)

* fix(desktop): correct onboarding provider list and agent creation API contract

- ProviderStep: expand from 3 to 16 providers in 4 groups (Popular, Cloud, Local, Regional)
  with correct provider_type values and api_base defaults
- AgentStep: fix API payload — use agent_key (slug), provider (name string),
  agent_type=predefined with description in other_config
- use-agents: fix field mapping — agent_key, display_name from backend response
- Clean up failed provider on verify error

* fix(desktop): add missing providers — Bailian Coding, Z.ai Coding, Ollama Cloud

* fix(desktop): match web dashboard brand colors + persist onboarding in store

- Replace Magic Blue theme with web dashboard's warm blue OKLCH palette
- Move onboarding state from localStorage to Zustand persist store
- Add "Run Setup Wizard" option in TopBar settings menu to re-trigger onboarding
- Fix light mode theme activation (explicit :root:not(.dark) overrides)

* fix(desktop): apply dark theme before first paint + better error messages

- Add class="dark" default on <html> + inline script to read persisted theme
  before React hydrate (prevents light flash on dark mode)
- Improve provider test error message for network failures

* feat(desktop): add GoClaw logo from web UI to onboarding + topbar

* fix(desktop): add Vite proxy for dev mode to avoid CORS gateway errors

- Proxy /v1, /ws, /health to localhost:18790 in Vite dev server
- Use relative URLs in dev mode (import.meta.env.DEV) so proxy handles CORS
- Production build uses direct gateway URL from Wails binding

* fix(desktop): GoClaw dock icon + fix duplicate provider slug on re-test

- Convert goclaw-icon.svg to 1024x1024 PNG for macOS dock icon
- Fix provider creation: handle existing slug by finding and updating
- Track build/appicon.png in git (exclude only build/bin/)

* fix(sqlitestore): UUID text/blob mismatch in scopeClause breaks all queries

scopeClause passed uuid.UUID (16-byte array) to SQLite ? placeholder,
but tenant_id column stores TEXT strings. SQLite compared BLOB vs TEXT
→ no match → all scoped queries returned empty results.

Fix: pass scope.TenantID.String() to ensure TEXT comparison.

Also includes:
- macOS dock icon (.icns from GoClaw logo)
- Onboarding auto-detect existing providers/agents
- Debug logging for token + API URL

* fix(desktop): CORS for dev mode + direct gateway URL

- Wails dev server (port 34115) doesn't proxy API calls, causing 405
- Frontend now connects directly to gateway URL from Wails binding
- Added GOCLAW_DESKTOP=1 env → enables CORS middleware on gateway
- desktopCORS wraps mux with Access-Control-Allow-* headers + OPTIONS
- Simplified provider test flow: list-then-create/update

* fix(desktop): split onboarding into 6 steps matching web UI flow

Web UI flow: create provider → select model + verify → create agent.
Desktop was incorrectly trying to verify before creating provider.

Changes:
- ProviderStep: now only creates/saves provider (no verify)
- NEW ModelVerifyStep: loads models from provider, test connection
- AgentStep: receives pre-selected model, shows read-only model field
- OnboardingWizard: 6 steps with auto-detect skip logic
- Auto-detect: has agents→Ready, has providers→ModelVerify, nothing→Provider

* feat(desktop): add Combobox component, use searchable model selector

- New Combobox: searchable dropdown with custom value support
- ModelVerifyStep: replace plain input/select with Combobox
- Models loaded from GET /v1/providers/{id}/models API
- Allows typing custom model name if API returns empty list

* fix(desktop): rename icon to iconfile.icns matching Wails convention

* refactor(desktop): overhaul UI/UX, fix onboarding, fix event handling

Desktop app major refactor:

UI/UX:
- Chat-focused layout with floating panels on dotted canvas
- Sidebar: agent list + sessions grouped by date (no resource counts)
- Modern input bar: rounded pill with attach/send buttons
- User bubble: card style matching web UI (not solid color)
- Thinking block: collapsible, max-height, proper label
- ErrorBoundary wrapping app
- Dock icon: regenerated with transparent bg + dark rounded frame

Onboarding:
- 3-step flow matching web UI (Provider → Model → Agent)
- Proper create vs update (check DB before POST)
- SetupStepper with step circles + connectors
- Agent presets from web UI (Fox Spirit, Artisan, Astrologer)
- Auto-detect existing setup via use-bootstrap-status hook

Event handling (verified from Go source):
- chunk: payload.content (not payload.chunk)
- thinking: payload.content (not payload.thinking)
- tool.call: payload.id/name (not toolId/toolName)
- tool.result: payload.is_error/content (not error field)
- run.completed: usage.prompt_tokens/completion_tokens
- New: block.reply, run.retrying handlers

Backend fixes:
- SQLite scanTime helper for modernc.org/sqlite text timestamps
- X-GoClaw-User-Id header in desktop API client (not X-User-ID)
- user_id: system (owner role in desktop single-user mode)
- CORS: allow X-GoClaw-User-Id header
- SQLite busy timeout: 5s → 10s
- Snapshot SQL: cross-DB compatible (FILTER→CASE, ::BIGINT→CAST)
- Promise.allSettled for bootstrap status (one fail doesn't block other)

* feat(desktop): chat polish, SQLite fixes, summoning modal, bootstrap guard

Chat Polish (Phase 1):
- ToolCallBlock: Wrench/Zap icons, phase badges, arg summary, grouped rendering
- ThinkingBlock: auto-expand on stream, Brain icon, cursor pulse
- MessageBubble: isStreaming prop, streaming cursor, grouped tool calls
- ImageLightbox: fullscreen overlay, gallery nav, keyboard shortcuts, download
- MediaBlock: grid layout, click-to-open lightbox, hover overlay
- ActivityIndicator: phase-specific icons (Brain/Wrench/RefreshCw)
- ChatCanvas: track lastAssistantId for streaming, EmptyState with prompts
- Filter [System] nudge messages and tool-role messages from chat history

SQLite Fixes:
- sessions_list: fix time.Time scan failure (3 sites) using sqliteTime scanner
- snapshots: fix ON CONFLICT expression mismatch with unique index
- snapshots: fix GetLatestBucket using nullSqliteTime
- pool: explicit PRAGMAs (busy_timeout=15s, WAL, synchronous=NORMAL)
- schema: seed master tenant (was missing, causing FK violations)
- schema: incremental migration framework (version-gated patches)

Onboarding:
- SummoningModal: port from web UI with framer-motion animations
- AgentStep: show summoning modal after create, continue button
- App: auto-detect empty DB and reset onboarded flag
- ChatCanvas: loading spinner while agent loads after onboarding

Bootstrap Guard:
- After auto-cleanup of BOOTSTRAP.md, check if USER.md is still empty
- Inject reminder if agent cleared BOOTSTRAP but didn't fill USER.md

Session Management:
- Load chat history on session click (was missing useEffect)
- Fix race condition: don't clear messages on session switch (atomic replace)
- SidebarFooter: center New Chat button text

* fix(desktop): session management, delete confirm, event listener race fix

- New Chat: only clears state, no empty session creation (sendMessage auto-creates)
- Delete session: hover X button with ConfirmDialog confirmation
- Event listener: use sessionKeyRef instead of closure to prevent stale events
- Remove "skip to dashboard" link from onboarding
- Add reusable ConfirmDialog + ConfirmDeleteDialog common components

* feat(desktop): settings view with tabbed layout (Phase 4)

- SettingsView: tab container with header, close button, canvas-dots bg
- SettingsTabBar: 9 tabs (Appearance, Providers, Agents, MCP, Skills, Tools, Cron, Traces, About)
- AppearanceTab: dark/light theme toggle, language + timezone placeholders
- AboutTab: version, edition limits, runtime info
- ui-store: activeView, settingsTab, openSettings(), closeSettings()
- AppShell: switch between chat and settings views
- SidebarFooter: gear icon opens settings (was "Run Setup Wizard")
- Keyboard: Cmd+, opens settings, Escape closes
- Branding: "GoClaw Lite" in sidebar header
- Agent status: online (green) instead of idle for desktop
- Tab content wrapped in solid bg card with border

* feat(desktop): provider management CRUD in settings (Phase 5)

- use-providers hook: list, create, update, delete, verify via HTTP API
- ProviderList: list view with Add button, empty state
- ProviderRow: status dot, type badge, edit/delete actions
- ProviderFormDialog: create/edit with type selector, masked API key, test connection
- Wire ProviderList into Settings Providers tab

* fix(desktop): remove Test Connection from provider form (requires model + provider ID)

* feat(desktop): agent management CRUD in settings (Phase 6)

- AgentData/AgentInput types matching web UI contracts
- use-agent-crud hook: list, create, update, delete, resummon (5 agent limit)
- AgentCard: emoji, status dot, provider/model/type badges, edit/delete/resummon
- AgentFormDialog: provider Combobox + model Combobox (from /v1/providers/{id}/models)
  - Create: agent_type selector, "Check & Create" verifies model before create
  - Edit: no type change, no re-verify needed
  - Personality textarea for predefined agents
- AgentList: grid, edition limit warning, create triggers SummoningModal
- Delete uses ConfirmDeleteDialog (type name to confirm)
- Sidebar agent list refreshes after CRUD

* feat(desktop): agent detail panel with full config (Phase 6 polish)

Agent detail panel (fullscreen overlay covering sidebar):
- PersonalitySection: emoji editor, display name, description, status select, default toggle, agent key display
- ModelBudgetSection: provider/model Combobox with verify-before-save, context window, max tool iterations
- EvolutionSection: self_evolve toggle with info callout (predefined agents only)
- Sticky save bar: backdrop blur, cancel/save buttons, spinner on save
- Save blocked if provider/model changed but not verified

AgentList: card click opens detail panel, create dialog separate
AgentFormDialog: create-only, Check & Create with verify

* fix(desktop): resummon confirm, summoning z-index, save bar UX

- Resummon requires confirm dialog before triggering
- SummoningModal z-index z-50 → z-[70] (above detail panel z-[60])
- Save bar: show error inline, "Verify model first" when blocked

* feat(desktop): agent detail quality polish — files tab, memory config, rich cards

Types (synced with web UI):
- AgentData: added owner_id, workspace, restrict_to_workspace, frontmatter, context_window, max_tool_iterations as required fields
- MemoryConfig, CompactionConfig interfaces
- BootstrapFile type for WS file operations

AgentCard (matching web UI agent-card.tsx):
- Star icon for default agent
- Animated pulse badge for summoning status
- Self-evolve sparkle indicator (orange when active)
- Frontmatter/expertise with line-clamp-3
- Context window display (e.g. "200K ctx")
- Safe emoji extraction, UUID name detection

AgentDetailPanel:
- Tab navigation: Overview + Files tabs
- Overview: Personality + ModelBudget + Memory + Evolution sections
- Files tab: WS-based file editor (agents.files.list/get/set)
  - File sidebar with selection
  - Textarea editor with dirty tracking
  - Save button with spinner
- Sticky save bar with backdrop blur (overview only)
- Resummon with confirm dialog

MemorySection (matching web UI memory-section.tsx):
- Enable/disable toggle
- 6 config fields: max_results, min_score, max_chunk_len, chunk_overlap, vector_weight, text_weight
- "Using global defaults" when disabled

* feat(desktop): agent files — hide USER/HEARTBEAT, add Edit with AI

- Hide USER.md, USER_PREDEFINED.md, HEARTBEAT.md from files tab (managed by bootstrap/cron)
- Add "Edit with AI" button → RegenerateDialog → POST /v1/agents/{id}/regenerate
- Auto-select first file on load
- Show file size in sidebar
- Pass agentId to files tab for regenerate API call

* fix(desktop): remove memory config section (no embedding in SQLite)

* fix(desktop): remove misleading bytes display from file sidebar

* fix(desktop): replace native checkboxes/selects with custom Switch + Combobox

- New Switch component matching Radix UI toggle style
- Replace all native <input type="checkbox"> with Switch in agent/provider forms
- Replace native <select> for status with Combobox
- All interactive elements have cursor-pointer

* fix(desktop): evolution callout colors — use opacity-based for both themes

* fix(desktop): global cursor-pointer for all interactive elements

* fix(desktop): improve dark mode contrast for text and status colors

- text-secondary: 0.62 → 0.68 lightness (better readability on dark bg)
- text-muted: 0.52 → 0.58 lightness (was below WCAG AA 4.5:1 minimum)
- success: 0.45 → 0.55 (green was too dim on dark bg)
- warning: 0.65 → 0.70 (slightly brighter)
- idle: 0.52 → 0.58 (match text-muted)

* fix(desktop): revert color values to exact web UI match (0.62/0.52/0.45)

* fix(desktop): agent card badge colors — match web UI badge variants exactly

* feat(desktop): add MCP servers + Builtin Tools settings tabs

Phase 7 implementation:

MCP Tab:
- Full CRUD with 5-server edition limit
- Form dialog with transport-conditional fields (stdio/SSE/streamable-http)
- Test Connection with inline success/error feedback
- Agent grants dialog (grant/revoke per agent)
- Tools discovery dialog (view server tools)
- KeyValueEditor with sensitive field masking (auth/token/secret)

Tools Tab:
- Category-grouped list of 41 seeded builtin tools
- Toggle enable/disable with optimistic update
- Specialized settings forms: web_fetch extractor chain, media provider chain
- Generic JSON editor fallback for other tools
- Provider/model Combobox selection for media tools

Common:
- RefreshButton component with 500ms min spin animation
- KeyValueEditor with password masking for sensitive keys

Fixes:
- SQLite builtin_tools scan: use scanTimePair() for timestamps
- Session click while in settings: now closes settings view
- Dark mode text contrast: bumped text-secondary/text-muted lightness
- Light mode text contrast: darkened text-secondary/text-muted
- Focus ring thickness: ring-2 → ring-1 globally
- Misleading "memory layering (Postgres)" log label

* feat(desktop): add Skills tab, agent skill grants, emoji avatar, agent form redesign

Skills:
- Skills settings tab with upload ZIP, toggle, delete, runtime check
- Agent skill grants section in agent detail panel (toggle per agent)
- SQLite SkillManageStore interface compliance fixes

Agent form:
- 2-column layout, wider modal (max-w-3xl)
- 6 personality presets (Fox Spirit, Artisan, Astrologer, Researcher, Writer, Coder)
- Separate Verify Model + Summon buttons
- Always predefined type (removed open option)

Fixes:
- SQLite time.Time scan: mcp_servers, mcp_grants, activity_logs
- Combobox portal with scroll/resize tracking
- AgentAvatar shows emoji from other_config
- uploadFile sends X-GoClaw-User-Id header + generic type
- isApiClientReady guard prevents ErrorBoundary crash
- Verify model field: valid (not success)

* fix(sqlitestore): comprehensive timestamp scan sweep + UI fixes

SQLite timestamp sweep (13 files, 25+ sites):
All time.Time direct scans replaced with sqliteTime/scanTimePair/nullSqliteTime.
Files: activity, teams, teams_tasks, teams_tasks_activity, config_permissions,
agents_access, tracing_spans, tracing_scan, channel_instances, tenants,
pending_messages, api_keys, heartbeat.

UI fixes:
- Agent switching: clear active session + chat when agent changes
- MCP table: vertical align middle on row cells

* fix(sqlitestore): fix json.RawMessage scan + sqliteVal for dynamic updates

- mcp_servers_access: scan json columns via string intermediates (SQLite
  TEXT → json.RawMessage incompatible, use string then convert)
- helpers: add sqliteVal() to auto-marshal map/slice/struct to JSON
  string in execMapUpdate/execMapUpdateWhereTenant — fixes agent save
  500 error when updating other_config, tools_config, etc.
- Add AgentMcpSection: toggle MCP server grants per agent in detail panel
- Clean up debug logging from McpGrantsDialog

* feat(desktop): add i18n (react-i18next) + toast system + language/timezone pickers

i18n:
- Install react-i18next + i18next
- Copy 12 web locale namespaces (en/vi/zh) + desktop.json namespace
- Create i18n/index.ts with browser language detection + localStorage persist
- Replace ~300 hardcoded strings across 40+ components with t() calls
- Language picker in ChatTopBar (top-right) + Settings > Appearance
- Timezone picker in ChatTopBar with search + Intl.supportedValuesOf fallback
- All 6 agent presets fully translated (vi/zh) with prompts from locale files
- Agent Key never translated (uses stable English slugs)

Toast:
- Zustand toast store (success/error/warning/default, 4s auto-dismiss)
- Toaster component (bottom-right, z-100, slide-in animation)
- Toast calls in all CRUD hooks (agents, providers, MCP, skills, tools)

* feat(desktop): add Cron Jobs + Traces settings tabs with syntax highlighting

Cron Jobs (Phase 8):
- WS RPC hook (cron.list/create/delete/toggle/run/runs)
- CronList table with schedule formatting, status badges, run/toggle/delete
- CronFormDialog with slug name, agent selector, 3-way schedule (every/cron/once)
- CronRunsDialog for execution history

Traces (Phase 9):
- HTTP REST hook (GET /v1/traces with pagination + agent filter)
- TraceList table with duration, tokens, spans, relative time
- TraceDetailDialog with metadata, collapsible input/output, flat span list
- Syntax-highlighted JSON/code previews via react-syntax-highlighter

* fix(sqlitestore): fix tracing scan — endTime *time.Time → nullSqliteTime

Both trace and span scan functions used *time.Time for nullable end_time
column, which fails on SQLite TEXT timestamps. Changed to nullSqliteTime
with Valid check before assigning pointer.

* fix(desktop): fix cron schedule type label — map 'at' kind to 'once' i18n key

* fix(desktop): prevent flash on cron/traces refresh — keep data while refetching

* feat(desktop): add file attachment rendering — FileButton + FilePreviewDialog

- FileButton: compact attachment button with emoji icon, filename, size, download
- FilePreviewDialog: modal with type-detected preview (image/video/audio/markdown/code/text)
- MarkdownRenderer: override a/img for /v1/files/ links → FileButton + resolved URLs
- MediaBlock: non-image files render as FileButton instead of plain links
- api.ts: add getBaseUrl() for file URL resolution

* fix(desktop): resolve file URLs in chat — toFileUrl for media_refs + relative paths

- use-chat: add toFileUrl() to convert raw paths to /v1/files/{basename} URLs
  for both run.completed media and history media_refs
- MarkdownRenderer: detect relative file paths (./path/file.ext) via isFileLink
  in addition to /v1/files/ links, resolve all to gateway URL

* fix(desktop): authenticated file serving — media cache + blob URLs

Security fix: all /v1/files/ requests now use Bearer auth via fetchFile().
No more raw <a href> or <img src> with unauthenticated file URLs.

- Add media-cache.ts (blob cache with 5-min TTL, dedup inflight fetches)
- Add use-media-url.ts hook (returns cached blob URL for authenticated media)
- Add AuthImage component (loads images via auth blob) + downloadFile helper
- Update MarkdownRenderer: file images use AuthImage, downloads use downloadFile
- Update FileButton: authenticated download via blob
- Update FilePreviewDialog: authenticated fetch for text/preview content
- Clean filename display (strip timestamps + query params)

* fix(desktop): fix file attachments — use media_refs.id for URL, auth all media

Root cause: media_refs from backend has {id, mime_type, kind} but no path/url.
The id IS the filename basename. Fixed toFileUrl to use ref.id as fallback.

Also: AuthImage for lightbox, authenticated audio/video in MediaBlock,
clean filename display (strip timestamps).

* fix(desktop): file serving, traces, and media rendering improvements

Backend:
- Store MediaRef.Path in loop_finalize for direct file serving
- Use full path (not basename) in gateway_managed event signing
- Add fuzzyMatchInDir for LLM-hallucinated filenames
- Add findInWorkspace support for agent dirs and ws/ directory
- Add POST /v1/files/sign endpoint for client-side URL signing
- Always save LLM span input_preview (not just verbose mode)
- Truncate previews from tail (keep recent context), limit 2000 chars
- Add exact filename hint to create_image/video/audio tool results

Desktop frontend:
- File attachments: FileButton, FilePreviewDialog, AuthImage with blob cache
- Media cache: sign URLs via API for non-ft URLs, Bearer auth fallback
- Download via Wails SaveFile binding (native Save As dialog)
- OpenFile + DownloadURL Wails Go bindings
- Traces: rewrite with span tree hierarchy, formatTokens (90.8K),
  formatDuration with start/end fallback, expandable spans
- Traces: export via DownloadURL, copy with checkmark state
- Code preview: JSON = oneDark syntax highlight, text = light pre block
- MarkdownRenderer: baseUrl prop for relative image resolution
- ImageLightbox: preventDefault on keyboard nav (no macOS beep)
- ErrorBoundary: reload instead of re-render on retry
- Combobox: compact sizing (py-1.5, text-sm)
- FileButton: fix nested button HTML violation

* feat(desktop): team tasks kanban board + edition policy + file preview fixes

Phase 11 implementation:

Backend:
- TeamActionPolicy interface — lite/full edition gating for team_tasks tool
- Filter blocked actions from schema enum + early guard in Execute()
- System prompt: edition-specific team member guidance
- Skip skill_manage/publish_skill registration + seeding in lite
- teams.create: require at least 1 member
- files.go: 2-layer path isolation (workspace boundary + tenant scope)

Desktop UI:
- Kanban board with 6 status columns + framer-motion layout animation
- Task detail modal with collapsible description/result sections
- Team create dialog with styled member checkboxes
- Sidebar: teams section with create button + Lite edition badge
- Chat TaskPanel: compact active tasks with real-time WS updates
- Real-time: debounced get-light fetch (300ms) + progress patch (1s)
- Edition comparison modal (Lite vs Standard feature table)
- Custom dropdown filter replacing native select
- i18n: teams namespace (en/vi/zh)

Fixes:
- WS params: camelCase (teamId, taskId, sessionKey) matching backend
- FileButton: span wrapper (HTML nesting) + createPortal for dialog
- FilePreviewDialog: defensive filename normalization for extension check
- use-chat: prevSessionRef init null — fix blank chat on view switch
- Scrollbar: 3px auto-hide
- Agent click in sidebar returns to chat view

* feat: Update frontend build assets, change the default agent thinking level to 'low', and update Go module dependencies.
2026-03-27 09:28:42 +07:00

576 lines
22 KiB
Go

package cmd
import (
"context"
"fmt"
"log/slog"
"os"
"path/filepath"
"time"
"github.com/google/uuid"
"github.com/nextlevelbuilder/goclaw/internal/bootstrap"
"github.com/nextlevelbuilder/goclaw/internal/bus"
"github.com/nextlevelbuilder/goclaw/internal/config"
mcpbridge "github.com/nextlevelbuilder/goclaw/internal/mcp"
"github.com/nextlevelbuilder/goclaw/internal/permissions"
"github.com/nextlevelbuilder/goclaw/internal/providers"
"github.com/nextlevelbuilder/goclaw/internal/sandbox"
"github.com/nextlevelbuilder/goclaw/internal/edition"
"github.com/nextlevelbuilder/goclaw/internal/skills"
"github.com/nextlevelbuilder/goclaw/internal/store"
"github.com/nextlevelbuilder/goclaw/internal/store/pg"
"github.com/nextlevelbuilder/goclaw/internal/tools"
"github.com/nextlevelbuilder/goclaw/internal/tracing"
"github.com/nextlevelbuilder/goclaw/internal/tts"
"github.com/nextlevelbuilder/goclaw/pkg/browser"
"github.com/nextlevelbuilder/goclaw/pkg/protocol"
)
// setupToolRegistry creates the tool registry and registers all tools.
// Returns the registry, exec approval manager, MCP manager, sandbox manager,
// browser manager (caller must defer Close), web fetch tool, TTS tool,
// permission policy engine, tool policy engine, data directory, and resolved agent defaults.
func setupToolRegistry(
cfg *config.Config,
workspace string,
providerRegistry *providers.Registry,
) (
toolsReg *tools.Registry,
execApprovalMgr *tools.ExecApprovalManager,
mcpMgr *mcpbridge.Manager,
sandboxMgr sandbox.Manager,
browserMgr *browser.Manager,
webFetchTool *tools.WebFetchTool,
ttsTool *tools.TtsTool,
permPE *permissions.PolicyEngine,
toolPE *tools.PolicyEngine,
dataDir string,
agentCfg config.AgentDefaults,
) {
// Create tool registry with all tools
toolsReg = tools.NewRegistry()
agentCfg = cfg.ResolveAgent("default")
// Sandbox manager (optional — routes tools through Docker containers)
if sbCfg := cfg.Agents.Defaults.Sandbox; sbCfg != nil && sbCfg.Mode != "" && sbCfg.Mode != "off" {
if err := sandbox.CheckDockerAvailable(context.Background()); err != nil {
slog.Warn("sandbox disabled: Docker not available",
"configured_mode", sbCfg.Mode,
"error", err,
)
} else {
resolved := sbCfg.ToSandboxConfig()
sandboxMgr = sandbox.NewDockerManager(resolved)
slog.Info("sandbox enabled", "mode", string(resolved.Mode), "image", resolved.Image, "scope", string(resolved.Scope))
}
}
// Register file tools + exec tool (with sandbox routing via FsBridge if enabled)
if sandboxMgr != nil {
toolsReg.Register(tools.NewSandboxedReadFileTool(workspace, agentCfg.RestrictToWorkspace, sandboxMgr))
toolsReg.Register(tools.NewSandboxedWriteFileTool(workspace, agentCfg.RestrictToWorkspace, sandboxMgr))
toolsReg.Register(tools.NewSandboxedListFilesTool(workspace, agentCfg.RestrictToWorkspace, sandboxMgr))
toolsReg.Register(tools.NewSandboxedEditTool(workspace, agentCfg.RestrictToWorkspace, sandboxMgr))
toolsReg.Register(tools.NewSandboxedExecTool(workspace, agentCfg.RestrictToWorkspace, sandboxMgr))
} else {
toolsReg.Register(tools.NewReadFileTool(workspace, agentCfg.RestrictToWorkspace))
toolsReg.Register(tools.NewWriteFileTool(workspace, agentCfg.RestrictToWorkspace))
toolsReg.Register(tools.NewListFilesTool(workspace, agentCfg.RestrictToWorkspace))
toolsReg.Register(tools.NewEditTool(workspace, agentCfg.RestrictToWorkspace))
toolsReg.Register(tools.NewExecTool(workspace, agentCfg.RestrictToWorkspace))
}
// Memory tools — PG-backed; always registered (PG memory is always available)
toolsReg.Register(tools.NewMemorySearchTool())
toolsReg.Register(tools.NewMemoryGetTool())
toolsReg.Register(tools.NewKnowledgeGraphSearchTool())
slog.Info("memory + knowledge graph tools registered (PG-backed)")
// Browser automation tool
if cfg.Tools.Browser.Enabled {
var opts []browser.Option
if cfg.Tools.Browser.RemoteURL != "" {
opts = append(opts, browser.WithRemoteURL(cfg.Tools.Browser.RemoteURL))
slog.Info("browser tool enabled", "remote", cfg.Tools.Browser.RemoteURL)
} else {
opts = append(opts, browser.WithHeadless(cfg.Tools.Browser.Headless))
slog.Info("browser tool enabled", "headless", cfg.Tools.Browser.Headless)
}
if cfg.Tools.Browser.ActionTimeoutMs > 0 {
opts = append(opts, browser.WithActionTimeout(time.Duration(cfg.Tools.Browser.ActionTimeoutMs)*time.Millisecond))
}
if cfg.Tools.Browser.IdleTimeoutMs > 0 {
opts = append(opts, browser.WithIdleTimeout(time.Duration(cfg.Tools.Browser.IdleTimeoutMs)*time.Millisecond))
} else if cfg.Tools.Browser.IdleTimeoutMs < 0 {
// Explicitly disable idle reaper with negative value
opts = append(opts, browser.WithIdleTimeout(0))
}
if cfg.Tools.Browser.MaxPages > 0 {
opts = append(opts, browser.WithMaxPages(cfg.Tools.Browser.MaxPages))
}
browserMgr = browser.New(opts...)
toolsReg.Register(browser.NewBrowserTool(browserMgr))
}
// Web tools (web_search + web_fetch)
webSearchTool := tools.NewWebSearchTool(tools.WebSearchConfig{
BraveEnabled: cfg.Tools.Web.Brave.Enabled,
BraveAPIKey: cfg.Tools.Web.Brave.APIKey,
DDGEnabled: cfg.Tools.Web.DuckDuckGo.Enabled,
})
if webSearchTool != nil {
toolsReg.Register(webSearchTool)
slog.Info("web_search tool enabled")
}
webFetchTool = tools.NewWebFetchTool(tools.WebFetchConfig{
Policy: cfg.Tools.WebFetch.Policy,
AllowedDomains: cfg.Tools.WebFetch.AllowedDomains,
BlockedDomains: cfg.Tools.WebFetch.BlockedDomains,
})
toolsReg.Register(webFetchTool)
slog.Info("web_fetch tool enabled", "policy", cfg.Tools.WebFetch.Policy, "blocked", len(cfg.Tools.WebFetch.BlockedDomains))
// Vision fallback tool (for non-vision providers like MiniMax)
toolsReg.Register(tools.NewReadImageTool(providerRegistry))
toolsReg.Register(tools.NewCreateImageTool(providerRegistry))
// Audio generation tool (MiniMax music + ElevenLabs sound effects)
toolsReg.Register(tools.NewCreateAudioTool(providerRegistry,
cfg.Tts.ElevenLabs.APIKey, cfg.Tts.ElevenLabs.BaseURL))
// TTS (text-to-speech) system — always create TtsTool so config reload can populate it later
ttsMgr := setupTTS(cfg)
if ttsMgr == nil {
ttsMgr = tts.NewManager(tts.ManagerConfig{})
}
ttsTool = tools.NewTtsTool(ttsMgr)
toolsReg.Register(ttsTool)
if ttsMgr.HasProviders() {
slog.Info("tts enabled", "provider", ttsMgr.PrimaryProvider(), "auto", string(ttsMgr.AutoMode()))
}
// Tool rate limiting (per session, sliding window)
if cfg.Tools.RateLimitPerHour > 0 {
toolsReg.SetRateLimiter(tools.NewToolRateLimiter(cfg.Tools.RateLimitPerHour))
slog.Info("tool rate limiting enabled", "per_hour", cfg.Tools.RateLimitPerHour)
}
// Credential scrubbing (enabled by default, can be disabled via config)
if cfg.Tools.ScrubCredentials != nil && !*cfg.Tools.ScrubCredentials {
toolsReg.SetScrubbing(false)
slog.Info("credential scrubbing disabled")
}
// MCP servers (config-based: shared across all agents)
if len(cfg.Tools.McpServers) > 0 {
mcpMgr = mcpbridge.NewManager(toolsReg, mcpbridge.WithConfigs(cfg.Tools.McpServers))
if err := mcpMgr.Start(context.Background()); err != nil {
slog.Warn("mcp.startup_errors", "error", err)
}
slog.Info("MCP servers initialized", "configured", len(cfg.Tools.McpServers), "tools", len(mcpMgr.ToolNames()))
}
// Exec approval system — always active (deny patterns + safe bins + configurable ask mode)
{
approvalCfg := tools.DefaultExecApprovalConfig()
// Override from user config (backward compat: explicit values take precedence)
if eaCfg := cfg.Tools.ExecApproval; eaCfg.Security != "" {
approvalCfg.Security = tools.ExecSecurity(eaCfg.Security)
}
if eaCfg := cfg.Tools.ExecApproval; eaCfg.Ask != "" {
approvalCfg.Ask = tools.ExecAskMode(eaCfg.Ask)
}
if len(cfg.Tools.ExecApproval.Allowlist) > 0 {
approvalCfg.Allowlist = cfg.Tools.ExecApproval.Allowlist
}
execApprovalMgr = tools.NewExecApprovalManager(approvalCfg)
// Wire approval to exec tools in the registry
if execTool, ok := toolsReg.Get("exec"); ok {
if aa, ok := execTool.(tools.ApprovalAware); ok {
aa.SetApprovalManager(execApprovalMgr, "default")
}
}
slog.Info("exec approval enabled", "security", string(approvalCfg.Security), "ask", string(approvalCfg.Ask))
}
// --- Enforcement: Policy engines ---
// Permission policy engine (role-based RPC access control)
permPE = permissions.NewPolicyEngine(cfg.Gateway.OwnerIDs)
// Tool policy engine (7-step tool filtering pipeline)
toolPE = tools.NewPolicyEngine(&cfg.Tools)
// Data directory for Phase 2 services
dataDir = cfg.ResolvedDataDir()
os.MkdirAll(dataDir, 0755)
// Block exec from accessing sensitive directories (data dir, .goclaw, config file).
// Prevents `cp /app/data/config.json workspace/` and similar exfiltration.
// Exception: .goclaw/skills-store/ is allowed (skills may contain executable scripts).
if execTool, ok := toolsReg.Get("exec"); ok {
if et, ok := execTool.(*tools.ExecTool); ok {
et.DenyPaths(dataDir, ".goclaw/")
et.AllowPathExemptions(".goclaw/skills-store/")
// Harden: block access to internal workspace files via shell commands.
// Prevents `cat ../config.json`, `cat memory.db` etc. from user workspaces.
et.DenyPaths(
filepath.Join(workspace, "memory.db"),
filepath.Join(workspace, "memory.db-wal"),
filepath.Join(workspace, "memory.db-shm"),
filepath.Join(workspace, "config.json"),
filepath.Join(workspace, "delegate"),
filepath.Join(dataDir, "goclaw.db"),
filepath.Join(dataDir, "goclaw.db-wal"),
filepath.Join(dataDir, "goclaw.db-shm"),
)
if cfgPath := os.Getenv("GOCLAW_CONFIG"); cfgPath != "" {
et.DenyPaths(cfgPath)
}
}
}
// Block filesystem tools from accessing internal system files within the workspace.
// Shared-workspace agents have workspace = dataDir root, exposing config.json,
// memory.db, .media/, delegate/ etc. via list_files/read_file.
// Non-shared agents are already isolated by resolvePath boundary check, but
// deny paths add defense-in-depth.
internalDenyPaths := []string{
"config.json", "memory.db", "memory.db-wal", "memory.db-shm",
"goclaw.db", "goclaw.db-wal", "goclaw.db-shm",
"memory/", ".media/", ".uploads/", "delegate/",
}
// read_file: allow .media/ access (uploaded documents accessed via AllowPaths
// for backward compat; new uploads go to per-user .uploads/ within workspace).
readFileDenyPaths := []string{
"config.json", "memory.db", "memory.db-wal", "memory.db-shm",
"goclaw.db", "goclaw.db-wal", "goclaw.db-shm",
"memory/", "delegate/",
}
if rf, ok := toolsReg.Get("read_file"); ok {
if t, ok := rf.(*tools.ReadFileTool); ok {
t.DenyPaths(readFileDenyPaths...)
}
}
if wf, ok := toolsReg.Get("write_file"); ok {
if t, ok := wf.(*tools.WriteFileTool); ok {
t.DenyPaths(internalDenyPaths...)
}
}
if lf, ok := toolsReg.Get("list_files"); ok {
if t, ok := lf.(*tools.ListFilesTool); ok {
t.DenyPaths(internalDenyPaths...)
}
}
if ed, ok := toolsReg.Get("edit"); ok {
if t, ok := ed.(*tools.EditTool); ok {
t.DenyPaths(internalDenyPaths...)
}
}
return
}
// wireTracingAndCron sets up tracing collector, snapshot worker, and cron config
// on an already-created store set. Shared between PG and SQLite build variants.
func wireTracingAndCron(
cfg *config.Config,
stores *store.Stores,
msgBus *bus.MessageBus,
dataDir string,
) (*tracing.Collector, *tracing.SnapshotWorker) {
var traceCollector *tracing.Collector
if stores.Tracing != nil {
traceCollector = tracing.NewCollector(stores.Tracing)
traceCollector.OnFlush = func(traceIDs []uuid.UUID) {
ids := make([]string, len(traceIDs))
for i, id := range traceIDs {
ids[i] = id.String()
}
msgBus.Broadcast(bus.Event{
Name: protocol.EventTraceUpdated,
Payload: map[string]any{"trace_ids": ids},
})
}
traceCollector.Start()
slog.Info("LLM tracing enabled")
}
// Start snapshot worker for hourly usage aggregation
var snapshotWorker *tracing.SnapshotWorker
if stores.Snapshots != nil {
snapshotWorker = tracing.NewSnapshotWorker(stores.DB, stores.Snapshots)
snapshotWorker.Start()
// Backfill historical data in background
go func() {
count, err := snapshotWorker.Backfill(context.Background())
if err != nil {
slog.Warn("snapshot backfill failed", "error", err)
} else if count > 0 {
slog.Info("snapshot backfill complete", "hours", count)
}
}()
}
// Wire cron config from config.json
cronRetryCfg := cfg.Cron.ToRetryConfig()
if stores.Cron != nil {
stores.Cron.SetOnJob(nil) // ensure initialized; actual handler set below
_ = cronRetryCfg // config available; cron store reads it internally
if cfg.Cron.DefaultTimezone != "" {
stores.Cron.SetDefaultTimezone(cfg.Cron.DefaultTimezone)
}
}
// Load secrets from config_secrets table before env overrides.
// Precedence: config.json → DB secrets → env vars (highest).
if stores.ConfigSecrets != nil {
if secrets, err := stores.ConfigSecrets.GetAll(context.Background()); err == nil && len(secrets) > 0 {
cfg.ApplyDBSecrets(secrets)
cfg.ApplyEnvOverrides()
slog.Info("config secrets loaded from DB", "count", len(secrets))
}
}
return traceCollector, snapshotWorker
}
// setupMemoryEmbeddings wires embedding provider to PGMemoryStore and triggers backfill.
// Resolves embedding provider from DB providers with settings.embedding.enabled.
func setupMemoryEmbeddings(
pgStores *store.Stores,
providerRegistry *providers.Registry,
) {
if pgStores.Memory != nil {
if embProvider := resolveEmbeddingProvider(pgStores.Providers, providerRegistry, pgStores.SystemConfigs); embProvider != nil {
pgStores.Memory.SetEmbeddingProvider(embProvider)
slog.Info("memory embeddings enabled", "provider", embProvider.Name(), "model", embProvider.Model())
// Backfill embeddings for existing chunks that were stored without vectors.
type backfiller interface {
BackfillEmbeddings(ctx context.Context) (int, error)
}
if bf, ok := pgStores.Memory.(backfiller); ok {
go func() {
bgCtx := context.Background()
count, err := bf.BackfillEmbeddings(bgCtx)
if err != nil {
slog.Warn("memory embeddings backfill failed", "error", err)
} else if count > 0 {
slog.Info("memory embeddings backfill complete", "chunks_updated", count)
}
}()
}
// Wire embedding provider into team store for semantic task search.
if pgTeamStore, ok := pgStores.Teams.(*pg.PGTeamStore); ok {
pgTeamStore.SetEmbeddingProvider(embProvider)
go func() {
if count, err := pgTeamStore.BackfillTaskEmbeddings(context.Background()); err != nil {
slog.Warn("task embeddings backfill failed", "error", err)
} else if count > 0 {
slog.Info("task embeddings backfill complete", "tasks_updated", count)
}
}()
}
// Wire embedding provider into KG store for entity semantic search.
if pgKG, ok := pgStores.KnowledgeGraph.(*pg.PGKnowledgeGraphStore); ok {
pgKG.SetEmbeddingProvider(embProvider)
go func() {
if count, err := pgKG.BackfillKGEmbeddings(context.Background()); err != nil {
slog.Warn("KG embeddings backfill failed", "error", err)
} else if count > 0 {
slog.Info("KG embeddings backfill complete", "entities_updated", count)
}
}()
}
} else {
slog.Warn("memory embeddings disabled (no API key), chunks stored without vectors")
}
}
}
// seedSystemConfigs ensures system_configs has all expected keys for all tenants.
// Inserts missing keys from config.json without overwriting existing values.
func seedSystemConfigs(sc store.SystemConfigStore, ts store.TenantStore, cfg *config.Config) {
syncSystemConfigs(sc, ts, cfg, true) // onlyMissing=true
}
// loadBootstrapFiles loads bootstrap files for the default agent's system prompt from DB.
// Seeds if empty; falls back to filesystem as last resort.
func loadBootstrapFiles(
pgStores *store.Stores,
workspace string,
agentCfg config.AgentDefaults,
) []bootstrap.ContextFile {
// Load bootstrap files for default agent's system prompt from DB.
// Seeds if empty; falls back to filesystem as last resort.
var contextFiles []bootstrap.ContextFile
if pgStores.Agents != nil {
bgCtx := context.Background()
defaultAgent, agErr := pgStores.Agents.GetByKey(bgCtx, "default")
if agErr == nil {
dbFiles := bootstrap.LoadFromStore(bgCtx, pgStores.Agents, defaultAgent.ID)
if len(dbFiles) > 0 {
contextFiles = dbFiles
slog.Info("bootstrap loaded from store", "count", len(dbFiles))
} else {
// DB empty → seed templates, then load
if _, seedErr := bootstrap.SeedToStore(bgCtx, pgStores.Agents, defaultAgent.ID, defaultAgent.AgentType); seedErr != nil {
slog.Warn("failed to seed bootstrap to store", "error", seedErr)
} else {
contextFiles = bootstrap.LoadFromStore(bgCtx, pgStores.Agents, defaultAgent.ID)
slog.Info("bootstrap seeded and loaded from store", "count", len(contextFiles))
}
}
}
}
if len(contextFiles) == 0 {
// DB fallback: load from workspace filesystem
rawFiles := bootstrap.LoadWorkspaceFiles(workspace)
truncCfg := bootstrap.TruncateConfig{
MaxCharsPerFile: agentCfg.BootstrapMaxChars,
TotalMaxChars: agentCfg.BootstrapTotalMaxChars,
}
if truncCfg.MaxCharsPerFile <= 0 {
truncCfg.MaxCharsPerFile = bootstrap.DefaultMaxCharsPerFile
}
if truncCfg.TotalMaxChars <= 0 {
truncCfg.TotalMaxChars = bootstrap.DefaultTotalMaxChars
}
contextFiles = bootstrap.BuildContextFiles(rawFiles, truncCfg)
slog.Info("bootstrap loaded from filesystem", "count", len(contextFiles))
}
// Debug: log bootstrap file loading results
{
var loadedNames []string
for _, cf := range contextFiles {
loadedNames = append(loadedNames, fmt.Sprintf("%s(%d)", cf.Path, len(cf.Content)))
}
slog.Info("bootstrap context files", "count", len(contextFiles), "files", loadedNames)
}
return contextFiles
}
// setupSkillsSystem creates the skills loader, registers skill tools, wires skills-store,
// seeds bundled skills, and enables embedding-based skill search.
func setupSkillsSystem(
cfg *config.Config,
workspace string,
dataDir string,
pgStores *store.Stores,
toolsReg *tools.Registry,
providerRegistry *providers.Registry,
msgBus *bus.MessageBus,
) (*skills.Loader, *tools.SkillSearchTool, string, string, string) {
var bundledSkillsDir string // resolved later; returned for HTTP handler fallback
// Skills loader + search tool
// Global skills live under ~/.goclaw/skills/ (user-managed), not data/skills/.
globalSkillsDir := os.Getenv("GOCLAW_SKILLS_DIR")
if globalSkillsDir == "" {
globalSkillsDir = filepath.Join(dataDir, "skills")
}
// Bundled skills: shipped with the Docker image at /app/bundled-skills/.
// Lowest priority — managed (skills-store) and user-uploaded skills override these.
builtinSkillsDir := os.Getenv("GOCLAW_BUILTIN_SKILLS_DIR")
if builtinSkillsDir == "" {
builtinSkillsDir = "/app/bundled-skills"
}
skillsLoader := skills.NewLoader(workspace, globalSkillsDir, builtinSkillsDir)
skillSearchTool := tools.NewSkillSearchTool(skillsLoader)
toolsReg.Register(skillSearchTool)
toolsReg.Register(tools.NewUseSkillTool())
slog.Info("skill_search tool registered", "skills", len(skillsLoader.ListSkills(context.Background())))
// Wire skills-store directory into filesystem loader so agents
// can discover uploaded skills in their system prompt and BM25 search index.
if pgStores.Skills != nil {
storeDirs := pgStores.Skills.Dirs()
if len(storeDirs) > 0 {
skillsLoader.SetManagedDir(storeDirs[0])
slog.Info("skills-store directory wired into loader", "dir", storeDirs[0])
// Seed system/bundled skills into DB
bundledSkillsDir = os.Getenv("GOCLAW_BUNDLED_SKILLS_DIR")
if bundledSkillsDir == "" {
// Check common locations: Docker default, then local dev
for _, candidate := range []string{"bundled-skills", "/app/bundled-skills", "skills"} {
if info, err := os.Stat(candidate); err == nil && info.IsDir() {
bundledSkillsDir = candidate
break
}
}
}
if bundledSkillsDir != "" {
if seederStore, ok := pgStores.Skills.(skills.SystemSkillStore); ok {
seeder := skills.NewSeeder(bundledSkillsDir, storeDirs[0], seederStore)
seeded, skipped, seededSkills, err := seeder.Seed(context.Background())
if err != nil {
slog.Warn("system skills seed failed", "error", err)
} else {
if seeded > 0 {
slog.Info("system skills seeded", "seeded", seeded, "skipped", skipped)
}
// Check dependencies asynchronously — does not block startup.
// Emits WS events per-skill so UI updates in realtime.
if len(seededSkills) > 0 {
seeder.CheckDepsAsync(seededSkills, msgBus)
}
}
}
}
}
}
// Publish skill tool — lets agents register created skills in the database.
// Disabled in lite edition: agents should not self-manage skills on desktop.
if pgStores.Skills != nil && edition.Current().TeamFullMode {
if manageStore, ok := pgStores.Skills.(store.SkillManageStore); ok {
storeDirs := pgStores.Skills.Dirs()
if len(storeDirs) > 0 {
toolsReg.Register(tools.NewPublishSkillTool(manageStore, storeDirs[0], dataDir, skillsLoader))
slog.Info("publish_skill tool registered")
toolsReg.Register(tools.NewSkillManageTool(manageStore, storeDirs[0], dataDir, skillsLoader))
slog.Info("skill_manage tool registered")
}
}
}
// Wire embedding-based skill search + per-agent access filtering
if pgStores.Skills != nil {
if sas, ok := pgStores.Skills.(store.SkillAccessStore); ok {
skillSearchTool.SetSkillAccessStore(sas)
}
if pgSkills, ok := pgStores.Skills.(*pg.PGSkillStore); ok {
if embProvider := resolveEmbeddingProvider(pgStores.Providers, providerRegistry, pgStores.SystemConfigs); embProvider != nil {
pgSkills.SetEmbeddingProvider(embProvider)
skillSearchTool.SetEmbeddingSearcher(pgSkills, embProvider)
slog.Info("skill embeddings enabled", "provider", embProvider.Name())
// Backfill embeddings for existing skills
go func() {
count, err := pgSkills.BackfillSkillEmbeddings(context.Background())
if err != nil {
slog.Warn("skill embeddings backfill failed", "error", err)
} else if count > 0 {
slog.Info("skill embeddings backfill complete", "skills_updated", count)
}
}()
}
}
}
return skillsLoader, skillSearchTool, globalSkillsDir, bundledSkillsDir, builtinSkillsDir
}