Commit Graph

102 Commits

Author SHA1 Message Date
viettranx 4e9f155a4c feat(agent): adaptive tool timing with slow tool notification
Track per-tool execution time statistics in session metadata. When a tool
call exceeds its adaptive threshold (2x historical max, min 120s default),
send a direct outbound notification to the user.

- ToolTimingMap: parse/serialize/record/threshold from session metadata
- StartSlowTimer: fires once per tool call, auto-cancels on completion
- Team config: slow_tool toggle (default on, always direct, never leader)
- UI: toggle in team settings with i18n (en/vi/zh)
- Store: add GetSessionMetadata to session store interface
2026-03-19 13:35:57 +07:00
Duc Nguyen 2cc9d68cdc fix(tts): config save, Edge provider, media dispatch + dark mode chat (#265)
* fix(tts): config save + Edge provider registration + dark mode chat bubbles

- Wrap TTS config payload in `raw` field for config.patch RPC (#229)
- Always register Edge TTS provider (free, no API key) instead of gating on `enabled` flag
- Fix low-contrast user message bubbles in dark mode chat

* fix(tts): skip duplicate media dispatch when temp file already delivered

When both the agent loop and the message tool dispatch the same TTS
temp file, the first dispatch succeeds and cleanup deletes it. Filter
out missing temp media files before sending to prevent "file not found"
errors and spurious error notifications on Telegram/Slack/Discord.

* feat(tts): include edge-tts in Docker image when Python enabled

Edge TTS is free (no API key) and serves as a universal TTS fallback.
Install it alongside Python in both ENABLE_PYTHON and ENABLE_FULL_SKILLS builds.

* chore(docker): expose build args from .env for compose builds

Pass ENABLE_OTEL, ENABLE_PYTHON, ENABLE_FULL_SKILLS as env-driven
build args so .env can control Docker build features without editing
docker-compose.yml directly.

* fix(tts): hot-reload TTS config on settings change via pub/sub

TTS providers were only registered at startup, so changing provider/API
key via the Web UI had no effect until container restart. Add a
tts-config-reload bus subscriber that rebuilds the TTS manager on
config changes, matching the pattern used by quota, cron, and web_fetch.
Always create a TtsTool at startup (even without providers) so the
reload subscriber can populate it when settings are first configured.

* fix(tts): protect TtsTool.UpdateManager with RWMutex to prevent data race

UpdateManager() can be called from the config reload goroutine while
Execute() reads t.manager concurrently from agent goroutines. Add
sync.RWMutex following the same pattern as WebFetchTool.UpdatePolicy().

Also update setupTTS doc comment which incorrectly stated it could
return nil — Edge TTS is now always registered.

---------

Co-authored-by: viettranx <viettranx@gmail.com>
2026-03-19 08:21:06 +07:00
viettranx 5b349db7eb feat(heartbeat): provider/model override + fix cache invalidation
- Add ProviderModelSelect to heartbeat config dialog (allowEmpty, verify button)
- Backend: accept providerName in HEARTBEAT.SET, resolve to UUID via GetProviderByName
- Add ModelOverride to RunRequest, used by Loop when set (cheaper model for heartbeat)
- Ticker passes heartbeat model override to agent RunRequest
- Fix: InvalidateCache after UpdateState so ListDue picks up new next_run_at immediately
- i18n: add sectionModel/modelHint keys (en/vi/zh)
2026-03-18 23:02:48 +07:00
Duc Nguyen dc51018563 fix: subagent provider routing + api_base fallback (#262)
* fix(subagent): inherit parent agent's provider instead of alphabetical fallback

Subagents previously used a fixed provider (alphabetically first from the
registry, often "anthropic") regardless of which provider the parent agent
used. This caused invalid combos like anthropic/glm-5 when a zai-coding
agent spawned subagents.

- Pass provider registry to SubagentManager for runtime resolution
- Inject parent provider name into context (WithParentProvider)
- Resolve activeProvider from parent context before LLM call
- Fix trace spans to show actual resolved provider, not default

* fix(providers): api_base fallback from config/env for DB providers

DB providers with empty api_base now inherit from config/env vars
(e.g., GOCLAW_ANTHROPIC_BASE_URL). Prevents proxy API keys from being
sent to the real provider API endpoint.

- Add APIBaseForType() method on ProvidersConfig
- registerProvidersFromDB falls back to config when api_base is empty
- ProvidersHandler uses resolveAPIBase() for model listing
- Add api_base, display_name, settings to provider validation whitelist

* fix(tracing): pass resolved provider name to subagent span emitters

- emitSubagentSpanStart now accepts providerName param instead of
  reading sm.provider.Name() — ensures root subagent span reflects
  the inherited parent provider, not the fallback default
- registerInMemory now uses resolveAPIBase() so DB providers with
  empty api_base inherit the config/env fallback (same as startup path)

---------

Co-authored-by: viettranx <viettranx@gmail.com>
2026-03-18 22:40:49 +07:00
viettranx 96cfd1bf08 feat(heartbeat): improve prompting, suppression, delivery targets and session cleanup
- Rewrite heartbeat prompt to instruct agent to EXECUTE checklist tasks, not echo them
- Simplify suppression: HEARTBEAT_OK present = always suppress, absent = always deliver
- Add delivery targets RPC (heartbeat.targets) for channel/chatId picker
- Sanitize backend errors — never expose raw SQL to client
- Add session cleanup for isolated heartbeat sessions after run
- Cap StaggerOffset at 10% of interval to avoid user-visible delay
- Fix Upsert to persist next_run_at correctly
2026-03-18 16:37:36 +07:00
viettranx 08a2d95c0c feat: agent heartbeat system — periodic proactive check-ins (#245)
Phase 1 (Core):
- Migration 000022: agent_heartbeats, heartbeat_run_logs, agent_config_permissions tables
- HeartbeatStore + ConfigPermissionStore interfaces with PG implementations
- HeartbeatTicker: background poll → active hours filter → queue-aware skip → run → smart suppression → deliver/log
- Heartbeat tool: status/get/set/toggle/set_checklist/get_checklist/test/logs actions
- Permission check with wildcard scope matching + TTL cache (60s)
- RPC methods: heartbeat.get/set/toggle/test/logs/checklist.get/checklist.set
- HEARTBEAT.md routed via context file interceptor (read/write for both open + predefined agents)
- Session keys: agent:{id}:heartbeat or agent:{id}💓{ts} (isolated)
- PromptMinimal for heartbeat sessions (like cron/subagent)
- Event broadcasting + cache invalidation via bus (heartbeat + config_perms)
- Gateway wiring: ticker init, event wiring, graceful shutdown

Phase 2 (Integration):
- wakeMode: CronPayload.WakeHeartbeat triggers heartbeat after cron job completes
- Queue-aware: Scheduler.HasActiveSessionsForAgent() skips busy agents
- Stagger: deterministic FNV offset spreads heartbeats across interval
- lightContext: RunRequest.LightContext skips context files, only injects checklist
- System prompt distinguishes cron (user-scheduled tasks) vs heartbeat (autonomous monitoring)
2026-03-18 13:11:44 +07:00
viettranx 49441f7305 refactor: remove dead delegate code, rename lane/channel to team/teammate
- Remove handleDelegateAnnounce() dead code (no sender emits delegate:* messages)
- Remove delegate tool reference from intent_classify.go
- Rename LaneDelegate → LaneTeam with backward-compat env var fallback
- Rename ChannelDelegate → ChannelTeammate across all team tool files
- Comment out lifecycle guards in team_tasks_lifecycle.go (TODO: reviewer workflow)
- Update string literals in cron.go, task_ticker.go
- Gate tool_status placeholder_update to non-streaming runs only
- Skip FinalizeStream on tool.call to prevent mid-run content loss
2026-03-18 11:04:45 +07:00
viettranx 843b550651 feat: runtime packages UI, pkg-helper, configurable shell deny groups (#244)
Runtime package management with security hardening:

- pkg-helper: root-privileged daemon for apk install/uninstall via Unix socket
- HTTP API: /v1/packages (list/install/uninstall/runtimes), admin role required for writes
- Shell deny groups: 15 configurable groups (per-agent overrides via context)
- Packages UI: Web page for managing system/pip/npm packages with confirmation dialogs
- Docker: privilege separation (root entrypoint → su-exec drop), init for zombie reaping
- Security: umask socket creation, persist file validation, deny pattern hardening
  (Node.js fetch/http, Python from/import, curl localhost, sensitive env vars)
- Auth: empty gateway token → admin role (dev/single-user mode)
2026-03-17 19:50:26 +07:00
viettranx b735c16d93 feat(teams): split dispatched/assigned events + add completed notifications
- Change agent-side broadcasts from EventTeamTaskAssigned to
  EventTeamTaskDispatched (post-turn, fallback, unblock, retry)
- Add completed notification with leader-completion skip logic
- Add Completed field to TeamNotifyConfig with *bool backwards compat
- Differentiate dispatched messages: unblocked vs regular dispatch
- Add EventTeamTaskDispatched to audit event mapper
2026-03-17 18:02:54 +07:00
viettranx aeadb20ba7 fix(teams): deduplicate task notifications and batch with debounce
Remove premature EventTeamTaskAssigned broadcast in executeCreate() that
caused duplicate "assigned to" Telegram notifications. Assignment
notification now only fires at actual dispatch (post-turn, fallback, or
unblocked).

Add TeamNotifyQueue (2s debounce, cap 20) to batch rapid-fire task
notifications per chat — reduces N messages to 1 when leader dispatches
multiple tasks at once. In leader mode this also reduces agent turns
from N to 1.

Also fix: ResetTaskStatus now clears progress_percent/progress_step on
retry, and retry broadcast includes TaskNumber/Subject for correct
notification formatting.
2026-03-17 14:29:52 +07:00
viettranx 97cacfe68b feat(teams): member task progress reminder + fix broken progress notifications
- Fix progress event payload missing TaskNumber, Subject, OwnerAgentKey,
  ProgressPercent, ProgressStep — notifications were rendering empty
- Fix progress notification format to include task name (consistent with
  dispatched/failed) and guard empty ProgressStep
- Change percent tool schema from number to integer for clarity
- Add pre-run member task reminder injecting task context before LLM loop
- Add mid-loop progress nudge every 10 iterations with suggested percent
  based on iteration ratio (handles maxIter=0 unlimited case)
- Enhance leader cross-session reminder to show progress % when available
- Strengthen TEAM.md member guidance: focus, result quality, progress rules
- Add progress bar to task list table view (matches kanban card pattern)
2026-03-17 12:43:09 +07:00
viettranx eee79d111e feat(teams): granular progress notifications with direct/leader mode
- Replace progress_notifications toggle with granular config:
  dispatched (on), progress (on), failed (on) + delivery mode
- Direct mode: outbound to channel, no AI processing
- Leader mode: inject into leader session with NO-ACTION instructions
- Add consumer.team-notify subscriber for event forwarding
- Enrich TeamTaskEventPayload with TaskNumber, ProgressPercent/Step
- Add auto-status system prompt section
- UI: card-select for delivery mode (Zap/Bot icons), 3 toggles
2026-03-16 22:46:51 +07:00
viettranx b0bd4d6198 fix(pairing): fix browser approval stuck + security hardening
Squash-merge PR #225 with security fixes:

- Fix browser pairing stuck on "Waiting for approval" (stale closure:
  useState → useRef for senderID in pairing-form)
- Fix auto-kick after pairing (RequireAuth now accepts senderID,
  onAuthFailure skips logout for paired browser sessions)
- Allow browser-paired users to access HTTP APIs via X-GoClaw-Sender-Id
  header with fail-closed IsPaired check
- Remove ad-hoc IsInternalOrBrowser(), use channels.IsInternalChannel()
- Log failed HTTP pairing auth attempts for security monitoring
- Pass senderID to HttpClient for authenticated HTTP requests
2026-03-16 20:09:44 +07:00
viettranx 0857321a6b fix(providers): correct Anthropic prompt caching + add datetime tool
- Move cache_control from request root (ignored by API) to per-block
  placement on last system block and last tool definition
- Change system prompt time format to date-only for better cache stability
- Add builtin datetime tool for precise timestamps (cron, memory, etc.)
- Add atMs past-time validation in cron handleUpdate (was only in handleAdd)
- Update cron description to guide model to use datetime tool first
2026-03-16 08:14:03 +07:00
viettranx e138ac7676 fix(teams): validate blocked_by terminal state + improve leader orchestration prompt
- Add terminal-state check in executeCreate(): reject blocked_by
  referencing completed/cancelled/failed tasks with actionable error
- Add full validation in executeUpdate(): batch query via GetTasksByIDs,
  check existence + team membership + terminal state
- Add GetTasksByIDs batch query to TeamStore interface + pg implementation
- Refactor: modularize gateway, skills store, and team tools into
  focused files
- Update TEAM.md leader prompt: prefer delegation, plan full task graph
  upfront, create tasks in order with blocked_by UUIDs
2026-03-15 23:16:16 +07:00
viettranx 84b1b07634 refactor(config): centralize hardcoded ~/.goclaw paths via config resolution
Replace all hardcoded ~/.goclaw path constructions with configurable
sources (cfg.ResolvedDataDir() for service dirs, cfg.Agents.Defaults.Workspace
for agent workspaces). This fixes data persistence issues in Docker
deployments where paths differ from local dev.

- Add DataDir field to Config with ResolvedDataDir() resolver
- Add ResolvedDataDirFromEnv() package-level helper for packages without Config
- Populate StoreConfig.SkillsStorageDir (was never set, caused hardcoded fallback)
- Agent workspaces now use subdirectory format (workspace/{key}) for volume compatibility
- Remove dead GOCLAW_SESSIONS_STORAGE env/config (sessions moved to PostgreSQL)
- Fix deploy-stg.sh trailing space after backslash + remove deprecated GOCLAW_MODE
- Add GOCLAW_SKILLS_DIR override in docker-compose for volume persistence
2026-03-15 21:20:46 +07:00
Goon 75c570e951 feat(security): credentialed exec + HTTP RBAC + API key cache (#197)
- Secure CLI credential injection via AES-256-GCM encrypted env vars
- API key management with fine-grained RBAC scopes
- resolveAuth/requireAuth middleware across all 25+ HTTP handlers
- In-memory API key cache with TTL, negative caching, pubsub invalidation
- Sandbox-first execution (fails if unavailable, no silent fallback)
- Credential scrubbing, constant-time token comparison, Admin-only CLI creds
- SQL migration 000020: secure_cli_binaries + api_keys tables
- 14 unit tests for cache and RBAC with race detector

Closes #197
2026-03-15 20:13:18 +07:00
viettranx f236d721a9 refactor(teams): redesign team detail page with kanban board layout
- Add board/ components: kanban board, columns, cards, toolbar, dialogs
- Add Zustand board store for task state management
- Simplify task-detail-dialog and task-list components
- Refactor team-detail-page to board-based layout
- Update team-version-modal with improved UI
- Clean up team-settings-tab
- Add i18n strings for board UI (en/vi/zh)
- Update workspace path resolution in gateway and teams_workspace
2026-03-15 17:28:54 +07:00
Viet Tran 9a9744077e refactor(teams): v2 system cleanup — remove legacy tools, fix followup, add events API (#210)
Major refactoring of the team system with multiple improvements:

## Removed legacy delegation tools
- Delete `delegate.go`, `delegate_async.go`, `delegate_sync.go`, `delegate_events.go`,
  `delegate_policy.go`, `delegate_prep.go`, `delegate_state.go`, `delegate_search_tool.go`
- Delete `evaluate_loop_tool.go`, `handoff_tool.go`
- Remove all references and registrations from tool manager and policy
- Clean up TEAM_PLAYBOOK_IDEAS.md and TEAM_SYSTEM.md (moved to docs)

## Rename await_reply → ask_user
- Rename action `await_reply` → `ask_user`, `clear_followup` → `clear_ask_user`
- Rename functions `executeAwaitReply` → `executeAskUser`, `executeClearFollowup` → `executeClearAskUser`
- Update system prompt with stronger wording to prevent model misuse
- Model was confusing "await_reply" with general waiting; "ask_user" is unambiguous

## Fix auto-followup false positives
- Add `HasActiveMemberTasks(ctx, teamID, excludeAgentID)` store method
- Guard `autoSetFollowup()` in consumer: skip when lead has active member tasks
- Prevents auto-followup when lead is orchestrating teammates (not waiting for user)

## Task identifier zero-padding
- Change format from `T-1-xxxx` → `T-001-xxxx` (3-digit minimum)

## Refactor workspace WS handlers to filesystem-only
- Rewrite `teams.workspace.list/read/delete` to use pure filesystem (os.ReadDir/ReadFile/Remove)
- Remove DB dependency from workspace WS handlers
- Consistent with storage handler and workspace tools
- Simplify TeamWorkspaceFile type and frontend hook

## Add team events listing API
- New WS method `teams.events.list` with team_id, limit, offset params
- New HTTP endpoint `GET /v1/teams/{id}/events` with bearer auth
- New `ListTeamEvents(ctx, teamID, limit, offset)` store method
- JOIN with team_tasks for team-wide event filtering

## Extract team access policy
- New `team_access_policy.go` — centralized team tool access control

## Migration 000019: team_id columns
- Add team_id foreign key columns to relevant tables

## Other improvements
- Add team_id propagation through agent loop, tracing, sessions
- Update i18n locale files (en/vi/zh) for new tool labels
- Update frontend builtin-tools page and require-setup component
- Bump RequiredSchemaVersion for migration 000019
2026-03-15 14:53:19 +07:00
viettranx 8ad425f5f8 fix: use resolved workspace dir for StorageHandler in Docker deployments
StorageHandler was hardcoded to browse ~/.goclaw/, which is empty in
Docker where volumes mount to separate paths (GOCLAW_WORKSPACE).
Use the already-resolved workspace variable so the Storage page
correctly shows workspace files regardless of deployment layout.
2026-03-14 14:21:59 +07:00
Viet Tran 1a42dc93a6 feat(teams): team system v2 with bug fixes, workspace scope, versioning, and prompt optimization (#183)
* feat(workspace): add team shared workspace for file collaboration

- Add workspace_write and workspace_read tools for agents to share files across team members
- Create team_workspaces DB table with migration 000017 (file metadata, pinning, tags)
- Implement PostgreSQL store layer for workspace CRUD operations
- Add RPC handlers for workspace list/read/delete from web UI
- Build React workspace tab with file listing, content preview, and delete
- Propagate workspace channel/chatID scope through delegation chain
- Auto-allow workspace tools in agent tool policy when agent belongs to a team
- Inject team workspace guidance into system prompt for team agents
- Add /reset command handler for clearing session history
- Harden MCP bridge context middleware to reject headers when no gateway token
- Add i18n strings for workspace UI in en/vi/zh locales

* feat(teams): add comprehensive task management with followup reminders and recovery

- Add task followup/reminder system with auto-set on lead agent reply and auto-clear when user responds on channel
- Add task recovery ticker to re-dispatch stale/pending tasks periodically
- Add task scopes, filtering by status/channel/chatID, and task events
- Add WS RPC handlers for task CRUD, assignments, comments, events, and bulk operations (teams_tasks.go)
- Add task detail dialog, settings UI for followup config, and scope filtering in web dashboard
- Add migrations 000018 (team_tasks_v2) and 000019 (task_followup)
- Extend team_tasks_tool with await_reply, clear_followup actions
- Auto-complete/fail team tasks when delegate agent finishes
- Add workspace file listing and team tool manager enhancements

* docs(teams): add team system architecture and playbook ideas documentation

- Add TEAM_SYSTEM.md with full architecture design covering task management, shared workspace, and delegation engine subsystems
- Add TEAM_PLAYBOOK_IDEAS.md outlining future team coordination layers (playbook, member capabilities, auto-learned patterns)
- Document data models, status flows, tool actions, followup reminder system, task ticker, execution locking, and workspace scope model

* fix(teams): resolve 6 critical bugs in team task system

- Fix unblock SQL: check array_length after array_remove (not before)
- Enforce single-team leadership in team creation
- Add requireLead() for approve/reject tool actions
- Validate cross-team dependency references in blocked_by
- Add team_id to handoff route for multi-team isolation
- Set blocked_by DEFAULT '{}' to prevent NULL array issues

* refactor(workspace): use stable userID as scope key instead of connection UUID

Workspace scope changed from (team_id, channel, chat_id) to (team_id, userID).
Fixes workspace fragmentation across WS tab refreshes and reconnections.

* feat(teams): add V1/V2 versioning with feature gating and optimized prompts

- IsTeamV2() helper gates advanced features (locking, followup, review, audit)
- V2 tool actions rejected for V1 teams with clear error message
- Ticker, gateway consumer, delegation hooks respect version flag
- TEAM.md renders v1/v2 sections conditionally
- Tool descriptions and params optimized (~38% token reduction)
- UI: version toggle in settings, V2 Beta badge, conditional rendering
- i18n: version modal keys for en/vi/zh

* fix(migration): use VARCHAR(255) for user ID columns and add metadata JSONB

- assignee_user_id, user_id, actor_id: TEXT → VARCHAR(255)
- Add metadata JSONB to team_task_comments and team_task_attachments

---------

Co-authored-by: Nam Nguyen Ngoc <namnn.0911@gmail.com>
2026-03-13 22:41:32 +07:00
Luan Vu b73f66d99b fix(tools): make MessageTool media path resolution workspace-aware (#169)
MessageTool.parseMediaPath() was hardcoded to only allow files in /tmp/,
while all other filesystem tools (read_file, write_file, edit, exec) use
workspace-aware resolvePath() with restrict_to_workspace enforcement.

This meant agents could create files in their workspace via write_file
but couldn't send them as attachments — only /tmp/ files from
create_image/create_audio worked.

Replace parseMediaPath() with resolveMediaPath() that:
- Reuses resolvePath() for consistent security (symlink, hardlink, traversal)
- Honors per-agent workspace + restrict_to_workspace from context
- Still allows /tmp/ as fallback (for create_image, create_audio, etc.)
- Supports relative paths resolved against workspace
- Updates tool description so LLM knows about MEDIA: prefix

Co-authored-by: Luvu182 <208665161+Luvu182@users.noreply.github.com>
2026-03-12 20:43:52 +07:00
viettranx 9115169c03 feat: expand audit logging via pub/sub event pattern
Replace direct ActivityStore injection with event-driven audit system.
Handlers emit audit events via msgBus.Broadcast(), a single subscriber
with buffered channel persists to activity_logs table.

Coverage expanded from 3 agent CRUD actions to ~65 audit points across
all HTTP handlers and WebSocket RPC methods including agents, providers,
skills, MCP servers, cron, sessions, teams, pairing, and more.
2026-03-12 18:34:56 +07:00
Viet Tran ace07509b7 feat(skills): system skills integration — toggle, dep checking, per-item install (#161)
* feat(infra): add runtime package support for skills

Install nodejs, npm, pandoc, github-cli + pre-install Python packages
(openpyxl, pandas, python-pptx, markitdown) and Node packages
(docx, pptxgenjs). Configure runtime dirs for agent pip/npm installs
with PIP_TARGET, NPM_CONFIG_PREFIX, NODE_PATH to enable dynamic
package installation in read-only container environment.

* feat(infra): add bundled skills with runtime package support

- Add 5 bundled skills: docx, pdf, pptx, xlsx, skill-creator from container skills-store
- Wire GOCLAW_BUILTIN_SKILLS_DIR env var in gateway and CLI
- Support optional runtime packages alongside dynamic skill loading
- Update Dockerfile to COPY bundled-skills at /app/bundled-skills/
- Add PIP_CACHE_DIR in docker-entrypoint.sh for clean pip installs
- Document bundled skills in 14-skills-runtime.md section 6

* feat(infra): remove ai-multimodal skill directory from bundled skills

Remove the ai-multimodal skill package as part of consolidating runtime
package support for bundled skills. This directory is no longer needed
in the bundled skills structure.

* feat(ci): add semantic release and Docker Hub publishing

Add go-semantic-release workflow to auto-create semver tags on merge to
main. Extend docker-publish to push all variants to both GHCR and
Docker Hub (digitop/goclaw).

* feat(skills): add system skills infrastructure with is_system column, dep scanning, and seeder

- Migration 000017: add is_system boolean column with partial index
- Store layer: UpsertSystemSkill, delete protection, IsSystemSkill
- ListAccessible auto-includes system skills (no grants needed)
- ListWithGrantStatus returns is_system field
- Dependency scanner: auto-detect deps from scripts/ or skill-manifest.json
- Dependency checker: verify system binaries, Python/Node packages
- Seeder: seed bundled skills into DB on startup (idempotent via hash)
- Gateway wiring: GOCLAW_BUNDLED_SKILLS_DIR env for bundled skills
- HTTP: delete guard (403), slug conflict check (409), rescan-deps endpoint
- UI: System badge, hide delete for system skills, rescan deps button
- Agent skills tab: "Always available" for system skills
- i18n: en/vi/zh keys for system skills, deps scanning

* feat(skills): conditional system prompt, skill manifests, and Zip Slip fix

- System prompt: only show package list when python3/node are available
- Add skill-manifest.json for pdf, docx, xlsx, pptx bundled skills
- Fix Zip Slip vulnerability in office/unpack.py (all 3 copies)

* refactor(skills): extract shared office code to _shared/ and deduplicate

Move office scripts (pack, unpack, validate, schemas, validators) from
duplicated copies in docx/xlsx/pptx to skills/_shared/office/ with
symlinks. Remove soffice.py (non-functional in containers) and update
SKILL.md references to use soffice binary directly. Update seeder
copyDir to follow symlinks.

Removes ~45K lines of duplicate code across 3 skills.

* fix(skills): address code review findings for system skills integration

- H1: Remove dead symlink branch in copyDir (filepath.Walk follows symlinks)
- H3: Fix rescan-deps to query ALL skills (including archived) and re-activate
  when deps become available; add ListAllSkills() + Status field to SkillInfo
- H4: Add Status field to SkillCreateParams, stop overloading Visibility
- M1: Batch Python/Node dep checks into single subprocess per runtime
- M4: Add rows.Err() check in ListSkills to prevent caching partial results

* feat(skills): async dep checking with realtime WS events

Split Seed() into sync DB upsert + async CheckDepsAsync() goroutine.
Gateway startup no longer blocks on Python/Node subprocess dep checks.

- Seed() returns seeded skills list, all initially status="active"
- CheckDepsAsync() runs in background, emits skill.deps.checked per-skill
- skill.deps.complete event emitted when all checks finish
- Each failed dep check: archives skill + BumpVersion() for immediate
  cache invalidation so next agent turn picks up the change
- UI: use-query-invalidation listens to skill.deps.* events → auto-refresh
  skills list in realtime

* feat(skills): system skills integration with toggle, dep checking, and per-item install

- Add is_system, deps, enabled columns to skills table (migration 017)
- Seed bundled core skills (pdf, docx, pptx, xlsx, skill-creator) on startup
- PYTHONPATH-based dep detection — eliminates false positives from local modules
- Per-item dep install UI with individual status (installing/success/error)
- Enable/disable toggle for core and custom skills (independent of dep status)
- Re-run dep check when skill is toggled back on
- Inline skill thresholds: 40 skills / 5000 tokens before switching to search mode
- Fix UpsertSystemSkill: backfill null file_hash without bumping DB version
- Remove redundant skill-manifest.json files (replaced by deps JSONB column)
- Show author from frontmatter in custom skills tab
- Runtime checker for python3/pip3/node/npm availability
- WS events for dep checking/installing progress
- docs: add 15-core-skills-system.md, 16-skill-publishing.md

---------

Co-authored-by: Goon <duy@wearetopgroup.com>
2026-03-12 09:20:41 +07:00
viettranx ec34f488df feat(tools): add tool alias registry for Claude Code skill compatibility
Add alias support to the tool Registry so Claude Code skills can reference
Anthropic tool names (Read, Write, Edit, Bash, etc.) and have them resolve
to GoClaw canonical tools (read_file, write_file, edit, exec, etc.).

- Registry: aliases map, RegisterAlias(), resolve(), Aliases()
- Policy engine: auto-includes alias defs when canonical tool passes filter
- System prompt: alias entries in coreToolSummaries + missing use_skill
- Legacy toolAliases migrated to Registry.RegisterAlias() at startup
2026-03-11 17:33:28 +07:00
Luan Vu 1b99406012 fix: resolve embedding provider from DB registry + per-agent config (#134)
The embedding provider resolution only matched 3 hardcoded names
(openai, openrouter, gemini), silently failing for DB-stored providers
like "openai-embedding". This caused memory chunks to be stored
without vectors even when a valid embedding provider was configured.

Changes:
- resolveEmbeddingProvider: fallback to provider registry for DB-stored
  provider names when hardcoded match fails
- gateway startup: read per-agent memory config from DB (priority over
  config file defaults) for embedding provider resolution
- memory IndexDocument: log embedding errors instead of swallowing them
- memory admin ListChunks: return full chunk text instead of truncating
  to 200 chars, avoiding confusing partial content in the UI

Co-authored-by: Luvu182 <208665161+Luvu182@users.noreply.github.com>
2026-03-11 14:31:00 +07:00
Viet Tran 73389d2715 fix(ui): align usage data contracts, add timezone setting, and fix empty usage page (#146)
- Fix 6 data contract mismatches between Go backend JSON tags and React
  frontend TypeScript interfaces (field renames, response envelope changes)
- Add timezone selector to topbar with 12 common timezone options
- Replace date-fns formatting with native Intl.DateTimeFormat for
  timezone-aware chart labels (reduces bundle ~20KB)
- Add missing SnapshotTimeSeries fields (memory_docs, memory_chunks,
  kg_entities, kg_relations) that caused empty usage page
- Add error banner to usage page for API error visibility
- Sanitize backend error messages in usage HTTP handlers
- Add batch chunking (max 3000 rows) for snapshot upserts
- Remove userId display from topbar
- Add usage analytics i18n strings for en/vi/zh
2026-03-11 14:22:03 +07:00
Viet Tran 0926d053b0 feat: add token usage tracking, cost analytics, budget enforcement, wake API, and activity audit trail (#142)
- A1+C2: Include token usage in run.completed event payload for WS clients
- A2: Cost tracking with model pricing config, cost calculation, and cost summary API
- A3: Budget enforcement per agent with monthly budget limits (migration 000015)
- C1: External wake/trigger API (POST /v1/agents/{id}/wake) for orchestrators
- C3: Activity audit trail with structured logging and queryable API
- UI: Activity page, cost stat card on overview, budget section in agent detail
- i18n: Complete en/vi/zh translations for all new features
2026-03-11 12:52:12 +07:00
Thieu Nguyen 8ad580521d refactor: deprecate standalone mode, managed mode is now default (#126)
* refactor: remove managed/standalone mode distinction from codebase

Standalone mode is deprecated; managed mode is now the only mode.
Remove redundant "managed mode" qualifiers from comments, docs,
and error messages. Error strings now reference "database stores"
instead of "managed mode" for clarity.

* improve(onboard): streamline onboard process and env setup

Simplify onboard wizard, extract helpers to dedicated file,
update env example and entrypoint for default managed mode,
clean up prepare-env script, update i18n catalogs.
2026-03-11 07:27:38 +07:00
viettranx b9e9e6e34a refactor(media): migrate builtin tool settings from legacy flat to chain format
- Update seed defaults to use chain format {"providers":[...]}
- Add startup auto-migration for existing legacy flat settings in DB
- Remove legacy flat format parsing from parseChainSettings()
2026-03-10 22:54:52 +07:00
Nam Nguyen Ngoc 9a0557c7a3 fix(tools): allow read_file to access CLI workspaces directory (#122)
- Add ~/.goclaw/cli-workspaces/ to read_file allowed paths
- Enables agents to read working files from CLI workspace sessions

Co-authored-by: Nam Nguyen Ngoc <namnn.0911@gmail.com>
2026-03-10 20:58:10 +07:00
viettranx 23f1957c56 feat(channels): wire contact auto-collector across all channel handlers
Pass contactCollector through channel manager to all channel handlers
(Telegram, Discord, Feishu, Slack, Zalo) so contacts are automatically
collected when users interact with the agent.
2026-03-10 18:46:44 +07:00
Luan Vu 25fd9c9d6d feat(cron): configurable default timezone for cron expressions (#117)
* feat(cron): configurable default timezone for cron expressions

Cron expressions (e.g. "0 8 * * *") are evaluated relative to a timezone.
Without an explicit per-job timezone, they default to the server's system
timezone, which may not match the user's local time — especially in Docker
containers (default UTC) or multi-region deployments.

This adds a `default_timezone` setting to `CronConfig` (IANA format, e.g.
"Asia/Ho_Chi_Minh") that is applied as fallback when a cron job has no
explicit `schedule.tz`. The setting is configurable via the UI config page
(Integrations → Cron Scheduler) and hot-reloads on config changes.

Backend:
- Add `DefaultTimezone` field to `CronConfig`
- Add `SetDefaultTimezone()` to `CronStore` interface + PG implementation
- Apply default TZ in `AddJob()` when `schedule.TZ` is empty
- Wire at startup + subscribe to config change events for hot reload
- Update cron tool description so LLM knows about gateway default

Frontend:
- Add timezone dropdown (20 common IANA timezones) to Cron config section
- Add i18n keys for en, vi, zh

* fix(cron): apply default timezone to existing jobs via computeNextRun

Pass defaultTZ as fallback to computeNextRun so existing cron jobs
(with timezone = NULL in DB) also use the gateway's configured default
timezone when computing next_run_at. This ensures old jobs benefit
from the timezone setting without needing a DB migration or backfill.

---------

Co-authored-by: Luvu182 <208665161+Luvu182@users.noreply.github.com>
2026-03-10 18:44:28 +07:00
viettranx 1ddd2e547a feat(compaction): configurable provider, model & max_tokens for pending message compaction
Add Provider, Model, MaxTokens to PendingCompactionConfig so users can
override the LLM used for pending message summarization via the config
UI. Falls back to agent's provider/model when not set. Increase default
max_tokens from 512 to 4096. Add allowEmpty prop to ProviderModelSelect
to prevent auto-selecting first provider when empty means "use default".
2026-03-10 14:27:15 +07:00
viettranx 3b6bf645f3 feat(channels): pending message compaction — fix provider, wire auto-compact, add global config & UI
- Fix compact endpoint using random provider instead of agent's configured provider+model
- Wire auto-compaction for all 5 channel types (telegram, discord, slack, feishu, zalo_personal)
  via PendingCompactable interface and InstanceLoader
- Add global PendingCompactionConfig (threshold, keep_recent) to ChannelsConfig
- Wire global config through InstanceLoader and PendingMessagesHandler
- Increase compaction timeout from 45s to 180s for slow providers
- Add pending compaction config card to Behavior tab in config page
- Add HowItWorksCard (expanded by default) and toast notifications to pending messages page
- Add i18n support for all new strings (en/vi/zh)
2026-03-10 12:49:12 +07:00
SpencerSwagger 885814ddd1 fix: spawn tool not registered when providers configured via database (#103) 2026-03-10 09:54:27 +07:00
viettranx bdb60de7ae chore: upgrade Go 1.25 → 1.26 and apply go fix modernizations
- Update go.mod and Dockerfile to Go 1.26
- Apply `go fix ./...` stdlib modernizations across 170+ files
- Add `go fix` to post-implementation checklist in CLAUDE.md
- Fix go fix misapplied rewrite in loop_history.go
2026-03-10 00:09:15 +07:00
viettranx e593b9cf22 feat(channels): real-time agent activity status & intent classification
- Add tool status display on channels during tool execution (streaming preview + reactions)
- Emit agent.activity events at phase transitions (thinking, tool_exec, compacting)
- Enrich delegation progress with per-member activity and tool info
- Add LLM-based intent classifier for DM status queries when agent is busy
  - Keyword fast-path for cancel/status patterns (no LLM cost)
  - Falls back to LLM classification with 5s timeout
  - Supports status_query (immediate reply) and cancel (abort run) intents
- Register/unregister runs in makeSchedulerRunFunc for channel inbound tracking
- Add sessionRuns secondary index in Router for O(1) IsSessionBusy lookups
- Add intent_classify config toggle (global default + per-agent override)
- Add tool_status config toggle for channel tool status display
- Add i18n keys and translations (en/vi/zh) for status messages
- Add web UI config toggles for intent_classify and tool_status
2026-03-09 23:58:56 +07:00
viettranx 63eff188ad feat(kg): add knowledge graph with LLM extraction, traversal, and graph visualization
- KnowledgeGraphStore interface + PostgreSQL implementation (recursive CTE traversal, 5s timeout)
- LLM entity extraction pipeline triggered on memory writes (background goroutine)
- knowledge_graph_search agent tool with search + traversal modes
- HTTP API: CRUD entities, traverse, extract, stats, graph endpoints
- Web UI: KG tab on memory page with table/graph toggle, entity detail, manual extraction
- Force-directed graph visualization using @xyflow/react + d3-force
- Builtin tool seed with configurable provider/model/confidence settings
2026-03-09 17:11:20 +07:00
Nam Nguyen Ngoc 11bed0cc01 fix(mcp-bridge): per-session security context + media forwarding (#91)
* fix(mcp-bridge): add per-session agent context and HMAC verification

- Add per-session MCP config with X-Agent-ID/X-User-ID headers instead
  of shared global config file
- Sign bridge context headers with HMAC-SHA256 to prevent forgery
- Add bridgeContextMiddleware to verify signatures on MCP bridge requests
- Store MCP configs in ~/.goclaw/mcp-configs/ outside agent workDir
- Use atomic writes (tmp + rename) for MCP config files
- Fix provider rename leaving ghost registry entries
- Remove provider_type from mutable fields on update
- Tighten temp dir permissions from 0755 to 0700

* feat(mcp-bridge): propagate channel routing context through MCP bridge

- Pass channel, chat_id, and peer_kind from agent loop to CLI provider options
- Inject X-Channel, X-Chat-ID, X-Peer-Kind headers in bridge context middleware
- Add BridgeContext struct to bundle per-call context for MCP config generation
- Include channel routing headers in per-session MCP config files
- Expose "message" tool via MCP bridge for cross-channel messaging
- Add extract helpers for new option keys in claude_cli_session.go

* feat(mcp-bridge): forward media attachments to outbound message bus

- Wire MessageBus into gateway server and MCP bridge handler
- Publish tool result media files to outbound bus for channel delivery
- Extract channel/chatID/peerKind from tool context for proper routing
- Add mimeFromExt helper for content-type detection on attachments

* feat(mcp-bridge): inject per-agent DB-backed MCP servers into Claude CLI config

- Add MCPServerLookup type to resolve agent-specific MCP servers from DB
- Wire MCPServerStore through provider registration and HTTP handler
- Extract mcpServerEntryToConfig helper to deduplicate transport config logic
- Add JSON-to-Go helpers (jsonToStringSlice, jsonToStringMap) for DB fields
- Merge per-agent MCP servers at config write time without overriding static entries

* fix(mcp-bridge): use Media struct fields and prefer explicit MimeType

- Map Media.Path to attachment URL instead of treating Media as string
- Use Media.MimeType when available, fall back to extension-based detection

* refactor(providers): deduplicate option extractors and extract bridge media forwarding

- Replace per-field extractors (extractSessionKey, extractAgentID, etc.) with generic extractStringOpt/extractBoolOpt
- Add bridgeContextFromOpts helper to build BridgeContext in one call
- Extract forwardMediaToOutbound from inline block in makeToolHandler
- Change NewBridgeServer msgBus param from variadic to explicit pointer

* fix(providers): validate provider_type on update instead of silently dropping it

- Add explicit validation against ValidProviderTypes with 400 response
- Remove silent delete(updates, "provider_type") that hid invalid values
- Caller now receives clear error when submitting unsupported provider_type

* fix(providers): add header injection validation to MCP bridge headers

- Extend CRLF/null-byte checks to agentID, channel, chatID, and peerKind
- Previously only userID had header injection prevention
- Prevents HTTP header injection via crafted values in MCP config

* fix(mcp-bridge): sign all context fields in HMAC and remove legacy code

- Sign all 5 bridge context fields (agentID|userID|channel|chatID|peerKind)
  in HMAC instead of only agentID|userID to prevent channel routing forgery
- Propagate context.Context into MCPServerLookup to respect request
  cancellation instead of using context.Background()
- Remove legacy BuildCLIMCPConfig, WithClaudeCLIMCPConfig, mcpConfigPath,
  and mcpCleanup (dead code since system is PG-only)
- Use mime.TypeByExtension before custom fallback in mimeFromExt
- Add debug log when media forwarding is skipped due to missing context
- Add thread-safety comment to SetMCPServerLookup

---------

Co-authored-by: Nam Nguyen Ngoc <namnn.0911@gmail.com>
Co-authored-by: viettranx <viettranx@gmail.com>
2026-03-09 15:23:56 +07:00
viettranx b284f963f5 feat(memory): add memory management page with CRUD, search, and indexing
Add full-stack memory document management:
- Backend: extend MemoryStore with admin queries (ListAllDocumentsGlobal,
  GetDocumentDetail, ListChunks), HTTP handler with auth middleware
- Frontend: memory page with agent/scope filters, document table with
  pagination, view/edit dialog with content and chunks tabs, create dialog
  with scope selection, semantic search dialog
- UI fixes: reduce input/textarea focus ring width, prevent ring clipping
  in dialog scroll containers, widen memory dialogs on desktop
2026-03-09 14:54:41 +07:00
viettranx 5f7ca84876 feat(channels): persist pending messages to PostgreSQL with Web UI
- Add channel_pending_messages table with UUID v7 PK, sender_id tracking
- Implement PendingMessageStore interface with batched flush (3s/20 msgs)
- Add LLM-based auto-compaction when entries exceed threshold (50)
- Wire persistent history into all channel factories (Telegram, Discord, Slack, Feishu, Zalo)
- Extract channel type constants (TypeTelegram, TypeDiscord, etc.) to eliminate magic strings
- Add HTTP API endpoints for pending messages management (list, view, compact, clear)
- Add Pending Messages dashboard page with group titles resolved from session metadata
- Track sender_id across entire pipeline (migration → store → history → handlers)
2026-03-09 12:39:43 +07:00
viettranx 967f7ae46f refactor: split gateway, consumer, onboard, and agent loop into smaller files
Extract helper functions and move existing functions to dedicated files:
- cmd/gateway.go → gateway_channels_setup.go (channel registration, RPC wiring, event subscribers)
- cmd/gateway_consumer.go → gateway_consumer_helpers.go, gateway_consumer_process.go
- cmd/gateway_managed.go → gateway_http_handlers.go (wireHTTP)
- cmd/onboard.go → onboard_resolve.go (API key resolution helpers)
- internal/agent/loop.go → loop_run.go (Run entry point)

No logic changes — only code movement between files within the same package.
2026-03-09 10:49:58 +07:00
viettranx 3c03d1f482 feat(cron): add detail page, running status, realtime events, and pagination
- Add /cron/:id detail page with job info, payload, run history
- Make cron.run async: respond immediately, execute in background
- Set last_status="running" before execution, emit CronEvent via WS
- Add CronEvent (running/completed/error) broadcast to all WS clients
- Add server-side pagination to GetRunLog (offset + total count)
- Show loading spinner on Run button when job is running (list + detail)
- Enrich CronRunLogEntry with duration, input/output tokens
- Make job names clickable in overview card → /cron/:id
- Fix refresh button animation using isFetching instead of isPending
2026-03-09 09:44:57 +07:00
Duc Nguyen e05a4018c9 fix: use platform type instead of instance name in system prompt + Zalo group routing (#90)
* fix(agent): use ChannelType in system prompt for proper channel context

The system prompt was using the channel instance name (e.g. "zep-lao") instead
of the platform type (e.g. "zalo_personal"), causing the LLM to not understand
which messaging platform it's running on. This led to context confusion where
the bot would ask users which channel to send to instead of using the current one.

Changes:
- Add ChannelType field to RunRequest and SystemPromptConfig
- Thread channel type from consumer/cron → agent loop → system prompt
- Add WithToolChannelType/ToolChannelTypeFromCtx for tool context
- Register channel types for both config-based and DB-loaded instances
- Fix Zalo group thread type detection with approvedGroups cache
- Update cron handler to resolve channel type for cron-triggered runs

* refactor(channels): add Type() to Channel interface, remove channelTypes map

Move channel type from a separate map in Manager to the Channel interface
itself. BaseChannel.Type() falls back to Name() for config-based channels
where name == type. Extracts resolveChannelType helper to DRY up 6
repeated resolution blocks across consumer and cron handlers.

* feat(zalo): add pending group history for conversation context

Zalo personal groups now record non-@mentioned messages in a ring buffer
(default 50, configurable via history_limit). When the bot IS mentioned,
pending history is flushed as context — matching Telegram/Discord/Feishu.

Separated mention gating from policy gating in checkGroupPolicy for
cleaner control flow.
2026-03-09 08:30:45 +07:00
SpencerSwagger 3f850bc713 fix(docker): remove gateway config check, unnecessary anymore (#88) 2026-03-09 07:09:15 +07:00
Duc Nguyen 137a986d4f feat(channels): add Slack channel (#83)
* feat(channels): add Slack channel via Socket Mode (#37)

Implement Slack integration using Socket Mode (xapp-/xoxb- tokens):
- Event-driven messaging via app_mention + message events
- Policy checks: open, pairing, allowlist, disabled (DM + group)
- Thread participation with configurable TTL
- Markdown-to-mrkdwn formatting pipeline
- Streaming support (edit-in-place + native ChatStreamer)
- SSRF-protected file downloads
- Debounce, dedup, reactions, group history context
- 170 unit tests (format, helpers, stream, SSRF)

Fix BaseChannel.HandleMessage allowlist to also check chatID,
enabling group allowlist with channel IDs across all channels.

Closes #37

* feat(slack): add file/media support and edit-to-mention handling

- Wire inbound file download into handleMessage (images, audio, documents)
- Add media.go with resolveMedia, classifyMime, buildMediaTags
- Extract shared ExtractDocumentContent to channels/media_utils.go (DRY with Telegram)
- Support file_share and message_changed subtypes
- Handle edit-to-mention: respond when user edits old message to add @bot
- Add MediaMaxBytes config field (default 20MB)
- Fix debounce media accumulation (was silently dropping files)
- Add 60s HTTP client timeout on file downloads
- Refactor downloadFile signature for slack.File compatibility
2026-03-09 07:02:37 +07:00
viettranx 6345df3136 feat(tools): add create_audio tool for music and sound effects
New tool supporting MiniMax music generation, ElevenLabs sound effects,
and Suno music (stub). Registers Suno as a valid provider type.

- create_audio.go: tool definition with provider chain for music, direct ElevenLabs for SFX
- create_audio_minimax.go: MiniMax music API (/music_generation)
- create_audio_elevenlabs.go: ElevenLabs sound effects API (/v1/sound-generation)
- create_audio_suno.go: stub for future Suno integration
- Add ProviderSuno type to store and gateway registration
2026-03-08 20:10:00 +07:00
viettranx 5536313335 feat(tools): add use_skill marker tool for skill activation observability
Add a no-op use_skill tool that generates tool.call/tool.result events
in tracing spans and realtime, making skill activations visible in
observability. The actual skill loading still happens via read_file.

Web UI renders use_skill events with a distinct Zap icon and skill name
instead of the generic wrench icon.
2026-03-08 16:45:26 +07:00
viettranx 62c28e1b3e feat(channels): unified media pipeline for Discord, Feishu, and WebSocket
Extract shared media utilities (MediaInfo, BuildMediaTags, TranscribeAudio,
DetectMIMEType) into internal/channels/media/ and refactor Telegram to use
them. Add full inbound/outbound media support to Discord and Feishu channels
(STT transcription, document extraction, media tags, voice agent routing).
Add WebSocket media upload/serve endpoints and MIME-aware media tags in
chat.send. Split large channel files for maintainability.
2026-03-08 16:39:46 +07:00