Commit Graph

19 Commits

Author SHA1 Message Date
Luan Vu a7f5acc1e3 fix(security): harden SQL injection, MCP prompt injection, sandbox fallback, and file serving (#246)
- execMapUpdate: validate column names with strict regex to prevent SQL injection
- HTTP update handlers: add field allowlists (agents, providers, custom_tools, mcp, channel_instances)
- pqStringArray: properly escape array elements to prevent PostgreSQL array literal injection
- scanStringArray: handle quoted elements in PostgreSQL array format
- MCP bridge: wrap tool results as external/untrusted content to prevent prompt injection
- File serving: block access to sensitive system directories (/etc, /proc, /sys, etc.)
- Sandbox: fail closed when Docker unavailable instead of silent fallback to host
- Shell deny: fix base64 --decode bypass, add host exec 1MB output limit
- ILIKE queries: escape % and _ wildcards in knowledge_graph, custom_tools, channel_instances

Co-authored-by: Luvu182 <208665161+Luvu182@users.noreply.github.com>
2026-03-18 07:42:38 +07:00
viettranx 843b550651 feat: runtime packages UI, pkg-helper, configurable shell deny groups (#244)
Runtime package management with security hardening:

- pkg-helper: root-privileged daemon for apk install/uninstall via Unix socket
- HTTP API: /v1/packages (list/install/uninstall/runtimes), admin role required for writes
- Shell deny groups: 15 configurable groups (per-agent overrides via context)
- Packages UI: Web page for managing system/pip/npm packages with confirmation dialogs
- Docker: privilege separation (root entrypoint → su-exec drop), init for zombie reaping
- Security: umask socket creation, persist file validation, deny pattern hardening
  (Node.js fetch/http, Python from/import, curl localhost, sensitive env vars)
- Auth: empty gateway token → admin role (dev/single-user mode)
2026-03-17 19:50:26 +07:00
teexiii 99dd363b13 feat(mcp): lazy-activate deferred tools on direct call in search mode (#235)
* feat: Implement MCP manager for server connections, tool registration, and deferred tool loading for agents.

* feat: Add tests for deferred tool activation logic within the tool registry and agent loop.

* fix(mcp): prevent deny list bypass via lazy activation + fix idempotency race

- Add PolicyEngine.IsDenied() to check deny patterns (incl. group: expansion)
  before allowing lazily-activated deferred tools to execute
- Check IsDenied() in both single-tool and parallel execution paths in loop.go
- Make ActivateToolIfDeferred idempotent by checking activatedTools before
  returning false, preventing concurrent goroutines from being blocked
- Add tests for deny-on-first-call, group deny patterns, and idempotent
  concurrent activation

---------

Co-authored-by: viettranx <viettranx@gmail.com>
2026-03-17 13:18:27 +07:00
viettranx 3f2b6e258e chore(teams): remove deprecated delegation tools
Remove delegate_search, evaluate_loop, handoff from:
- Seed data, system prompt, i18n keys/catalogs, channel events
- Consumer handler (handleHandoffAnnounce), handoff route lookup
- HandoffRouteData struct + PG implementation
- Protocol events, MCP bridge comment
- Web UI locale files (en/vi/zh)
2026-03-16 22:46:18 +07:00
viettranx 27fb900510 refactor(tools): remove workspace_read/workspace_write, use file tools for team workspace
Remove dedicated workspace tools in favor of making existing file tools
(read_file, write_file, list_files, edit) team-workspace-aware.

- Delete workspace_tool_read.go and workspace_tool_write.go
- Clean up workspace_dir.go: export WorkspaceDir, remove dead code
  (workspaceRelPath, sanitizeFilePath, inferMimeType, templates, etc.)
- Remove workspace tool registration from gateway_managed.go
- Remove workspace tool references from policy, subagent, MCP bridge
- Add PathAllowable/PathDenyable to types.go for interface abstraction
2026-03-16 20:05:26 +07:00
Viet Tran 1a42dc93a6 feat(teams): team system v2 with bug fixes, workspace scope, versioning, and prompt optimization (#183)
* feat(workspace): add team shared workspace for file collaboration

- Add workspace_write and workspace_read tools for agents to share files across team members
- Create team_workspaces DB table with migration 000017 (file metadata, pinning, tags)
- Implement PostgreSQL store layer for workspace CRUD operations
- Add RPC handlers for workspace list/read/delete from web UI
- Build React workspace tab with file listing, content preview, and delete
- Propagate workspace channel/chatID scope through delegation chain
- Auto-allow workspace tools in agent tool policy when agent belongs to a team
- Inject team workspace guidance into system prompt for team agents
- Add /reset command handler for clearing session history
- Harden MCP bridge context middleware to reject headers when no gateway token
- Add i18n strings for workspace UI in en/vi/zh locales

* feat(teams): add comprehensive task management with followup reminders and recovery

- Add task followup/reminder system with auto-set on lead agent reply and auto-clear when user responds on channel
- Add task recovery ticker to re-dispatch stale/pending tasks periodically
- Add task scopes, filtering by status/channel/chatID, and task events
- Add WS RPC handlers for task CRUD, assignments, comments, events, and bulk operations (teams_tasks.go)
- Add task detail dialog, settings UI for followup config, and scope filtering in web dashboard
- Add migrations 000018 (team_tasks_v2) and 000019 (task_followup)
- Extend team_tasks_tool with await_reply, clear_followup actions
- Auto-complete/fail team tasks when delegate agent finishes
- Add workspace file listing and team tool manager enhancements

* docs(teams): add team system architecture and playbook ideas documentation

- Add TEAM_SYSTEM.md with full architecture design covering task management, shared workspace, and delegation engine subsystems
- Add TEAM_PLAYBOOK_IDEAS.md outlining future team coordination layers (playbook, member capabilities, auto-learned patterns)
- Document data models, status flows, tool actions, followup reminder system, task ticker, execution locking, and workspace scope model

* fix(teams): resolve 6 critical bugs in team task system

- Fix unblock SQL: check array_length after array_remove (not before)
- Enforce single-team leadership in team creation
- Add requireLead() for approve/reject tool actions
- Validate cross-team dependency references in blocked_by
- Add team_id to handoff route for multi-team isolation
- Set blocked_by DEFAULT '{}' to prevent NULL array issues

* refactor(workspace): use stable userID as scope key instead of connection UUID

Workspace scope changed from (team_id, channel, chat_id) to (team_id, userID).
Fixes workspace fragmentation across WS tab refreshes and reconnections.

* feat(teams): add V1/V2 versioning with feature gating and optimized prompts

- IsTeamV2() helper gates advanced features (locking, followup, review, audit)
- V2 tool actions rejected for V1 teams with clear error message
- Ticker, gateway consumer, delegation hooks respect version flag
- TEAM.md renders v1/v2 sections conditionally
- Tool descriptions and params optimized (~38% token reduction)
- UI: version toggle in settings, V2 Beta badge, conditional rendering
- i18n: version modal keys for en/vi/zh

* fix(migration): use VARCHAR(255) for user ID columns and add metadata JSONB

- assignee_user_id, user_id, actor_id: TEXT → VARCHAR(255)
- Add metadata JSONB to team_task_comments and team_task_attachments

---------

Co-authored-by: Nam Nguyen Ngoc <namnn.0911@gmail.com>
2026-03-13 22:41:32 +07:00
Thieu Nguyen 8ad580521d refactor: deprecate standalone mode, managed mode is now default (#126)
* refactor: remove managed/standalone mode distinction from codebase

Standalone mode is deprecated; managed mode is now the only mode.
Remove redundant "managed mode" qualifiers from comments, docs,
and error messages. Error strings now reference "database stores"
instead of "managed mode" for clarity.

* improve(onboard): streamline onboard process and env setup

Simplify onboard wizard, extract helpers to dedicated file,
update env example and entrypoint for default managed mode,
clean up prepare-env script, update i18n catalogs.
2026-03-11 07:27:38 +07:00
viettranx bdb60de7ae chore: upgrade Go 1.25 → 1.26 and apply go fix modernizations
- Update go.mod and Dockerfile to Go 1.26
- Apply `go fix ./...` stdlib modernizations across 170+ files
- Add `go fix` to post-implementation checklist in CLAUDE.md
- Fix go fix misapplied rewrite in loop_history.go
2026-03-10 00:09:15 +07:00
viettranx e327a880d4 feat(agent): improve system prompt structure for better LLM adherence
- Move SOUL.md + IDENTITY.md to primacy zone (top of prompt) with
  recency reinforcement at bottom to prevent persona drift in long chats
- Separate MCP tools into dedicated section with real descriptions
  (truncated to 200 chars) instead of generic "(custom tool)"
- Add MCP tool priority instruction: prefer MCP over core tools
- Raise MCP inline threshold from 30 to 40 tools
- Add knowledge_graph_search to Memory Recall section (conditional)
- Add recency reminders for memory_search and knowledge_graph_search
- Add knowledge_graph_search to coreToolSummaries
2026-03-09 18:33:12 +07:00
Nam Nguyen Ngoc 11bed0cc01 fix(mcp-bridge): per-session security context + media forwarding (#91)
* fix(mcp-bridge): add per-session agent context and HMAC verification

- Add per-session MCP config with X-Agent-ID/X-User-ID headers instead
  of shared global config file
- Sign bridge context headers with HMAC-SHA256 to prevent forgery
- Add bridgeContextMiddleware to verify signatures on MCP bridge requests
- Store MCP configs in ~/.goclaw/mcp-configs/ outside agent workDir
- Use atomic writes (tmp + rename) for MCP config files
- Fix provider rename leaving ghost registry entries
- Remove provider_type from mutable fields on update
- Tighten temp dir permissions from 0755 to 0700

* feat(mcp-bridge): propagate channel routing context through MCP bridge

- Pass channel, chat_id, and peer_kind from agent loop to CLI provider options
- Inject X-Channel, X-Chat-ID, X-Peer-Kind headers in bridge context middleware
- Add BridgeContext struct to bundle per-call context for MCP config generation
- Include channel routing headers in per-session MCP config files
- Expose "message" tool via MCP bridge for cross-channel messaging
- Add extract helpers for new option keys in claude_cli_session.go

* feat(mcp-bridge): forward media attachments to outbound message bus

- Wire MessageBus into gateway server and MCP bridge handler
- Publish tool result media files to outbound bus for channel delivery
- Extract channel/chatID/peerKind from tool context for proper routing
- Add mimeFromExt helper for content-type detection on attachments

* feat(mcp-bridge): inject per-agent DB-backed MCP servers into Claude CLI config

- Add MCPServerLookup type to resolve agent-specific MCP servers from DB
- Wire MCPServerStore through provider registration and HTTP handler
- Extract mcpServerEntryToConfig helper to deduplicate transport config logic
- Add JSON-to-Go helpers (jsonToStringSlice, jsonToStringMap) for DB fields
- Merge per-agent MCP servers at config write time without overriding static entries

* fix(mcp-bridge): use Media struct fields and prefer explicit MimeType

- Map Media.Path to attachment URL instead of treating Media as string
- Use Media.MimeType when available, fall back to extension-based detection

* refactor(providers): deduplicate option extractors and extract bridge media forwarding

- Replace per-field extractors (extractSessionKey, extractAgentID, etc.) with generic extractStringOpt/extractBoolOpt
- Add bridgeContextFromOpts helper to build BridgeContext in one call
- Extract forwardMediaToOutbound from inline block in makeToolHandler
- Change NewBridgeServer msgBus param from variadic to explicit pointer

* fix(providers): validate provider_type on update instead of silently dropping it

- Add explicit validation against ValidProviderTypes with 400 response
- Remove silent delete(updates, "provider_type") that hid invalid values
- Caller now receives clear error when submitting unsupported provider_type

* fix(providers): add header injection validation to MCP bridge headers

- Extend CRLF/null-byte checks to agentID, channel, chatID, and peerKind
- Previously only userID had header injection prevention
- Prevents HTTP header injection via crafted values in MCP config

* fix(mcp-bridge): sign all context fields in HMAC and remove legacy code

- Sign all 5 bridge context fields (agentID|userID|channel|chatID|peerKind)
  in HMAC instead of only agentID|userID to prevent channel routing forgery
- Propagate context.Context into MCPServerLookup to respect request
  cancellation instead of using context.Background()
- Remove legacy BuildCLIMCPConfig, WithClaudeCLIMCPConfig, mcpConfigPath,
  and mcpCleanup (dead code since system is PG-only)
- Use mime.TypeByExtension before custom fallback in mimeFromExt
- Add debug log when media forwarding is skipped due to missing context
- Add thread-safety comment to SetMCPServerLookup

---------

Co-authored-by: Nam Nguyen Ngoc <namnn.0911@gmail.com>
Co-authored-by: viettranx <viettranx@gmail.com>
2026-03-09 15:23:56 +07:00
viettranx 01e40a92b0 feat(mcp): add dynamic tool search and shared connection pool
MCP tool search mode: when an agent has >30 MCP tools, tools are
deferred and a single mcp_tool_search meta-tool is registered instead.
Uses BM25 index for keyword search, activating matched tools on demand.

Shared connection pool: introduces Pool that maintains one physical
connection per MCP server name, shared across all agents via refcount.
Eliminates N×M duplicate connections (N agents × M servers).
2026-03-07 18:52:37 +07:00
viettranx 0473b9a334 feat(mcp): add test connection, server tools endpoint, and grants UX
Backend:
- Add POST /v1/mcp/servers/test endpoint reusing DiscoverTools()
- Add GET /v1/mcp/servers/{id}/tools for discovered tool listing
- Add MCPToolLister interface and wire through gateway

Frontend:
- Add Test Connection button with success/error feedback in MCP form
- Replace agent ID text input with agent select dropdown in grants dialog
- Replace tool allow/deny text inputs with multi-select from server tools
- Add MCP tools dialog for viewing connected server tools
- Widen MCP dialogs to sm:max-w-xl
2026-03-07 14:55:28 +07:00
viettranx b2c4d543aa feat(providers): add Claude CLI provider with MCP bridge (#61)
Add Claude CLI as an LLM provider (subscription-based, no API key needed).
The CLI manages session history, tool execution, and context while GoClaw
forwards messages and streams responses.

Key features:
- Claude CLI provider with session persistence (--resume)
- MCP bridge server exposing GoClaw tools to CLI via streamable-http
- Security hooks (shell deny patterns, workspace path restrictions)
- Per-session mutex preventing concurrent CLI calls
- Onboard wizard for Claude CLI setup and auth verification
- Web UI for adding/managing Claude CLI provider with auth status
- Provider registry Close() for proper shutdown cleanup

Security:
- CLI path validation (only "claude" or absolute paths from DB)
- Token auth middleware for MCP bridge endpoint
- Shell injection prevention in hook scripts (single-quoted paths)
- Relative path resolution before workspace boundary checks
- Resource leak prevention on provider replace/unregister

Co-authored-by: nhokboo <nhokboo@users.noreply.github.com>
2026-03-07 02:06:39 +07:00
Viet Tran 6895e369f6 refactor: remove standalone mode, consolidate to managed-only (PostgreSQL) (#70)
- Remove standalone mode code: file-based stores, standalone gateway,
  heartbeat service, SQLite memory, standalone docker-compose
- Rename docker-compose.managed.yml → docker-compose.postgres.yml
- Clean up ~130 Go comments referencing "managed mode" qualifier
- Simplify docker-compose.yml env vars (providers/channels via web UI)
- Update .env.example to essential vars only (token + encryption key)
- Add setup wizard UI (provider → agent → channel bootstrap flow)
- Add logs.tail WebSocket handler for live log streaming
- Add cursor-pointer to interactive UI components
- Clean up config page (remove standalone-only sections)
- Update README and docs for managed-only architecture
2026-03-06 18:51:11 +07:00
viettranx 78abdec887 feat(mcp+skills): per-agent registry isolation, skill access filtering, managed skill lifecycle (#57)
Security: fix cross-agent MCP tool leak by cloning tool registry before MCP registration.

MCP: enforce mcp_ prefix on all tool names, add cache invalidation on server/grant changes,
add grant management endpoints, add group:mcp policy support for per-agent allowlisting.

Skills: persist full YAML frontmatter, auto-promote/demote visibility on grant/revoke,
simplify versioning, handle ZIP wrapper directories, expand tilde in skillsDir path.

Fixes: wrap DeleteSkill cascade in transaction, use atomic NOT EXISTS for revoke-demote,
create cancel context before storing server in map.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 23:14:31 +07:00
Thieu Nguyen 363838c309 fix(sandbox): rename from openclaw to goclaw for project name consistency (#35)
Update sandbox image name, container prefix, Docker labels, and MCP client info
from 'openclaw' to 'goclaw' to match the repository project name.

Changes:
- internal/sandbox/sandbox.go: image and container prefix
- internal/sandbox/docker.go: Docker label and fallback prefix
- docker-compose.sandbox.yml: image reference in comments and env
- internal/mcp/manager_connect.go: MCP client info name
2026-03-02 07:38:23 +07:00
viettranx 50a90aa8c6 refactor: split large Go files (>350 lines) into smaller same-package files
Pure file reorganization — no logic changes, no renames, no refactoring.
Functions moved to new files in the same package for maintainability.

Split 13 files across 6 packages into 25 new files:
- store/pg: teams.go → teams_tasks/delegation/messaging.go; mcp_servers.go → mcp_servers_access.go
- tools: delegate.go → delegate_state/policy/events.go; subagent.go → subagent_exec/config.go;
  web_search.go → web_search_brave/ddg.go; web_fetch.go → web_fetch_convert.go;
  sessions.go → sessions_history/send.go
- providers: anthropic.go → anthropic_stream/request.go
- mcp: manager.go → manager_connect/tools/util.go
- channels/feishu: bot.go → bot_parse/policy.go; larkclient.go → larkclient_messaging.go
- cmd: gateway_consumer.go → gateway_cron.go; agent_chat.go → agent_chat_client/standalone.go

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 17:27:28 +07:00
Michael 406f83160e fix(mcp): treat "method not found" ping responses as healthy (#1)
MCP servers that don't implement the `ping` method return a "method not
found" error. The health loop previously treated this as a failure,
triggering exponential backoff reconnects and making tools temporarily
unavailable.

Now uses case-insensitive matching to detect "method not found" responses
and treats the server as healthy (it did respond, just doesn't support
ping). Resets reconnect attempts and clears error state.
2026-02-27 16:02:53 +07:00
Viet Tran f3f4c67b36 Initial commit: GoClaw AI agent gateway
Multi-agent AI gateway with WebSocket RPC, HTTP API, and messaging channel integrations.
Go port of OpenClaw with multi-tenant PostgreSQL, per-user isolation, security hardening,
and production observability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 14:58:07 +07:00