goclaw/docs/18-acp-provider.md

18 - ACP Provider (Agent Client Protocol)

The ACP provider enables GoClaw to orchestrate external coding agents (Claude Code, Codex CLI, Gemini CLI, Kiro, or any ACP-compatible agent) as subprocesses via JSON-RPC 2.0 over stdio. One provider covers all ACP agents through config-driven agent registry.

References: ACP Spec · ACP Schema · Issue #189 · PR #190

---

1. Architecture

flowchart TD
    AL["GoClaw Agent Loop"] -->|"Chat / ChatStream"| ACP["ACPProvider<br/>(acp_provider.go)"]
    ACP -->|"GetOrSpawn"| PP["ProcessPool<br/>(process.go)"]
    PP -->|"spawn binary"| PROC["Subprocess<br/>(stdin/stdout pipes)"]
    PROC <-->|"JSON-RPC 2.0<br/>newline-delimited"| CONN["Conn<br/>(jsonrpc.go)"]

    CONN -->|"initialize"| AGT["Agent<br/>(claude, codex, gemini...)"]
    CONN -->|"session/new"| AGT
    CONN -->|"session/prompt"| AGT
    CONN -->|"session/cancel"| AGT

    AGT -->|"fs/readTextFile"| TB["ToolBridge<br/>(tool_bridge.go)"]
    AGT -->|"fs/writeTextFile"| TB
    AGT -->|"terminal/*"| TERM["Terminal Registry<br/>(terminal.go)"]
    AGT -->|"permission/request"| TB

    TB -->|"workspace sandbox"| FS["Filesystem"]
    TERM -->|"deny patterns + allowlist"| CMD["Command Execution"]

Key principles:

• GoClaw is an ACP client — it spawns and controls agent subprocesses
• Each subprocess is a long-lived OS process communicating via stdin/stdout
• Security enforced at the tool bridge layer: workspace sandboxing, deny patterns, permission modes

---

2. Wire Protocol: JSON-RPC 2.0 over Stdio

Transport

Messages are newline-delimited JSON on stdin/stdout. Each message is a complete JSON object followed by \n. The Conn type (jsonrpc.go, 217 lines) handles bidirectional communication.

GoClaw (Client)                     Agent (Server)
     │                                    │
     │──── {"jsonrpc":"2.0","id":1,  ────►│  Request
     │      "method":"initialize",        │
     │      "params":{...}}               │
     │                                    │
     │◄─── {"jsonrpc":"2.0","id":1,  ─────│  Response
     │      "result":{...}}               │
     │                                    │
     │◄─── {"jsonrpc":"2.0",         ─────│  Notification (no id)
     │      "method":"session/update",    │
     │      "params":{...}}               │
     │                                    │
     │◄─── {"jsonrpc":"2.0","id":42, ─────│  Agent→Client Request
     │      "method":"fs/readTextFile",   │
     │      "params":{"path":"..."}}      │
     │                                    │
     │──── {"jsonrpc":"2.0","id":42, ────►│  Client→Agent Response
     │      "result":{"content":"..."}}   │

Message Format

type jsonrpcMessage struct {
    JSONRPC string          `json:"jsonrpc"`         // always "2.0"
    ID      *int64          `json:"id,omitempty"`    // present for requests/responses, absent for notifications
    Method  string          `json:"method,omitempty"`
    Params  json.RawMessage `json:"params,omitempty"`
    Result  json.RawMessage `json:"result,omitempty"`
    Error   *jsonrpcError   `json:"error,omitempty"`
}

Key Conn Methods

Method	Purpose
Call(ctx, method, params, &result)	Send request, block until response (with context timeout)
Notify(method, params)	Fire-and-forget notification
Start()	Spawn readLoop goroutine for incoming messages
Done()	Channel closed when read loop exits (process died)

Buffer sizing: Scanner uses 256KB initial / 10MB max per message — handles large file contents in tool bridge responses.

ID sequencing: Atomic Int64 counter, no lock contention.

---

3. Session Lifecycle

sequenceDiagram
    participant C as GoClaw (Client)
    participant A as Agent (Subprocess)

    C->>A: initialize {clientInfo, capabilities}
    A->>C: initialize response {agentInfo, capabilities}

    C->>A: session/new {}
    A->>C: session/new response {sessionId}

    loop Per user message
        C->>A: session/prompt {sessionId, content}
        A-->>C: session/update {kind:"message", content}
        A-->>C: session/update {kind:"message", content}
        A->>C: session/prompt response {stopReason}
    end

    Note over C,A: Optional: Cancel
    C-->>A: session/cancel {sessionId}

Phase 1: Initialize

Client declares capabilities (filesystem read/write, terminal support). Agent responds with identity and capabilities (audio, image, embedded context).

// Client sends:
InitializeRequest{
    ClientInfo: ClientInfo{Name: "goclaw", Version: "1.0"},
    Capabilities: ClientCaps{
        Fs:       &FsCaps{Read: true, Write: true},
        Terminal: &TerminalCaps{Create: true},
    },
}

Phase 2: New Session

Creates an isolated session on the agent. Returns sessionId used in all subsequent prompt calls.

Phase 3: Prompt Loop

Send user content blocks (text + images). Agent streams session/update notifications with message deltas, tool call progress, and plan updates. Prompt completes with a response containing stopReason.

Phase 4: Cancel (Optional)

Cooperative cancellation via session/cancel notification. Agent may take time to stop.

---

4. Content Handling

ContentBlock Types

type ContentBlock struct {
    Type     string `json:"type"`               // "text", "image", "audio"
    Text     string `json:"text,omitempty"`      // text content
    Data     string `json:"data,omitempty"`      // base64 for image/audio
    MimeType string `json:"mimeType,omitempty"`  // e.g. "image/png"
}

Request Extraction (GoClaw → Agent)

1. Extract system prompt + user message from ChatRequest.Messages
2. Prepend system prompt to first user message (ACP has no separate system message API)
3. Attach images as separate content blocks with base64 data

Response Collection (Agent → GoClaw)

1. Accumulate SessionUpdate notifications during prompt execution
2. Collect text blocks into response content string
3. Map stopReason to GoClaw finish reason:
   • "maxContextLength" → "length"
   • All others → "stop"

SessionUpdate Structure

type SessionUpdate struct {
    Kind    string         `json:"kind"`    // "message", "toolCall", "plan"
    Content []ContentBlock `json:"content,omitempty"`
    ToolCall *ToolCallUpdate `json:"toolCall,omitempty"`
}

type ToolCallUpdate struct {
    ID      string         `json:"id"`
    Name    string         `json:"name"`
    Status  string         `json:"status"`  // "running", "completed"
    Content []ContentBlock `json:"content,omitempty"`
}

---

5. Process Pool

ProcessPool (process.go, 237 lines) manages subprocess lifecycle.

Spawn Flow

GetOrSpawn(sessionKey)
  ├→ Check cached process (sync.Map)
  │   └→ Found + alive → return
  ├→ Acquire per-key spawn mutex (prevent thundering herd)
  └→ spawn():
      ├→ exec.Command(binary, args...)
      ├→ cmd.Env = filterACPEnv(os.Environ())  // strip secrets
      ├→ Create stdin/stdout pipes
      ├→ cmd.Stderr = limitedWriter(4KB)
      ├→ cmd.Start()
      ├→ NewConn(stdin, stdout, toolBridge.Handle, notifyHandler)
      ├→ conn.Start()  // begin readLoop
      ├→ Initialize()  // ACP handshake
      ├→ NewSession()  // create session
      ├→ Monitor exit in background goroutine
      └→ Store in pool

Idle Reaping

Every 30 seconds, the reaper checks all processes:

for each process in pool:
    if process.inUse > 0: skip          // active prompt running
    if time.Since(lastActive) > idleTTL:
        process.cmd.Process.Kill()       // SIGKILL
        remove from pool

Crash Recovery

If a process exits unexpectedly (detected via <-proc.exited channel), the next GetOrSpawn call automatically spawns a replacement. The active prompt is lost — caller receives an error.

Concurrency Controls

Mechanism	Purpose
sync.Map for processes	Lock-free concurrent access
Per-key spawn mutex	Prevent duplicate spawns for same session
inUse atomic flag	Reaper skips active processes
lastActive timestamp	Tracks idle time for reaping
Session-level mutex in ACPProvider	Serializes prompts per session

---

6. Tool Bridge (Agent → Client Requests)

ToolBridge (tool_bridge.go, 204 lines) handles all agent-initiated requests with security enforcement.

Request Routing

Method	Handler	Description
fs/readTextFile	readFile()	Read file within workspace
fs/writeTextFile	writeFile()	Write file within workspace
terminal/createTerminal	createTerminal()	Spawn command subprocess
terminal/terminalOutput	terminalOutput()	Get current output
terminal/waitForTerminalExit	waitForExit()	Block until exit (10-min timeout)
terminal/releaseTerminal	releaseTerminal()	Clean up resources
terminal/killTerminal	killTerminal()	Force-terminate
permission/request	handlePermission()	Permission check

Permission Modes

Mode	Reads	Writes	Terminal	Permission Requests
approve-all	✅	✅	✅	✅ (default)
approve-reads	✅	❌	❌	Per-type
deny-all	❌	❌	❌	❌

Workspace Sandbox

All file paths validated via resolvePath():

func resolvePath(path string) (string, error) {
    abs := filepath.Join(workspace, path)
    real, _ := filepath.EvalSymlinks(abs)  // resolve symlinks
    if !strings.HasPrefix(real, workspace) {
        slog.Warn("security.acp_path_escape", ...)
        return "", fmt.Errorf("path outside workspace")
    }
    return real, nil
}

Symlink resolution prevents ../../etc/passwd attacks even when symlinks point outside workspace.

---

7. Terminal System

Terminal (terminal.go, 212 lines) manages command execution within the tool bridge.

Security Layers

1. Binary Allowlist (63 binaries):

sh, bash, zsh, fish, node, npm, npx, pnpm, yarn, bun, deno,
python, python3, pip, pip3, uv, ruby, gem, go, cargo, rustc,
java, javac, mvn, gradle, dotnet, git, gh, docker, kubectl,
make, cmake, gcc, g++, clang, curl, wget, jq, yq, tar, zip,
unzip, gzip, cat, head, tail, less, grep, rg, find, ls, mv,
cp, mkdir, rm, chmod, touch, sed, awk, sort, wc, diff, tee

2. Deny Patterns: Regex patterns from GoClaw's DefaultDenyPatterns are applied to the full command string (binary + args).

3. Working Directory Sandbox: Terminal cwd validated against workspace boundary.

cappedBuffer

Thread-safe circular buffer that retains only the last N bytes (default 10MB):

type cappedBuffer struct {
    mu   sync.Mutex
    data []byte
    max  int
}
// On overflow: keeps tail (recent output), discards head

Used for both stdout and stderr capture. Prevents unbounded memory growth from verbose agent output.

---

8. Environment Filtering

Before spawning any agent subprocess, filterACPEnv() strips sensitive environment variables:

Prefix-based (12 prefixes):

GOCLAW_, CLAUDE_, ANTHROPIC_, OPENAI_, DATABASE_, AWS_,
GOOGLE_, AZURE_, GITHUB_, DOCKER_, STRIPE_, SSH_

Exact-match (15 keys):

DB_DSN, PGPASSWORD, PGUSER, PGHOST, PGDATABASE, PGPORT,
REDIS_URL, MONGO_URI, NPM_TOKEN, SENTRY_AUTH_TOKEN,
SENTRY_DSN, DATADOG_API_KEY, TWILIO_AUTH_TOKEN,
SENDGRID_API_KEY, SLACK_TOKEN

This prevents credential leakage to untrusted agent binaries.

---

9. Configuration

Config File (config.json)

{
  "providers": {
    "acp": {
      "binary": "claude",        // agent binary (must be in PATH)
      "args": ["--profile", "goclaw"],  // optional spawn args
      "model": "claude",         // default model name for routing
      "work_dir": "/workspace",  // base workspace directory
      "idle_ttl": "5m",          // process idle timeout
      "perm_mode": "approve-all" // "approve-all" | "approve-reads" | "deny-all"
    }
  }
}

Database Registration

Create via Providers API or Web UI:

Field	Value
provider_type	"acp"
api_base	Binary name or absolute path ("claude", "/usr/local/bin/codex")
settings	{"args": [...], "idle_ttl": "5m", "perm_mode": "approve-all", "work_dir": "..."}

Binary validation: Only claude, codex, gemini, or absolute paths are accepted for DB-based registration. Verified via exec.LookPath().

Gateway Wiring

// Config-based: resolved at startup
registerACPFromConfig(registry, cfg.Providers.ACP)

// DB-based: resolved from llm_providers table
registerACPFromDB(registry, providerData)

Both paths:

1. Verify binary exists via exec.LookPath
2. Parse IdleTTL duration
3. Resolve WorkDir (default: ~/.goclaw/acp-workspaces)
4. Create NewACPProvider(binary, args, workDir, idleTTL, denyPatterns, opts...)

Live Reload

DB-based providers support live reload via pubsub. When a provider is created/updated/deleted in the Web UI, a cache.invalidate event triggers re-registration without gateway restart.

---

10. Streaming vs Non-Streaming

Chat (Non-Streaming)

func (p *ACPProvider) Chat(ctx, req) → *ChatResponse

1. Lock session mutex
2. GetOrSpawn process
3. Prompt(content, onUpdate) — blocks until complete
4. Collect all text deltas into strings.Builder
5. Return ChatResponse{Content: text, FinishReason: mapped}

ChatStream

func (p *ACPProvider) ChatStream(ctx, req, onChunk) → *ChatResponse

1. Lock session mutex
2. Set up cancel listener (session/cancel on context cancellation)
3. GetOrSpawn process
4. Prompt(content, onUpdate) with callback:
   • Extract text blocks from each SessionUpdate
   • Emit StreamChunk{Content: delta} via onChunk
5. On completion: emit StreamChunk{Done: true}
6. Return accumulated ChatResponse

---

11. Error Handling

Scenario	Behavior
Binary not found	Log warning, skip provider registration
Subprocess crash mid-prompt	Active prompt fails; next GetOrSpawn respawns
Malformed JSON-RPC	Log debug, skip message, continue reading
Path escape attempt	Log security.acp_path_escape, return error to agent
Terminal binary not in allowlist	Return error to agent
Terminal deny pattern match	Return error to agent
Context cancelled (ChatStream)	Send session/cancel, return partial response
Idle timeout	Reaper kills process; respawned on next request
Permission denied	Return error based on perm_mode
Large output (>10MB terminal)	cappedBuffer retains tail only

---

12. File Reference

File	Lines	Purpose
internal/providers/acp_provider.go	227	Provider interface: Chat, ChatStream, content extraction
internal/providers/acp/types.go	189	ACP protocol types: Initialize, Session, ContentBlock
internal/providers/acp/jsonrpc.go	217	Bidirectional JSON-RPC 2.0 over stdio
internal/providers/acp/process.go	237	Subprocess pool: spawn, reap, crash recovery
internal/providers/acp/session.go	71	Session lifecycle: init → new → prompt → cancel
internal/providers/acp/tool_bridge.go	204	Agent→client request handler with sandbox
internal/providers/acp/terminal.go	212	Terminal subprocess lifecycle + cappedBuffer
internal/providers/acp/helpers.go	81	Environment filtering, limitedWriter
internal/config/config_channels.go	—	ACPConfig struct definition
internal/store/provider_store.go	—	ProviderACP = "acp" constant
cmd/gateway_providers.go	—	Config + DB registration wiring

---

Cross-References

Document	Relevant Content
02-providers.md	ACP overview section (§10)
03-tools-system.md	Shell deny patterns reused by ToolBridge
09-security.md	Defense-in-depth layers
01-agent-loop.md	Chat/ChatStream provider contract