goclaw/docs/20-api-keys-auth.md

20 — API Keys & Authentication

GoClaw supports two authentication mechanisms: a single gateway token (configured at startup) and multiple API keys with fine-grained RBAC scopes. Both work across HTTP REST and WebSocket RPC.

---

1. Gateway Token

The gateway token is set in config.json under gateway.token. It grants full admin access.

{
  "gateway": {
    "token": "my-secret-token"
  }
}

Used as a Bearer token:

Authorization: Bearer my-secret-token

Or in WebSocket connect:

{"method": "connect", "params": {"token": "my-secret-token", "user_id": "admin"}}

Security

The gateway token is compared using constant-time comparison (crypto/subtle.ConstantTimeCompare) in both HTTP (auth.go:tokenMatch) and WebSocket (router.go:handleConnect) to prevent timing attacks. The comparison reveals no information about where the provided token first differs from the expected token.

---

2. API Keys

API keys provide scoped, revocable access for CI/CD, integrations, and third-party clients.

Key Format

goclaw_a1b2c3d4e5f6789012345678901234567890abcdef

• Prefix: goclaw_ (6 chars)
• Random: 32 hex characters (128 bits of entropy)
• Display prefix: goclaw_ + first 8 hex chars (shown in UI after creation)

Security Model

Keys are hashed with SHA-256 before storage — the raw key is never persisted. This follows the same pattern as GitHub Personal Access Tokens.

raw key → SHA-256 → stored hash

On authentication, the incoming token is hashed and looked up in the api_keys table via a partial index on non-revoked keys.

Show-Once Pattern

The raw key is returned only once in the create response. All subsequent list/get calls show only the prefix field (8 hex characters representing the first 4 bytes of the random component). Users must copy the key immediately after creation, as it cannot be retrieved again.

---

3. RBAC Scopes

Each API key is assigned one or more scopes that determine what operations it can perform. Scopes are validated against the set of valid scopes at creation time.

Scope	Description
operator.admin	Full access — equivalent to gateway token, can manage API keys and SecureCLI configs
operator.read	Read-only access to agents, sessions, skills, chat history, and metadata
operator.write	Read + write access — can send chat messages, manage sessions, trigger cron jobs
operator.approvals	Manage shell command execution approvals (approve/deny exec requests)
operator.pairing	Manage browser device pairings (list, revoke paired devices)

Role Derivation

The highest-privilege scope determines the effective role via RoleFromScopes() in permissions/policy.go:

if admin scope present           → RoleAdmin
if write/approvals/pairing scope → RoleOperator
if read scope only               → RoleViewer
default                          → RoleViewer

The derived role is then used by the PolicyEngine.CanAccess() method to gate RPC method access (see 19 — WebSocket RPC).

---

4. Authentication Flow

Prioritized Auth Paths

GoClaw tries authentication methods in this priority order:

1. Gateway token (exact match via constant-time comparison) → RoleAdmin
2. API key (SHA-256 hash lookup in api_keys table) → role from scopes
3. Browser pairing (sender ID must be paired with "browser" device type) → RoleOperator (HTTP only; requires X-GoClaw-Sender-Id header)
4. No auth configured (backward compatibility: if no gateway token is set) → RoleOperator
5. No valid auth found → 401 Unauthorized

HTTP Request Flow

flowchart TD
    A[Incoming HTTP request] --> B{Authorization header?}
    B -->|No| C{X-GoClaw-Sender-Id header?}
    B -->|Yes, extract Bearer token| D{Match gateway token?}
    D -->|Yes| E[RoleAdmin]
    D -->|No| F[Hash token + lookup in api_keys]
    F --> G{Found + valid?}
    G -->|Yes| H[Derive role from scopes]
    G -->|No| I{Gateway token configured?}
    I -->|Yes| J[401 Unauthorized]
    I -->|No| K[RoleOperator - backward compat]
    C -->|Check paired device| L{Device paired?}
    L -->|Yes| M[RoleOperator]
    L -->|No| J
    E --> N[Authenticate request]
    H --> N
    K --> N
    M --> N

WebSocket Connect Flow

The same auth paths apply for WebSocket connect messages. The connection parameter token is checked against the gateway token first, then API keys, then browser pairing.

API Key Caching

API keys are cached in-memory with a 5-minute TTL to reduce database load. The cache:

• Maps SHA-256 hash → APIKeyData + derived role
• Supports negative caching (tracks failed lookups) to prevent repeated DB queries for invalid tokens
• Caps negative cache to 10,000 entries to prevent memory exhaustion from token spraying attacks
• Is invalidated globally via pubsub on key creation, revocation, or update

Last-Used Tracking

On successful API key authentication, last_used_at is updated asynchronously (fire-and-forget goroutine with 5-second timeout) to avoid blocking the request path. This tracks API usage without slowing down authentication.

---

5. Authentication Priority & Backward Compatibility

HTTP Request Headers

• Bearer token: Authorization: Bearer <token> — checked first for gateway token or API key
• User ID: X-GoClaw-User-Id: <user-id> — optional external user identifier (max 255 chars)
• Browser pairing: X-GoClaw-Sender-Id: <sender-id> — identifies a previously-paired browser device
• Locale: Accept-Language — user's preferred language (en, vi, zh; default: en)

Backward Compatibility

If no gateway token is configured (gateway.token is empty in config.json), all unauthenticated requests are granted RoleOperator access. This enables self-hosted deployments without strict authentication. Once a gateway token is configured, all requests must authenticate or use browser pairing.

---

5. Database Schema

CREATE TABLE api_keys (
    id            UUID PRIMARY KEY,
    name          VARCHAR(100) NOT NULL,
    prefix        VARCHAR(8)   NOT NULL,              -- first 8 chars for display
    key_hash      VARCHAR(64)  NOT NULL UNIQUE,       -- SHA-256 hex digest (unique constraint)
    scopes        TEXT[]       NOT NULL DEFAULT '{}', -- e.g. {'operator.admin','operator.read'}
    expires_at    TIMESTAMPTZ,                        -- NULL = never expires
    last_used_at  TIMESTAMPTZ,
    revoked       BOOLEAN      NOT NULL DEFAULT false,
    created_by    VARCHAR(255),                       -- user ID who created the key
    created_at    TIMESTAMPTZ  NOT NULL DEFAULT now(),
    updated_at    TIMESTAMPTZ  NOT NULL DEFAULT now()
);

-- Fast lookup by hash (only active, non-revoked keys)
CREATE INDEX idx_api_keys_key_hash ON api_keys (key_hash) WHERE NOT revoked;

-- Fast lookup by prefix (for display/search in web UI)
CREATE INDEX idx_api_keys_prefix ON api_keys (prefix);

The partial index on key_hash ensures only active (non-revoked) keys are searched during authentication, keeping lookups fast regardless of how many keys have been revoked over time. The key_hash column has a UNIQUE constraint, so only one key can have a given hash.

---

6. API Endpoints

HTTP REST

Method	Path	Description
GET	/v1/api-keys	List all keys (masked)
POST	/v1/api-keys	Create key
POST	/v1/api-keys/{id}/revoke	Revoke key

WebSocket RPC

Method	Description
api_keys.list	List all keys (masked)
api_keys.create	Create key
api_keys.revoke	Revoke key

All API key management operations require admin access (gateway token or API key with operator.admin scope).

Create Request

{
  "name": "ci-deploy",
  "scopes": ["operator.read", "operator.write"],
  "expires_in": 2592000
}

Field	Type	Required	Description
name	string	Yes	Human-readable label
scopes	string[]	Yes	Permission scopes
expires_in	int	No	TTL in seconds (omit for no expiry)

Create Response

{
  "id": "01961234-5678-7abc-def0-123456789012",
  "name": "ci-deploy",
  "prefix": "goclaw_a1b2c3d4",
  "key": "goclaw_a1b2c3d4e5f6789012345678901234567890abcdef",
  "scopes": ["operator.read", "operator.write"],
  "expires_at": "2026-04-14T12:00:00Z",
  "created_at": "2026-03-15T12:00:00Z"
}

List Response

[
  {
    "id": "01961234-...",
    "name": "ci-deploy",
    "prefix": "goclaw_a1b2c3d4",
    "scopes": ["operator.read", "operator.write"],
    "expires_at": "2026-04-14T12:00:00Z",
    "last_used_at": "2026-03-15T14:30:00Z",
    "revoked": false,
    "created_by": "admin",
    "created_at": "2026-03-15T12:00:00Z"
  }
]

Note: key field is absent in list responses. Only prefix is shown.

---

7. Backward Compatibility

The gateway token continues to work exactly as before. API keys are an additional authentication path — no breaking changes to existing integrations.

Auth method	Before	After
Gateway token	Admin access	Admin access (unchanged)
API key	N/A	Scoped access per key
Device pairing	Operator access	Operator access (unchanged)

---

8. SecureCLI — CLI Credential Injection

SecureCLI is a feature that allows GoClaw to automatically inject credentials into CLI tools (e.g., gh, gcloud, aws) without requiring the agent to handle plaintext secrets. Credentials are stored encrypted at rest and injected at process startup.

Use Case

When an agent needs to run gh auth, gcloud auth, or other authenticated CLI commands, the admin can configure a SecureCLI binary with encrypted environment variables. The agent never sees the raw credentials — they are injected directly into the child process environment via Direct Exec Mode.

Database Schema

CREATE TABLE secure_cli_binaries (
    id              UUID PRIMARY KEY,
    binary_name     TEXT NOT NULL,        -- "gh", "gcloud", "aws", etc.
    binary_path     TEXT,                 -- optional absolute path (auto-resolved if null)
    description     TEXT,
    encrypted_env   BYTEA NOT NULL,       -- AES-256-GCM encrypted JSON
    deny_args       JSONB DEFAULT '[]',   -- regex patterns to block subcommands
    deny_verbose    JSONB DEFAULT '[]',   -- verbose flag patterns
    timeout_seconds INTEGER DEFAULT 30,
    tips            TEXT,                 -- injected into TOOLS.md context for agents
    agent_id        UUID,                 -- null = global (all agents), else agent-specific
    enabled         BOOLEAN DEFAULT true,
    created_by      TEXT,
    created_at      TIMESTAMPTZ,
    updated_at      TIMESTAMPTZ
);

HTTP REST API

Method	Path	Description
GET	/v1/cli-credentials	List all SecureCLI configurations
POST	/v1/cli-credentials	Create a new SecureCLI credential config
GET	/v1/cli-credentials/{id}	Get a specific SecureCLI config
PUT	/v1/cli-credentials/{id}	Update a SecureCLI config
DELETE	/v1/cli-credentials/{id}	Delete a SecureCLI config
POST	/v1/cli-credentials/{id}/test	Dry-run test (requires admin)
GET	/v1/cli-credentials/presets	List preset templates for common CLIs

Features

• Agent-specific or global: Configs can be scoped to a single agent or shared across all agents (agent_id = null)
• Encryption: Environment variables are encrypted with AES-256-GCM (same as other secrets)
• Deny patterns: Regex patterns to block dangerous subcommands (e.g., prevent gh auth logout)
• Timeout: Per-config timeout override for long-running CLI operations
• Tips: Admin-provided hints injected into agent TOOLS.md context to guide which CLI commands are safe

Example Configuration

{
  "binary_name": "gh",
  "binary_path": "/usr/local/bin/gh",
  "description": "GitHub CLI for PRs and issues",
  "encrypted_env": {
    "GH_TOKEN": "ghp_xxxxxxxxxxxx"
  },
  "deny_args": ["auth", "logout"],
  "deny_verbose": ["--verbose", "-v"],
  "timeout_seconds": 60,
  "tips": "Use 'gh pr list' to search PRs, 'gh issue create' to open issues. Avoid 'auth' and 'logout' commands.",
  "agent_id": null
}

---

9. Web UI

The API Keys management page is accessible from the sidebar under System > API Keys (admin only).

Features:

• Create dialog: Name, scope checkboxes, optional expiry (1 day, 7 days, 30 days, 90 days, never)
• Show-once dialog: Displays the raw key with copy button after creation
• List view: Searchable, paginated table with name, prefix, scopes, expiry, last used, status
• Revoke: Confirmation dialog before revoking a key

---

10. Usage Examples

cURL with API Key

# List agents (read scope required)
curl -H "Authorization: Bearer goclaw_a1b2c3d4..." \
     http://localhost:9090/v1/agents

# Send chat message (write scope required)
curl -X POST -H "Authorization: Bearer goclaw_a1b2c3d4..." \
     -H "Content-Type: application/json" \
     -d '{"model":"goclaw:my-agent","messages":[{"role":"user","content":"Hello"}]}' \
     http://localhost:9090/v1/chat/completions

WebSocket with API Key

{"id": 1, "method": "connect", "params": {
  "token": "goclaw_a1b2c3d4e5f6...",
  "user_id": "ci-bot"
}}

Create Key via API

curl -X POST -H "Authorization: Bearer gateway-admin-token" \
     -H "Content-Type: application/json" \
     -d '{"name":"ci","scopes":["operator.read","operator.write"]}' \
     http://localhost:9090/v1/api-keys

---

11. File Reference

File	Purpose
internal/crypto/apikey.go	Key generation + SHA-256 hashing
internal/store/api_key_store.go	Store interface + APIKeyData struct
internal/store/secure_cli_store.go	SecureCLI store interface + SecureCLIBinary struct
internal/store/pg/api_keys.go	PostgreSQL API key implementation
internal/store/pg/secure_cli.go	PostgreSQL SecureCLI implementation
internal/http/api_keys.go	HTTP API handler for API keys
internal/http/secure_cli.go	HTTP API handler for SecureCLI
internal/http/auth.go	HTTP auth middleware (resolveAPIKey, tokenMatch)
internal/http/api_key_cache.go	In-memory API key cache with TTL + pubsub invalidation
internal/gateway/router.go	WebSocket connect auth (API key path)
internal/gateway/methods/api_keys.go	WebSocket RPC methods for API keys
internal/permissions/policy.go	RBAC policy engine + role derivation + scope validation
migrations/000020_secure_cli_and_api_keys.up.sql	Database migration (api_keys + secure_cli_binaries)
ui/web/src/pages/api-keys/	Web UI components