mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-09-04 00:17:17 +00:00
dev
23
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Goon
|
d64a31ebdb | fix(usage): enforce caps on auxiliary llm calls | ||
|
Duy Nguyen
|
5a189330f9 | fix(test): support windows test execution | ||
|
Duy /zuey/andGitHub
|
4472c607b8 |
feat(workstation): Remote Workstation Runtime — SSH exec + security + audit (#4)
* feat(packages): add update flow for GitHub binaries (#900) Closes #900. Proactive update-check + atomic swap for GitHub-installed binaries on the Runtime & Packages page. Interfaces prepared for pip/npm/apk extension in Phase 2. - UpdateCache + UpdateRegistry + PackageLocker (ctx-aware keyed mutex) - GitHubUpdateChecker: ETag-aware, distinct /latest vs /list ETag keys, semver-correct ordering via golang.org/x/mod/semver, non-semver fallback that refuses to downgrade, pre-release + stable candidate fusion for the v1.0.0-rc.1 -> v1.0.0 transition - GitHubUpdateExecutor: two-phase .bak swap with hadBackup-aware rollback, manifest save retry (3x, 100ms/500ms/1s backoff), nil-safe meta access, explicit ScratchDir, 0755 set pre-rename - HTTP: GET /v1/packages/updates (SWR), POST /v1/packages/updates/refresh, POST /v1/packages/update, POST /v1/packages/updates/apply-all (always 200, failed[] is error source). Master-scope gated. - WS events package.update.{checked,started,succeeded,failed} forwarded to owner clients via event_filter.go - Frontend: useUpdates hook + 3 components (summary bar, update-all modal, row button), master-scope-gated disabled state - i18n: 8 backend keys + 17 frontend keys x en/vi/zh - Config: packages.github_token (reserved), updates_check_ttl, scratch_dir - 45+ new tests, race-clean, BenchmarkCheckAll10Packages ~1.1ms/op warm * docs(packages): document update flow + Phase 1 completion - packages-github.md: "Updating Installed Packages" section with UI + API contract, troubleshooting runbook (corrupt cache, rate-limit, scratch dir, mid-swap recovery) - 17-changelog.md + CHANGELOG.md: Phase 1 entry - 14-skills-runtime.md: cross-ref to update flow - journal entry capturing CRIT fixes (double-write, lock-key mismatch, rollback false-alarm) + design wins (keyed locks, red-team pre-flight) * feat(workstation): remote workstation runtime — SSH exec + security + audit Adds generic Remote Workstation Runtime enabling agents to execute commands on user-owned SSH workstations. Includes registry (DB + API + UI), SSH backend with connection pool and circuit breaker, workstation.exec + claude_remote tools, NFKC + binary-name allowlist security, and audit logging. Standard edition only. Closes #941. * fix(workstation): address 3 critical + 5 important code review findings - C1: Add json:"-" to Metadata/DefaultEnv fields; use SanitizedView() in all API responses to prevent SSH private key leakage - C2: Wire CheckEnv into PermCheckFn; LD_PRELOAD/PATH injection now blocked - C3: SSH Setenv fallback — prepend `export K=V;` when server rejects Setenv - I1: BackendCache sync.RWMutex → sync.Mutex (fix data race on lastUsed) - I2: Validate metadata shape in handleUpdate before store write - I3: Include command in exec-done event; activity sink uses actual cmd hash - I4: Wrap pool release in sync.Once (idempotent double-call safety) - I5: Verify workstation tenant ownership before adding permissions * fix(packages): bypass HTTPS+IP validation in update executor tests Test httptest servers bind to http://127.0.0.1 which fails both the HTTPS scheme check and literal-IP SSRF guard. Add testSkipDownloadValidation flag (same pattern as existing withTestDownloadHosts) to skip full URL validation in test context. * fix(workstation): address Claude review findings — tenant isolation + pool leak + dead code - Activity list: add workstation ownership check before listing (prevents cross-tenant activity enumeration via known UUID) - SSH pool: clean up p.sem + p.circuits maps in CloseWorkstation, prune, and Close to prevent unbounded map growth - RPC handlers: return ErrInvalidRequest on JSON unmarshal failure instead of silently using zero-value params - Remove unused containsControlChars function in normalize.go - HTTP tests: add 10s context timeout to prevent CI package timeout * fix(workstation): DefaultEnv JSON parse, backend cache leak, perm ownership check - DefaultEnv: replace KEY=VALUE text parse with json.Unmarshal (stored as JSON by HTTP handler, was silently ignored) - BackendCache: close losing backend on concurrent cache miss to prevent pruneLoop goroutine leak - Backend interface: add Close() error method; SSHBackend delegates to pool.Close() - handlePermList: add wsStore.GetByID ownership check (prevents cross-tenant UUID enumeration returning empty array vs 404) - scanRows: log scan errors instead of silently skipping * fix(workstation): wire activity sink shutdown + remove misleading comment - WireActivitySink: capture cleanup func, register in gateway shutdown (was discarded → retention goroutine leaked + buffered rows lost) - Add Stop() to WorkstationActivityStore interface (PG+SQLite already had it) - wireWorkstationTools returns cleanup func; gateway.go defers it - Remove misleading "re-validate env" comment in allowlist.go Check() * ci: bump unit test timeout from 90s to 120s hooks/handlers package (goja script tests) consumes ~85s on cold CI runners, leaving insufficient headroom for HTTP retry tests with 1s backoff. 120s provides adequate breathing room without masking real deadlocks. * fix: compile errors in integration tests + allowlist docstring - packages_update_test: add missing lockKey arg to registry.Apply - mcp_grant_revoke_test: remove unused fakeMCPClient struct - allowlist.go: fix Check() docstring to match actual 3-step pipeline * fix(test): relax mcp grant revoke assertion for pre-Phase02 state Execute-time grant checking not yet wired — test correctly gets an error but the message is "no active client" (nil clientPtr) rather than "grant revoked". Accept any error as valid regression guard. * chore: trigger CI on digitopvn/goclaw fork * ci: retrigger workflows * fix(permissions): classify workstation methods in RBAC policy |
||
|
viettranx
|
b676ee1351 |
refactor: adopt Go 1.26+ standard library modernizations
- Use maps.Copy in hooks dispatcher instead of manual map loop - Remove implicit loop variable capture in router_abort_test (Go 1.26 semantics) - Use range without index in http_test where index unused - Use min() builtin in script_test instead of manual min computation |
||
|
|
4b02c27307 |
feat: native image_generation for Codex + OpenAI-compat providers (#1002)
* refactor(providers): migrate ToolDefinition.Function to pointer + add image response fields
ToolDefinition.Function becomes *ToolFunctionSchema with omitempty so native tool types (image_generation, web_search, etc.) can be declared without a function body. All 9 internal construction sites updated. CleanToolSchemas refactored — function-shape cleaning extracted into cleanFunctionSchema helper, outer pass-through handles native tools.
Added image response fields needed by the next commits: ChatResponse.Images, StreamChunk.Images, ImageContent.Partial (distinguishes partial frames from final images).
* feat(providers): native image_generation for Codex + OpenAI-compat tracks
Codex native (POST /codex/responses): emit image_generation tool object in request tools[] (type, action, model, output_format, partial_images). Handle SSE events response.image_generation_call.partial_image + response.output_item.done (type image_generation_call) + response.completed output[] walk for non-stream. Dedup per item_id. Extend codexSSEEvent/codexItem with output_format, result, partial_image_b64, partial_image_index.
OpenAI-compat (/v1/chat/completions): serialize ToolDefinition{Type:'image_generation'} as {type:'image_generation'} pass-through. Parse choices[0].message.images[] + delta.images[] (data URLs) via new parseDataURL helper; append to ChatResponse.Images.
ProviderCapabilities.ImageGeneration flag; Codex provider + adapter set true. Other providers default false.
* feat(agent,http,store): persist assistant images + tri-level image_generation gate
Agent loop tri-level gate: (provider capability) AND (AgentConfig.AllowImageGeneration, default true, stored in other_config.allow_image_generation) AND (request lacks x-goclaw-no-image-gen header). Gate in loop_tool_filter.go appends ToolDefinition{Type:'image_generation'} only when all three pass. Per-request opt-out parsed in chat_completions.go and propagated via RunRequest.NoImageGen.
Media persistence: persistAssistantImages writes final images (Partial:false) to {workspace}/media/{sha256}.{ext}, returns MediaRef entries, clears inline Images[] from the message. Idempotent on hash, traversal-safe, symlink-guarded. Invoked from pipeline.FinalizeStage via new Deps.PersistAssistantImages callback — covers both stream-final and non-stream paths.
Agent store reads AllowImageGeneration from other_config JSONB with absent/nil = true default (matches V3Flags pattern). No DB migration — code-only default.
* feat(ui/web): image_generation toggle + streaming placeholder + download filename
Composer chip 'Images' visible only when active agent's provider has ImageGeneration capability. Per-agent localStorage persistence via useImageGenToggle hook. When off, sends noImageGen:true to WS chat method (maps to x-goclaw-no-image-gen on upstream HTTP call path).
ActiveRunZone renders a skeleton placeholder while streaming partial_image frames arrive. MediaGallery assigns generated-YYYYMMDD-HHmmss.png as the download filename for hex/UUID PNGs.
i18n keys added to en/vi/zh chat.json: imageGenToggle, imageGenGenerating, imageGenDownloadName. 8 vitest tests for the toggle hook.
* docs: add Image Generation section to codebase-summary + changelog entry
Documents the new native image_generation pipeline across providers layer (Codex + OpenAI-compat), agent gate, media persistence, and web UI surface.
* fix(ui/web): match Codex-routed providers for image_generation toggle
Image-gen toggle visibility was hard-coded to provider id 'chatgpt_oauth' but real Codex-routed agents in production use provider ids like 'cliproxy-codex'. The toggle never rendered.
Replace the Set-has check with a small helper that accepts the literal ids plus any provider string containing 'codex' (case-insensitive). Same logic applied in both chat-input.tsx (composer chip) and chat-page.tsx (streaming placeholder gate).
Verified against a live Codex-routed agent: toggle now renders, noImageGen:true propagates on toggle-off.
* docs(pr-1002): targeted-mode UX evidence report
Captures the UI integration surface for native image_generation against a live Codex-routed agent on the remote dev backend.
Includes: composer toggle chip (rendered), streaming skeleton placeholder, and honest failure-path capture showing the legacy create_image builtin fallback. Self-contained HTML report in .github/pr-assets/1002/index.html.
* fix(permissions): classify sessions.compact as write method
CI RBAC-drift test (TestMethodRole_DriftCoverage_AllProtocolMethodsClassified) was failing because the new sessions.compact method added upstream was not classified in any of isPublicMethod / isAdminMethod / isWriteMethod / isReadMethod.
Sessions compaction mutates session history (compacts messages into summaries), so it belongs with the other sessions.* write methods.
* fix(tests): remove duplicate contains() in integration package
Both tts_gemini_live_test.go and mcp_grant_revoke_test.go declared a file-local func contains(s, substr string) bool with identical bodies, causing 'contains redeclared in this block' at compile time in the integration job.
Replace all call sites with strings.Contains (same semantics, stdlib) and drop the duplicates. No behavior change.
* feat(providers): NativeImageProvider interface + Codex implementation
Defines a provider-level contract (NativeImageProvider.GenerateImage) that OAuth-backed providers can implement to serve image generation without exposing static API credentials. Re-uses the PR's Track A native wire format (POST /codex/responses with image_generation tool, item.result decoding, SSE fallback).
CodexProvider + CodexAdapter implement it. Also adds MediaRef.Prompt field so downstream layers can propagate the generating prompt alongside the asset.
* feat(tools): route create_image through NativeImageProvider for OAuth providers
create_image.callProvider now checks for a NativeImageProvider implementation before the credentialProvider interface. When the provider chain points at a Codex-family provider (no static API key), the tool delegates to the provider's GenerateImage which executes the native ChatGPT Responses API image_generation flow.
Resolves 'provider X does not expose API credentials required for image generation' errors for openai-codex / cliproxy-codex chains. On success the tool embeds the user's prompt as a PNG tEXt 'Description' chunk (file-local pngEmbedPrompt helper to avoid tools→agent import cycle), writes the image to /tmp, and threads the prompt through result.MediaPrompts for downstream MediaRef propagation.
* feat(agent,pipeline): propagate image prompt through MediaRef + PNG tEXt embed helper
Adds EmbedPNGPrompt public helper in internal/agent/png_metadata.go that inserts a tEXt 'Description' chunk (plus 'Software: goclaw') into PNG byte streams before the IEND chunk. Non-PNG inputs are passed through unchanged — the helper never errors on unknown formats.
FinalizeStage wires MediaResult.Prompt (from create_image tool output) onto MediaRef.Prompt so the UI can render the generating prompt alongside the image. Per-image prompt list threaded via pipeline RunState.
* feat(ui/web): show generating prompt as caption under each image in MediaGallery
When a MediaRef carries a prompt, MediaGallery renders it as a muted italic caption (line-clamp-2) beneath the image with the full text in the title tooltip. Caption is hidden when the prompt is absent so non-assistant images (user uploads, legacy data) look unchanged.
MediaItem + session media_refs types extended with an optional prompt field; the chat-message adapter threads ref.prompt through when converting WS payloads to UI state.
* fix(providers/codex): stream:true + instructions for native image_generation
The ChatGPT Responses API on /codex/responses rejects two things hard:
- stream:false → HTTP 400 "Stream must be set to true"
- missing instructions → HTTP 400 "Instructions are required"
buildNativeImageRequestBody now sets stream:true and a purpose-specific instructions string ("Generate an image matching the user's description using the image_generation tool. Return only the image; do not describe it in text."). The existing parseNativeImageSSE path was already in place for stream parsing; routing changed from the non-stream branch to the SSE branch.
Regression assertions added to TestCodexGenerateImage_BuildsNativeRequest so these two fields can't silently regress.
* feat(providers,tools,ui): image_model whitelist (gpt-image-2 default, gpt-image-1.5 legacy)
Replaces the hardcoded "gpt-image-2" literal in buildNativeImageRequestBody with a user-configurable field threaded through NativeImageRequest.ImageModel. The whitelist is enforced by ValidateImageModel which rejects anything outside {gpt-image-2, gpt-image-1.5} with a clear error — prevents silent upstream 400s from model names the Responses API would reject.
create_image.callProvider reads params.image_model from the chain entry and threads it through. Empty / absent falls back to DefaultImageModel (gpt-image-2).
UI: added an 'Image model' select inside the existing openai-codex Settings panel on the Create Image Provider Chain dialog. Options: Default · gpt-image-2 (recommended) and Legacy · gpt-image-1.5. i18n keys in en/vi/zh tools.json under builtin.mediaChain.
Tests: TestCodexGenerateImage covers default/legacy/rejected model cases; TestCreateImageTool_ThreadsImageModel covers params→request threading with empty/legacy/explicit sub-cases.
* fix(tools): raise media chain default timeout to 600s/1 retry for image gen
gpt-image-2 on complex prompts (dense Vietnamese text, infographic layouts) legitimately takes 4–8 minutes to complete. The previous default of 120s × 2 retries routinely died mid-generation with 'context deadline exceeded' — the upstream run was still producing bytes when our ctx cancelled.
Default is now Timeout: 600 / MaxRetries: 1. Retries dropped to 1 because image generation is stateful per upstream run: a mid-flight timeout leaves orphan server work, and retrying a fresh generation doubles cost for no gain. Surface the failure fast so operators can widen the timeout instead.
Operators can still set a tighter value explicitly via the Chain dialog.
* refactor: remove user-facing Images toggle, keep admin-level AllowImageGeneration
The per-request opt-out toggle (composer chip + streaming placeholder + noImageGen header plumbing) was a support footgun — users toggle OFF, forget, then can't generate images and think it's broken. Removed in full.
Kept: AgentConfig.AllowImageGeneration (admin kill-switch, stored in other_config.allow_image_generation, default true). Tri-level gate simplifies to two tiers: provider capability AND agent config allows.
Removed: useImageGenToggle hook, IMAGE_GEN_PROVIDER_IDS set in chat-input, supportsImageGenProvider helper, agentProvider/agentKey props on ChatInput, showImageGenPlaceholder prop on MessageBubble/ActiveRunZone/ChatThread, noImageGen param on use-chat-send, parseNoImageGen in chat_completions.go, NoImageGen on RunRequest, no_image_gen_header_test.go, imageGenToggle/imageGenGenerating i18n keys. Kept imageGenDownloadName — used by MediaGallery for generated-\*.png filename resolution.
* docs(pr-1002): refreshed UX trace + updated codebase notes
Replaces the earlier stealth-state evidence with a clean three-capture trace from a real successful run: inline image + prompt caption, MediaGallery lightbox expansion, Chain dialog with the new Image model dropdown open. Skill-routing rows scrubbed from the capture — they reflect per-agent skill setup, not anything this PR introduces.
codebase-summary + changelog: reflect final state (toggle removed, image_model selector, 600s default chain timeout, gpt-image-2 as quality baseline).
* fix(pipeline): preserve mid-loop image_generation output across iterations
FinalizeStage previously read state.Think.LastResponse.Images, which holds
only the last iteration's response. If the LLM emitted image_generation_call
in iteration N alongside a function_call, then responded text-only in N+1,
the image from N was silently dropped on finalize.
Accumulate final (non-partial) images into state.Observe.AssistantImages
across every iteration via ObserveStage, and source FinalizeStage from the
accumulator instead of LastResponse. Partial streaming frames are filtered
defensively; response.Images is cleared on drain to prevent double-counting
on re-exec.
* test(pipeline): regression coverage for mid-loop image accumulation
Six ObserveStage cases covering image accumulation semantics:
- single-iter image-only, image+tool_call same iter, mid-loop image
surviving text-only final iter, multiple images across iters, partial
frame filtering, nil-response safety.
Two FinalizeStage cases verifying accumulator is the source of truth:
- PersistsFromObserveAccumulator: image in Observe + empty LastResponse
must still be persisted via PersistAssistantImages.
- NoPersistWhenAccumulatorEmpty: no call when no images were emitted.
---------
Co-authored-by: viettranx <viettranx@gmail.com>
|
||
|
viettranx
|
c81cf6cf39 |
feat(security): MCP validation + hooks context isolation
Phase 02 - MCP input validation: - Add command allowlist (node, python, npx, uvx, etc.) - Block dangerous args (--eval, -e, -c, exec(), etc.) - SSRF protection via security.Validate() with DNS rebinding check - Fail-closed env var allowlist for headers - Integrate validation in create/update/import handlers Phase 03 - Hooks async context leak: - Use context.WithoutCancel to preserve TenantID/UserID in async hooks - Increase audit write timeout 2s → 5s - Increase circuit breaker store timeout 1s → 2s Also: Add security guard tests for skills upload |
||
|
viettranx
|
ee328d4266 |
refactor: consolidate hooks table migration and drop deprecated agent_id column
- Consolidated PG migration 054: agent_hooks rename + junction table + deprecated column drop - Updated SQLite schema v19 with final consolidated migration - Removed obsolete schema rebuild files (v21, v23) - Updated Go store layer (pg/hooks.go, sqlitestore/hooks.go) - Updated integration tests to use new table names (hooks, hook_agents) - Updated TypeScript protocol, UI components, and i18n strings - Updated gateway methods to reflect schema changes Tables: agent_hooks → hooks, agent_hook_agents → hook_agents |
||
|
viettranx
|
df82fafb28 |
feat(hooks): name field + beta explainer modal + UI polish
- PG migration 000054 + SQLite v21→v22: add nullable name column - HookConfig.Name in Go struct, PG/SQLite scan/insert, WS handlers - Builtin seed writes spec ID as hook name - UI: name input in create/edit form, displayed in hook list row - Beta card: "Learn more" button opens modal explaining hooks×skills×MCP - Default handler_type changed from http to script - Script editor section: max-h-[50vh] with scroll, border always visible - i18n: all new keys in en/vi/zh |
||
|
viettranx
|
feaf7b9e40 |
feat(hooks/builtin): default_disabled flag + flip pii-redactor off
Input-mutation hooks should be opt-in. Adds an explicit YAML toggle so
operators turn them on after verifying the regex / policy matches their
content, instead of getting silent mutation on first install.
- internal/hooks/builtin/loader.go: Spec.DefaultDisabled (yaml
default_disabled). Reverse-named so zero value preserves prior
default-on behavior for any spec that omits the field.
- internal/hooks/builtin/seed.go:
- Fresh insert path honors the flag → row created with enabled=false
- Version-bump path: when an UPDATE crosses the boundary that newly
introduces default_disabled=true, flip enabled→false once. Subsequent
boots without a version bump leave the user's manual re-enable alone
(the switch fall-through does nothing on equal versions). Logs
hooks.builtin_default_off_applied for audit.
- internal/hooks/builtin/builtins.yaml: pii-redactor v1 → v2 with
default_disabled: true. Existing installs auto-flip to disabled on
next boot via the version-bump branch above; users re-enable through
the UI toggle when ready.
- Tests:
- TestSeed_DefaultDisabledOnFreshInsert (insert respects flag)
- TestSeed_DefaultDisabledFlipsOnVersionBump (one-shot flip + user
re-enable preserved on the next no-bump boot)
|
||
|
viettranx
|
f3d6c99dda |
fix(hooks): close source-tier forge + i18n + concurrency hardening
Code-review findings from Wave 1 audit (commits 097776c5..f76a681e).
Critical (security):
- C1 internal/gateway/methods/hooks.go: parseHookConfigParams strips
caller-supplied source/id/created_by/version. Without this a tenant
admin could POST {"source":"builtin"} and escalate their UI hook into
the dispatcher's builtin capability tier (which is allowed to mutate
event input). Default Source resolves to "ui" via Validate.
- C2 internal/gateway/methods/hooks.go: handleUpdate also strips source +
created_by from the patch map. Same forge surface via PATCH instead of
POST.
High:
- H1 internal/gateway/methods/hooks.go: Update + Delete handlers wrap
hooks.ErrBuiltinReadOnly into the i18n key MsgHookBuiltinReadOnly so
users get a localized "builtin hooks are read-only" message instead of
the raw English sentinel string.
- H3 internal/hooks/handlers/script.go: bound tenantSems map with
opportunistic sweep — once map crosses 64 entries, idle slots
(no in-flight grants AND last-used > 1h ago) are reclaimed under the
same lock the acquire path already takes. Goroutine-free; common-path
cost unchanged.
Medium:
- M1 internal/hooks/dispatcher.go: builtinAllowlistLookup uses
atomic.Pointer instead of plain package var. Fixes the parallel-test
race where one test installs a lookup while another reads.
- M3 ui/web/index.html: drop http: https: from CSP connect-src — keep
'self' + ws:/wss: only. Same-origin XHR/WS still works; cross-origin
HTTP is no longer wildcard-permitted.
Tests:
- internal/gateway/methods/hooks_source_strip_test.go pins the C1+C2
strip behavior at the parse layer with a regression case that posts
every forge field and asserts each is zeroed.
|
||
|
viettranx
|
33d188f01f |
feat(hooks/migration): auto-disable legacy command hooks on Standard boot
Phase 07 runtime migration. Post-Wave-1 Standard deployments that pre-dated the edition gate may still carry enabled command-type hook rows in the DB; those keep firing via the dispatcher after the UI stops letting users create new ones. This helper flips them off at startup so live traffic never fires a disabled-by-policy handler again. - internal/hooks/migration_command_autodisable.go: DisableLegacyCommandHooks(ctx, hookStore, edition). Standard only; Lite returns early. Master-scope context + List(Enabled=true) → skip non-command + source='builtin' rows → Update enabled=false. Per-row WARN log with hook_id + tenant_id for audit; INFO summary when n>0. Idempotent: second boot finds nothing. - cmd/gateway_managed.go: invoke after builtin.Seed, BEFORE handler registration so no HTTP/WS traffic races the migration. - Tests (fake store, 5 cases): * Standard with mixed rows → only command+source=ui gets disabled * http + script/builtin + command/builtin rows untouched * Idempotent second run → n=0 * Lite edition → n=0, nothing touched * nil store → n=0, no error * List error → propagated up Note: Phase 04/05 never seed command-typed builtins, but the carve-out is kept defensively so a future accidental builtin command row can't be disabled by this migration. |
||
|
viettranx
|
a6592fd65f |
feat(hooks/builtin): ship pii-redactor exemplar — email + phone masking
First real builtin under Phase 04 infrastructure. Registers two DB rows
(user_prompt_submit + pre_tool_use) on startup, runs the embedded JS with
source-tier gate + dispatcher dotted-path allowlist.
- internal/hooks/builtin/pii-redactor.js (ES5.1, goja-compatible):
handle(event) returns updatedInput only when email or E.164-ish phone
actually matched. Redacts rawInput + toolInput.{command,query,content}.
- builtins.yaml: pii-redactor entry (version 1, priority 900, timeout 2s,
on_timeout=allow — missing redaction < blocking conversation).
- Drop _placeholder.js; pii-redactor.js is now the first real //go:embed match.
- Export builtin.Source(name) so handler tests load embedded JS without
going through Load()/cache state.
- Tests:
* loader_test: registry contains pii-redactor after Load().
* script_pii_redactor_test (handler-level, real goja runtime): email
masked in rawInput; +E164 phone masked; toolInput.command redacted,
toolInput.path left alone (not in mutable_fields); no PII → no
mutation; redact(redact(s)) == redact(s).
* dispatcher_test: 3 dotted-path walker cases (toolInput.command only
lets that key through; rawInput-only doesn't let forged toolInput
override original event keys; toolInput wildcard merges all keys).
Bench: ~242µs/op on 1 KiB Apple M4 Max, well under <1ms target.
|
||
|
viettranx
|
8ee540dc7e |
feat(hooks/builtin): embedded registry + UPSERT seed + store-layer readonly
Phase 04 of hooks Wave 1. Adds infrastructure for shipping canonical hook rows with the binary; Phase 05 plants the first real builtin (pii-redactor). - internal/hooks/builtin: loader (//go:embed yaml+js), Seed() with version reconciliation (newer embed overwrites DB; downgrade warn-only; preserves user's enabled toggle), AllowlistFor() for per-id mutable-field lookup. Stable UUIDv5 namespace 082ab084-a25f-52b4-a4a4-eb8a816bd9a8 keys rows across restarts so Seed is idempotent on every boot. - internal/hooks/store.go: ErrBuiltinReadOnly sentinel + WithSeedBypass ctx marker (seeder bypasses the guard; users cannot). - PG + SQLite hook stores: Update/Delete now reject any patch other than enabled=* on source='builtin' rows. Fail-closed on GetByID errors. - Dispatcher: SetBuiltinAllowlistLookup setter lets the gateway wire the registry-backed strict allowlist; unwired tests keep Phase 03 permissive default. Gateway installs a strip-all lookup first so a Load() failure fails closed instead of opening mutation to the permissive default. - i18n: MsgHookBuiltinReadOnly (en/vi/zh). - yaml.v3 promoted to direct dep. - Tests: loader parse + namespace stability + seed idempotency (3 boots → N rows), version bump, downgrade detection, operator BuiltinDisable escape hatch, PG + SQLite readonly guard round-trip. |
||
|
viettranx
|
b7592bcacf |
feat(hooks/script): integrate script handler — FireResult + source-tier gate + migration
Wave 1 Phase 03. Wires the Phase 02 Goja handler into the dispatcher, widens
the DB schema for script + builtin sources, and refactors Dispatcher.Fire to
return a FireResult so builtin-source hooks can mutate event input.
Dispatcher:
- FireResult{Decision, UpdatedToolInput, UpdatedRawInput} replaces the plain
Decision return. stdDispatcher.runSync keeps a local evMut copy so a
builtin-source hook's updatedInput flows to downstream hooks in the chain
and to the caller; non-builtin (source=ui) script mutations are dropped +
WARN-logged (defense-in-depth; source tier enforced at the dispatcher, not
only at the handler).
- applyBuiltinMutation + placeholder builtinAllowlistFor (rawInput, toolInput)
— Phase 04 overrides the allowlist from builtins.yaml.
- noopDispatcher returns FireResult{DecisionAllow}.
- Script mutation tests: TestDispatcher_ScriptMutation_BuiltinSourceApplies,
TestDispatcher_ScriptMutation_UISourceDenied.
Pipeline + callers (14 Fire sites refactored):
- FireHook wrapper returns FireResult; context_stage.go:52 applies
UpdatedRawInput → state.Input.Message; tool_stage.go:52 applies
UpdatedToolInput → tc.Arguments before ExecuteToolCall.
- delegate_bridge + delegate_tool keep the Decision branch; Updated* ignored.
- dispatcher_test / delegate_bridge_test / delegate_tool_hooks_test /
integration test helpers updated for the new shape.
Validation:
- edition_gate allows HandlerScript on every edition (sandboxed, no shell
escape surface).
- config.validateHandler rejects empty source, source > 32 KiB, goja compile
error; validateTimeout rejects on_timeout=ask|defer (reserved).
- Six new config tests cover the script path.
Migrations:
- PG migration 000053 relaxes handler_type + source CHECKs and drops
uq_hooks_{global,tenant,agent} — scripts routinely want many small hooks
per event. RequiredSchemaVersion → 53.
- SQLite SchemaVersion → 21; patch 20 is a SELECT 1 placeholder because
SQLite can't ALTER a CHECK — rebuildAgentHooksV21 runs outside the
migration tx (parallel to backfillV16) to rename/recreate the table with
widened CHECKs. schema.sql fresh-DB path matches.
- H9 fix: PGHookStore.Create + SqliteHookStore.Create honor caller-provided
cfg.ID (uuid.Nil falls back to UUIDv7), unblocking Phase 04 idempotent
UUIDv5 seed. TestCreateHonorsFixedID on both stores.
Config:
- config.HooksConfig{ScriptConcurrency, ScriptPerTenantConcurrency,
ScriptCacheSize, BuiltinDisable}; buildHookHandlers wires the script
handler with the caps from appCfg.Hooks.
Gates green: go build ./... + sqliteonly, go test -race -count=1 across
hooks/pipeline/sqlitestore.
|
||
|
viettranx
|
a556161d0d |
feat(hooks/script): goja ES5.1 script handler + deny-all sandbox
Wave 1 Phase 02 — pure runtime + sandbox. Dispatcher wiring, edition gate,
and DB migration land in Phase 03.
Handler:
- internal/hooks/handlers/script.go — ScriptHandler with two-layer semaphore
(global 10 / per-tenant 3), hashicorp/golang-lru v2 program cache (cap 500),
InvalidateHook hook for Phase 03, watchdog goroutine interrupting on ctx
cancel, sanitized error text (strips \n + goja " at <frame>" inline tails)
- internal/hooks/handlers/script_sandbox.go — deny-all/allowlist hardening,
SetMaxCallStackSize(256), prototype-chain nullify running BEFORE deny pass
so Function.prototype.constructor can still be reached
- internal/hooks/handlers/script_runtime.go — LRU compile, JSON round-trip +
deep-freeze event binding, 4KiB truncating stdout capture, strict return
parser enforcing {decision, reason?, updatedInput?}
Types:
- HandlerScript HandlerType = "script"
- DecisionAsk / DecisionDefer reserved (Wave 1 treats as block + warn)
- hooks.ScriptResult + WithScriptResult / ScriptResultFrom ctx helpers for
Phase 03 dispatcher to apply builtin-source UpdatedInput
Tests (go test -race green):
- script_test.go — 12 unit cases (happy/error/timeout/ask/defer/truncate/H1)
- script_sandbox_corpus_test.go — 27 cases: escape primitives (constructor
chain, Reflect, Proxy, Symbol, Promise, __proto__ walk, eval, Function,
Date chain, JSON replacer, toJSON DoS), resource bombs (recursion, loop,
memory, ReDoS), typeof sanity (Reflect/Proxy/Symbol/Promise/GoError/
globalThis/eval/Function undefined), mutation defense (toolInput/rawInput
frozen, Go-side map unchanged)
Deps: github.com/dop251/goja, github.com/hashicorp/golang-lru/v2.
Gates: go vet clean, go test -race clean, go build ./... + sqliteonly clean.
|
||
|
viettranx
|
553340b3a9 |
feat(hooks/prompt): LLM-driven prompt handler with per-turn cap
Adds PromptHandler that runs an LLM over matched hook events (user prompts, tool calls) to produce decisions (allow/block/modify). Features: - Budget integration: per-tenant token spend enforced via budget.Store - Per-turn counter seeded in ContextStage and propagated through state.Ctx so PreToolUse fires under the same cap as UserPromptSubmit - prompt_template required (non-empty) at validate time to prevent misconfigured hooks from silently no-oping - Injection-hardened: tool_input/raw_input are quoted into system prompt rather than concatenated - Resolver abstraction (RegistryResolver) picks model from provider registry with SystemConfigs fallback |
||
|
viettranx
|
ab50e571c5 |
feat(hooks/budget): per-tenant token spend tracking subsystem
Atomic budget store for prompt-handler LLM spend, with PG and SQLite implementations. Enforces per-hook token caps to prevent runaway cost from matcher-bypass scenarios. |
||
|
viettranx
|
fb3552f7e5 |
fix(hooks/security): add SSRF-safe http client missed by phase 2 commit
Phase 2 (a587231b) advertised SSRF hardening for the http hook handler but the supporting `internal/security` package was never created, so the production HTTPHandler fell back to a bare http.Client and admin-config webhooks could probe loopback / link-local / RFC1918 / cloud-metadata. - internal/security/ssrf.go: Validate(rawURL) parses + resolves once, rejects loopback/link-local/private/multicast/unspecified + 169.254.169.254; NewSafeClient(timeout) returns an http.Client whose DialContext pins the resolved IP from context (defense-in-depth re-checks the dialed IP) and refuses redirects (CheckRedirect = ErrUseLastResponse) - internal/hooks/handlers/http_handler.go: call Validate before each request, attach pinned IP via security.WithPinnedIP(ctx, ip) - cmd/gateway_managed.go: construct HTTPHandler with Client: security.NewSafeClient(10*time.Second) - tests: 13 new ssrf_test.go cases (every block category + redirect refused + dial pinned); existing http_test.go retrofitted with security.SetAllowLoopbackForTest helper for httptest.NewServer - plan: phase-02 Step 2a marked done with reference to this commit Refs: GitHub Issue #875 |
||
|
viettranx
|
97f9784443 |
feat(hooks): phase 2 — handlers, pipeline wiring, ssrf-safe http client
- add command + http handlers (internal/hooks/handlers) with edition gating and Authorization-header decryption via crypto.Decrypt - add SSRF-safe dialer (internal/security/ssrf.go): DNS-once + pinned IP, blocks loopback / link-local / private ranges - wire dispatcher into pipeline stages: ContextStage fires SessionStart (async) + UserPromptSubmit (sync); ToolStage fires PreToolUse (sync, COW staging for updatedInput) + PostToolUse (async); FinalizeStage fires Stop (async) - bridge delegate events to dispatcher (SubagentStart / SubagentStop) - construct dispatcher in cmd/gateway_managed.go with both handlers and thread it through agent.ResolverDeps + delegateTool.SetHookDispatcher - unit tests for both handlers + integration tests covering HTTP allow with audit row, HTTP block, command Lite-only gate, delegate bridge subscribe - defer worker pool optimization (Phase 2 Step 3) to Phase 4 Refs: GitHub Issue #875 |
||
|
viettranx
|
3fe5ae0b50 |
feat(hooks): introduce agent lifecycle hooks foundation with fail-closed blocking semantics
Phase 1 of agent-hooks-system establishes: - Hook types, matchers, and CEL evaluation engine for event-driven extensibility - Edition-gated command handlers (Lite disabled) with dedup_key-indexed audit log - Dual-DB store layer (PostgreSQL + SQLite) with transaction boundary enforcement - Blocking-event semantics: hook execution failures propagate to agent loop, graceful retry - Tenant-isolated audit trail with per-hook execution context tracing - Integration tests verifying store correctness and hook dispatch atomicity Lays groundwork for post-v3.0 phase-02 (per-tenant onboarding hooks). |
||
|
viettranx
|
4678065887 |
refactor: remove dead quality gates / hook engine code
The delegation system this depended on was previously removed, leaving internal/hooks/ as dead code with zero imports. Remove the entire hook engine, UI config section, protocol types, i18n keys, and all documentation references. |
||
|
viettranx
|
3f2b6e258e |
chore(teams): remove deprecated delegation tools
Remove delegate_search, evaluate_loop, handoff from: - Seed data, system prompt, i18n keys/catalogs, channel events - Consumer handler (handleHandoffAnnounce), handoff route lookup - HandoffRouteData struct + PG implementation - Protocol events, MCP bridge comment - Web UI locale files (en/vi/zh) |
||
|
viettranx
|
6066adc15a | feat: Implement agent delegation, quality gates, and a new hooks evaluation system. |