Commit Graph
1497 Commits
Author SHA1 Message Date
viettranx 955b7a512f docs: update architecture and changelog for context-tokens and compact-quality fixes
- Update docs/00-architecture-overview.md with TokenCounter and sessions.metadata details
- Update docs/codebase-summary.md with overhead accounting and compaction logic
- Update docs/project-changelog.md with v3.11.0 context-tokens accuracy fixes
2026-04-23 08:31:53 +07:00
viettranx cd4a8bd10f fix(store): persist last_prompt_tokens via sessions.metadata for accurate UI display
- Store last_prompt_tokens in sessions.metadata JSONB (PostgreSQL + SQLite)
- Update SessionsList queries to retrieve metadata and provide token display values
- Add fallback heuristic for sessions without metadata (estimated from history)
- Add tests: sessions_list_heuristic_test.go, sessions_list_metadata_tokens_test.go
- Add integration test: sessions_display_tokens_integration_test.go
2026-04-23 08:31:53 +07:00
viettranx eb6723d674 fix(pipeline): include tool-schema tokens in overhead + dynamic compact max_tokens
- Add TokenCounter.CountToolSchemas() to measure JSON schema size for all tools
- Include tool schemas in OverheadTokens calculation for accurate context usage
- Implement dynamic max_tokens: in/25 clamp [1024, 8192] for compaction
- Add characterization tests: count_tool_schemas_test.go
- Add overhead verification tests: context_stage_overhead_test.go, context_stage_tool_overhead_test.go
- Add integration tests: context_stage_integration_test.go
- Add compact tests: loop_compact_dynamic_max_test.go, loop_compact_max_tokens_test.go
- Add sanitize tests: loop_history_sanitize_max_tokens_test.go
- Add integration test: loop_compact_integration_test.go
2026-04-23 08:31:53 +07:00
viettranx 04a9938f4f feat(tts): wire tenant timeout + fix Gemini text-only 400
- HTTP synthesize + test-connection now read tenant tts.timeout_ms
  (default 120s, was hardcoded 15s/10s). Gemini client default also
  bumped 30s→120s so both layers align when tenant config unset.
- Inline prefix "Speak naturally: " prepended to single-voice text;
  multi-speaker transcripts pass through unchanged.
- ErrTextOnlyResponse sentinel for 400 "text generation" bodies;
  single-voice retries once with stronger prefix. Narrowed needle
  list avoids false positives on unrelated 400s.
- SynthesizeWithFallbackAdapted now returns errors.Join so sentinel
  survives fallback chain; HTTP 422 mapping + locale-translated
  ForLLM in agent tool (EN/VI/ZH catalogs).
- Default Gemini model bumped to gemini-3.1-flash-tts-preview.
2026-04-23 08:31:53 +07:00
viettranx 4d6ebe9c58 feat(telegram): inject bot self-identity into agent system prompt
After stripBotMention removes the bot's own @mention from content, the LLM
still lacks knowledge of its own platform handle. In multi-bot groups (e.g.
"@botA @botB do X") the other bot's mention remains in the content and the
agent incorrectly treats it as the intended target, replying NO_REPLY.

Capture the bot's first_name from GetMe during channel Start and expose it
alongside the username via a new MetaChannelSelfIdentity metadata key. The
consumer appends the formatted hint ("You are @{username} ({display_name})
on this Telegram channel.") to the agent's extraSystemPrompt so the LLM can
reliably identify itself across single- and multi-bot scenarios.

Falls back to "You are @{username} on this Telegram channel." when the
display name is not available, and no-ops when the username has not been
resolved yet (startup race).
2026-04-23 08:31:53 +07:00
viettranx e392fca14d fix(telegram): strip own @mention from inbound content
Agents previously saw their own Telegram handle (e.g. "@viet_super_bot")
in user messages and mistook it for a different bot, replying NO_REPLY.
The username was only used for the mention gate, never removed from the
content passed to the LLM.

Slack and Feishu already strip their own bot mentions (handlers_mention.go
stripBotMention, bot_parse.go resolveMentions); Telegram was the odd one out.

Implementation:
- Add stripBotMention helper with leading/trailing word-boundary anchors so
  inline matches inside words (e.g. contact@viet_super_bot.com) are not
  falsely stripped.
- Apply in handleMessage right after the mention gate, before pairing/media
  processing, so history recording for unmentioned messages keeps raw text.
- Restore "[empty message]" placeholder when a message consisting only of
  "@botname" becomes empty after stripping.
2026-04-23 08:31:53 +07:00
viettranx 3a85d625a6 feat(tools): add send_file for delivering existing workspace files
- new send_file(path, caption?) tool with DenyPaths guard and duplicate-delivery check
- patch message(MEDIA:) to Mark DeliveredMedia on send success (closes cross-tool dup gap)
- register in gateway wiring + builtin seed
- add to systemprompt coreToolSummaries; clarify write_file deliver=true description
- 16 tests green (PG + SQLite builds clean, invariants green)
2026-04-23 08:31:53 +07:00
4b02c27307 feat: native image_generation for Codex + OpenAI-compat providers (#1002)
* refactor(providers): migrate ToolDefinition.Function to pointer + add image response fields

ToolDefinition.Function becomes *ToolFunctionSchema with omitempty so native tool types (image_generation, web_search, etc.) can be declared without a function body. All 9 internal construction sites updated. CleanToolSchemas refactored — function-shape cleaning extracted into cleanFunctionSchema helper, outer pass-through handles native tools.

Added image response fields needed by the next commits: ChatResponse.Images, StreamChunk.Images, ImageContent.Partial (distinguishes partial frames from final images).

* feat(providers): native image_generation for Codex + OpenAI-compat tracks

Codex native (POST /codex/responses): emit image_generation tool object in request tools[] (type, action, model, output_format, partial_images). Handle SSE events response.image_generation_call.partial_image + response.output_item.done (type image_generation_call) + response.completed output[] walk for non-stream. Dedup per item_id. Extend codexSSEEvent/codexItem with output_format, result, partial_image_b64, partial_image_index.

OpenAI-compat (/v1/chat/completions): serialize ToolDefinition{Type:'image_generation'} as {type:'image_generation'} pass-through. Parse choices[0].message.images[] + delta.images[] (data URLs) via new parseDataURL helper; append to ChatResponse.Images.

ProviderCapabilities.ImageGeneration flag; Codex provider + adapter set true. Other providers default false.

* feat(agent,http,store): persist assistant images + tri-level image_generation gate

Agent loop tri-level gate: (provider capability) AND (AgentConfig.AllowImageGeneration, default true, stored in other_config.allow_image_generation) AND (request lacks x-goclaw-no-image-gen header). Gate in loop_tool_filter.go appends ToolDefinition{Type:'image_generation'} only when all three pass. Per-request opt-out parsed in chat_completions.go and propagated via RunRequest.NoImageGen.

Media persistence: persistAssistantImages writes final images (Partial:false) to {workspace}/media/{sha256}.{ext}, returns MediaRef entries, clears inline Images[] from the message. Idempotent on hash, traversal-safe, symlink-guarded. Invoked from pipeline.FinalizeStage via new Deps.PersistAssistantImages callback — covers both stream-final and non-stream paths.

Agent store reads AllowImageGeneration from other_config JSONB with absent/nil = true default (matches V3Flags pattern). No DB migration — code-only default.

* feat(ui/web): image_generation toggle + streaming placeholder + download filename

Composer chip 'Images' visible only when active agent's provider has ImageGeneration capability. Per-agent localStorage persistence via useImageGenToggle hook. When off, sends noImageGen:true to WS chat method (maps to x-goclaw-no-image-gen on upstream HTTP call path).

ActiveRunZone renders a skeleton placeholder while streaming partial_image frames arrive. MediaGallery assigns generated-YYYYMMDD-HHmmss.png as the download filename for hex/UUID PNGs.

i18n keys added to en/vi/zh chat.json: imageGenToggle, imageGenGenerating, imageGenDownloadName. 8 vitest tests for the toggle hook.

* docs: add Image Generation section to codebase-summary + changelog entry

Documents the new native image_generation pipeline across providers layer (Codex + OpenAI-compat), agent gate, media persistence, and web UI surface.

* fix(ui/web): match Codex-routed providers for image_generation toggle

Image-gen toggle visibility was hard-coded to provider id 'chatgpt_oauth' but real Codex-routed agents in production use provider ids like 'cliproxy-codex'. The toggle never rendered.

Replace the Set-has check with a small helper that accepts the literal ids plus any provider string containing 'codex' (case-insensitive). Same logic applied in both chat-input.tsx (composer chip) and chat-page.tsx (streaming placeholder gate).

Verified against a live Codex-routed agent: toggle now renders, noImageGen:true propagates on toggle-off.

* docs(pr-1002): targeted-mode UX evidence report

Captures the UI integration surface for native image_generation against a live Codex-routed agent on the remote dev backend.

Includes: composer toggle chip (rendered), streaming skeleton placeholder, and honest failure-path capture showing the legacy create_image builtin fallback. Self-contained HTML report in .github/pr-assets/1002/index.html.

* fix(permissions): classify sessions.compact as write method

CI RBAC-drift test (TestMethodRole_DriftCoverage_AllProtocolMethodsClassified) was failing because the new sessions.compact method added upstream was not classified in any of isPublicMethod / isAdminMethod / isWriteMethod / isReadMethod.

Sessions compaction mutates session history (compacts messages into summaries), so it belongs with the other sessions.* write methods.

* fix(tests): remove duplicate contains() in integration package

Both tts_gemini_live_test.go and mcp_grant_revoke_test.go declared a file-local func contains(s, substr string) bool with identical bodies, causing 'contains redeclared in this block' at compile time in the integration job.

Replace all call sites with strings.Contains (same semantics, stdlib) and drop the duplicates. No behavior change.

* feat(providers): NativeImageProvider interface + Codex implementation

Defines a provider-level contract (NativeImageProvider.GenerateImage) that OAuth-backed providers can implement to serve image generation without exposing static API credentials. Re-uses the PR's Track A native wire format (POST /codex/responses with image_generation tool, item.result decoding, SSE fallback).

CodexProvider + CodexAdapter implement it. Also adds MediaRef.Prompt field so downstream layers can propagate the generating prompt alongside the asset.

* feat(tools): route create_image through NativeImageProvider for OAuth providers

create_image.callProvider now checks for a NativeImageProvider implementation before the credentialProvider interface. When the provider chain points at a Codex-family provider (no static API key), the tool delegates to the provider's GenerateImage which executes the native ChatGPT Responses API image_generation flow.

Resolves 'provider X does not expose API credentials required for image generation' errors for openai-codex / cliproxy-codex chains. On success the tool embeds the user's prompt as a PNG tEXt 'Description' chunk (file-local pngEmbedPrompt helper to avoid tools→agent import cycle), writes the image to /tmp, and threads the prompt through result.MediaPrompts for downstream MediaRef propagation.

* feat(agent,pipeline): propagate image prompt through MediaRef + PNG tEXt embed helper

Adds EmbedPNGPrompt public helper in internal/agent/png_metadata.go that inserts a tEXt 'Description' chunk (plus 'Software: goclaw') into PNG byte streams before the IEND chunk. Non-PNG inputs are passed through unchanged — the helper never errors on unknown formats.

FinalizeStage wires MediaResult.Prompt (from create_image tool output) onto MediaRef.Prompt so the UI can render the generating prompt alongside the image. Per-image prompt list threaded via pipeline RunState.

* feat(ui/web): show generating prompt as caption under each image in MediaGallery

When a MediaRef carries a prompt, MediaGallery renders it as a muted italic caption (line-clamp-2) beneath the image with the full text in the title tooltip. Caption is hidden when the prompt is absent so non-assistant images (user uploads, legacy data) look unchanged.

MediaItem + session media_refs types extended with an optional prompt field; the chat-message adapter threads ref.prompt through when converting WS payloads to UI state.

* fix(providers/codex): stream:true + instructions for native image_generation

The ChatGPT Responses API on /codex/responses rejects two things hard:

- stream:false → HTTP 400 "Stream must be set to true"

- missing instructions → HTTP 400 "Instructions are required"

buildNativeImageRequestBody now sets stream:true and a purpose-specific instructions string ("Generate an image matching the user's description using the image_generation tool. Return only the image; do not describe it in text."). The existing parseNativeImageSSE path was already in place for stream parsing; routing changed from the non-stream branch to the SSE branch.

Regression assertions added to TestCodexGenerateImage_BuildsNativeRequest so these two fields can't silently regress.

* feat(providers,tools,ui): image_model whitelist (gpt-image-2 default, gpt-image-1.5 legacy)

Replaces the hardcoded "gpt-image-2" literal in buildNativeImageRequestBody with a user-configurable field threaded through NativeImageRequest.ImageModel. The whitelist is enforced by ValidateImageModel which rejects anything outside {gpt-image-2, gpt-image-1.5} with a clear error — prevents silent upstream 400s from model names the Responses API would reject.

create_image.callProvider reads params.image_model from the chain entry and threads it through. Empty / absent falls back to DefaultImageModel (gpt-image-2).

UI: added an 'Image model' select inside the existing openai-codex Settings panel on the Create Image Provider Chain dialog. Options: Default · gpt-image-2 (recommended) and Legacy · gpt-image-1.5. i18n keys in en/vi/zh tools.json under builtin.mediaChain.

Tests: TestCodexGenerateImage covers default/legacy/rejected model cases; TestCreateImageTool_ThreadsImageModel covers params→request threading with empty/legacy/explicit sub-cases.

* fix(tools): raise media chain default timeout to 600s/1 retry for image gen

gpt-image-2 on complex prompts (dense Vietnamese text, infographic layouts) legitimately takes 4–8 minutes to complete. The previous default of 120s × 2 retries routinely died mid-generation with 'context deadline exceeded' — the upstream run was still producing bytes when our ctx cancelled.

Default is now Timeout: 600 / MaxRetries: 1. Retries dropped to 1 because image generation is stateful per upstream run: a mid-flight timeout leaves orphan server work, and retrying a fresh generation doubles cost for no gain. Surface the failure fast so operators can widen the timeout instead.

Operators can still set a tighter value explicitly via the Chain dialog.

* refactor: remove user-facing Images toggle, keep admin-level AllowImageGeneration

The per-request opt-out toggle (composer chip + streaming placeholder + noImageGen header plumbing) was a support footgun — users toggle OFF, forget, then can't generate images and think it's broken. Removed in full.

Kept: AgentConfig.AllowImageGeneration (admin kill-switch, stored in other_config.allow_image_generation, default true). Tri-level gate simplifies to two tiers: provider capability AND agent config allows.

Removed: useImageGenToggle hook, IMAGE_GEN_PROVIDER_IDS set in chat-input, supportsImageGenProvider helper, agentProvider/agentKey props on ChatInput, showImageGenPlaceholder prop on MessageBubble/ActiveRunZone/ChatThread, noImageGen param on use-chat-send, parseNoImageGen in chat_completions.go, NoImageGen on RunRequest, no_image_gen_header_test.go, imageGenToggle/imageGenGenerating i18n keys. Kept imageGenDownloadName — used by MediaGallery for generated-\*.png filename resolution.

* docs(pr-1002): refreshed UX trace + updated codebase notes

Replaces the earlier stealth-state evidence with a clean three-capture trace from a real successful run: inline image + prompt caption, MediaGallery lightbox expansion, Chain dialog with the new Image model dropdown open. Skill-routing rows scrubbed from the capture — they reflect per-agent skill setup, not anything this PR introduces.

codebase-summary + changelog: reflect final state (toggle removed, image_model selector, 600s default chain timeout, gpt-image-2 as quality baseline).

* fix(pipeline): preserve mid-loop image_generation output across iterations

FinalizeStage previously read state.Think.LastResponse.Images, which holds
only the last iteration's response. If the LLM emitted image_generation_call
in iteration N alongside a function_call, then responded text-only in N+1,
the image from N was silently dropped on finalize.

Accumulate final (non-partial) images into state.Observe.AssistantImages
across every iteration via ObserveStage, and source FinalizeStage from the
accumulator instead of LastResponse. Partial streaming frames are filtered
defensively; response.Images is cleared on drain to prevent double-counting
on re-exec.

* test(pipeline): regression coverage for mid-loop image accumulation

Six ObserveStage cases covering image accumulation semantics:
- single-iter image-only, image+tool_call same iter, mid-loop image
  surviving text-only final iter, multiple images across iters, partial
  frame filtering, nil-response safety.

Two FinalizeStage cases verifying accumulator is the source of truth:
- PersistsFromObserveAccumulator: image in Observe + empty LastResponse
  must still be persisted via PersistAssistantImages.
- NoPersistWhenAccumulatorEmpty: no call when no images were emitted.

---------

Co-authored-by: viettranx <viettranx@gmail.com>
2026-04-23 08:22:39 +07:00
viettranx a7962d182a feat(pipeline): session compaction overflow recovery (#958)
- Add ZAI/GLM context overflow patterns to error_classify.go
- ThinkStage detects overflow, triggers emergency compaction + 1 retry
- Wire ReserveTokensFloor config to pipeline budget calculation
- Send user-friendly error on channel RunFailed events
- Add sessions.compact WebSocket method for manual truncation
2026-04-20 08:49:38 +07:00
viettranx 1b8627077a release: TTS expansion v3.10.0
Merge 10 commits from dev:
- TTS provider capabilities baseline + Gemini provider
- Phase 1: per-provider params expansion (Gemini, ElevenLabs, MiniMax)
- Phase 2: PortalVoicePicker extraction (Gemini portal UX)
- Phase 3: Basic/Advanced collapsible DynamicParamForm
- Phase 4: agent override allow-list (speed/emotion/style)
- Post-review: adapter triplication collapsed via AgentOverridableAs
- UX: TTS section spacing tightened
- Voice labels: Gemini style, MiniMax gender/language heuristics
2026-04-20 07:37:33 +07:00
viettranx 2c00d50a6e feat(tts): parse MiniMax voice gender + language from naming convention
MiniMax /v1/get_voice does not return explicit gender/language fields,
but the system-voice naming convention exposes them:

- voice_id prefix `male-*` / `female-*` (legacy Chinese voices)
- voice_name suffix `*_Man` / `*_Boy` / `*_Lord` / `*_Speaker` → male
- voice_name suffix `*_Lady` / `*_Girl` / `*_Belle` / `*_Lass` → female
- voice_name prefix `English_*` / `Chinese_*` / `Japanese_*` /
  `Korean_*` / `French_*` / etc. → language

`parseMinimaxLabels` is best-effort — returns nil for unparseable IDs
(e.g. `moss_audio_<uuid>` cloned voices), so the picker shows no badge
rather than a wrong one. The picker already renders `voice.labels` as
badges; nothing to wire on the FE side.

8-case unit test covers the matrix.
2026-04-20 07:32:35 +07:00
viettranx f13524ec08 feat(tts): show Gemini voice character (style) as a label badge
Google does not publish gender for the 30 prebuilt Gemini voices —
inclusive design — but does publish a one-word style descriptor
(Bright, Firm, Smooth, Mature, Lively, etc.). Surface it so users can
pick by character.

- `audio.VoiceOption` gains `Labels map[string]string` (provider-
  specific descriptors). ElevenLabs already returns labels via API;
  Gemini now hardcodes `{"style": "..."}` per voice from the official
  catalog.
- Web + desktop `mapCapVoiceToPortal` passes labels through. Both
  pickers' `LABEL_KEYS` now include `"style"`, so the existing badge
  rendering picks it up — first 1–2 entries shown next to the name.
- TS `VoiceOption` interface (web + desktop) gains optional `labels`.

No gender field is invented for Gemini voices.
2026-04-20 07:28:15 +07:00
viettranx 5b009caa95 fix(tts): tighten Card section spacing so titles sit close to content
Card primitive uses `flex flex-col gap-6` which adds 24px between
header and content regardless of `pb-3` on CardHeader. Override the
flex-gap to `gap-3` on each TTS section so the section number/title
visually attaches to its body.

Affects: provider-setup, credentials-section, voice/model card on
tts-page, test-playground, behavior-section.
2026-04-20 07:22:32 +07:00
viettranx 82ca0f117b refactor(tts): collapse adapter triplication; honest concurrency comment
Post-review cleanup of Phase 4. Closes Finding #9 properly and corrects
the Finding #13 documentation lie surfaced in the code-review report.

Capability schema:
- Replace `AgentOverridable bool` with `AgentOverridableAs string` on
  ParamSchema. Empty string = not overridable; non-empty = the generic
  key alias (`"speed"`, `"emotion"`, `"style"`).
- Each provider declaration now carries the alias inline, so the
  generic↔native mapping has a single TS-readable source.

Frontend:
- Web `tts-override-block.tsx` drops the inline `GENERIC_TO_NATIVE`
  literal and derives the bidirectional adapter from the filtered
  capability params (each param self-describes its alias). Adapter
  tests rewritten around the new shape.
- Desktop `AgentDetailPanel.tsx` drops the 45-line inline IIFE in
  favour of a new `<TtsOverrideFineTune>` component that uses the
  same alias-based mapping.

Backend:
- Move `AgentTTSParamsAllowedKeys` + `ValidateAgentTTSParams` to
  `internal/audio/agent_params_adapter.go`. HTTP `validate.go` and WS
  `gateway/methods/agents_update.go` both delegate, eliminating the
  duplicated `{speed, emotion, style}` literal.

Cleanup:
- Delete orphan i18n keys `MsgTtsParamInvalidJSON` and
  `MsgTtsParamDependsOn` from `keys.go` + en/vi/zh catalogs (no
  in-code references; DependsOn is FE-only, JSON parse failures
  already surface via slog).

Documentation:
- `prompt-settings-section.tsx` Finding #13 comment rewritten to
  honestly describe the best-effort merge into a fresh local copy of
  the cached `otherConfig` prop. Concurrent-tab clobber remains
  possible — server-side JSON-merge-patch endpoint planned for v2.

Tests: 9 backend suites (race), web 217/217, desktop build clean,
both Go build tags pass.
2026-04-20 07:14:56 +07:00
viettranx 247344d689 feat(tts): per-agent params override (speed/emotion/style) with adapter
Phase 4 — final phase of the TTS params/layout/agent-override plan.

Adds a 3-key allow-list (`speed`, `emotion`, `style`) per agent stored
in `agents.other_config.tts_params`. Backend resolves and merges into
`opts.Params` PER ATTEMPT inside the fallback loop so each provider
sees its own native shape — never the primary's keys when fallback
runs (Finding #1 critical).

Backend:
- `AgentOverridable bool` on `audio.ParamSchema`. UI filter reads this
  flag from /v1/tts/capabilities; no separate TS literal mirror —
  capabilities API is the single source of truth (Finding #9).
- `audio.AdaptAgentParams(generic, provider)` maps the 3 generic keys
  to provider-native paths (e.g. `speed` → `voice_settings.speed` for
  ElevenLabs, flat `speed` for OpenAI/MiniMax, dropped for Edge/Gemini).
- `Manager.SynthesizeWithFallbackAdapted` adapts inside the loop so
  fallback providers receive correctly-shaped params.
- `manager_auto.go` and `tools/tts.go` Execute do per-attempt adaptation
  on the tenant + direct + fallback call sites.
- Drop log bumped to `slog.Info("tts.agent.params.dropped", ...)` for
  audit trail when a generic key isn't supported by the active provider.
- Cross-check test asserts every adapter switch case has at least one
  capability ParamSchema with `AgentOverridable: true`, and vice versa.

Security (red-team findings):
- Allow-list ENFORCED at write path: `validateAgentTTSParams` in HTTP
  `handleUpdate` AND WS `agents_update` rejects any `tts_params` key
  outside `{speed, emotion, style}` (Finding #5).
- 64KB body cap on agent PUT via `http.MaxBytesReader` (Finding #6).
- Explicit tenant-scope guard after `agents.GetByID` (Finding #12).
- Concurrent-tab clobber: handleSave merges `tts_params` into a fresh
  copy of `otherConfig` rather than reusing stale state (Finding #13).
- Rate-limit verified — RoleAdmin gate sufficient for v1 (Finding #15).

Frontend (web + desktop):
- `TtsOverrideBlock` rewritten: filters capability params to
  `agent_overridable === true`, renders via `DynamicParamForm`. Hides
  entirely for providers with no overridable params (Edge, Gemini).
- Bidirectional adapter (generic ↔ capability-native form state) so
  agent storage stays in generic keys while UI works in native paths.
  25 round-trip tests cover all 5 providers.
- Desktop `AgentDetailPanel` gains an inline fine-tune section gated
  on `globalProvider`, reusing the desktop `DynamicParamForm`.

i18n: `tts.override.params.title` ("Fine-tune") added to web + desktop
en/vi/zh.

Tests: all 9 backend suites green (race), web 214/214, desktop build
clean, both Go build tags pass.
2026-04-20 06:46:19 +07:00
viettranx 85ebbcfbd5 feat(tts): split DynamicParamForm into Basic and Advanced sections
Phase 3 of the TTS params/layout/agent-override plan.

Reads the `Group` field shipped in Phase 1 to partition each provider's
params into Basic (always visible) and Advanced (collapsed by default,
chevron toggle, count badge). Edge providers with no advanced params
hide the toggle entirely.

- `partitionSchema(schema)` splits params; unknown group values fall
  into Basic (forward-compat).
- `evaluateDependsOn` runs against the full shared form state, so
  cross-section dependencies (e.g. MiniMax `audio.bitrate` depends on
  basic-section `audio.format`) work without scoping.
- Count badge uses `t("tts.advanced.count", {count})` with i18next
  pluralization (`_one` for English).
- Desktop mirror with inline SVG chevron (no lucide-react dep).

i18n (web + desktop):
- New `tts.advanced.count` + `tts.advanced.count_one` keys in en/vi/zh.
- `tts.advanced.title` already present; verified.

TS contract: `ParamSchema.group?: string` added to web + desktop
`tts-capabilities.ts`. Plain TS interface — no Zod strict-validator
guard needed (verified via grep).

10 new unit tests cover partition order, unknown-group fallback, count
badge respecting DependsOn, and the cross-group MiniMax case.
189/189 web tests pass.
2026-04-20 06:28:17 +07:00
viettranx bd95022efe feat(tts): extract PortalVoicePicker and route Gemini through it
Phase 2 of the TTS params/layout/agent-override plan.

Web (`ui/web/src/components/voice-picker.tsx`):
- Extract `PortalVoicePicker` (search + VoiceRow + portal dropdown)
  from the `DynamicVoicePicker` inner body.
- Introduce `PortalVoice` shape + `mapCapVoiceToPortal` adapter so
  capability `VoiceOption` and `useVoices()` results both feed in.
- Dispatcher gains a Gemini route: static voices + provider==="gemini"
  → PortalVoicePicker with the static catalog. Other static providers
  (OpenAI) continue to use `StaticVoicePicker` (Radix Select).
- `DynamicVoicePicker` reduced to a thin fetch wrapper around
  `useVoices()` + `useRefreshVoices()`.

Desktop mirror (`ui/desktop/frontend/src/components/agents/voice-picker.tsx`):
- Same extraction + Gemini route. OpenAI keeps the Combobox-driven
  static path to preserve native UX.

Tests: 9 new cases covering Gemini→Portal dispatch, mapping shape,
absent `preview_url`/`labels`, plus the unchanged ElevenLabs/MiniMax/
OpenAI paths. 179/179 web tests pass.

No backend change. No new i18n keys (existing voice_picker.* reused).
2026-04-20 06:23:09 +07:00
viettranx ee57ec0fdc feat(tts): expand per-provider params + validation + golden fixtures
Phase 1 of the TTS params/layout/agent-override plan.

Capability schema:
- `ParamSchema.Group` field (`"basic"` default, `"advanced"` when set).
- Tag existing advanced params across openai/elevenlabs/minimax/gemini.

Gemini:
- Expose `temperature` (basic, 0.0–2.0, default 1.0, subtle-effect note),
  `seed`, `presencePenalty`, `frequencyPenalty` (advanced, experimental).
- Merge into `generationConfig` via explicit-presence resolvers so
  nil-params bodies stay byte-equivalent.

ElevenLabs:
- `output_format` enum (27 variants, default `mp3_44100_128`, advanced).
- `FormatMeta` lookup drives SynthResult MIME + extension.
- URL built via `net/url.Values.Encode()`; regex pre-validation for
  `output_format` (`^[a-z0-9_]+$`) and `language_code`
  (`^[a-z]{2,3}(-[A-Z]{2})?$`) blocks query-string injection.
- Telegram opus contract preserved: `opts.Format=="opus"` forces
  `audio/ogg; codecs=opus` regardless of user-set `output_format`.

MiniMax:
- `language_boost` (basic enum), `subtitle_enable` (basic bool),
  `pronunciation_dict` (advanced text, 8KB cap, wrapped as
  `{"tone":[...]}`). Parse failure logs length only + omits.

Validation:
- `audio.ValidateParams` enforces Min/Max/Enum + rejects unknown keys.
- Wired into `/v1/tts/synthesize` and `/v1/tts/config` write paths.
- `loadParamsBlob` capped at 16KB.
- i18n keys `MsgTtsParamOutOfRange`, `MsgTtsParamInvalidJSON`,
  `MsgTtsParamUnknownKey` added to en/vi/zh catalogs.

Tests:
- Golden `testdata/default_body.golden.json` per provider (gemini,
  elevenlabs, minimax, openai) checked in; invariant tests diff
  against file instead of self-referential capture.
- Round-trip tests for each new param + Telegram opus contract +
  URL-injection attempts.
2026-04-20 00:20:50 +07:00
viettranx 613b6e38d7 feat(tts): provider capabilities schema + Gemini TTS + dynamic param forms
Baseline groundwork for TTS expansion plan. Introduces a capabilities
system that the follow-up plan (phase-01..04) builds on.

Backend:
- `audio.ParamSchema` / `ProviderCapabilities` types + per-provider
  capabilities.go for edge, elevenlabs, minimax, openai, gemini.
- Gemini TTS provider (client, models, voices, wav encoder,
  multi-speaker via SpeakerVoice, audio-tag aware prompts).
- TTSOptions gains `Params map[string]any` (read-only) + `Speakers`.
- `VoiceListProvider` interface decouples HTTP voice handler from
  provider-specific impls.
- `nested_keys.go` resolves dot-separated param paths for nested
  provider bodies (voice_settings.stability etc.).
- Characterization + defaults-invariant tests per provider lock
  nil-params byte-equivalence before new params land.
- `/v1/tts/capabilities` HTTP endpoint + integration coverage.
- Dual-read tests (PG + SQLite) for tts_config.

Frontend (web + desktop):
- `DynamicParamForm` with depends-on evaluation, split into
  fields/logic modules. Slider primitive added.
- `AudioTagPicker` + `MultiSpeakerEditor` for Gemini.
- `voice-picker` refactored toward portal UX; combobox tightened.
- tts-capabilities API client + typed hooks.
- i18n catalogs (en/vi/zh) expanded; parity tests guard key drift.
- TTS page reorganised (voice-model-section removed; playground +
  credentials + provider-setup split cleanly).

Docs: codebase-summary, project-changelog, tts-provider-capabilities.
2026-04-20 00:05:19 +07:00
viettranx 7639a8c013 feat(pruning): enable context pruning by default with cache-ttl mode
- Change default mode from "off" to "cache-ttl" (enabled by default)
- Increase softTrimRatio from 0.25 to 0.3 to match TS defaults
- Update tests to reflect new default-enabled behavior
2026-04-19 21:00:13 +07:00
viettranx 5bc73d557c Merge remote-tracking branch 'origin/main' into dev 2026-04-19 16:20:30 +07:00
viettranx 7f7cf99da1 docs: final consistency sweep and normalization
- Fix prose body citation in 15-core-skills-system (removed :410 line ref)
- Normalize File Reference schema in 08-scheduling-cron and 10-tracing-observability to Module/Path/Purpose format
- Add missing hint line to 03-tools-system File Reference section
- Clean Go method-call symbols from 01-agent-loop Mermaid diagrams (Router + Resolver)
2026-04-19 16:12:57 +07:00
viettranx b6a7e7d754 docs: condense agent-loop and agent-teams body text
- 01-agent-loop: replace Go function names in sanitize step details and
  mermaid labels with behavioral descriptions; clean resolver resolved
  properties, router cache/run-tracking, and team workspace context
  variables of symbol references
- 11-agent-teams: compress mailbox 3-action table + Use Cases into
  3 narrative paragraphs; replace WorkspaceDir Go code block with
  prose description
2026-04-19 15:41:44 +07:00
viettranx cd1b3f267b docs: condense security, rpc, auth, evolution body text
- 09-security: compress 9-bullet edge-case list to 6-row Attack Surface/Mitigation table
- 20-api-keys-auth: replace auth.go:tokenMatch + router.go:handleConnect citations with behavior descriptions; remove RoleFromScopes() file ref
- 21-agent-evolution: remove resolver.go:346 code block + loop.go/loop_history.go line citations, replace with behavioral descriptions
- 19-websocket-rpc: already clean (no body-text .go:line refs)
2026-04-19 15:41:16 +07:00
viettranx 463d3d1c5d docs: replace file-reference lists with module tables (batch B) 2026-04-19 15:38:26 +07:00
viettranx 653df6fb52 docs: replace file-reference lists with module tables (batch A)
Replace trailing File Reference sections in 7 docs with 3-4 row
module-level tables. Column schema: Module | Path | Purpose.
Adds 1-line grep hint at end of each section. Also fixes one
body .go:line citation in 05-channels-messaging.md.
2026-04-19 15:37:59 +07:00
viettranx d7e2883926 docs: consolidate HTTP API into single v3-only reference
Merge docs/22-v3-http-endpoints.md into docs/18-http-api.md, adding
episodic memory, evolution metrics, vault, orchestration, and v3 flags
sections. Drop File Reference table in favour of 4-row module table.
Add missing live routes (TTS, backup/restore, CLI agent grants, cancel-
summon, system-prompt-preview, vault full CRUD, github-releases,
shell-deny-groups). Delete docs/22-v3-http-endpoints.md and update
cross-references in doc 21.
2026-04-19 15:33:24 +07:00
viettranx 985de60d93 docs: condense tools-system to concept-focused reference
Rewrote docs/03-tools-system.md from 987 lines to 519 lines.
Removed Go type/const definitions, function citations, 47-file
path listing, and static regex patterns. Replaced with concept
tables, complete tool inventory cross-checked against live
internal/tools/ registrations, and 3-row module table.
2026-04-19 15:32:39 +07:00
viettranx d6ae866321 docs: condense ws-team-events to base+delta format 2026-04-19 15:31:58 +07:00
viettranx adc2a1fafe docs: delete stale internal changelog, redundant with root CHANGELOG.md
Root CHANGELOG.md is the user-facing single source of truth. docs/17-changelog.md
was an internal verbose log duplicating git history.

Also removed docs/multi-tenant-architecture.bk.md (gitignored, pre-v3 backup
superseded by 23-multi-tenant-architecture.md).

Phase 01 of plans/260419-1344-docs-audit-and-condense.
2026-04-19 15:21:01 +07:00
6d7389539a fix(vault): prevent vault_read id-namespace collision (#959)
* fix(vault): prevent vault_read id-namespace collision

vault_search was leaking KG/episodic entity ids into result sets even when
narrow `types` were requested, and callers then passed those ids to
vault_read which returned a generic "document not found". The cause was
threefold:

1. `types` filter was only applied to the vault fan-out; KG and episodic
   ran unconditionally. Now gated by shouldFanout(types, key).
2. vault_search output lacked a per-source tool hint. Each result now ends
   with " → use <tool>" naming the correct follow-up (vault_read,
   knowledge_graph_search, or memory_search).
3. vault_read miss returned "document not found" without checking whether
   the id belonged to a foreign namespace. It now probes KG then episodic
   and returns a namespace-specific redirect error. Stores are injected
   via SetKGStore/SetEpisodicStore, nil-safe, tenant-scoped.

Adds red→green characterization tests plus an end-to-end integration
scenario seeding a vault doc + KG entity with identical basenames.

* test(agent): bump none-mode prompt size budget to 3100

vault_read wiring (#948) added ~95 chars to read_file tool summary,
pushing none-mode prompt from <3000 to 3075 chars. Bump budget to
3100 (~775 tokens) to match the intentional addition.

* test(integration): ensure data_migrations table exists in reset helper

The reset helper runs before RunPendingHooks, but RunPendingHooks is
what normally creates data_migrations. On a fresh CI database the
DELETE fails with 'relation does not exist'. Create the table
defensively so reset works regardless of execution order.

* refactor(vault): per-source id fields + wire episodic into search

Align vault_search output fields with downstream tool input params:
doc_id (vault_read), entity_id (knowledge_graph_search), episodic_id
(memory_expand). Prevents LLMs from pattern-matching a generic `id:`
and misrouting a foreign-namespace uuid into vault_read. Fallback
redirect in vault_read now quotes id + names the correct param so
the LLM can self-correct in one turn.

Also wire stores.Episodic into VaultSearchService (stale comment
claimed pending-impl; PGEpisodicStore has existed and been in use
since v3). Unifies search fan-out with vault_read namespace probe.

---------

Co-authored-by: viettranx <viettranx@gmail.com>
2026-04-19 15:00:07 +07:00
Viet TranandGitHub 7b83e890aa feat(ui): a11y + toggle + viewport-close for voice picker (#964)
- aria-haspopup/aria-expanded/aria-controls on trigger; role="listbox"
  + aria-label on dropdown; role="option" + aria-selected on rows. Listbox
  id from useId() so multiple pickers don't collide.
- Clicking the trigger while open now closes it (toggle), not a no-op.
- Close on window scroll/resize: the fixed-position dropdown doesn't
  reflow with the viewport, so pinning it in the wrong place is worse
  than closing.
2026-04-19 14:29:20 +07:00
David MizrahiandGitHub 597e59ac04 fix(tts): make ElevenLabs voice picker selectable with portaled dropdown (#960) 2026-04-19 14:25:59 +07:00
Viet TranandGitHub fe90f6fed9 fix(tools): clean up Windows exec merge leftovers + expand credentialed env (#963)
- shell.go: remove commented-out git conflict markers; restore load-bearing
  rationale for plain exec.Command vs CommandContext (process-group kill).
- credentialed_exec.go: pass through SYSTEMROOT, TEMP, APPDATA and other
  Windows runtime vars that native CLIs (gh, az, aws, npm) require. Missing
  SYSTEMROOT breaks networking/registry APIs in most Win32 programs. Switch
  to USERNAME (Windows) vs USER (POSIX) per platform convention.
2026-04-19 14:23:26 +07:00
steelstringandGitHub 2bc8e909f1 Fix/desktop lite edition cmd exec failure (#961)
* This fixes a data integrity issue affecting fresh installations of Desktop Lite and any environment where agent.other_config and serveral columns are not pre-initialized. This will stop Desktop Lite edtion from saving any agent setting.

The root cause is missing default initialization of JSON field in update flow, leading to NOT NULL violations in both SQLite and Postgres.

Fix ensures consistent behavior across Lite and Standard deployments.

* Fix: Destop Lite Windows edition exec failure
The executing of local commands use sh wrongly.
2026-04-19 14:20:21 +07:00
viettranx b967055560 feat(channels): shared WriterLabel helper with prompt-injection sanitization 2026-04-18 22:49:51 +07:00
viettranx 4bc7ca1320 fix(store): tenant-scope GetContactsBySenderIDs to prevent cross-tenant leak 2026-04-18 22:49:48 +07:00
viettranx 9333fee5b6 feat(agent): add affirmative writer hint + sender identity to group permissions prompt 2026-04-18 22:43:01 +07:00
viettranx 6e935b3314 fix(agent): fallback 'User <id>' in group writer prompt when metadata empty 2026-04-18 22:38:16 +07:00
viettranx d4a08c818f fix(agent): broaden NO_REPLY detection to cover decorative variants
Models occasionally emit NO_REPLY with trailing underscores, quotes,
markdown emphasis, or short explanations (e.g. `NO_REPLY_`,
`**NO_REPLY**`, `NO_REPLY: offline`). Previous word-boundary check
treated `_` as a word char so `NO_REPLY_` slipped through and was
delivered to end users.

Rewrite IsSilentReply to strip decorative wrappers, then prefix-match
case-insensitively. Token is only rejected when glued to another
word character (e.g. `NO_REPLYING`). Intentional divergence from
upstream #19537 — documented in the doc comment.
2026-04-18 21:46:26 +07:00
viettranx 59a8c25349 revert(http): remove admin backfill endpoint — lazy heal on /writers is enough 2026-04-18 21:43:19 +07:00
viettranx 78e2882335 feat(ui): unknown writer label fallback + i18n (en/vi/zh) 2026-04-18 21:35:54 +07:00
viettranx f1c1a466f0 feat(http): admin backfill endpoint for legacy writer metadata 2026-04-18 21:35:50 +07:00
viettranx d9c453cd49 feat(telegram): lazy self-heal writer metadata on /writers + fallback label 2026-04-18 21:35:48 +07:00
viettranx e66badb803 fix(permissions): auto-enrich file_writer metadata on grant from Web UI 2026-04-18 21:35:46 +07:00
viettranx cadca23cab feat(channels): add MemberResolver interface + Telegram getChatMember impl 2026-04-18 21:35:44 +07:00
viettranx 9f68f3ee0d feat(pancake): auto-react allow/deny scope filter + UI toggle
Extends #919 with per-channel scope control for Facebook auto-react.

- AutoReactOptions pointer struct (allow/deny post & user IDs)
- filterAutoReact + containsString helpers; deny overrides allow
- UI surfaces features.auto_react toggle + 4 tags fields
- Nil options = react-all (backward compatible)
2026-04-18 17:55:11 +07:00
viettranx 2abb317e02 docs: update tools system and changelog for web_search refactor
Update docs/03-tools-system.md to document:
- Per-tenant web_search provider chain resolution
- Tenant settings schema (provider_order, per-provider disable)
- Cache TTL and event-driven invalidation
- Config secrets lookup for API keys

Update docs/17-changelog.md with web_search tenant-scoping feature.
2026-04-18 17:47:39 +07:00
viettranx 18e48cd0fc test(integration): add web_search tenant isolation and migration hook tests
Add comprehensive integration tests:
- web_search_tenant_isolation_test: verify per-tenant provider chains, cache
  hit/miss behavior, event-driven invalidation
- web_search_migrate_hook_test: verify migration of inline keys from
  config.json5 and builtin_tool_tenant_configs.settings to config_secrets

Tests cover multi-tenant scenarios, cache expiry, concurrent execution.
2026-04-18 17:47:36 +07:00
viettranx 96b75f0413 refactor(ui): remove web_search config section
Remove ToolsWebSection from config page and clean up i18n keys for web
search provider configuration (duckduckgo, exa, tavily, brave). Web search
now configured entirely via builtin_tool_tenant_configs REST API.

Delete tools-web-section.tsx component and update locale catalogs (en/vi/zh).
2026-04-18 17:47:33 +07:00