docs: phase 02 completion — voice/audio system docs + architecture updates

Update architecture overview with streaming TTS provider layer. Expand tools
system docs with voice/model resolution. Update HTTP and WebSocket RPC
documentation for voice endpoints. Record Phase 02 completion in changelog.
Update CLAUDE.md with voice picker and streaming TTS patterns.
This commit is contained in:
viettranx
2026-04-14 23:35:44 +07:00
parent a7c8170c4a
commit cf16cf53db
6 changed files with 181 additions and 12 deletions
+4
View File
@@ -200,10 +200,14 @@ Apply before finalizing any multi-phase plan. Trust-but-verify between scout →
10. **Context key style convention** — check existing `context.go` pattern before introducing new key types. Mixed = code smell.
11. **Alias/shim coverage** — enumerate ALL exported symbols via `go doc <pkg>`. Add compile-time signature guards.
12. **Verify pass MANDATORY after rewrite** — spawn fresh Explore/grep to audit planner output. Don't trust self-validation.
13. **No fabricated Go identifiers** — every type/function/package referenced in plan must be grep-verified to exist (`grep -rn '^func <name>\|^type <name>' <pkg>`). Plausible-sounding APIs (`Keyring`, `StartSpan`, `Validator`, `security.SSRF`) are RED FLAGS — conventions differ per codebase. Rule of thumb: if you can't cite `file_path:line_number` for a symbol, you're fabricating. Apply especially when planner says "reuse existing X" — re-verify X exists before writing into plan.
14. **Plausible-but-wrong API families** — watch for: OTel-style `StartSpan/EndSpan` (codebase may be emit-based); wrapper types like `Keyring/Vault/Manager` (codebase may use free functions); `internal/security` or `internal/auth` packages (often scattered instead of centralized). When unsure, `go doc <pkg>` lists actual exported surface.
**Pattern to avoid:** user asks → planner writes → report "done".
**Safer pattern:** user asks → scout → planner writes → audit-verify → report.
**Concrete red-team practice:** After planner completes, run `code-reviewer`/`brainstormer` agent in audit mode with explicit instruction "spot-check 15+ factual claims against live codebase". Caught fabricated `internal/security/`, `crypto.Keyring`, `tracing.StartSpan` in Agent Hooks plan (see `plans/260414-2229-agent-hooks-system/reports/audit-260414-2310-plan-review.md`).
## Post-Implementation Checklist
After implementing or modifying Go code, run these checks:
+23
View File
@@ -475,6 +475,29 @@ Six distinct workspace scenarios:
- **L0 Auto-Injection**: Top-K vault entries injected into system prompt as "relevant context from vault"
- **Filesystem Sync**: Vault entries exported as `.md` files for manual editing, re-imported with change tracking
### Audio & Voice System (ElevenLabs + Streaming TTS)
**Provider architecture** (`internal/audio/`):
- **`Manager`**: Central orchestrator dispatching TTS/STT/Music/SFX requests to pluggable providers
- **`TTSProvider` interface**: Core text-to-speech contract (blocking, buffered response)
- **`StreamingTTSProvider` interface**: Optional interface for ElevenLabs `/stream` endpoint (chunked audio via `io.ReadCloser`)
- **Implementations**: ElevenLabs, OpenAI, Edge, MiniMax (phase-gated; STT/Music/SFX partial)
**Voice discovery & caching**:
- **Voice cache** (`internal/audio/voice_cache.go`): In-memory LRU (cap 1000 tenants, TTL 1h) shared by HTTP `/v1/voices` + WS `voices.list` handlers. Thread-safe with `sync.Mutex` (LRU updates require write lock)
- **Cache miss recovery**: HTTP handler auto-fetches from ElevenLabs; WS handler requires prior cache warm (provider resolution deferred to Phase 3)
- **Agent audio context** (`store.WithAgentAudio` / `AgentAudioFromCtx`): Immutable snapshot bundle (AgentID + OtherConfig JSONB) injected by dispatcher before tool dispatch; consumed by `TtsTool.Execute` for voice/model resolution
**Agent-level configuration** (`agents.other_config` JSONB):
- `tts_voice_id`: ElevenLabs voice ID (e.g., "pMsXgVXv3BLzUgSXRplE")
- `tts_model_id`: Model choice (eleven_v3, eleven_flash_v2_5, eleven_multilingual_v2, eleven_turbo_v2_5)
- Resolution precedence: CLI args → agent config → tenant override → provider default
**Web UI voice picker** (`ui/web/src/components/voice-picker.tsx`):
- Combobox with BM25 search, preview playback button (HTML `<audio>`)
- Handles preview CDN 403 (expiry) via `onError` → auto-refresh cache
- Embedded in PromptSettingsSection, bound to `other_config.tts_voice_id`
---
## Cross-References
+8 -1
View File
@@ -812,10 +812,17 @@ func (t *MyTool) Execute(ctx context.Context, params MyParams) (*Result, error)
#### tts (tenant override shape)
```json
{
"primary": "elevenlabs"
"primary": "elevenlabs",
"default_voice_id": "pMsXgVXv3BLzUgSXRplE",
"default_model": "eleven_flash_v2_5"
}
```
**Fields:**
- `primary`: Provider selection (elevenlabs, openai, edge, minimax)
- `default_voice_id`: Tenant-level voice ID fallback (ElevenLabs); overridable per-agent via `agent.other_config.tts_voice_id`
- `default_model`: Tenant-level model choice (eleven_v3, eleven_flash_v2_5, eleven_multilingual_v2, eleven_turbo_v2_5); overridable per-agent via `agent.other_config.tts_model_id`
**Secret vs non-secret split:**
- Non-secret (provider priorities, max_results, allowed_domains): `builtin_tool_tenant_configs.settings` (editable via UI)
- Secret (API keys): `config_secrets` table (encrypted, tenant-scoped)
+16
View File
@@ -48,6 +48,22 @@ Unified audio provider management via new `internal/audio/` package with pluggab
**Impact**: Existing TTS flows fully compatible. New code can import `internal/audio` directly. STT/Music/SFX wiring deferred to Phase 3-4.
#### ElevenLabs Audio Manager Enhancements — Phase 2 (2026-04-14)
Voice discovery and agent-level audio config via new backend endpoints, in-memory cache, and web UI picker. Bundles producer/consumer context pattern (`store.WithAgentAudio`) for seamless voice/model resolution throughout the tool execution pipeline.
**What changed:**
- **Voice cache** (`internal/audio/voice_cache.go`): In-memory LRU (cap 1000 tenants) with TTL 1h, shared between HTTP + WS handlers, thread-safe under concurrent access
- **Streaming TTS interface** (`internal/audio/types.go`): New `StreamingTTSProvider` optional interface for ElevenLabs `/v1/text-to-speech/{voice_id}/stream` chunked playback
- **ElevenLabs enhancements**: Model allowlist (11_v3, eleven_flash_v2_5, eleven_multilingual_v2, eleven_turbo_v2_5), `SynthesizeStream()` method, `ListVoices()` via `/v1/voices`
- **Agent audio context** (`internal/store/context.go`): New `WithAgentAudio` / `AgentAudioFromCtx` bundle; producer wires snapshot at dispatcher level (internal/agent/), consumer (`TtsTool.Execute`) reads voice/model overrides from agent config
- **Agent config extension** (`agents.other_config` JSONB): New `tts_voice_id` and `tts_model_id` fields with resolution precedence: args → agent → tenant → provider default
- **HTTP + WS endpoints**: GET /v1/voices (cached), POST /v1/voices/refresh (admin-only), WS method `voices.list` + `voices.refresh`
- **Web voice picker** (`ui/web/src/components/voice-picker.tsx`): Combobox with search, preview button (HTML audio + onError → refresh), embedded in PromptSettingsSection
- **i18n**: 10 new frontend keys (voice_label, voice_placeholder, voice_refresh, voice_preview, model_label, etc.) + 2 new backend keys (MsgTtsUnknownModel, MsgVoicesListFailed) across en/vi/zh
**Impact**: Existing TTS callers fully compatible (backward-compat via Phase 1 alias layer). Web UI gains voice discovery + per-agent voice/model overrides. Zero breaking changes. Integration test validates producer+consumer context flow.
#### Trace Stop/Abort Redesign — Cascading 4-Layer Fix (2026-04-14)
The Stop button on the traces page now reliably aborts running traces. Previous implementation had independent race conditions across HTTP streaming, agent router, trace persistence, and UI polling; this redesign fixes all four layers atomically.
+68 -11
View File
@@ -798,7 +798,64 @@ Workspace file management.
---
## 22. Media
## 22. Voices & Audio
Voice discovery for TTS providers (ElevenLabs). All endpoints are tenant-scoped and require tenant admin or operator role.
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/v1/voices` | List available voices (in-memory cached, TTL 1h) |
| `POST` | `/v1/voices/refresh` | Force refresh voice cache (admin-only) |
### `GET /v1/voices`
Fetch available TTS voices for the current tenant's provider.
**Query Parameters:**
- None
**Response** (200 OK):
```json
[
{
"voice_id": "pMsXgVXv3BLzUgSXRplE",
"name": "Alice",
"preview_url": "https://...",
"category": "premade",
"labels": {
"use_case": "conversational",
"accent": "american"
}
}
]
```
**Errors:**
- 401: Missing or invalid token
- 403: Insufficient permissions (requires tenant admin/operator)
- 500: Provider error (e.g., ElevenLabs API unreachable)
**Caching:**
- Responses are in-memory cached per tenant with TTL 1h
- Cache is shared across all HTTP + WebSocket handlers
- Cache miss triggers immediate fetch from provider
### `POST /v1/voices/refresh`
Invalidate the voice cache for the current tenant, forcing a fresh fetch on the next request.
**Admin-only endpoint.** Useful after voice updates or CDN expiry issues.
**Request body:** (empty)
**Response** (202 Accepted):
```json
{ "message": "voice cache invalidated" }
```
---
## 23. Media
| Method | Path | Description |
|--------|------|-------------|
@@ -807,7 +864,7 @@ Workspace file management.
---
## 23. Files
## 24. Files
| Method | Path | Description |
|--------|------|-------------|
@@ -818,7 +875,7 @@ Auth via Bearer token or `?token=` query param (for `<img>` tags). MIME type aut
---
## 24. API Keys
## 25. API Keys
Admin-only endpoints for managing gateway API keys. See [20 — API Keys & Auth](20-api-keys-auth.md) for the full authentication and authorization model.
@@ -856,7 +913,7 @@ Admin-only endpoints for managing gateway API keys. See [20 — API Keys & Auth]
---
## 25. OAuth
## 26. OAuth
| Method | Path | Description |
|--------|------|-------------|
@@ -943,7 +1000,7 @@ Notes:
---
## 26. Edition
## 27. Edition
| Method | Path | Description |
|--------|------|-------------|
@@ -951,7 +1008,7 @@ Notes:
---
## 27. Tenants
## 28. Tenants
Multi-tenant management (admin only).
@@ -967,7 +1024,7 @@ Multi-tenant management (admin only).
---
## 28. System Configs
## 29. System Configs
Key-value system configuration store.
@@ -980,7 +1037,7 @@ Key-value system configuration store.
---
## 29. Team Workspace & Attachments
## 30. Team Workspace & Attachments
| Method | Path | Description |
|--------|------|-------------|
@@ -990,7 +1047,7 @@ Key-value system configuration store.
---
## 30. Shell Deny Groups
## 31. Shell Deny Groups
| Method | Path | Description |
|--------|------|-------------|
@@ -998,7 +1055,7 @@ Key-value system configuration store.
---
## 31. System
## 32. System
| Method | Path | Description |
|--------|------|-------------|
@@ -1017,7 +1074,7 @@ Key-value system configuration store.
---
## 32. MCP Bridge
## 33. MCP Bridge
Exposes GoClaw tools to Claude CLI via streamable HTTP at `/mcp/bridge`. Only listens on localhost. Protected by gateway token with HMAC-signed context headers.
+62
View File
@@ -501,6 +501,68 @@ Multi-tenant management (admin only).
---
## 17.1. Voices (Voice Discovery)
Discover available TTS voices for the tenant's configured provider.
| Method | Description |
|--------|-------------|
| `voices.list` | Fetch available voices (in-memory cached, TTL 1h) |
| `voices.refresh` | Force cache invalidation (admin-only) |
### `voices.list` Request
```json
{
"method": "voices.list",
"id": 1
}
```
**Response** (200 OK):
```json
{
"id": 1,
"result": [
{
"voice_id": "pMsXgVXv3BLzUgSXRplE",
"name": "Alice",
"preview_url": "https://...",
"category": "premade",
"labels": {
"use_case": "conversational",
"accent": "american"
}
}
]
}
```
**Errors:**
- `code: -1`: Provider error (e.g., ElevenLabs API unreachable)
- `code: -2`: Cache miss + no provider context available (desktop edition in Phase 2; HTTP handler resolves provider dynamically)
### `voices.refresh` Request
Admin-only. Invalidate tenant cache, forcing fresh fetch on next list.
```json
{
"method": "voices.refresh",
"id": 2
}
```
**Response** (200 OK):
```json
{
"id": 2,
"result": { "message": "voice cache invalidated" }
}
```
---
## 18. Browser Automation
| Method | Description |