diff --git a/docs/00-architecture-overview.md b/docs/00-architecture-overview.md index d2a5b6f7..8f46c4cb 100644 --- a/docs/00-architecture-overview.md +++ b/docs/00-architecture-overview.md @@ -481,7 +481,7 @@ V3 introduces a **pluggable 8-stage pipeline** (replacing the monolithic `runLoo | Stage | Phase | Responsibility | |-------|-------|-----------------| -| **ContextStage** | Setup (once) | Inject agent/user/workspace context, compute per-user files | +| **ContextStage** | Setup (once) | Inject agent/user/workspace context, compute per-user files, calculate token overhead (system prompt, tools, etc.) | | **ThinkStage** | Iteration | Build system prompt, filter tools by policy, call LLM | | **PruneStage** | Iteration | Context pruning (2-pass: soft trim → hard clear), run memory flush if compaction triggered | | **ToolStage** | Iteration | Execute tool calls (parallel goroutines for multiple calls) | diff --git a/docs/codebase-summary.md b/docs/codebase-summary.md index 4fcba5f9..f3fc3a32 100644 --- a/docs/codebase-summary.md +++ b/docs/codebase-summary.md @@ -143,7 +143,7 @@ Native `image_generation` support in the Codex provider (`POST /codex/responses` ## Key Conventions - **Store layer:** Interface-based; PG (`store/pg/`) + SQLite (`store/sqlitestore/`). Raw SQL, `$1/$2` params. -- **Session token display:** v3 compaction now uses dynamic max_tokens; session token display reads from `sessions.metadata.last_prompt_tokens`. +- **Session token display:** v3 compaction now uses dynamic max_tokens (`in/25` clamped `[1024,8192]`); session token display reads from `sessions.metadata.last_prompt_tokens` and `last_message_count`. Tool schemas counted via `TokenCounter.CountToolSchemas()` and included in ContextStage overhead. - **Context propagation:** `store.WithLocale`, `store.WithUserID`, `store.WithTenantID`, etc. - **Security logs:** `slog.Warn("security.*")` for all security events. - **SSRF prevention:** `validateProviderURL()` in `internal/http/tts_validate.go`. diff --git a/docs/project-changelog.md b/docs/project-changelog.md index 84fbccb0..24c867d5 100644 --- a/docs/project-changelog.md +++ b/docs/project-changelog.md @@ -37,6 +37,32 @@ Implementation is evidence-backed against the native ChatGPT Responses API event ## 2026-04-20 +### Pipeline: accurate context token tracking + dynamic compaction + +**Features** + +- **Session token display from metadata:** `sessions.metadata` now carries `last_prompt_tokens` and `last_message_count` on finalize. List query reads from metadata; fallback to octet/rune heuristic when absent. Fixes stale token display across session re-opens. +- **Tool-schema token accounting:** `TokenCounter.CountToolSchemas(model, tools)` new method counts tool definitions serialized as JSON. Tool-schema tokens included in `OverheadTokens` at ContextStage. +- **Dynamic compaction max_tokens:** Compaction `max_tokens` now derived from `in/25` with clamp `[1024, 8192]`. Applied to both summarization flow (`loop_compact.go`) and history sanitization (`loop_history_sanitize.go`). Replaces static 4096 limit — adapts budget to context size. + +**Code** + +- `internal/store/pg/sessions_list.go` — read/write `last_prompt_tokens` and `last_message_count` in metadata. +- `internal/store/sqlitestore/sessions*.go` — parity SQLite store updates. +- `internal/tokencount/token_counter.go` — `CountToolSchemas` interface method + `tiktoken_counter.go` impl. +- `internal/pipeline/context_stage.go` — include tool overhead in `OverheadTokens`. +- `internal/agent/loop_compact.go` — `dynamicSummaryMax` function; apply to compaction call. +- `internal/agent/loop_history_sanitize.go` — apply dynamic max to sanitization. + +**Tests** + +- `internal/tokencount/count_tool_schemas_test.go` — tool schema token counting. +- `internal/agent/loop_compact_dynamic_max_test.go` — dynamic max_tokens clamping. +- `internal/pipeline/context_stage_tool_overhead_test.go` — tool overhead integration. +- `internal/store/sqlitestore/sessions_display_tokens_integration_test.go` — metadata round-trip. + +--- + ### TTS: timeout tenant-config + Gemini text-only 400 fix **Features & Fixes**