chore: add plan and phase reports for D1 + cron rollout

2026-04-17 17:21:30 +00:00 · 2026-04-15 13:29:48 +07:00
parent f5e03cfff2
commit 6a4829e45b
13 changed files with 920 additions and 0 deletions
--- a/plans/reports/docs-manager-260415-1304-phase-06-docs.md
+++ b/plans/reports/docs-manager-260415-1304-phase-06-docs.md
@@ -0,0 +1,132 @@
+# Phase 06 Documentation — Completion Report
+
+**Date:** 2026-04-15  
+**Status:** DONE  
+**Plan:** D1 + Cron Infra + JSDoc + Trading History (Phase 06)
+
+## Summary
+
+Executed Phase 06 of the D1+Cron infra plan — created comprehensive documentation for D1 + cron usage and updated all existing docs to reflect current implementation. All module authors now have clear guidance for adopting persistent storage and scheduled jobs without needing to read framework internals.
+
+## Files Created
+
+1. **`docs/using-d1.md`** (~320 LOC)
+   - When to choose D1 vs KV (decision matrix included)
+   - Module initialization pattern: `init({ db, sql, env })`
+   - Table naming convention: `{moduleName}_{table}`
+   - Writing migrations: `src/modules/<name>/migrations/NNNN_*.sql` with lexical sorting
+   - SQL API reference: `run()`, `all()`, `first()`, `prepare()`, `batch()`
+   - Migration execution: `npm run db:migrate` with `--local` and `--dry-run` flags
+   - Testing with `FakeD1` from `tests/fakes/fake-d1.js`
+   - First-time D1 setup: `wrangler d1 create` workflow
+   - Worked example: simple counter with migration + handler
+
+2. **`docs/using-cron.md`** (~300 LOC)
+   - Cron declaration: `crons: [{ schedule, name, handler }]` in module export
+   - Handler signature: `(event, { db, sql, env })` receives module context
+   - Cron expression syntax: 5-field standard with Cloudflare docs reference
+   - Critical: manual `wrangler.toml` registration required (`[triggers] crons`)
+   - Error isolation: one handler failing doesn't block others
+   - Execution limits: 15-minute wall-clock per task
+   - Local testing: `curl "http://localhost:8787/__scheduled?cron=<schedule>"`
+   - Multiple modules can share a schedule (fan-out)
+   - Worked example: trade retention cleanup cron
+   - Worked example: stats recalculation
+
+## Files Updated
+
+1. **`docs/adding-a-module.md`** (+95 LOC)
+   - New section: "Optional: D1 Storage" — init pattern, migrations, npm run db:migrate
+   - New section: "Optional: Scheduled Jobs" — crons array, wrangler.toml requirement
+   - Cross-links to `using-d1.md` and `using-cron.md`
+   - Updated testing section to mention `fake-d1.js`
+   - Referenced trading module as full example (D1 + crons)
+
+2. **`docs/architecture.md`** (+100 LOC)
+   - Module contract code block: added `sql` parameter and `crons` array
+   - Storage section: separate subsections for KVStore and SqlStore
+   - New scheduled event flow diagram (flow from Cloudflare cron to handler dispatch)
+   - Added `dispatchScheduled` and cron isolation details
+
+3. **`docs/codebase-summary.md`** (+30 LOC)
+   - Module table: added Storage + Crons columns
+   - Key data flows: added scheduled job flow diagram
+   - Deploy pipeline: noted `npm run db:migrate` step
+   - Test coverage: updated to mention D1 coverage + fake-d1.js
+
+4. **`docs/code-standards.md`** (+40 LOC)
+   - Module conventions: updated to show `sql` parameter and null-guards
+   - New section: "JSDoc & Type Definitions" — central location `src/types.js`, when to add, validation rules
+   - Noted ESLint (`eslint src`) runs alongside Biome
+
+5. **`CLAUDE.md`** (project root) (+15 LOC)
+   - Commands section: added `npm run db:migrate` with all flags
+   - Module contract: updated to show `sql`, `env`, and `crons`
+   - Cross-linked to `using-d1.md` and `using-cron.md`
+
+6. **`docs/deployment-guide.md`** (+80 LOC)
+   - Prerequisites: added D1 to "Cloudflare account with" checklist
+   - New section: "Cloudflare D1 Database" (optional but recommended) — `wrangler d1 create`, paste UUID, run migrations
+   - Renumbered sections (KV is now section 2, secrets is now section 3, local config is now section 4)
+   - New section in Deploy: "Cron Configuration" — explain wrangler.toml `[triggers] crons` requirement
+   - Updated first-time deploy flow: added D1 setup + migration steps
+
+7. **`README.md`** (+80 LOC)
+   - Why section: added D1 + crons to feature bullets
+   - Architecture snapshot tree: added `types.js`, D1 files (`cf-sql-store.js`, `create-sql-store.js`), cron files (`cron-dispatcher.js`, `validate-cron.js`), migrations example, fake-d1.js, migrate.js
+   - Local dev: added `npm run db:migrate` and noted `/__scheduled` endpoint
+   - Deploy section: added D1 setup and migration steps to first-time flow
+   - Further reading: added links to `using-d1.md` and `using-cron.md`, updated plan references
+
+8. **`plans/260415-1010-d1-cron-infra/phase-06-docs.md`**
+   - Status: Todo → Complete
+   - All todos: checked off
+
+9. **`plans/260415-1010-d1-cron-infra/plan.md`**
+   - Status: Draft → Complete
+   - Phase 06: Todo → Complete
+
+## Verification
+
+All doc references verified against shipped code:
+
+- ✓ `createSqlStore(moduleName, env)` returns `null` when `env.DB` not bound (checked `src/db/create-sql-store.js`)
+- ✓ `sql.run/all/first/prepare/batch` API matches `src/db/sql-store-interface.js` JSDoc
+- ✓ Table prefix pattern `{moduleName}_{table}` enforced by `createSqlStore`
+- ✓ Migration runner at `scripts/migrate.js` walks `src/modules/*/migrations/*.sql`, applies via `wrangler d1 execute`, tracks in `_migrations`
+- ✓ Cron handler signature `(event, { db, sql, env })` matches `src/modules/cron-dispatcher.js`
+- ✓ Trading module exports `crons` with schedule `"0 17 * * *"` (verified `src/modules/trading/index.js`)
+- ✓ `wrangler.toml` has `[triggers] crons = ["0 17 * * *"]` matching module declaration
+- ✓ `src/index.js` exports both `fetch` and `scheduled(event, env, ctx)` handlers
+- ✓ `src/types.js` defines all central typedefs (Env, Module, Command, Cron, ModuleContext, etc.)
+- ✓ `validateCron` enforces name regex and schedule format (verified `src/modules/validate-cron.js`)
+
+## Documentation Quality
+
+- **Tone consistency:** All new docs match existing style (clear, code-first, practical examples)
+- **Cross-linking:** New docs link to each other + existing docs; no orphaned pages
+- **Code examples:** All examples based on actual shipped code (trading module, fake-d1 tests, migration runner)
+- **Completeness:** Covers happy path (module author perspective) + guard clauses (null-safety, error handling)
+- **Searchability:** Docs well-organized by topic (when to use, how to implement, testing, examples, troubleshooting)
+
+## Deliverables Checklist
+
+- [x] `docs/using-d1.md` created
+- [x] `docs/using-cron.md` created
+- [x] `docs/adding-a-module.md` updated with cron + D1 sections
+- [x] `docs/architecture.md` updated with storage details + scheduled flow
+- [x] `docs/codebase-summary.md` updated (module table, flows, coverage)
+- [x] `docs/code-standards.md` updated (JSDoc section, module conventions)
+- [x] `docs/deployment-guide.md` updated (D1 setup, migration steps, cron registration)
+- [x] `CLAUDE.md` updated (module contract, commands)
+- [x] `README.md` updated (feature bullets, architecture tree, deploy flow, further reading)
+- [x] Plan phase 06 marked complete
+- [x] Overall plan marked complete
+
+All 9 deliverable targets from phase-06-docs.md completed.
+
+## Concerns
+
+None. All deliverables shipped to spec. Documentation is current, accurate, and immediately actionable for future module authors. No stale references, broken links, or incomplete examples.
+
+**Status:** DONE
--- a/plans/reports/fullstack-260415-1052-phase-01-d1-setup.md
+++ b/plans/reports/fullstack-260415-1052-phase-01-d1-setup.md
@@ -0,0 +1,27 @@
+# Phase 01 — D1 Setup — Implementation Report
+
+## Files Created
+- `src/db/sql-store-interface.js` — JSDoc typedefs for SqlStore/SqlRunResult
+- `src/db/cf-sql-store.js` — CFSqlStore class wrapping env.DB prepare/run/all/first/batch
+- `src/db/create-sql-store.js` — factory; returns null when env.DB absent, SqlStore otherwise; exposes tablePrefix
+- `tests/fakes/fake-d1.js` — in-memory D1 fake with seed(), runLog, queryLog; naive table extraction from SQL text
+- `tests/db/create-sql-store.test.js` — 13 tests: validation, tablePrefix, run/all/first/batch round-trips
+- `scripts/migrate.js` — custom migration runner; walks src/modules/*/migrations/*.sql sorted, tracks applied in _migrations table, supports --dry-run and --local flags
+
+## Files Modified
+- `src/modules/registry.js` — added createSqlStore import; passes `sql: createSqlStore(mod.name, env)` into init alongside db
+- `wrangler.toml` — added [[d1_databases]] block; database_id = REPLACE_ME_D1_UUID (requires manual fill after `wrangler d1 create`)
+- `package.json` — added `"db:migrate": "node scripts/migrate.js"`; chained into deploy: `wrangler deploy && npm run db:migrate && npm run register`
+
+## Deviations
+- `dispatcher.js`: no change needed — it delegates entirely to buildRegistry which already handles init; spec note resolved.
+- `basename` import in migrate.js kept (unused but Biome didn't flag it as unused import — left for future use). Actually removed by format — clean.
+
+## Test Results
+- npm test: 118/118 pass (12 files)
+- npm run lint: clean (0 errors)
+- register:dry: exits with "not found" for .env.deploy — expected in dev, no code regression
+
+## Notes
+- `wrangler.toml` placeholder `REPLACE_ME_D1_UUID` must be replaced with real UUID from `npx wrangler d1 create miti99bot-db` before deploying.
+- fake-d1 is a minimal fake (no SQL parser); tests that need real SQL semantics should use better-sqlite3 or Miniflare.
--- a/plans/reports/fullstack-260415-1052-phase-02-cron-wiring.md
+++ b/plans/reports/fullstack-260415-1052-phase-02-cron-wiring.md
@@ -0,0 +1,26 @@
+# Phase 02 Report — Cron Wiring
+
+## Files Modified/Created
+
+- **Created** `src/modules/validate-cron.js` — validates `{ name, schedule, handler }` entries; 5/6-field cron regex check
+- **Created** `src/modules/cron-dispatcher.js` — `dispatchScheduled(event, env, ctx, registry)` fan-out with per-handler try/catch
+- **Modified** `src/modules/registry.js` — added `validateCron` import; cron validation + duplicate-name check in `loadModules`; `CronEntry[]` collection in `buildRegistry`; `registry.crons` exposed in typedef and return value
+- **Modified** `src/bot.js` — added `getRegistry(env)` export; shares same memoized registry with `fetch` handler
+- **Modified** `src/index.js` — added `scheduled(event, env, ctx)` export; calls `getRegistry` then `dispatchScheduled`
+- **Modified** `wrangler.toml` — added `[triggers] crons = []` placeholder with authoring instructions
+- **Created** `tests/modules/validate-cron.test.js` — 9 tests (valid entry, bad name, bad schedule, bad handler)
+- **Created** `tests/modules/cron-dispatcher.test.js` — 6 tests (schedule match, no-match, fan-out, error isolation, ctx pass-through, empty registry)
+- **Modified** `tests/modules/registry.test.js` — added 6 cron collection tests (empty, collect, fan-out, duplicate name, non-array, invalid entry)
+
+## Tests
+
+- All 139 tests pass (`14 passed` files)
+- 0 new lint errors introduced (17 eslint errors are pre-existing: KVNamespace/D1Database/Request undefined types + CRLF line-ending biome format noise from Windows git autocrlf)
+
+## Deviations
+
+- `getRegistry(env)` added to `bot.js` rather than importing `buildRegistry` directly in `index.js` — avoids bypassing the Bot init path and ensures single shared memoized registry across `fetch` + `scheduled`.
+- Test fixture module names use kebab-case (`mod-a`, `mod-b`) to satisfy `createStore`'s `^[a-z0-9_-]+$` constraint (initial camelCase caused 2 failures, fixed immediately).
+
+**Status:** DONE
+**Summary:** Cron Triggers wired end-to-end — module contract extended with `crons[]`, dispatcher dispatches per schedule, registry collects + validates, `scheduled()` exported from worker entry. All tests green, no new lint issues.
--- a/plans/reports/fullstack-260415-1052-phase-03-trading-history.md
+++ b/plans/reports/fullstack-260415-1052-phase-03-trading-history.md
@@ -0,0 +1,38 @@
+# Phase 03 — Trading Trade History: Implementation Report
+
+## Files Modified / Created
+
+| File | Action |
+|---|---|
+| `src/modules/trading/migrations/0001_trades.sql` | Created — schema + 2 indexes |
+| `src/modules/trading/history.js` | Created — `recordTrade`, `listTrades`, `formatTradesHtml`, `createHistoryHandler` |
+| `src/modules/trading/handlers.js` | Modified — `handleBuy`/`handleSell` accept optional `onTrade` callback |
+| `src/modules/trading/index.js` | Modified — accept `sql` in init, wire `onTrade`, register `/history` command |
+| `tests/modules/trading/history.test.js` | Created — 21 tests |
+| `plans/260415-1010-d1-cron-infra/phase-03-trading-history.md` | Status → Complete, todos ticked |
+| `plans/260415-1010-d1-cron-infra/plan.md` | Phase 03 → Complete |
+
+## Tasks Completed
+
+- [x] Migration SQL (`trading_trades` + 2 indexes)
+- [x] `recordTrade` — inserts row, logs+swallows on failure, skips silently when sql=null
+- [x] `listTrades` — camelCase mapping, limit clamp [1..50], returns [] when sql=null
+- [x] `formatTradesHtml` — HTML-escaped symbols, BUY/SELL labels, Telegram HTML mode
+- [x] `createHistoryHandler` — parses N, defaults to 10, clamps to 50
+- [x] Wired into buy/sell via `onTrade` callback pattern (keeps handlers.js clean)
+- [x] `/history` registered as public command in index.js
+
+## Tests Status
+
+- Type check: N/A (plain JS)
+- Unit tests: **160/160 pass** (21 new in history.test.js)
+- Lint: **clean** (Biome + ESLint)
+
+## Design Notes
+
+- `onTrade` callback pattern chosen over passing `sql` directly into handlers — keeps handlers.js unaware of D1, easier to test in isolation.
+- `createHistoryHandler` takes `sql` at factory time; `index.js` uses a lazy wrapper `(ctx) => createHistoryHandler(sql)(ctx)` so the module-level `sql` variable (set in `init`) is captured correctly after startup.
+- `recordTrade` failure path: try/catch logs `console.error` and returns — portfolio KV remains source of truth.
+
+**Status:** DONE
+**Summary:** Phase 03 complete — trade history table, `/history` command, buy/sell wiring, 21 tests all green, lint clean.
--- a/plans/reports/fullstack-260415-1052-phase-04-retention-cron.md
+++ b/plans/reports/fullstack-260415-1052-phase-04-retention-cron.md
@@ -0,0 +1,35 @@
+# Phase 04 — Retention Cron Implementation Report
+
+**Date:** 2026-04-15
+**Status:** DONE
+
+## Files Modified
+
+- `tests/fakes/fake-d1.js` — enhanced with SQL-semantic SELECT/DELETE support (DISTINCT, ORDER BY ts DESC, LIMIT/OFFSET with bind params, DELETE WHERE id IN)
+- `src/modules/trading/retention.js` — created; per-user + global trim handler, overridable caps
+- `src/modules/trading/index.js` — imported `trimTradesHandler`, added `crons` array
+- `wrangler.toml` — `crons = ["0 17 * * *"]`
+- `tests/modules/trading/retention.test.js` — created; 9 tests covering all scenarios
+- `plans/260415-1010-d1-cron-infra/phase-04-retention-cron.md` — Status → Complete, todos ticked
+- `plans/260415-1010-d1-cron-infra/plan.md` — phase 04 → Complete
+
+## Tasks Completed
+
+- [x] `retention.js` with per-user + global trim (hybrid SELECT-then-DELETE approach)
+- [x] Caps exported as `PER_USER_CAP=1000`, `GLOBAL_CAP=10000`; overridable via optional arg for small-seed tests
+- [x] Cron entry wired in trading module (`schedule: "0 17 * * *"`, `name: "trim-trades"`)
+- [x] `wrangler.toml` schedule added
+- [x] 9 retention tests: per-user trim, small-user no-op, exact-cap no-op, multi-user, global trim, combined pass, idempotence, sql=null guard
+
+## Tests Status
+
+- Unit tests: 169/169 passed (all files)
+- Lint: clean (Biome + ESLint, 54 files)
+
+## Key Decision
+
+fake-d1 is SQL-less — DELETE naively cleared entire table. Chose option (b): hybrid SELECT-IDs-then-DELETE-by-id-list in `retention.js`. Enhanced fake-d1 to support targeted `DELETE WHERE id IN (?,...)` and `SELECT ... ORDER BY ts DESC LIMIT/OFFSET` with bind param resolution (`?` tokens resolved against binds iterator). This keeps tests meaningful without adding better-sqlite3 dependency.
+
+## Concerns
+
+None.
--- a/plans/reports/fullstack-260415-1052-phase-05-jsdoc.md
+++ b/plans/reports/fullstack-260415-1052-phase-05-jsdoc.md
@@ -0,0 +1,51 @@
+# Phase 05 — JSDoc Pass Report
+
+**Date:** 2026-04-15
+**Phase:** phase-05-jsdoc-pass
+**Plan:** plans/260415-1010-d1-cron-infra/
+
+## Files Modified
+
+| File | Change |
+|------|--------|
+| `package.json` | Added `eslint ^10.2.0`, `eslint-plugin-jsdoc ^62.9.0` devDeps; updated `lint` script |
+| `eslint.config.js` | Created — flat config, JSDoc-only rules, `definedTypes` for CF/custom globals |
+| `src/types.js` | Created — central typedef module: `Env`, `Command`, `Cron`, `ModuleContext`, `Module`, `Trade` + re-exports of `KVStore`, `SqlStore`, `Portfolio` |
+| `src/db/kv-store-interface.js` | `Object` → `object` in all typedefs |
+| `src/db/sql-store-interface.js` | `Object` → `object` in all typedefs |
+| `src/modules/registry.js` | `Object` → `object`; fixed invalid TS destructure syntax in `init` property type |
+| `src/modules/validate-command.js` | `Object` → `object` |
+| `src/modules/validate-cron.js` | `Object` → `object` (Phase 02 owns impl; only typedef fixed) |
+| `src/modules/trading/portfolio.js` | `Object` → `object` |
+| `src/modules/trading/symbols.js` | `Object` → `object` |
+| `src/modules/index.js` | Removed `{@link loadModules}` curly-brace syntax misread as a type |
+
+Files already fully annotated (no changes needed): `src/index.js`, `src/bot.js`, `src/db/cf-kv-store.js`, `src/db/create-store.js`, `src/db/cf-sql-store.js`, `src/db/create-sql-store.js`, `src/modules/dispatcher.js`, `src/modules/trading/handlers.js`, `src/modules/trading/prices.js`, `src/modules/trading/format.js`, `src/modules/trading/stats-handler.js`.
+
+## Tasks Completed
+
+- [x] Install `eslint`, `eslint-plugin-jsdoc`
+- [x] `eslint.config.js` with JSDoc-only rules
+- [x] Update `lint` script
+- [x] `src/types.js` central typedef file
+- [x] All targeted files annotated / typedef-corrected
+- [x] `npm run lint` — clean (Biome + ESLint both exit 0)
+
+## Tests Status
+
+- Biome: pass
+- ESLint: pass (0 errors, 0 warnings)
+- Unit tests: 139/139 pass (14 test files)
+
+## Issues Encountered
+
+1. `eslint-plugin-jsdoc` uses `definedTypes` (rule option) not `definedNames` (settings key) — corrected in `eslint.config.js`.
+2. Several files had CRLF endings introduced by `node` script edits — resolved via `biome format --write`.
+3. `{@link loadModules}` inside `@file` JSDoc was parsed as a type reference by the plugin — removed curly braces.
+4. Registry `BotModule.init` used TypeScript destructure syntax `({ db, sql, env }: {...})` which `jsdoc/valid-types` rejects — changed to plain `(ctx: {...}) => Promise<void>`.
+5. `validate-cron.js` is Phase 02-owned but had an `Object` typedef that caused lint errors — fixed only the typedef line (no logic changes).
+
+---
+
+**Status:** DONE
+**Summary:** `npm run lint` exits 0 (Biome + ESLint clean), 139 tests pass, no runtime changes. Central `src/types.js` typedef file created; all JSDoc issues corrected across 11 files.