5 Commits

Author SHA1 Message Date
tiennm99 ea7df56e2d feat(db,cron): phase 04 — dual-write wrappers + factory routing + drift verifier + e2e
The integration phase. Wires Phase 02 (MongoKVStore) and Phase 03
(MongoTradesStore + MongoSqlStore shim) into the request path behind
two env flags so KV and Atlas run side-by-side until cutover.

Storage routing
- DualKVStore + DualSqlStore: Promise.allSettled writes to BOTH backends,
  reads from primary only. Secondary failures log + enqueue onto a KV
  retry queue (__retry:mongo-failed:* / __retry:mongo-sql-failed:*).
  Primary failure throws. _kind="dual" sentinel for test seam.
- create-store.js + create-sql-store.js: full flag matrix (STORAGE_PRIMARY
  ∈ {kv,mongo}, DUAL_WRITE ∈ {0,1}, MONGODB_URI presence) with
  STUB_SENTINEL short-circuit for deploy-time. Post-cutover shape commented
  inline so Phase 07 simplification is mechanical.

Stub mongo for register
- scripts/stub-kv.js: STUB_SENTINEL constant + duck-typed stubMongo
  (no-op connect/close, throwing collection access). Replaces the
  originally-planned string sentinel which would have stalled register.js
  on serverSelectionTimeoutMS if ever passed to MongoClient (code-reviewer #2).
- scripts/register.js: stub env passes MONGODB_URI=STUB_SENTINEL,
  STORAGE_PRIMARY="kv", DUAL_WRITE="0". Asserted via vi.spyOn that
  MongoClient.prototype.connect is never reached.

Drift verifier cron (1/hr)
- src/cron/drift-verifier.js: drains both retry queues by re-attempting
  secondary writes, deletes on success. Spot-checks parity by sampling
  DRIFT_SAMPLE_N keys per module, hashing, logging mismatches.
- src/modules/cron-dispatcher.js: SYSTEM_CRONS array dispatched alongside
  module crons. Keeping system cron out of registry.crons preserves
  existing module-cron length tests and is the cleaner design.
- wrangler.toml: vars STORAGE_PRIMARY/DUAL_WRITE/DRIFT_SAMPLE_N + cron
  schedule "0 * * * *" added.

Trading wiring
- src/modules/registry.js: builds new MongoTradesStore(env) when Mongo
  is in play and threads it as tradesStore into trading module's init
  context. Trading module already accepted optional tradesStore (Phase 03
  backwards-compat) — D1 path remains for STORAGE_PRIMARY=kv + DUAL_WRITE=0.

Tests + verification
- tests/db/dual-kv-store.test.js, dual-sql-store.test.js: write-both,
  secondary-fail-logs+enqueues, primary-fail-throws, reads-primary-only,
  _kind sentinel.
- tests/db/stub-mongo-sentinel.test.js: spy on MongoClient.connect,
  assert zero calls across all flag-matrix combos.
- tests/cron/drift-verifier.test.js: queue drain, skip paths, error safety.
- tests/e2e/storage-roundtrip.test.js: wordle KV dual-write +
  trading MongoTradesStore against fake-mongo.

Tests: 577 → 638 (+61). register:dry passes without Atlas. Lint clean.

Concerns
- Drift-verifier parity-spot-check tests assert queue-drain only;
  full mismatch detection needs real Atlas (Vitest ES-module caching
  blocks reliable prototype patching). Verifier logic verified by
  inspection.
2026-04-26 09:02:07 +07:00
tiennm99 99cd8449ec feat(db,trading): phase 03 — MongoTradesStore + trading refactor + SqlStore shim
Replaces the originally-planned SQL-pattern dispatcher with a direct refactor:
trading/history.js + retention.js call MongoTradesStore methods explicitly
instead of routing strings through a regex dispatcher. Cleaner abstraction;
eliminates the "7th SQL statement silently breaks" risk flagged in code-review.

- src/db/mongo-trades-store.js: 6 explicit methods (insert, byUser,
  distinctUsers, oldRowsForUser, oldRows, deleteByIds). Lazy index init
  for (user_id, ts desc), (ts desc), and sparse (legacy_id).
- src/db/mongo-sql-store.js: thin SqlStore shim returning {changes:1,
  last_row_id:0} (number, NOT hex) to satisfy the existing
  tests/db/create-sql-store.test.js:48-52 contract. Exists purely for
  factory branching; trading code calls MongoTradesStore directly.
  Unsupported SQL throws loud.
- trading/history.js + retention.js + index.js: accept optional
  tradesStore in init args. Falls back to existing D1 sql path when
  tradesStore absent — keeps trading working on D1 until Phase 04
  wires dual-write.
- legacy_id: null on runtime inserts. Sparse index + field reserved
  for backfill (Phase 05) to preserve original D1 integer IDs for
  historical join-ability.

Pre-refactor grep gates (all PASS):
- exactly 6 SQL statements in src/modules/trading/
- zero arithmetic on .id (.id [+-*/<>])
- last_row_id consumed by zero callers in trading

Tests: 529 → 577 (+48). Lint clean.
2026-04-26 08:48:18 +07:00
tiennm99 5b00cae76e feat(db): phase 02 — MongoKVStore + memoized client + fake-mongo
Implements the KVStore interface against MongoDB Atlas with full behavioral
parity vs CFKVStore (null-on-missing, swallow-corrupt-JSON, idempotent delete,
throw-on-undefined-putJSON). Not wired into the request path yet — Phase 04
adds dual-write wrappers and factory routing.

- src/db/mongo-client.js: memoized MongoClient + getDb(env). On connect()
  reject, nulls both client and connectPromise so next call retries cleanly
  (regression-tested). Catches MongoServerSelectionError and emits a
  structured warning before rethrow so callers can map to 503.
- src/db/mongo-kv-store.js: KVStore impl. get/getJSON filter on expiresAt
  at read time to close the up-to-60s TTL-sweeper stale-read window vs
  CFKVStore. list() returns keys WITH prefix preserved (parity — wrapper
  in create-store.js:65 strips). Cursor pagination via sorted _id +
  limit(N+1), NOT skip(). Lazy ensureIndex per (collection, isolate)
  tracked in module-scope Set.
- src/db/mongo-list-cursor.js: extracted cursor encode/decode to keep
  mongo-kv-store.js under 200 LOC.
- tests/fakes/fake-mongo.js: Map-backed fake covering the surface needed
  by both Phase 02 (KVStore) and Phase 03 (MongoTradesStore).
- tests/db/mongo-kv-store.test.js: 26 tests, including TTL stale-read
  regression (1s TTL + time advance), 2-level prefix list regression,
  cursor pagination, connect-reject retry, MongoServerSelectionError
  structured log.

Tests: 503 → 529 (+26). Lint clean.
2026-04-26 08:48:01 +07:00
tiennm99 83c6892d6e feat: add D1 storage layer with per-module migration runner
- SqlStore interface + CF D1 wrapper + per-module factory (table prefix convention)
- init signature extended to ({ db, sql, env }); sql is null when DB binding absent
- custom migration runner walks src/modules/*/migrations/*.sql, tracks applied in _migrations table
- npm run db:migrate with --dry-run and --local flags; chained into deploy
- fake-d1 test helper with subset of SQL semantics for retention and history tests
2026-04-15 13:21:53 +07:00
tiennm99 c4314f21df feat: scaffold plug-n-play telegram bot on cloudflare workers
grammY-based bot with a module plugin system loaded from the MODULES env
var. Three command visibility levels (public/protected/private) share a
unified command namespace with conflict detection at registry build.

- 4 initial modules (util, wordle, loldle, misc); util fully implemented,
  others are stubs proving the plugin system end-to-end
- util: /info (chat/thread/sender ids) + /help (pure renderer over the
  registry, HTML parse mode, escapes user-influenced strings)
- KVStore interface with CFKVStore and a per-module prefixing factory;
  getJSON/putJSON convenience helpers; other backends drop in via one file
- Webhook at POST /webhook with secret-token validation via grammY's
  webhookCallback; no admin HTTP surface
- Post-deploy register script (npm run deploy = wrangler deploy && node
  --env-file=.env.deploy scripts/register.js) for setWebhook and
  setMyCommands; --dry-run flag for preview
- 56 vitest unit tests across 7 suites covering registry, db wrapper,
  dispatcher, help renderer, validators, and HTML escaper
- biome for lint + format; phased implementation plan under plans/
2026-04-11 09:49:06 +07:00