feat(db,cron): phase 04 — dual-write wrappers + factory routing + drift verifier + e2e

The integration phase. Wires Phase 02 (MongoKVStore) and Phase 03
(MongoTradesStore + MongoSqlStore shim) into the request path behind
two env flags so KV and Atlas run side-by-side until cutover.

Storage routing
- DualKVStore + DualSqlStore: Promise.allSettled writes to BOTH backends,
  reads from primary only. Secondary failures log + enqueue onto a KV
  retry queue (__retry:mongo-failed:* / __retry:mongo-sql-failed:*).
  Primary failure throws. _kind="dual" sentinel for test seam.
- create-store.js + create-sql-store.js: full flag matrix (STORAGE_PRIMARY
  ∈ {kv,mongo}, DUAL_WRITE ∈ {0,1}, MONGODB_URI presence) with
  STUB_SENTINEL short-circuit for deploy-time. Post-cutover shape commented
  inline so Phase 07 simplification is mechanical.

Stub mongo for register
- scripts/stub-kv.js: STUB_SENTINEL constant + duck-typed stubMongo
  (no-op connect/close, throwing collection access). Replaces the
  originally-planned string sentinel which would have stalled register.js
  on serverSelectionTimeoutMS if ever passed to MongoClient (code-reviewer #2).
- scripts/register.js: stub env passes MONGODB_URI=STUB_SENTINEL,
  STORAGE_PRIMARY="kv", DUAL_WRITE="0". Asserted via vi.spyOn that
  MongoClient.prototype.connect is never reached.

Drift verifier cron (1/hr)
- src/cron/drift-verifier.js: drains both retry queues by re-attempting
  secondary writes, deletes on success. Spot-checks parity by sampling
  DRIFT_SAMPLE_N keys per module, hashing, logging mismatches.
- src/modules/cron-dispatcher.js: SYSTEM_CRONS array dispatched alongside
  module crons. Keeping system cron out of registry.crons preserves
  existing module-cron length tests and is the cleaner design.
- wrangler.toml: vars STORAGE_PRIMARY/DUAL_WRITE/DRIFT_SAMPLE_N + cron
  schedule "0 * * * *" added.

Trading wiring
- src/modules/registry.js: builds new MongoTradesStore(env) when Mongo
  is in play and threads it as tradesStore into trading module's init
  context. Trading module already accepted optional tradesStore (Phase 03
  backwards-compat) — D1 path remains for STORAGE_PRIMARY=kv + DUAL_WRITE=0.

Tests + verification
- tests/db/dual-kv-store.test.js, dual-sql-store.test.js: write-both,
  secondary-fail-logs+enqueues, primary-fail-throws, reads-primary-only,
  _kind sentinel.
- tests/db/stub-mongo-sentinel.test.js: spy on MongoClient.connect,
  assert zero calls across all flag-matrix combos.
- tests/cron/drift-verifier.test.js: queue drain, skip paths, error safety.
- tests/e2e/storage-roundtrip.test.js: wordle KV dual-write +
  trading MongoTradesStore against fake-mongo.

Tests: 577 → 638 (+61). register:dry passes without Atlas. Lint clean.

Concerns
- Drift-verifier parity-spot-check tests assert queue-drain only;
  full mismatch detection needs real Atlas (Vitest ES-module caching
  blocks reliable prototype patching). Verifier logic verified by
  inspection.
This commit is contained in:
2026-04-26 09:02:07 +07:00
parent 99cd8449ec
commit ea7df56e2d
15 changed files with 1924 additions and 24 deletions
+12 -2
View File
@@ -19,7 +19,7 @@
*/
import { buildRegistry, resetRegistry } from "../src/modules/registry.js";
import { stubAi, stubKv } from "./stub-kv.js";
import { STUB_SENTINEL, stubAi, stubKv } from "./stub-kv.js";
const TELEGRAM_API = "https://api.telegram.org";
@@ -71,8 +71,18 @@ async function main() {
// Build the registry against the same code the Worker uses. Stub KV
// satisfies the binding so createStore() does not throw.
// MONGODB_URI = STUB_SENTINEL ensures create-store / create-sql-store
// factories short-circuit before constructing any MongoClient.
// STORAGE_PRIMARY + DUAL_WRITE lock the factories to the CF-only path.
resetRegistry();
const reg = await buildRegistry({ MODULES: modules, KV: stubKv, AI: stubAi });
const reg = await buildRegistry({
MODULES: modules,
KV: stubKv,
AI: stubAi,
MONGODB_URI: STUB_SENTINEL,
STORAGE_PRIMARY: "kv",
DUAL_WRITE: "0",
});
const commands = [...reg.publicCommands.values()].map(({ cmd }) => ({
command: cmd.name,
+38
View File
@@ -8,6 +8,14 @@
* are assumed read-only (or tolerant of missing state) at registration time.
* If a future module writes inside init(), update the matching stub to
* swallow writes safely.
*
* `STUB_SENTINEL` is passed as `env.MONGODB_URI` so the create-store /
* create-sql-store factories short-circuit BEFORE constructing any MongoClient.
* This ensures zero Atlas connections during `npm run register:dry`.
*
* `stubMongo` is a duck-typed no-op that satisfies the MongoClient surface
* used by MongoKVStore. It is NOT a real MongoClient instance. It must never
* be used in production — sentinel check in the factories prevents that.
*/
/** @type {KVNamespace} */
@@ -44,3 +52,33 @@ export const stubAi = {
return { data: [] };
},
};
/**
* Sentinel value passed as `env.MONGODB_URI` during deploy-time registry
* builds. Factories check for this value FIRST and return CF-only stores
* immediately — no MongoClient is ever constructed.
*
* @type {string}
*/
export const STUB_SENTINEL = "__stub_mongo__";
/**
* Duck-typed no-op MongoClient surface. Satisfies the shape used by
* MongoKVStore / MongoTradesStore without performing any network IO.
* Used only when `env.MONGODB_URI === STUB_SENTINEL`.
*/
export const stubMongo = {
db() {
return {
collection() {
throw new Error("stubMongo: no IO at deploy time");
},
};
},
async connect() {
return undefined;
},
async close() {
return undefined;
},
};