mirror of
https://github.com/tiennm99/miti99bot-js.git
synced 2026-05-14 11:52:59 +00:00
tiennm99
0859356ec7
Operator-run migration scripts for KV→Mongo and D1→trading_trades, plus a parity verifier and a rollback wiper. Pure local Node — no Worker code, no /__admin/* routes, no new Worker secrets. Complies with docs/architecture.md §10. Scripts - backfill-kv-to-mongo.js: paginates CF KV REST API per module, fetches values, $setOnInsert upsert into per-module Mongo collection. Resumes from .backfill-cursor-<module>.json on restart. Throttles 50 ops/sec. expiresAt derived from KV metadata.expiration (debugger #10). --dry-run and --module flags for incremental work. - backfill-d1-to-mongo.js: wrangler d1 execute --remote --json → parse → insertMany batches into trading_trades, preserving original integer id as legacy_id (code-reviewer #13). Pre-flight aborts if collection non-empty unless --force. - verify-mongo-parity.js: count parity ±1%, SHA256 value compare, expiresAt ±5min bucket. Full-scan when <10K docs, sqrt-sample capped at 500 otherwise (code-reviewer #21). Trading: full-scan on legacy_id/ts/user_id/symbol/qty. - wipe-mongo.js: rollback helper. deleteMany across all collections with readline confirm. --yes for CI. - lib/migration-helpers.js: shared sleep, sha256, checkpoint I/O, cfKvList/cfKvGet, MongoClient singleton, sample strategy. Surface updates - .env.deploy.example: CF account/token/namespace placeholders. - package.json: backfill:kv[:dry], backfill:d1[:dry], verify:mongo, wipe:mongo scripts. - check-secret-leaks.js: SECRETS array gains CLOUDFLARE_API_TOKEN + CLOUDFLARE_ACCOUNT_ID for defense-in-depth. - .gitignore: .backfill-cursor-*.json excluded. Tests: 638 → 667 (+29 pure-logic tests for sha256, checkpoint round-trip, count-diff, sample-size, fetch-mocked CF REST). Lint clean. Operator-run sequence (after Phase 06 deploy): npm run backfill:kv:dry # preview npm run backfill:kv npm run backfill:d1:dry npm run backfill:d1 npm run verify:mongo # exit 0 = parity ok
31 lines
1.5 KiB
Bash
31 lines
1.5 KiB
Bash
# Post-deploy registration env, consumed by `scripts/register.js` via
|
|
# `node --env-file=.env.deploy`. Copy to .env.deploy (gitignored) and fill in.
|
|
#
|
|
# TELEGRAM_BOT_TOKEN + TELEGRAM_WEBHOOK_SECRET must match the values set via
|
|
# `wrangler secret put` so the Worker and Telegram agree on the same secret.
|
|
TELEGRAM_BOT_TOKEN=
|
|
TELEGRAM_WEBHOOK_SECRET=
|
|
|
|
# Public URL of the deployed Worker (no trailing slash). Known after the first
|
|
# `wrangler deploy`. Example: https://miti99bot.your-subdomain.workers.dev
|
|
WORKER_URL=
|
|
|
|
# Same MODULES value as wrangler.toml [vars]. Duplicated here so the register
|
|
# script can derive the public command list without parsing wrangler.toml.
|
|
MODULES=util,wordle,loldle,misc,trading,lolschedule,semantle,doantu,twentyq
|
|
|
|
# MongoDB Atlas connection string. Used by the `mongodb` driver inside the Worker
|
|
# AND by local backfill / verify scripts. MUST match the value set via
|
|
# `wrangler secret put MONGODB_URI` for the Worker. Same secret-mirror protocol
|
|
# as TELEGRAM_BOT_TOKEN / TELEGRAM_WEBHOOK_SECRET.
|
|
# Format: mongodb+srv://miti99bot-worker:<pass>@<host>/miti99bot?retryWrites=true&w=majority
|
|
MONGODB_URI=
|
|
|
|
# Cloudflare API creds for backfill scripts (Phase 05). Read-only scope is enough.
|
|
# Create at: dash.cloudflare.com → My Profile → API Tokens → Create Token.
|
|
# Permissions needed: Account → Workers KV Storage → Read; Account → D1 → Read.
|
|
CLOUDFLARE_ACCOUNT_ID=
|
|
CLOUDFLARE_API_TOKEN=
|
|
# KV namespace ID (production). Same value as wrangler.toml [[kv_namespaces]] id.
|
|
KV_NAMESPACE_ID=
|