tiennm99/store-scraper-bot
1
0
Fork 0
mirror of https://github.com/tiennm99/store-scraper-bot.git synced 2026-08-20 06:24:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
19 Commits 1 Branch 0 Tags
8554b72b0b0fde59fa800e3f75c39df0e9bb74da
Commit Graph
4 Commits
Author SHA1 Message Date
tiennm99 8554b72b0b refactor: drop Mongo class discriminator + delete src/models/ 
Inline trivial factory bodies into the repos and scrapers that used them.
The class:/_id: fields were Java-Mongo parity artifacts that nothing
in this codebase reads — Redis docs with the old fields still parse
fine; the next write drops them.
 2026-05-09 22:30:48 +07:00
tiennm99 c2dd35b75f feat: migrate to Vercel + Upstash with KEY_PREFIX namespacing 
Phases 1-5 of consolidate-vercel-upstash plan. Replaces Cloudflare
Workers + KV with Vercel serverless functions + Upstash Redis. Inlines
app-store-scraper / google-play-scraper npm libs (drops the
store-scraper.vercel.app HTTP roundtrip). KEY_PREFIX (default
'store-scraper-bot:') namespaces all Redis keys so the Upstash DB can
be safely shared with other Vercel projects.

- vercel.json + .vercelignore + Vercel-aware package.json scripts
- api/webhook.js + api/cron.js Vercel functions (with shared
  src/app-builder.js); cron auth fails closed when CRON_SECRET unset
- src/repository/upstash.js replaces kv.js; all 4 repos take a handle
  bundling client + prefix
- scripts/migrate-atlas-to-upstash.js writes legacy Java Atlas state
  directly to Upstash with --dry-run + --include-cache flags
- .env.example refreshed for the new env surface

Phases 6 (Vercel deploy + webhook cutover) and 7 (Docker + wrangler
cleanup) remain operator-driven post-deploy.
 2026-05-09 20:07:07 +07:00
tiennm99 bff1d324f5 feat: Cloudflare Workers code port (deploy pending) 
Refactors source to be Worker-shaped. No live deploy yet — sister deploy plan
runs Atlas provisioning + smoke later.

- wrangler.toml with nodejs_compat_v2, daily UTC 0 cron (= 7am Asia/Ho_Chi_Minh)
- package.json: drop node-telegram-bot-api, node-cron, dotenv, pino,
  pino-pretty; add wrangler devDep; bump to 0.2.0
- src/bot/telegram-api.js: raw fetch wrapper for Telegram Bot API
- src/bot/dispatch.js: per-message dispatcher extracted from polling loop
- src/repository/mongodb.js: memoized MongoClient per warm isolate, typed
  MongoUnavailable error, fast-fail timeouts
- src/repository/store.js: factory binding env once
- All 4 repositories converted to factory shape
- src/api/{apple,google}-scraper.js: take store instead of importing repos
- src/index.js: Worker entry exporting { fetch, scheduled }; webhook validates
  X-Telegram-Bot-Api-Secret-Token; ack-then-waitUntil pattern
- src/scheduler/scheduler.js: trimmed; runDailyCheck only (no node-cron)
- src/config.js, src/logger.js: env-driven, console.log JSON output
- scripts/register-webhook.js: setWebhook + setMyCommands; --dry-run supported
- scripts/check-secret-leaks.js: lint blocks console.log(env.<SECRET>)
- plans/260426-2015-cloudflare-worker-code-port: this code port plan
- plans/260426-2327-cloudflare-deploy-and-smoke: sister deploy plan

Validated via node --check on all 32 source files; lint clean. Real deploy
gates (bundle size, cold-start CPU) run in deploy plan.
 2026-04-26 23:36:39 +07:00
tiennm99 a656a18ce8 feat: initial JavaScript port of store-scraper-bot 
Node.js 20+ ESM port mirroring Java/Go implementations.

- 13 Telegram commands matching Java identifiers
- MongoDB schema parity (common, group, apple_app, google_app collections)
- Apple/Google scrapers calling store-scraper.vercel.app with 10-min cache
- Daily 7am Vietnam-time cron with weekend-silent mode
- HTML table renderer matching Java/Go output
- Docker + Compose (prod and dev)

Untested end-to-end against live Telegram or upstream API.
 2026-04-26 20:16:00 +07:00