Commit Graph
117 Commits
Author SHA1 Message Date
tiennm99 0169c09b0f docs: add Phase 06 (Atlas → KV data migration) to KV plan
Adds the phase doc that drove the migration script and updates plan.md
with the new phase, dependency edges (02 → 06 → 04), and removes the
prior "Out of scope: Data migration" line.
2026-05-05 21:15:27 +07:00
tiennm99 d33bfeae3d feat: add Atlas → KV one-shot migration script
Operator runs `npm run migrate` (reads admin + group docs from Atlas)
followed by `npm run migrate:bulk` (uploads via wrangler kv bulk put).
Cache collections are skipped by default since they auto-rebuild from
upstream APIs; --include-cache flag migrates them with TTL preserved.

- mongodb is added as a devDependency only — never enters the Worker
  bundle, the Worker still talks to KV exclusively.
- scripts/.atlas-export.json is gitignored (contains exported state).
- README documents the one-time runbook.
2026-05-05 21:15:18 +07:00
tiennm99 971c9c311f docs: add KV migration plan, supersede Atlas deploy plan
Adds the 260505-1425 plan that drove the KV swap (Phase 01 already
applied in the preceding commit; Phases 02-05 are operator-driven).
Marks 260426-2327-cloudflare-deploy-and-smoke as superseded since
the KV pivot was taken pre-emptively rather than after a hard-gate trip.
2026-05-05 20:39:12 +07:00
tiennm99 9898f408a2 feat: replace MongoDB driver with Cloudflare KV storage
Drops the mongodb dependency entirely; all four logical collections
(admin singleton, group, apple_app, google_app) now live in a single KV
namespace bound as STORE_KV with prefixed keys. Cache TTL is delegated
to KV via expirationTtl (clamped to the 60s minimum). Document shape,
field names, and Java parity at the doc level are preserved.

- Adds src/repository/kv.js helper (getJson/putJson/del with TTL clamp)
- Rewrites all four *-repository.js modules on top of KV
- Removes src/repository/mongodb.js and the MONGODB_URI env requirement
- Adds an early STORE_KV-binding guard in src/index.js
- Bumps to 0.3.0
2026-05-05 20:39:01 +07:00
tiennm99 00f49531e0 docs: add Java vs JS parity reports from /ck:xia
researcher report maps 47 Java files to 35 JS files (11 PARITY, 2
MINOR, 0 GAP, 3 EXTRA). xia synthesis records source manifest, decision
matrix, and risk score. No port work needed; remaining items are
operational.
2026-04-29 16:07:17 +07:00
tiennm99 00a6415778 chore: archive code-port plan, add plans/todo.md index
Code port (commit bff1d32) is done; move plan to plans/archive/.
New plans/todo.md is the forward-looking index pointing at the active deploy
plan, pre-flight checks, hard gates, and open questions for the operator.
2026-04-26 23:46:22 +07:00
tiennm99 797f040574 feat: Cloudflare Workers code port (deploy pending)
Refactors source to be Worker-shaped. No live deploy yet — sister deploy plan
runs Atlas provisioning + smoke later.

- wrangler.toml with nodejs_compat_v2, daily UTC 0 cron (= 7am Asia/Ho_Chi_Minh)
- package.json: drop node-telegram-bot-api, node-cron, dotenv, pino,
  pino-pretty; add wrangler devDep; bump to 0.2.0
- src/bot/telegram-api.js: raw fetch wrapper for Telegram Bot API
- src/bot/dispatch.js: per-message dispatcher extracted from polling loop
- src/repository/mongodb.js: memoized MongoClient per warm isolate, typed
  MongoUnavailable error, fast-fail timeouts
- src/repository/store.js: factory binding env once
- All 4 repositories converted to factory shape
- src/api/{apple,google}-scraper.js: take store instead of importing repos
- src/index.js: Worker entry exporting { fetch, scheduled }; webhook validates
  X-Telegram-Bot-Api-Secret-Token; ack-then-waitUntil pattern
- src/scheduler/scheduler.js: trimmed; runDailyCheck only (no node-cron)
- src/config.js, src/logger.js: env-driven, console.log JSON output
- scripts/register-webhook.js: setWebhook + setMyCommands; --dry-run supported
- scripts/check-secret-leaks.js: lint blocks console.log(env.<SECRET>)
- plans/260426-2015-cloudflare-worker-code-port: this code port plan
- plans/260426-2327-cloudflare-deploy-and-smoke: sister deploy plan

Validated via node --check on all 32 source files; lint clean. Real deploy
gates (bundle size, cold-start CPU) run in deploy plan.
2026-04-26 23:36:39 +07:00
tiennm99 3c97553c69 feat: initial JavaScript port of store-scraper-bot
Node.js 20+ ESM port mirroring Java/Go implementations.

- 13 Telegram commands matching Java identifiers
- MongoDB schema parity (common, group, apple_app, google_app collections)
- Apple/Google scrapers calling store-scraper.vercel.app with 10-min cache
- Daily 7am Vietnam-time cron with weekend-silent mode
- HTML table renderer matching Java/Go output
- Docker + Compose (prod and dev)

Untested end-to-end against live Telegram or upstream API.
2026-04-26 20:16:00 +07:00
tiennm99 ca1a409610 Clear and rewrite
Retire the Go port; history continues from the JavaScript implementation (store-scraper-bot).
2026-08-04 23:00:01 +07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
824279e1fe build(deps): bump golang.org/x/crypto from 0.26.0 to 0.45.0 (#1)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.26.0 to 0.45.0.
- [Commits](https://github.com/golang/crypto/compare/v0.26.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-23 19:17:21 +07:00
tiennm99 6bc7756596 docs: update reference to java-store-scraper-bot 2026-05-09 17:38:42 +07:00
tiennm99 a5cade43cf docs: point reference link at legacy-store-scraper-bot
Java reference impl renamed from store-scraper-bot to legacy-store-scraper-bot. The new canonical store-scraper-bot is the JS implementation.
2026-05-09 17:29:21 +07:00
tiennm99 fdb5bee896 docs: mark project as preview, AI-assisted, untested 2026-04-25 22:01:35 +07:00
tiennm99 e695d9b223 feat: port logic from Java original to align with current Java behavior
- Switch Telegram parse mode to HTML to match Java client.
- Rename command identifiers to Java's: delgroup/delapple/delgoogle/
  checkappscore/rawappleapp/rawgoogleapp.
- Move admin singleton to "common" collection at _id="admin".
- Store group _id as string-of-int64 to match Java AbstractModel schema.
- Add `class` discriminator field on persisted models.
- Expand AppleAppResponse (42 fields) and GoogleAppResponse (63 fields,
  with nested Category/Feature) to match Java records.
- Add AppleAppRequest (track id or bundle id) / GoogleAppRequest types.
- Implement Raw* commands as Telegram document attachments instead of
  truncated text.
- Group-authorization gate (admin.HasGroup(chatId)) on non-admin commands.
- Cache TTL via millis timestamp; round score to 1 decimal; weekend silent
  send in scheduler.
- Match Java table renderer: " │ " separators, "─┼─" rows every 5 lines.
- Prefer MONGODB_CONNECTION_STRING env (Java parity); auto-extract DB
  name from URI with fallback to "store-scraper-bot".

Note: This is an AI-assisted port (not human-verified). Behavior parity
with the Java implementation has not been tested end-to-end; treat as a
starting point.
2026-04-25 20:26:25 +07:00
tiennm99 cfc8d67641 Merge branch 'feature/claude-converted' 2025-12-03 20:46:02 +07:00
tiennm99 9b03eb007a feat(core): add logic converted from original repo by Claude Code 2025-11-15 18:54:06 +07:00
tiennm99 f243bc9b88 Update README.md 2025-12-01 00:08:58 +07:00
Tien Nguyen MinhandGitHub 165023f539 Initial commit 2025-11-15 18:52:19 +07:00
tiennm99 c5f3628f01 Clear and rewrite
Retire the Java implementation; history continues from the Go port (store-scraper-bot-go).
2026-08-04 22:59:44 +07:00
tiennm99 e794000df3 feat(mongodb): manual handle expire 2025-12-23 21:15:33 +07:00
tiennm99 d85dc72aab feat(deps): cleanup 2025-12-16 22:20:49 +07:00
tiennm99 d99798fae7 feat(repo): cleanup 2025-12-16 22:08:25 +07:00
tiennm99 bb4c12e29e fix(mongo): codec 2025-12-16 21:57:57 +07:00
tiennm99 b048cbc0a5 fix(deserialize): 2025-12-15 22:56:37 +07:00
tiennm99 c060aa891d feat(codec): try to auto handle (de)serialize 2025-12-15 22:49:57 +07:00
tiennm99 8394f909ae feat(mongo): cleanup 2025-12-15 22:46:21 +07:00
tiennm99 0eae89ac16 feat(db): use mongodb instead of couchbase (WIP) 2025-11-22 23:05:39 +07:00
tiennm99 dae676da69 fix(schedule): exception break schedule 2025-11-17 12:07:21 +07:00
tiennm99 23fd181c0f Update log4j2.xml 2025-11-14 18:55:22 +07:00
tiennm99 d5467f16bc Update log4j2.xml 2025-11-14 18:32:53 +07:00
tiennm99 56b6dc5cb3 feat(loki): try to enable loki again 2025-11-14 18:25:46 +07:00
tiennm99 61e1dcfe00 Update Dockerfile 2025-11-14 18:11:10 +07:00
tiennm99 5879bdda67 Update Dockerfile 2025-11-14 18:05:32 +07:00
tiennm99 99ab547bc5 Update Dockerfile 2025-11-14 18:00:57 +07:00
tiennm99 dda84a82cf Update Dockerfile 2025-11-14 17:57:14 +07:00
tiennm99 709a798836 Update Dockerfile 2025-11-14 17:49:49 +07:00
tiennm99 bccc6f647e feat(docker): use distTar instead of fatjar 2025-11-14 17:45:13 +07:00
tiennm99 7335c2757e feat(loki): remove loki 2025-11-14 01:29:02 +07:00
tiennm99 dfbabb964a Update build.gradle.kts 2025-11-14 01:26:04 +07:00
tiennm99 d8a945a619 Revert "Update Main.java"
This reverts commit e53b1884c4.
2025-11-14 01:25:35 +07:00
tiennm99 fb4401f345 Update Main.java 2025-11-14 01:22:08 +07:00
tiennm99 b413dd07d0 Update build.gradle.kts 2025-11-14 01:20:42 +07:00
tiennm99 574fadf08b Update build.gradle.kts 2025-11-14 01:09:39 +07:00
tiennm99 3f27a213e4 Update Log4j2 config and shadowJar transformer usage
Removed the custom 'packages' attribute from log4j2.xml configuration and updated the shadowJar task to use the class reference for Log4j2PluginsCacheFileTransformer. This streamlines configuration and aligns with recommended usage.
2025-11-14 01:05:24 +07:00
tiennm99 805db99045 Update log4j2 config and shadowJar task
Added 'packages' attribute to log4j2.xml Configuration for custom Log4j2 plugins. Removed mergeServiceFiles() from shadowJar task in build.gradle.kts, relying only on Log4j2PluginsCacheFileTransformer.
2025-11-14 01:01:32 +07:00
tiennm99 c03f2ec8f6 Update build.gradle.kts 2025-11-14 00:52:27 +07:00
tiennm99 671f1c64d9 feat(loki): add loki again
after fix shadow jar issue
2025-11-14 00:49:04 +07:00
tiennm99 7739590c4a Update log4j2.xml 2025-11-14 00:35:46 +07:00
tiennm99 9bf855c354 Revert "Update Dockerfile"
This reverts commit 07dbdfc125.
2025-11-14 00:31:25 +07:00
tiennm99 0fc2a4472d Update Dockerfile 2025-11-14 00:29:22 +07:00