tiennm99/store-scraper-bot
1
0
Fork 0
mirror of https://github.com/tiennm99/store-scraper-bot.git synced 2026-09-03 00:18:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
112 Commits 1 Branch 0 Tags
00a64157782c9df8713e9781437eef0d1480c6e2
Commit Graph
112 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tiennm99 00a6415778 chore: archive code-port plan, add plans/todo.md index 
Code port (commit bff1d32) is done; move plan to plans/archive/.
New plans/todo.md is the forward-looking index pointing at the active deploy
plan, pre-flight checks, hard gates, and open questions for the operator.
 2026-04-26 23:46:22 +07:00
tiennm99 797f040574 feat: Cloudflare Workers code port (deploy pending) 
Refactors source to be Worker-shaped. No live deploy yet — sister deploy plan
runs Atlas provisioning + smoke later.

- wrangler.toml with nodejs_compat_v2, daily UTC 0 cron (= 7am Asia/Ho_Chi_Minh)
- package.json: drop node-telegram-bot-api, node-cron, dotenv, pino,
  pino-pretty; add wrangler devDep; bump to 0.2.0
- src/bot/telegram-api.js: raw fetch wrapper for Telegram Bot API
- src/bot/dispatch.js: per-message dispatcher extracted from polling loop
- src/repository/mongodb.js: memoized MongoClient per warm isolate, typed
  MongoUnavailable error, fast-fail timeouts
- src/repository/store.js: factory binding env once
- All 4 repositories converted to factory shape
- src/api/{apple,google}-scraper.js: take store instead of importing repos
- src/index.js: Worker entry exporting { fetch, scheduled }; webhook validates
  X-Telegram-Bot-Api-Secret-Token; ack-then-waitUntil pattern
- src/scheduler/scheduler.js: trimmed; runDailyCheck only (no node-cron)
- src/config.js, src/logger.js: env-driven, console.log JSON output
- scripts/register-webhook.js: setWebhook + setMyCommands; --dry-run supported
- scripts/check-secret-leaks.js: lint blocks console.log(env.<SECRET>)
- plans/260426-2015-cloudflare-worker-code-port: this code port plan
- plans/260426-2327-cloudflare-deploy-and-smoke: sister deploy plan

Validated via node --check on all 32 source files; lint clean. Real deploy
gates (bundle size, cold-start CPU) run in deploy plan.
 2026-04-26 23:36:39 +07:00
tiennm99 3c97553c69 feat: initial JavaScript port of store-scraper-bot 
Node.js 20+ ESM port mirroring Java/Go implementations.

- 13 Telegram commands matching Java identifiers
- MongoDB schema parity (common, group, apple_app, google_app collections)
- Apple/Google scrapers calling store-scraper.vercel.app with 10-min cache
- Daily 7am Vietnam-time cron with weekend-silent mode
- HTML table renderer matching Java/Go output
- Docker + Compose (prod and dev)

Untested end-to-end against live Telegram or upstream API.
 2026-04-26 20:16:00 +07:00
tiennm99 ca1a409610 Clear and rewrite 
Retire the Go port; history continues from the JavaScript implementation (store-scraper-bot).
 2026-08-04 23:00:01 +07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 		824279e1fe build(deps): bump golang.org/x/crypto from 0.26.0 to 0.45.0 (#1) 
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.26.0 to 0.45.0.
- [Commits](https://github.com/golang/crypto/compare/v0.26.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-05-23 19:17:21 +07:00
tiennm99 6bc7756596 docs: update reference to java-store-scraper-bot 2026-05-09 17:38:42 +07:00
tiennm99 a5cade43cf docs: point reference link at legacy-store-scraper-bot 
Java reference impl renamed from store-scraper-bot to legacy-store-scraper-bot. The new canonical store-scraper-bot is the JS implementation.
 2026-05-09 17:29:21 +07:00
tiennm99 fdb5bee896 docs: mark project as preview, AI-assisted, untested 2026-04-25 22:01:35 +07:00
tiennm99 e695d9b223 feat: port logic from Java original to align with current Java behavior 
- Switch Telegram parse mode to HTML to match Java client.
- Rename command identifiers to Java's: delgroup/delapple/delgoogle/
  checkappscore/rawappleapp/rawgoogleapp.
- Move admin singleton to "common" collection at _id="admin".
- Store group _id as string-of-int64 to match Java AbstractModel schema.
- Add `class` discriminator field on persisted models.
- Expand AppleAppResponse (42 fields) and GoogleAppResponse (63 fields,
  with nested Category/Feature) to match Java records.
- Add AppleAppRequest (track id or bundle id) / GoogleAppRequest types.
- Implement Raw* commands as Telegram document attachments instead of
  truncated text.
- Group-authorization gate (admin.HasGroup(chatId)) on non-admin commands.
- Cache TTL via millis timestamp; round score to 1 decimal; weekend silent
  send in scheduler.
- Match Java table renderer: " │ " separators, "─┼─" rows every 5 lines.
- Prefer MONGODB_CONNECTION_STRING env (Java parity); auto-extract DB
  name from URI with fallback to "store-scraper-bot".

Note: This is an AI-assisted port (not human-verified). Behavior parity
with the Java implementation has not been tested end-to-end; treat as a
starting point.
 2026-04-25 20:26:25 +07:00
tiennm99 cfc8d67641 Merge branch 'feature/claude-converted' 2025-12-03 20:46:02 +07:00
tiennm99 9b03eb007a feat(core): add logic converted from original repo by Claude Code 2025-11-15 18:54:06 +07:00
tiennm99 f243bc9b88 Update README.md 2025-12-01 00:08:58 +07:00
Tien Nguyen MinhandGitHub 165023f539 Initial commit 2025-11-15 18:52:19 +07:00
tiennm99 c5f3628f01 Clear and rewrite 
Retire the Java implementation; history continues from the Go port (store-scraper-bot-go).
 2026-08-04 22:59:44 +07:00
tiennm99 e794000df3 feat(mongodb): manual handle expire 2025-12-23 21:15:33 +07:00
tiennm99 d85dc72aab feat(deps): cleanup 2025-12-16 22:20:49 +07:00
tiennm99 d99798fae7 feat(repo): cleanup 2025-12-16 22:08:25 +07:00
tiennm99 bb4c12e29e fix(mongo): codec 2025-12-16 21:57:57 +07:00
tiennm99 b048cbc0a5 fix(deserialize): 2025-12-15 22:56:37 +07:00
tiennm99 c060aa891d feat(codec): try to auto handle (de)serialize 2025-12-15 22:49:57 +07:00
tiennm99 8394f909ae feat(mongo): cleanup 2025-12-15 22:46:21 +07:00
tiennm99 0eae89ac16 feat(db): use mongodb instead of couchbase (WIP) 2025-11-22 23:05:39 +07:00
tiennm99 dae676da69 fix(schedule): exception break schedule 2025-11-17 12:07:21 +07:00
tiennm99 23fd181c0f Update log4j2.xml 2025-11-14 18:55:22 +07:00
tiennm99 d5467f16bc Update log4j2.xml 2025-11-14 18:32:53 +07:00
tiennm99 56b6dc5cb3 feat(loki): try to enable loki again 2025-11-14 18:25:46 +07:00
tiennm99 61e1dcfe00 Update Dockerfile 2025-11-14 18:11:10 +07:00
tiennm99 5879bdda67 Update Dockerfile 2025-11-14 18:05:32 +07:00
tiennm99 99ab547bc5 Update Dockerfile 2025-11-14 18:00:57 +07:00
tiennm99 dda84a82cf Update Dockerfile 2025-11-14 17:57:14 +07:00
tiennm99 709a798836 Update Dockerfile 2025-11-14 17:49:49 +07:00
tiennm99 bccc6f647e feat(docker): use distTar instead of fatjar 2025-11-14 17:45:13 +07:00
tiennm99 7335c2757e feat(loki): remove loki 2025-11-14 01:29:02 +07:00
tiennm99 dfbabb964a Update build.gradle.kts 2025-11-14 01:26:04 +07:00
tiennm99 d8a945a619 Revert "Update Main.java" 
This reverts commit e53b1884c4.
 2025-11-14 01:25:35 +07:00
tiennm99 fb4401f345 Update Main.java 2025-11-14 01:22:08 +07:00
tiennm99 b413dd07d0 Update build.gradle.kts 2025-11-14 01:20:42 +07:00
tiennm99 574fadf08b Update build.gradle.kts 2025-11-14 01:09:39 +07:00
tiennm99 3f27a213e4 Update Log4j2 config and shadowJar transformer usage 
Removed the custom 'packages' attribute from log4j2.xml configuration and updated the shadowJar task to use the class reference for Log4j2PluginsCacheFileTransformer. This streamlines configuration and aligns with recommended usage.
 2025-11-14 01:05:24 +07:00
tiennm99 805db99045 Update log4j2 config and shadowJar task 
Added 'packages' attribute to log4j2.xml Configuration for custom Log4j2 plugins. Removed mergeServiceFiles() from shadowJar task in build.gradle.kts, relying only on Log4j2PluginsCacheFileTransformer.
 2025-11-14 01:01:32 +07:00
tiennm99 c03f2ec8f6 Update build.gradle.kts 2025-11-14 00:52:27 +07:00
tiennm99 671f1c64d9 feat(loki): add loki again 
after fix shadow jar issue
 2025-11-14 00:49:04 +07:00
tiennm99 7739590c4a Update log4j2.xml 2025-11-14 00:35:46 +07:00
tiennm99 9bf855c354 Revert "Update Dockerfile" 
This reverts commit 07dbdfc125.
 2025-11-14 00:31:25 +07:00
tiennm99 0fc2a4472d Update Dockerfile 2025-11-14 00:29:22 +07:00
tiennm99 3ff0f56ea7 Update Dockerfile 2025-11-14 00:26:15 +07:00
tiennm99 a92acf0a4f Update Dockerfile 2025-11-14 00:24:11 +07:00
tiennm99 03c00c164a feat(loki): remove 2025-11-14 00:23:05 +07:00
tiennm99 d28ae8918b Update Dockerfile 2025-11-14 00:17:40 +07:00
tiennm99 e522e00ab1 Update Dockerfile 2025-11-13 23:52:27 +07:00