mirror of
https://github.com/tiennm99/store-scraper-bot.git
synced 2026-05-27 04:02:12 +00:00
chore: backlog cleanup — deps, CI, bot description, ops docs
- pin form-data/qs/tough-cookie via package.json overrides; clears 3 of 4 Dependabot alerts (request SSRF risk-accepted, no upstream fix) - add GitHub Actions CI (lint + syntax check) on push/PR - add /settings and /setdayswarning to setMyCommands - new npm run describe sets bot profile description via Bot API - README: drop stale preview warning, add Operations section
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- run: npm run lint
|
||||
# Update find roots when adding new top-level JS dirs.
|
||||
- name: Syntax check all JS
|
||||
run: find api scripts src -name '*.js' -print0 | xargs -0 -n1 node --check
|
||||
@@ -3,23 +3,17 @@
|
||||
JavaScript (Node.js) implementation. Ports [java-store-scraper-bot](https://github.com/tiennm99/java-store-scraper-bot).
|
||||
Runs on Vercel serverless functions with Upstash Redis as the data store.
|
||||
|
||||
> ⚠️ **Preview / unstable — use at your own risk.**
|
||||
> This port was produced largely with AI assistance and has **not** been tested
|
||||
> end-to-end against a live Telegram bot or the upstream Java implementation.
|
||||
> Behavior parity is intended but unverified. Do not run against a production database.
|
||||
|
||||
The Java version remains the reference implementation.
|
||||
|
||||
## Status
|
||||
|
||||
- Upstash Redis schema mirrors the Java/Go Mongo layout: keys `admin`,
|
||||
`group:{chatId}`, `apple:{appId}`, `google:{appId}` (last two TTL'd via Redis
|
||||
`EX`). Multi-tenant isolation via `KEY_PREFIX` (default `store-scraper-bot:`).
|
||||
- Telegram command identifiers match Java exactly: `/info`, `/addgroup`,
|
||||
`/delgroup`, `/listgroup`, `/addapple`, `/delapple`, `/addgoogle`,
|
||||
`/delgoogle`, `/listapp`, `/checkapp`, `/checkappscore`, `/rawappleapp`,
|
||||
`/rawgoogleapp`.
|
||||
- Telegram command identifiers match Java plus per-group settings:
|
||||
`/info`, `/addgroup`, `/delgroup`, `/listgroup`, `/addapple`, `/delapple`,
|
||||
`/addgoogle`, `/delgoogle`, `/listapp`, `/checkapp`, `/checkappscore`,
|
||||
`/rawappleapp`, `/rawgoogleapp`, `/settings`, `/setdayswarning`.
|
||||
- HTML parse mode; weekend-silent daily report; configurable upstream cache (default 10 min).
|
||||
- Per-group warning threshold override via `/setdayswarning` (falls back to `NUM_DAYS_WARNING_NOT_UPDATED` env default).
|
||||
- Inlined `app-store-scraper` + `google-play-scraper` (no external scraper service).
|
||||
|
||||
## Requirements
|
||||
@@ -68,6 +62,24 @@ npm run deploy # vercel deploy --prod && register webhook
|
||||
```
|
||||
|
||||
`npm run register` re-points the Telegram webhook at the URL in `.env.deploy:WORKER_URL`.
|
||||
`npm run describe` updates the bot's profile description / about-text (run once when copy changes).
|
||||
|
||||
## Operations
|
||||
|
||||
### Dashboards
|
||||
|
||||
- **Vercel project** — function logs, cron history, deploy status
|
||||
- **Upstash console** — Redis metrics, key browser, request latency
|
||||
|
||||
### Credential rotation (quarterly)
|
||||
|
||||
- **Upstash REST token** — regenerate in Upstash console, update `UPSTASH_REDIS_REST_TOKEN` in Vercel env, redeploy
|
||||
- **Telegram webhook secret** — generate new value, update `TELEGRAM_WEBHOOK_SECRET` in Vercel env, redeploy, then `npm run register`
|
||||
|
||||
### Dependency security
|
||||
|
||||
- Transitive vulnerabilities from `app-store-scraper → request` are pinned via `overrides` in `package.json` (`form-data`, `qs`, `tough-cookie`).
|
||||
- The unfixable `request` SSRF advisory is risk-accepted: only known endpoints (`itunes.apple.com`, `play.google.com`) are called; no user-controlled URLs reach `request`.
|
||||
|
||||
## Project Layout
|
||||
|
||||
|
||||
Generated
+295
-21
@@ -237,6 +237,35 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/call-bind-apply-helpers": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
|
||||
"integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-errors": "^1.3.0",
|
||||
"function-bind": "^1.1.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/call-bound": {
|
||||
"version": "1.0.4",
|
||||
"resolved": "https://registry.npmjs.org/call-bound/-/call-bound-1.0.4.tgz",
|
||||
"integrity": "sha512-+ys997U96po4Kx/ABpBCqhA9EuxJaQWDQg7295H4hBphv3IZg0boBKuwYpt4YXp6MZ5AmZQnU/tyMTlRpaSejg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"call-bind-apply-helpers": "^1.0.2",
|
||||
"get-intrinsic": "^1.3.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/caseless": {
|
||||
"version": "0.12.0",
|
||||
"resolved": "https://registry.npmjs.org/caseless/-/caseless-0.12.0.tgz",
|
||||
@@ -477,6 +506,20 @@
|
||||
"url": "https://github.com/fb55/domutils?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/dunder-proto": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
|
||||
"integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"call-bind-apply-helpers": "^1.0.1",
|
||||
"es-errors": "^1.3.0",
|
||||
"gopd": "^1.2.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/ecc-jsbn": {
|
||||
"version": "0.1.2",
|
||||
"resolved": "https://registry.npmjs.org/ecc-jsbn/-/ecc-jsbn-0.1.2.tgz",
|
||||
@@ -521,6 +564,51 @@
|
||||
"url": "https://github.com/fb55/entities?sponsor=1"
|
||||
}
|
||||
},
|
||||
"node_modules/es-define-property": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
|
||||
"integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/es-errors": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz",
|
||||
"integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/es-object-atoms": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz",
|
||||
"integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-errors": "^1.3.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/es-set-tostringtag": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
|
||||
"integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-errors": "^1.3.0",
|
||||
"get-intrinsic": "^1.2.6",
|
||||
"has-tostringtag": "^1.0.2",
|
||||
"hasown": "^2.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/es5-ext": {
|
||||
"version": "0.10.64",
|
||||
"resolved": "https://registry.npmjs.org/es5-ext/-/es5-ext-0.10.64.tgz",
|
||||
@@ -644,19 +732,68 @@
|
||||
}
|
||||
},
|
||||
"node_modules/form-data": {
|
||||
"version": "2.3.3",
|
||||
"resolved": "https://registry.npmjs.org/form-data/-/form-data-2.3.3.tgz",
|
||||
"integrity": "sha512-1lLKB2Mu3aGP1Q/2eCOx0fNbRMe7XdwktwOruhfqqd0rIJWwN4Dh+E3hrPSlDCXnSR7UtZ1N38rVXm+6+MEhJQ==",
|
||||
"version": "2.5.5",
|
||||
"resolved": "https://registry.npmjs.org/form-data/-/form-data-2.5.5.tgz",
|
||||
"integrity": "sha512-jqdObeR2rxZZbPSGL+3VckHMYtu+f9//KXBsVny6JSX/pa38Fy+bGjuG8eW/H6USNQWhLi8Num++cU2yOCNz4A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"asynckit": "^0.4.0",
|
||||
"combined-stream": "^1.0.6",
|
||||
"mime-types": "^2.1.12"
|
||||
"combined-stream": "^1.0.8",
|
||||
"es-set-tostringtag": "^2.1.0",
|
||||
"hasown": "^2.0.2",
|
||||
"mime-types": "^2.1.35",
|
||||
"safe-buffer": "^5.2.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.12"
|
||||
}
|
||||
},
|
||||
"node_modules/function-bind": {
|
||||
"version": "1.1.2",
|
||||
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
|
||||
"integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/get-intrinsic": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
|
||||
"integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"call-bind-apply-helpers": "^1.0.2",
|
||||
"es-define-property": "^1.0.1",
|
||||
"es-errors": "^1.3.0",
|
||||
"es-object-atoms": "^1.1.1",
|
||||
"function-bind": "^1.1.2",
|
||||
"get-proto": "^1.0.1",
|
||||
"gopd": "^1.2.0",
|
||||
"has-symbols": "^1.1.0",
|
||||
"hasown": "^2.0.2",
|
||||
"math-intrinsics": "^1.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/get-proto": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz",
|
||||
"integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"dunder-proto": "^1.0.1",
|
||||
"es-object-atoms": "^1.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/get-stream": {
|
||||
"version": "5.2.0",
|
||||
"resolved": "https://registry.npmjs.org/get-stream/-/get-stream-5.2.0.tgz",
|
||||
@@ -710,6 +847,18 @@
|
||||
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/gopd": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
|
||||
"integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/got": {
|
||||
"version": "11.8.6",
|
||||
"resolved": "https://registry.npmjs.org/got/-/got-11.8.6.tgz",
|
||||
@@ -758,6 +907,45 @@
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/has-symbols": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
|
||||
"integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/has-tostringtag": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
|
||||
"integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"has-symbols": "^1.0.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/hasown": {
|
||||
"version": "2.0.3",
|
||||
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.3.tgz",
|
||||
"integrity": "sha512-ej4AhfhfL2Q2zpMmLo7U1Uv9+PyhIZpgQLGT1F9miIGmiCJIoCgSmczFdrc97mWT4kVY72KA+WnnhJ5pghSvSg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"function-bind": "^1.1.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/htmlparser2": {
|
||||
"version": "10.1.0",
|
||||
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz",
|
||||
@@ -925,6 +1113,15 @@
|
||||
"es5-ext": "~0.10.2"
|
||||
}
|
||||
},
|
||||
"node_modules/math-intrinsics": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
|
||||
"integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/memoizee": {
|
||||
"version": "0.4.17",
|
||||
"resolved": "https://registry.npmjs.org/memoizee/-/memoizee-0.4.17.tgz",
|
||||
@@ -1019,6 +1216,18 @@
|
||||
"node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/object-inspect": {
|
||||
"version": "1.13.4",
|
||||
"resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz",
|
||||
"integrity": "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/once": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
|
||||
@@ -1124,12 +1333,18 @@
|
||||
}
|
||||
},
|
||||
"node_modules/qs": {
|
||||
"version": "6.5.5",
|
||||
"resolved": "https://registry.npmjs.org/qs/-/qs-6.5.5.tgz",
|
||||
"integrity": "sha512-mzR4sElr1bfCaPJe7m8ilJ6ZXdDaGoObcYR0ZHSsktM/Lt21MVHj5De30GQH2eiZ1qGRTO7LCAzQsUeXTNexWQ==",
|
||||
"version": "6.15.1",
|
||||
"resolved": "https://registry.npmjs.org/qs/-/qs-6.15.1.tgz",
|
||||
"integrity": "sha512-6YHEFRL9mfgcAvql/XhwTvf5jKcOiiupt2FiJxHkiX1z4j7WL8J/jRHYLluORvc1XxB5rV20KoeK00gVJamspg==",
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"side-channel": "^1.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.6"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/querystringify": {
|
||||
@@ -1192,19 +1407,6 @@
|
||||
"node": ">= 6"
|
||||
}
|
||||
},
|
||||
"node_modules/request/node_modules/tough-cookie": {
|
||||
"version": "2.5.0",
|
||||
"resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-2.5.0.tgz",
|
||||
"integrity": "sha512-nlLsUzgm1kfLXSXfRZMc1KLAugd4hqJHDTvc2hDIwS3mZAfMEuMbc03SujMF+GEcpaX/qboeycw6iO8JwVv2+g==",
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"psl": "^1.1.28",
|
||||
"punycode": "^2.1.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.8"
|
||||
}
|
||||
},
|
||||
"node_modules/requires-port": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/requires-port/-/requires-port-1.0.0.tgz",
|
||||
@@ -1264,6 +1466,78 @@
|
||||
"node": ">=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/side-channel": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.0.tgz",
|
||||
"integrity": "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-errors": "^1.3.0",
|
||||
"object-inspect": "^1.13.3",
|
||||
"side-channel-list": "^1.0.0",
|
||||
"side-channel-map": "^1.0.1",
|
||||
"side-channel-weakmap": "^1.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/side-channel-list": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.1.tgz",
|
||||
"integrity": "sha512-mjn/0bi/oUURjc5Xl7IaWi/OJJJumuoJFQJfDDyO46+hBWsfaVM65TBHq2eoZBhzl9EchxOijpkbRC8SVBQU0w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-errors": "^1.3.0",
|
||||
"object-inspect": "^1.13.4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/side-channel-map": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/side-channel-map/-/side-channel-map-1.0.1.tgz",
|
||||
"integrity": "sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"call-bound": "^1.0.2",
|
||||
"es-errors": "^1.3.0",
|
||||
"get-intrinsic": "^1.2.5",
|
||||
"object-inspect": "^1.13.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/side-channel-weakmap": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/side-channel-weakmap/-/side-channel-weakmap-1.0.2.tgz",
|
||||
"integrity": "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"call-bound": "^1.0.2",
|
||||
"es-errors": "^1.3.0",
|
||||
"get-intrinsic": "^1.2.5",
|
||||
"object-inspect": "^1.13.3",
|
||||
"side-channel-map": "^1.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/sshpk": {
|
||||
"version": "1.18.0",
|
||||
"resolved": "https://registry.npmjs.org/sshpk/-/sshpk-1.18.0.tgz",
|
||||
|
||||
@@ -13,6 +13,8 @@
|
||||
"deploy": "vercel deploy --prod && npm run register",
|
||||
"register": "node --env-file=.env.deploy scripts/register-webhook.js",
|
||||
"register:dry": "node --env-file=.env.deploy scripts/register-webhook.js --dry-run",
|
||||
"describe": "node --env-file=.env.deploy scripts/set-bot-description.js",
|
||||
"describe:dry": "node --env-file=.env.deploy scripts/set-bot-description.js --dry-run",
|
||||
"lint": "node scripts/check-secret-leaks.js"
|
||||
},
|
||||
"license": "Apache-2.0",
|
||||
@@ -21,5 +23,10 @@
|
||||
"@vercel/functions": "^3.5.1",
|
||||
"app-store-scraper": "^0.18.0",
|
||||
"google-play-scraper": "^10.1.2"
|
||||
},
|
||||
"overrides": {
|
||||
"form-data": "^2.5.4",
|
||||
"qs": "^6.14.1",
|
||||
"tough-cookie": "^4.1.3"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
phase: 1
|
||||
title: Dependabot overrides + audit
|
||||
status: completed
|
||||
priority: P1
|
||||
effort: 30m
|
||||
dependencies: []
|
||||
---
|
||||
|
||||
# Phase 1: Dependabot overrides + audit
|
||||
|
||||
## Overview
|
||||
|
||||
All 4 open alerts come transitively from `app-store-scraper@0.18.0 → request@2.88.2`. The `request` lib is abandoned (no patched version), but its transitive deps `qs`, `form-data`, `tough-cookie` have patches. Use npm `overrides` to force-pin the patched versions; accept the unfixable `request` SSRF advisory.
|
||||
|
||||
## Context (from preflight)
|
||||
|
||||
```
|
||||
app-store-scraper@0.18.0 (latest; still pulls request)
|
||||
└─ request@2.88.2 (abandoned — SSRF GHSA, no fix)
|
||||
├─ form-data@2.3.3 → patch to ^2.5.4 (CRITICAL: unsafe random boundary)
|
||||
├─ qs@6.5.5 → patch to ^6.14.1 (DoS via memory exhaustion)
|
||||
└─ tough-cookie@2.5.0 → patch to ^4.1.3 (Prototype Pollution)
|
||||
google-play-scraper@10.1.2
|
||||
└─ tough-cookie@4.1.4 (already patched, no action)
|
||||
```
|
||||
|
||||
The `request` SSRF alert has `fixed_in: null` — no upstream fix. Risk: low — we only call known endpoints (`itunes.apple.com`, `play.google.com`); no user-controlled URLs reach `request`. Document + dismiss in GitHub UI as "won't fix / risk-accepted".
|
||||
|
||||
## Architecture
|
||||
|
||||
`package.json` `overrides` field is npm's documented mechanism for forcing transitive dep versions. Apply at the top level (no nesting) — both `request` and `google-play-scraper` should use the patched versions.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
**Modify**
|
||||
- `package.json` — add `overrides` block
|
||||
- `package-lock.json` — regenerate
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Add `overrides` to `package.json`:
|
||||
```json
|
||||
"overrides": {
|
||||
"form-data": "^2.5.4",
|
||||
"qs": "^6.14.1",
|
||||
"tough-cookie": "^4.1.3"
|
||||
}
|
||||
```
|
||||
Carets so future patch bumps flow through.
|
||||
2. Run `npm install` to regenerate `package-lock.json`.
|
||||
3. Run `npm audit` — expect only the `request` SSRF alert remaining (unfixable). Everything else should clear.
|
||||
4. `npm ls form-data qs tough-cookie` — confirm overridden versions are in the tree.
|
||||
5. `npm run lint` to verify no incidental break.
|
||||
6. Open GitHub Dependabot UI and dismiss the `request` SSRF alert: "risk accepted, no user-controlled URLs reach `request`".
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] `npm audit` shows 0 vulnerabilities except the unfixable `request` SSRF
|
||||
- [ ] `npm ls form-data` shows `^2.5.4`
|
||||
- [ ] `npm ls qs` shows `^6.14.1`
|
||||
- [ ] `npm ls tough-cookie` shows `^4.1.3` everywhere
|
||||
- [ ] GitHub Dependabot page shows 0 open alerts (or only the dismissed `request` one)
|
||||
- [ ] Bot smoke-run on Vercel preview deploy still works
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Override breaks app-store-scraper:** lib's pinned `qs@~6.5.2` could rely on old behavior. `qs@6.5 → 6.14` is minor under semver but `qs` has had behavior tweaks (parameter parsing). Mitigation: smoke-test `/checkapp` against a real Apple app after install. If broken, narrow override to a patched 6.5.x line if one exists; else vendor a minimal apple-scraper using native `fetch`.
|
||||
- **form-data 2.3 → 2.5 bump:** patch-level under semver. Same mitigation.
|
||||
- **`request` SSRF unfixable:** documented + dismissed. Long-term: replace `app-store-scraper` with in-house fetch wrapper (out of scope).
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
phase: 2
|
||||
title: CI workflow on PR/push
|
||||
status: completed
|
||||
priority: P2
|
||||
effort: 30m
|
||||
dependencies: []
|
||||
---
|
||||
|
||||
# Phase 2: CI workflow on PR/push
|
||||
|
||||
## Overview
|
||||
|
||||
Add a single GitHub Actions workflow running on PR + push: `npm ci`, secret-leak lint, and `node --check` on every `.js` file. No tests exist yet so no test job. Cheap signal that someone's commit at least parses + doesn't add secrets.
|
||||
|
||||
## Architecture
|
||||
|
||||
One workflow file, single job, two minutes max runtime. Use Node 20 (matches `engines.node` in `package.json`). Cache npm via `actions/setup-node@v4`'s built-in cache.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
**Create**
|
||||
- `.github/workflows/ci.yml`
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Create `.github/workflows/ci.yml`:
|
||||
```yaml
|
||||
name: CI
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
jobs:
|
||||
verify:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- run: npm run lint
|
||||
- name: Syntax check all JS
|
||||
run: find api scripts src -name '*.js' -print0 | xargs -0 -n1 node --check
|
||||
```
|
||||
2. Commit + push → confirm green run in Actions tab.
|
||||
3. Add a status badge to README (optional, low value — skip unless trivial).
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Workflow file present at `.github/workflows/ci.yml`
|
||||
- [ ] First run on push to `main` is green
|
||||
- [ ] PRs show CI status check
|
||||
- [ ] Job completes < 2 minutes
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Bundle-size gate dropped:** original todo asked for "lint + bundle-size as PR check". Bundle size mattered when target was Cloudflare Workers (1MB script limit). On Vercel serverless, 250MB unzipped limit makes bundle size irrelevant. Skipped — YAGNI.
|
||||
- **`find` path coverage:** if a future top-level dir is added, the syntax check misses it. Mitigation: documented in workflow comment to update the find roots when adding new dirs. Acceptable — no current dynamism in repo layout.
|
||||
- **`npm ci` slowness:** mitigated by built-in npm cache. Cold runs ~30s, warm <10s.
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
phase: 3
|
||||
title: Bot description script
|
||||
status: completed
|
||||
priority: P3
|
||||
effort: 20m
|
||||
dependencies: []
|
||||
---
|
||||
|
||||
# Phase 3: Bot description script
|
||||
|
||||
## Overview
|
||||
|
||||
`/setdayswarning` and `/settings` need to also show up in the Telegram command picker — extend the existing `setMyCommands` list. Separately, set the bot's About + Description text via `setMyDescription` and `setMyShortDescription` so users see what it does in the chat header / profile.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two pieces:
|
||||
1. **Extend `register-webhook.js`** — add the two new commands (`settings`, `setdayswarning`) to the existing `COMMANDS` array. They'll be pushed on the next `npm run register`.
|
||||
2. **New one-shot script** `scripts/set-bot-description.js` — calls `setMyDescription` (long) + `setMyShortDescription` (140 chars), gated by env. Run manually after copy is locked.
|
||||
|
||||
Description copy (English; bot is operated in Vietnam but commands are English in the existing register script):
|
||||
|
||||
- **Short** (≤120 chars): `Track Apple App Store + Google Play app updates. Get notified when tracked apps go N days without an update.`
|
||||
- **Long** (≤512 chars): two paragraphs explaining tracking + the daily report + how to get started (`/info` → ask admin to `/addgroup` → `/addapple` etc.).
|
||||
|
||||
## Related Code Files
|
||||
|
||||
**Modify**
|
||||
- `scripts/register-webhook.js` — append `settings` + `setdayswarning` to `COMMANDS`
|
||||
|
||||
**Create**
|
||||
- `scripts/set-bot-description.js` — modeled on `register-webhook.js` (same env loading, same `--dry-run` flag)
|
||||
|
||||
**Modify**
|
||||
- `package.json` `scripts` — add `"describe": "node --env-file=.env.deploy scripts/set-bot-description.js"`
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Read existing `scripts/register-webhook.js` to confirm env-loading pattern (`.env.deploy` via `--env-file`).
|
||||
2. Append to `COMMANDS` array in `register-webhook.js`:
|
||||
```js
|
||||
{ command: 'settings', description: 'Show this group\'s settings' },
|
||||
{ command: 'setdayswarning', description: 'Set warning threshold (days)' },
|
||||
```
|
||||
3. Create `scripts/set-bot-description.js`:
|
||||
- Same env-required check as register-webhook (just `TELEGRAM_BOT_TOKEN`)
|
||||
- Defines `SHORT` and `LONG` string constants
|
||||
- Posts to `https://api.telegram.org/bot{TOKEN}/setMyDescription` with `{ description: LONG }`
|
||||
- Posts to `…/setMyShortDescription` with `{ short_description: SHORT }`
|
||||
- Honors `--dry-run` like the webhook script
|
||||
- Logs the resulting bot info via `getMe` for confirmation
|
||||
4. Add `describe` npm script.
|
||||
5. Test: `npm run register` (pushes new commands), then `npm run describe` (sets descriptions). Confirm in Telegram.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Telegram command picker in any group/DM shows `settings` and `setdayswarning`
|
||||
- [ ] BotFather-equivalent description is set (visible when user opens bot profile)
|
||||
- [ ] Both scripts honor `--dry-run`
|
||||
- [ ] `npm run describe` is idempotent (Telegram allows re-setting same value)
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Description length limits:** Telegram caps short at 120, long at 512. Mitigation: hard-coded copy known to fit; script can `assert` length before posting.
|
||||
- **Wrong language:** existing copy is English; if user wants Vietnamese, the script supports per-language via `language_code` param. Not adding now (YAGNI).
|
||||
- **No rollback:** setting is global per bot. To revert, re-run with previous text. Acceptable.
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
phase: 4
|
||||
title: Docs + ops reminders
|
||||
status: completed
|
||||
priority: P3
|
||||
effort: 20m
|
||||
dependencies: []
|
||||
---
|
||||
|
||||
# Phase 4: Docs + ops reminders
|
||||
|
||||
## Overview
|
||||
|
||||
Add a short "Operations" section to README covering quarterly Upstash credential rotation, links to Vercel + Upstash dashboards (the de-facto observability surface), and a one-liner on the dependabot policy. Also drop the outdated "Preview / unstable" warning at the top — bot is live and the Java cutover is done.
|
||||
|
||||
## Architecture
|
||||
|
||||
Single-file edit in README. No new docs file — operator notes are short enough to live where deploy + register flow already lives. Future extraction into `docs/operations.md` only if it grows past ~30 lines.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
**Modify**
|
||||
- `README.md` — drop "Preview / unstable" blockquote; add `## Operations` section after `## Run`
|
||||
- `plans/todo.md` — mark items resolved by phases 1–3 done; leave Tests + Observability as remaining
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Remove the `> ⚠️ Preview / unstable` blockquote (~5 lines near top of README).
|
||||
2. Add `## Operations` section after `## Run`:
|
||||
```markdown
|
||||
## Operations
|
||||
|
||||
### Dashboards
|
||||
- Vercel project — function logs, cron history, deploy status
|
||||
- Upstash console — Redis metrics, key browser, request latency
|
||||
|
||||
### Credential rotation (quarterly)
|
||||
- Upstash REST token — regenerate in Upstash console, update `UPSTASH_REDIS_REST_TOKEN` in Vercel env, redeploy
|
||||
- Telegram webhook secret — generate new value, update `TELEGRAM_WEBHOOK_SECRET` in Vercel env, redeploy, then `npm run register`
|
||||
|
||||
### Dependency security
|
||||
- Transitive vulnerabilities from `app-store-scraper → request` are pinned via `overrides` in `package.json`
|
||||
- The unfixable `request` SSRF advisory is risk-accepted: only known endpoints (itunes.apple.com, play.google.com) are called; no user-controlled URLs reach `request`
|
||||
```
|
||||
3. Update `plans/todo.md`: mark phase 1–3 items done; leave Tests + Observability with note that Observability is YAGNI given Vercel/Upstash dashboards.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] README "Preview / unstable" warning removed
|
||||
- [ ] README has Operations section covering dashboards, rotation, dep security
|
||||
- [ ] `plans/todo.md` reflects resolved items
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- **Calendar reminder:** prose-only docs don't trigger anyone. Mitigation: user adds a calendar event manually — out of scope for code.
|
||||
- **Doc rot:** rotation steps will go stale if Vercel/Upstash UIs change. Mitigation: keep wording vague ("regenerate in Upstash console") rather than click-by-click.
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
title: 'backlog cleanup: dependabot, CI, bot description'
|
||||
description: >-
|
||||
Knock out the small/concrete items in plans/todo.md. Tests + observability
|
||||
remain out of scope.
|
||||
status: completed
|
||||
priority: P2
|
||||
created: 2026-05-10T00:00:00.000Z
|
||||
---
|
||||
|
||||
# backlog cleanup: dependabot, CI, bot description
|
||||
|
||||
## Overview
|
||||
|
||||
Four small, independent items from `plans/todo.md` bundled into one plan because each is too small to justify its own. Phases are independent — can be merged separately or together.
|
||||
|
||||
## Goals & Non-Goals
|
||||
|
||||
**Goals**
|
||||
- Resolve all 4 open Dependabot alerts (1 critical + 3 medium) where a fix exists
|
||||
- Add minimal GitHub Actions CI (lint + syntax) on PR and push
|
||||
- Set Telegram bot description / short-description via Bot API
|
||||
- Document quarterly Upstash credential rotation
|
||||
|
||||
**Non-Goals (separate future plans)**
|
||||
- **Tests** — needs framework choice (vitest vs node:test), conventions, fixtures. Treat as its own multi-phase plan.
|
||||
- **Observability dashboard** — Vercel + Upstash already provide built-in dashboards. Custom-built dashboard is YAGNI until a real ops question arises that the built-ins can't answer.
|
||||
|
||||
## Phases
|
||||
|
||||
| Phase | Name | Status |
|
||||
|-------|------|--------|
|
||||
| 1 | [Dependabot overrides + audit](./phase-01-dependabot-overrides-audit.md) | Completed |
|
||||
| 2 | [CI workflow on PR/push](./phase-02-ci-workflow-on-pr-push.md) | Completed |
|
||||
| 3 | [Bot description script](./phase-03-bot-description-script.md) | Completed |
|
||||
| 4 | [Docs + ops reminders](./phase-04-docs-ops-reminders.md) | Completed |
|
||||
|
||||
## Dependencies
|
||||
|
||||
Phases are independent. No cross-plan dependencies.
|
||||
+11
-9
@@ -1,16 +1,18 @@
|
||||
# Outstanding Work
|
||||
|
||||
Bot live on Vercel + Upstash. Java bot ready for shutdown (auth gap on raw* commands fixed in 4fe4a78).
|
||||
Bot live on Vercel + Upstash. Java bot retired.
|
||||
|
||||
## Backlog
|
||||
|
||||
- [ ] Triage GitHub Dependabot alerts (1 critical + 6 moderate on default branch)
|
||||
- [ ] Tests (none exist)
|
||||
- [ ] Quarterly Upstash credential rotation reminder
|
||||
- [ ] Observability dashboard (Vercel + Upstash)
|
||||
- [ ] CI workflow (lint + bundle-size as PR check)
|
||||
- [ ] Telegram bot description / about-text via `setMyDescription`
|
||||
- [ ] Tests (no framework chosen yet — needs its own multi-phase plan)
|
||||
|
||||
## Archive
|
||||
## Done (see git history)
|
||||
|
||||
All migration plans + reports moved to [archive/](archive/) — Vercel + Upstash consolidation, two superseded Cloudflare attempts, java→js parity research.
|
||||
- ~~Triage Dependabot alerts~~ — overrides pinned in `package.json`; `request` SSRF risk-accepted (260510-0001)
|
||||
- ~~CI workflow~~ — `.github/workflows/ci.yml` (lint + syntax check) (260510-0001)
|
||||
- ~~Telegram bot description~~ — `npm run describe` (260510-0001)
|
||||
- ~~Operations docs~~ — README "Operations" section covers rotation + dashboards (260510-0001)
|
||||
|
||||
## Dropped (YAGNI)
|
||||
|
||||
- Observability dashboard — Vercel + Upstash dashboards already cover function logs, cron history, Redis metrics. Revisit only when a real ops question can't be answered by the built-ins.
|
||||
|
||||
@@ -33,6 +33,8 @@ const COMMANDS = [
|
||||
{ command: 'checkappscore', description: 'Check scores + ratings of tracked apps' },
|
||||
{ command: 'rawappleapp', description: 'Dump raw Apple API JSON for an app' },
|
||||
{ command: 'rawgoogleapp', description: 'Dump raw Google API JSON for an app' },
|
||||
{ command: 'settings', description: "Show this group's settings" },
|
||||
{ command: 'setdayswarning', description: 'Set warning threshold (days, 0 = default)' },
|
||||
];
|
||||
|
||||
async function tg(method, payload) {
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
#!/usr/bin/env node
|
||||
// Sets the bot's profile description (long) + short description.
|
||||
// Run via: npm run describe (reads .env.deploy)
|
||||
// Dry run: npm run describe -- --dry-run
|
||||
|
||||
const TOKEN = process.env.TELEGRAM_BOT_TOKEN;
|
||||
const DRY = process.argv.includes('--dry-run');
|
||||
|
||||
if (!TOKEN) {
|
||||
console.error('TELEGRAM_BOT_TOKEN is required');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const SHORT =
|
||||
'Track Apple App Store + Google Play app updates. Get notified when tracked apps go N days without an update.';
|
||||
|
||||
const LONG =
|
||||
'Tracks Apple App Store and Google Play apps and pings your group when an app has not been updated in over N days.\n\n' +
|
||||
'Get started: send /info in your group, ask the bot admin to /addgroup it, then /addapple <id> or /addgoogle <appId>. ' +
|
||||
'Use /settings to view per-group config and /setdayswarning <n> to override the warning threshold.';
|
||||
|
||||
if (SHORT.length > 120) {
|
||||
console.error(`Short description too long (${SHORT.length}/120)`);
|
||||
process.exit(1);
|
||||
}
|
||||
if (LONG.length > 512) {
|
||||
console.error(`Long description too long (${LONG.length}/512)`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
async function tg(method, payload) {
|
||||
if (DRY) {
|
||||
console.log(`[dry-run] ${method}`, JSON.stringify(payload, null, 2));
|
||||
return { ok: true, result: '(dry)' };
|
||||
}
|
||||
const res = await fetch(`https://api.telegram.org/bot${TOKEN}/${method}`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(payload),
|
||||
});
|
||||
const body = await res.json();
|
||||
if (!body.ok) {
|
||||
console.error(`${method} failed`, body);
|
||||
process.exit(1);
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
await tg('setMyShortDescription', { short_description: SHORT });
|
||||
await tg('setMyDescription', { description: LONG });
|
||||
const me = await tg('getMe', {});
|
||||
console.log('Bot:', JSON.stringify(me.result, null, 2));
|
||||
Reference in New Issue
Block a user