Delete the byte-oriented KVStore/VersionedStore abstraction and the
DynamoDB/memory KV backends. Add a generic typed store (DocStore[T] with
Provider/Collection/Typed) persisting each value as a flattened native
Mongo document (storedDoc[T] via bson inline) — no value envelope.
- MongoDB is the only runtime backend; memory kept for tests/local.
- All modules + deploynotify use typed stores; persisted structs carry
bson tags == json names (incl. nested lolschedule/wordle types).
- lolschedule wraps its array/scalar values in named structs.
- migrate-dynamo-to-mongo writes the flattened shape via Typed[bson.M]
with wrap rules; Scan/--dry-run/--verify retained.
Verified: go vet/build clean; full go test green hermetically and
in-container vs real Mongo 7 + DynamoDB Local (storage integration +
migrator e2e).
Persist the KV value field as a BSON string so it is directly readable in
the Atlas/Compass UI, matching DynamoDB's String storage. Reads still accept
BSON binary as a backward-compat fallback. Non-UTF-8 callers must encode
upstream, same constraint as DynamoDB; all current callers write JSON.
Mark phases 1–5 (MongoDB, cron, polling, migration, decommission) as completed.
Update phase files with finalized implementation notes and test coverage summaries.
Add 5-phase plan to self-host on Coolify (docker-compose) with a MongoDB
Atlas backend, an in-process cron scheduler, DynamoDB->Atlas migration,
and full AWS teardown. Red-teamed and validated; decommission scope
verified against live AWS/Cloudflare accounts. Include free-tier audit
and S3-elimination research reports. Ignore wrangler local cache.
Parallelizing the per-symbol fetches opened N simultaneous TLS handshakes into
an empty connection pool. On the memory-constrained Lambda (256MB ~0.15 vCPU)
those CPU-bound handshakes thrashed and each exceeded the per-fetch timeout, so
every ticker rendered "(no price)". Sequential fetches reuse the price client's
keep-alive connection (one handshake), which is why the code was sequential by
design.
- revert stock and coin stats loops to sequential (keep the reply-budget
sub-context and 3s per-fetch timeout)
- log per-symbol fetch failures instead of silently swallowing them
Stats handlers reused the single bounded update context for both upstream
price fetches and the final Telegram reply, while per-upstream HTTP timeouts
equalled the whole-handler budget. One slow upstream drained the deadline and
the reply failed with "context deadline exceeded" (observed on /stock_stats).
- add chathelper.FetchContext: fetches run under a child context that reserves
a tail of the deadline for the reply, which is sent on the original context
- cap kbs/coin/gold/vnappmob HTTP timeouts at 3s so one hung upstream cannot
consume the reply budget
- fetch held stock/coin prices concurrently (bounded) so total latency tracks
the slowest single fetch instead of their sum; per-symbol failures degrade to
a "no price" line and the summary still sends
Extend the stats module to track per-user counts in addition to the
existing per-command totals. CommandHook now receives the originating
*models.Update so the hook can attribute invocations to a user; only
the stats module consumes this hook today.
Schema (sort keys under pk="stats"):
count:<cmd> existing, per-command total
user:<id> new, per-user total with cached username
pair:<cmd>:<id> new, per (command, user) pair
Subcommands (all public):
/stats top commands (unchanged)
/stats users top users overall
/stats user <name> top commands for that user
/stats cmd <name> top users of that command
When the sender has no Telegram username the per-user/pair writes are
skipped — the global per-command counter still increments. View helpers
fan out KV reads in parallel to stay inside the webhook deadline. The
existing read-modify-write race is unchanged; closing it would need
atomic UpdateItem ADD on the KV interface.
Delete 7 plan dirs (6 implemented, 1 superseded) and 19 reports tied to
removed or non-existent plans. Active plans (aws-port, pre-deploy-wrapup,
cf-data-migration, trongtruonghop, iam-least-privilege, go-port-cloud-run)
and their referenced reports are preserved.
Captures the 2026-05-18 security review session output:
- plans/reports/code-reviewer-260518-1019-security-aws-infra.md
- plans/reports/code-reviewer-260518-1019-security-go-app.md
- plans/reports/researcher-260518-1019-security-dependencies.md
- docs/deploy-aws-free-tier-guide.md (adds free-tier hard rule +
accepted security trade-offs as project standards)
Plan for the two HIGH-severity findings (F1, F2) targeting
github-deploy-miti99bot OIDC role: plans/260518-1019-iam-least-privilege/.
Plan was red-team-reviewed (15 findings applied) and validate-interviewed
(4 decisions recorded). Zero unresolved contradictions. Implementation
not yet started; phase 1 is standalone and lowest risk.
Other audit findings (F3 CORS, F4 root handler, F5-F16) deferred to
future commits; rationale in audit report.
CloudFormation's AWS::Scheduler::Schedule Target schema has no property
for HTTPS universal invocation (URL, method, headers) — confirmed
against AWS docs. Switch to the legacy EventBridge Rule path which
supports HTTP targets natively via ApiDestination:
- AWS::Events::Connection: API_KEY auth, presents X-Cron-Token header.
ApiKeyValue stored in EventBridge service-linked secret on stack
update (no per-invoke SSM fetch, AWS-managed secret fees).
- AWS::Events::ApiDestination: POST to ${FunctionUrl}cron/lolschedule_daily_push.
- AWS::Events::Rule: cron(0 1 * * ? *) — daily 01:00 UTC / 08:00 ICT.
Targets ApiDestination with retry x2, 600s max age, DLQ to CronDLQ.
- EventBridgeInvokeRole replaces SchedulerExecutionRole (events.amazonaws.com
principal, events:InvokeApiDestination scoped to this destination only).
NoEcho CronSharedSecret CFN parameter restored; GHA fetches the SSM
SecureString and passes via --parameter-overrides so the value never
appears in template source or stack events.
Free-tier preserved: 1 invocation/day, well under EventBridge Rules +
ApiDestinations free quotas.
Reverts 585d996 + c70b9d0. Both CFN deploys failed at changeset
validation; prod stack was never mutated. Approach A
(AWS::Scheduler::Schedule with arn:aws:scheduler:::http-invoke target)
is unimplementable in pure CloudFormation — the Target schema has no
property for HTTP endpoint/method/headers, regardless of name.
Replacement landing in a follow-up commit.
Restored: plans/reports/brainstorm-260517-1411-eventbridge-schedule-fix.md
(useful design context even though Approach A invalidated).
Adds AWS::Scheduler::Schedule resource invoking the Function URL via
HTTPS universal target (arn:aws:scheduler:::http-invoke) at 01:00 UTC
daily. Token sourced from SSM via {{resolve:ssm-secure}}; failures
retried twice and DLQ'd to existing CronDLQ. Closes the deferred
phase-05 of 260510-0234-pre-deploy-wrapup.
Vietnamese disclaimer one-liner. Interpolates a target name (default "VNG")
and the sender — @username when set, else tg://user?id link with the display
name. User-supplied text HTML-escaped; reply routed through ReplyHTML to keep
forum-topic threading.
Captures the two parallel code-reviewer reports that drove the
audit-driven hygiene pass in the previous commit. Kept for future
reviewers and as the trail of which findings were applied vs.
deferred as cosmetic.
The go-telegram/bot v1.20.0 MatchTypeCommand does byte-exact equality
on the full bot_command entity, which in groups is "cmd@botname"
(length 15 for /help@miti99bot) — never equal to the registered name
"cmd". All 27 commands silently missed when used in group chats.
Swap RegisterHandler(..., MatchTypeCommand, ...) for a local matchFunc
that strips the @suffix before comparing. Telegram only routes
/cmd@otherbot to otherbot, so the suffix is safe to drop
unconditionally.
Add a structured `dispatch` log line in the webhook (update_id,
chat_id, chat_type, text preview) so the next silent-drop symptom
shows up in CloudWatch without code archaeology — the existing
[TGBOT] [UPDATE] line only prints struct pointers.
12 new TestMatchCommand sub-tests cover DM, group, mid-text, wrong
command, non-command entity, and bounds edge cases.
Pre-existing 9 errcheck violations introduced by d67517e blocked
golangci-lint on every CI run after the migration toolchain landed.
- Wrap defer Close() in anonymous func to match house style
(cf. internal/modules/lolschedule/api_client.go:162).
- Mark Fprint*/Fprintln to io.Writer with _, _ = (best-effort writes
to os.Stdout / bytes.Buffer; errors not actionable for callers).
No behavior change. go vet, golangci-lint, go test ./... all clean
locally.
Append two steps to .github/workflows/deploy.yml that POST setWebhook
and setMyCommands against the freshly-deployed Function URL, reading
credentials from SSM. Mirrors `make telegram-setup` but inlined to
avoid the Makefile's --profile admin assumption.
Token and webhook-secret are masked via ::add-mask:: before any echo.
Jobs fail loudly on Telegram API errors via `jq -e .ok`.
Mark the manual setWebhook snippets in docs/deploy-aws.md and
docs/deploy-aws-free-tier-guide.md as break-glass.
- phase-01: mark complete; record live D1+KV inventory outcome (9 durable
KV keys, trading source is KV not D1, retired modules skipped)
- phase-02: mark complete; toolchain verified end-to-end against prod CF
- phase-03: rewrite to flat KV copy (D1-transform branch dropped)
- plan.md: amend staging-only decision to permit pre-cutover bulk import
against the live table while it has no AWS-served writes
- runbook: lock matrix and trading source, document Phase 02 commands
Lock the Cloudflare-to-AWS migration matrix and runbook against the live DynamoDB runtime shape, and gate AWS cutover on verified migration parity instead of assuming a symmetric rollback.
Rename:
- Go module github.com/tiennm99/miti99bot-go → github.com/tiennm99/miti99bot
- CloudFormation stack miti99bot-aws-port → miti99bot
- Drop "port", "Cloud Run", "GCP", "cutover", "Phase NN" framing from
active code and docs — project reads as canonical AWS-Lambda from now on.
AWS deploy guide + flow fix:
- New docs/deploy-aws-free-tier-guide.md — Ubuntu 24.04 ARM64 onboarding
with project-local venv (pip awscli + sam-cli), SSM secrets via read -s,
idempotent OIDC provider + role creation, $1 budget alarm.
- Drop sam build from the pipeline — provided.al2023 + makefile builder
expects a Makefile in CodeUri (build/lambda/, the output dir), so the
step always fails. sam deploy --template-file template.yaml now reads
the raw template and zips build/lambda/ directly.
- Rollback section rewritten — use continue-update-rollback /
cancel-update-stack / git-SHA redeploy. Drop the broken
--use-previous-template recipe.
- DynamoDB free-tier row corrected (on-demand is 2.5M read / 1M write
request units, not 25 RCU/WCU).
Updated:
- README.md fully rewritten (drops port/legacy framing, lists modules,
points new users at the free-tier guide).
- aws/README.md retitled "AWS account setup", phase numbers stripped.
- Makefile / .github/workflows/deploy.yml — sam deploy flow.
- samconfig.toml — stack_name = "miti99bot".
- Go comments — Cloud Run → Lambda, Cloud Scheduler → EventBridge
Scheduler, Cloud Logging → CloudWatch Logs.
- Struct field GCPProject → FirestoreProject (env GOOGLE_CLOUD_PROJECT
unchanged).
Plus advisory reports under plans/reports/ from the code-reviewer +
researcher passes that informed the fixes.
Verified: go vet ./..., go build ./..., go test ./... all green.
- Add new modules: doantu, semantle, twentyq, ai (Gemini integration)
- Update module registry with new command registration
- Update tests and documentation for module system
- Update README with new module references
- Research reports comparing AWS/GCP free tiers and greenfield trade-offs
- New AWS port plan with 7 phases (infrastructure, deployment, integration)
- Mark GCP plan deploy phases as superseded by new AWS plan
Phase 11 partial of the go-port-cloud-run plan. Code-side
observability hooks ready ahead of Phase 01 GCP rollout.
- internal/server/log_middleware.go: HTTP middleware that wraps the
router and emits {msg:"req", method, path, status, ms} per
request. statusRecorder defaults to 200 when the inner handler
doesn't call WriteHeader (Go writes 200 implicitly on first body
write). Wired into server.New so /, /webhook, /cron/* all log.
- internal/metrics/counters.go: in-memory Registry with
IncCommand/IncError/IncAI. Atomic Int64 per name + RWMutex on the
map; steady-state increments are mutex-free. Periodic Run flushes
via the project logger every 60s and one final flush on ctx done.
Empty flush is silent (no-noise default).
- Dispatcher instrumented: every command invocation calls
metrics.IncCommand; every handler error calls
metrics.IncError("handler-error"). Logger keeps the full error
detail; counters keep the rate.
- cmd/server/main.go: go metrics.Run(rootCtx) so the flush loop
cancels with SIGTERM and emits the trailing window before exit.
Test coverage: 12 new tests (7 metrics, 3 middleware, 2 default-
registry round-trip). go test -race -count=1 ./... clean
(20 packages); golangci-lint clean.
Soak / cold-start measurement / log-based metrics setup deferred to
post-deployment (Phase 01 prerequisite).
Phase 6e (final sub-phase of port-plan Phase 06). LoL esports match
schedule via lolesports.com's persisted API.
- internal/modules/lolschedule:
- api_client.go: HTTP client with cache-first lookup (120s fresh
window, 60-min stale fallback). Cache record shape matches JS so
cross-runtime KV migration round-trips.
- parse_date.go: ICT-anchored date parser. Accepts dd-mm-yyyy,
dd/mm/yyyy, ddmmyyyy; trailing month/year may be omitted (default
to current ICT month/year). Rejects impossible dates (Apr 31, Feb
29 in non-leap, etc.).
- format.go: Today (grouped by league) and Week (grouped by league
-> day) renderers. Major-league filter (LCK/LPL/LEC/LCS/Worlds/
MSI/etc.) keeps replies under Telegram's 4096-char limit. All
user-influenced strings HTML-escaped.
- subscribers.go: Idempotent add/remove/list keyed by chat id.
- handlers.go: 5 commands (`/lolschedule [date]`,
`/lolschedule_today`, `/lolschedule_week`,
`/lolschedule_subscribe`, `/lolschedule_unsubscribe`).
- 22 tests across api-client (cache hit / miss / stale fallback /
hard fail / show filter / non-JSON), parse-date (full and
short formats, defaults, rejections, ICT anchor), format (event
line states, league ordering, week grouping, HTML escape, major
filter), subscribers (idempotent add/remove), handlers (HTML
reply, error path, subscribe/unsubscribe round-trip).
Daily-push cron deferred to Phase 09 (Cloud Scheduler). Subscribers
are still collected so the push lights up the moment the cron infra
lands. Deps doesn't currently expose a *bot.Bot reference; that is
the prerequisite that Phase 09 will solve.
go test -race -count=1 ./... clean (19 packages); golangci-lint clean.
Phase 6d of the go-port-cloud-run plan. Adds the fourth loldle
variant — guess the champion from a splash art (any skin, including
non-Default).
- internal/modules/loldlesplash: champions.go (embed splashes.json,
10557-line DDragon-sourced pool with SplashChampion + Skin types),
state.go (gameState gains a `skinId` field so the same splash
shows across guesses; default 4 guesses — splash is harder than
ability since non-Default skins are in rotation), handlers.go
(sendPhoto path uses the DDragon CDN splash URL via
models.InputFileString), loldlesplash.go (Module Factory).
- Reuses internal/modules/util/chathelper and internal/champname.
- 4 commands wired: loldle_splash (public), loldle_splash_giveup
(public), loldle_splash_stats (public), loldle_splash_setmax
(private).
- 13 tests: lookup (embed shape + DDragon URL prefix + Default skin
invariant), state (skinId round-trip + JS-wire-format decode +
default 4), handlers (sendPhoto with correct URL, win, unknown,
giveup with skin label, stats, setmax owner + non-owner).
go test -race -count=1 ./... clean (18 packages); golangci-lint
clean.
Phase 6c of the go-port-cloud-run plan. Adds the third loldle variant
— guess the champion from a single ability icon (passive or Q/W/E/R).
- internal/modules/loldleability: champions.go (embed
abilities.json, 5334-line DDragon-sourced pool with
AbilityChampion + Ability types), state.go (gameState gains a
`slot` field so the same icon shows across guesses in a round),
render-free handlers.go (sendPhoto path uses
models.InputFileString with the DDragon CDN URL directly — no
binary upload), loldleability.go (Module Factory).
- Reuses internal/modules/util/chathelper and internal/champname
(same shared layer the other variants use).
- 4 commands wired: loldle_ability (public), loldle_ability_giveup
(public), loldle_ability_stats (public), loldle_ability_setmax
(private).
- 14 tests: lookup (embed shape + DDragon URL prefix + slot
coverage), state (slot round-trip + JS-wire-format decode +
streak), handlers (sendPhoto with correct URL, win, unknown
champion, duplicate, giveup with slot label, stats, setmax owner
+ non-owner).
- gocyclo cap nudged 20 -> 22 to accommodate handleAbility's
pre-flight validation branch.
go test -race -count=1 ./... clean (17 packages); golangci-lint
clean.
Phase 6 of the 2026-05-09 review remediation plan. Bundle of small
hygiene fixes — none individually urgent but better folded together
than scattered across follow-ups.
- .golangci.yml: enable errcheck/govet/gosec/staticcheck/unused/
ineffassign/gocyclo/misspell/revive. Tuned to the codebase style
(no universal exported-doc requirement, gocyclo cap at 20 to
accommodate handler dispatch). 0 issues across the tree.
- ci.yml: add golangci-lint job + govulncheck (informational).
- Defensive guards:
- registry.go: Module.Name mismatch now errors at Build instead of
silently overwriting (TestBuild_RejectsFactoryNameMismatch).
- cmd/server/main.go: PORT env validated numerically + 0..65535.
- firestore_provider.go: For() re-validates module name; invalid
names return an invalidStore whose every op errors with
ErrInvalidModuleName.
- Dead code removal:
- wordle: gameTTLSeconds const + pickDaily/hashDJB2/todayUTC
helpers + their tests deleted (pickDaily was unused;
daily.go renamed pick_random.go).
- Dependency: golang.org/x/net v0.52.0 -> v0.54.0 (resolves
GO-2026-4918 HTTP/2 infinite-loop CVE).
- Deferred from the original phase plan: Docker digest pinning
(Dependabot handles), per-handler file splits (largest file 279 LOC;
splits would churn for marginal gain).
go test -race -count=1 ./... clean (15 packages); golangci-lint run
clean (0 issues).
Phase 5 of the 2026-05-09 review remediation plan. Closes the
handler-layer test gap (5 modules at 0% coverage in the audit) and
ends the storage-package's CI t.Skip on Firestore emulator tests.
- internal/testutil: Update fixture builders (NewPrivateMessage,
NewGroupMessage, NewSupergroupMessage, NewChannelMessage) plus a
RecordingBot that wraps the real go-telegram/bot.Bot with an
httptest server. The bot library hits the test server instead of
Telegram; multipart form fields are captured per call. Tests assert
on Sent() / LastSent() / AssertSentText().
- Handler tests added: misc (4), util (7), wordle (10), loldle (9),
loldleemoji (8). Cover happy paths, error paths, auth gates,
group-vs-private subject keying, KV side effects.
- Coverage 44.7% -> 69.8% (verified via -coverprofile). All packages
now report coverage in CI output.
- CI: ci.yml installs cloud-firestore-emulator beta component and
starts it on localhost:8090 before go test. Sets
FIRESTORE_EMULATOR_HOST + GOOGLE_CLOUD_PROJECT env so the storage
package's emulator-gated tests execute instead of skipping.
go test -race -count=1 ./... clean across all 15 packages locally.
tiennm99