Parallelizing the per-symbol fetches opened N simultaneous TLS handshakes into
an empty connection pool. On the memory-constrained Lambda (256MB ~0.15 vCPU)
those CPU-bound handshakes thrashed and each exceeded the per-fetch timeout, so
every ticker rendered "(no price)". Sequential fetches reuse the price client's
keep-alive connection (one handshake), which is why the code was sequential by
design.
- revert stock and coin stats loops to sequential (keep the reply-budget
sub-context and 3s per-fetch timeout)
- log per-symbol fetch failures instead of silently swallowing them
Stats handlers reused the single bounded update context for both upstream
price fetches and the final Telegram reply, while per-upstream HTTP timeouts
equalled the whole-handler budget. One slow upstream drained the deadline and
the reply failed with "context deadline exceeded" (observed on /stock_stats).
- add chathelper.FetchContext: fetches run under a child context that reserves
a tail of the deadline for the reply, which is sent on the original context
- cap kbs/coin/gold/vnappmob HTTP timeouts at 3s so one hung upstream cannot
consume the reply budget
- fetch held stock/coin prices concurrently (bounded) so total latency tracks
the slowest single fetch instead of their sum; per-symbol failures degrade to
a "no price" line and the summary still sends
- Change /coin_sell semantics from quantity to USD amount
- Add price-validity and dust-quantity guards to buy/sell
- Improve usage text and error messages
- Add tests for invalid amount, dust, zero price, and insufficient holdings
Replace goldprice.org (403s datacenter IPs) with a provider chain:
gold-api.com -> Swissquote public quotes -> NBP daily fixing (PLN/gram
converted via the shared FX table). Per-provider failures are logged;
a full-chain outage surfaces as a retryable error instead of the
silent no-price reply.
- prefix.go: return errors.ErrUnsupported when inner store lacks CompareAndSwap, enabling fail-fast instead of infinite retries
- Adds comprehensive CAS semantics test coverage for all KV backends:
* memory_kv_test.go: new tests for basic operations and CAS failures
* prefix_test.go: tests for wrapped CAS errors and unsupported operations
* dynamodb_kv_test.go, firestore_kv_test.go: CAS failure scenarios
- portfolio_test.go: test retry exhaustion, business-error short-circuit, concurrent updates, and fail-fast on unsupported CAS
The CF→AWS data migration (closed 2026-05-16) is long done and the
tooling isn't wired into any production path. Remove the one-shot binary,
its support package, and the migration runbook.
In live code, replace 'JS-parity' / 'same shape as JS' / 'cross-runtime
KV migration' comments with the real, stable reason for each behavior
(wire-format invariant, null-vs-zero distinction, CloudWatch alarm field
name, etc.). 24 files touched across lolschedule, loldle, wordle, twentyq,
trading, misc, util, server, metrics, ai, keylock.
- delete cmd/migrate_cf_data/
- delete internal/migration/
- delete docs/cf-to-aws-migration-runbook.md
Subscribers previously stored only chatID; the daily-push cron sent
without MessageThreadID so Telegram routed to General regardless of the
topic the user subscribed from.
- Storage shape: []Subscriber{ChatID, ThreadID} keyed by (ChatID, ThreadID)
so the same chat can subscribe in multiple topics independently.
- Decoder accepts legacy []int64 rows (ThreadID=0); successful mutations
rewrite the slot in the new shape, no manual migration needed.
- Cron forwards MessageThreadID on every send.
- Auto-prune classifies terminal errors: chat-wide kills (bot blocked,
deactivated, kicked, deleted, group upgraded) wipe every entry for the
ChatID; topic-only "have no rights to send" prunes just (ChatID, ThreadID).
Extend the stats module to track per-user counts in addition to the
existing per-command totals. CommandHook now receives the originating
*models.Update so the hook can attribute invocations to a user; only
the stats module consumes this hook today.
Schema (sort keys under pk="stats"):
count:<cmd> existing, per-command total
user:<id> new, per-user total with cached username
pair:<cmd>:<id> new, per (command, user) pair
Subcommands (all public):
/stats top commands (unchanged)
/stats users top users overall
/stats user <name> top commands for that user
/stats cmd <name> top users of that command
When the sender has no Telegram username the per-user/pair writes are
skipped — the global per-command counter still increments. View helpers
fan out KV reads in parallel to stay inside the webhook deadline. The
existing read-modify-write race is unchanged; closing it would need
atomic UpdateItem ADD on the KV interface.
Sequential List + N GetItem made /stats latency scale with the number
of tracked commands. On a cold Lambda container with ~25 commands the
cumulative DynamoDB round-trips can exceed the 10s webhook handler
deadline; the trailing chathelper.Reply -> b.SendMessage then fails on
a cancelled ctx and the dispatcher only logs the error, leaving the
user with no reply at all.
Fan the per-key GetJSONs out into goroutines joined by sync.WaitGroup.
Wall-clock latency collapses to ~1 round-trip while preserving the
per-key error isolation (a single GetJSON failure still drops only its
own row). Storage interface unchanged; no race regression vs. the prior
per-item write pattern.
- Remove ReservedConcurrentExecutions: 1 (account lacks unreserved
concurrency headroom; deploy rolled back). Accept best-effort
counter semantics as documented.
- Add /stats to aws/telegram-commands.json so it appears in the
Telegram client's / autocomplete after the next deploy.
Implement a new stats module for the Telegram bot that tracks per-command usage with persistent KV storage. The module provides a /stats command displaying usage sorted by popularity with a 4096-byte Telegram message cap. Includes CommandHook integration for post-dispatch tracking via background goroutine (2s bounded context), proper test coverage, and registry initialization. Updated server config with stats factory and reserved concurrent execution control to prevent TOCTOU issues.
New internal/deploynotify package fires a single Telegram DM to
BOT_OWNER_ID on the first cold start that observes a new gitSHA.
Dedup via a DynamoDB KV record so repeat cold starts of the same
version stay silent. Send-then-write order means a failed Telegram
call doesn't permanently silence retries.
gitSHA is baked into the binary via `-ldflags "-X main.gitSHA=..."`
from Makefile; empty SHA (non-make builds) silently disables the
feature. No new env vars or IAM permissions.
The default lolesports schedule page is anchored near "now", so a
calendar-aligned /lolschedule_week issued midweek was silently dropping
events from Mon–Wed: we only walked pages.newer. fetchSchedulePage now
also returns pages.older, and fetchEventsInRange walks older until the
earliest collected event is ≤ from, then walks newer until ≥ to. Page
budget bumped 3 → 8 to accommodate dense regular-season weeks.
Previously a rolling 7-day window from today; now snaps to the Monday
of the current ICT week so the reply matches what users mean by "this
week". Adds ictWeekStartOf helper plus week-boundary tests.
Vietnamese disclaimer one-liner. Interpolates a target name (default "VNG")
and the sender — @username when set, else tg://user?id link with the display
name. User-supplied text HTML-escaped; reply routed through ReplyHTML to keep
forum-topic threading.
Concurrency
- lolschedule: serialize subscriber Get→mutate→Put via state.subscribersMu;
the single-slot list was previously losing writes under concurrent
/lolschedule_subscribe.
- trading: PriceClient memoises its default *http.Client so /trade_stats
reuses TLS connections across held tickers.
Observability
- server/log_middleware: defer the req log line and recover panics so a
panicking cron handler still emits the structured req entry CloudWatch
filters on for 5xx alerting.
- server/router (cron): inner recover with cron-name context captures the
panicking job before the middleware's safety net does.
- telegram/webhook: rune-safe truncation in dispatch logs — Vietnamese,
Korean, and emoji previews no longer ship as garbled bytes.
- lolschedule/api_client: same rune-safe fix for error-body log truncation.
- telegram/webhook: gate the post-recover WriteHeader(200) so a panicking
handler that already touched w doesn't trigger superfluous-WriteHeader.
Correctness
- twentyq: clearGame error during solved-relaunch is logged instead of
silently swallowed (was a permanent deadlock vector on KV failure).
- misc /mstats: KV read failure replies "Could not load stats. Try again
later." to the user instead of returning into the dispatcher; matches the
pattern other modules use.
- migrate_cf_data trading-audit-dump: surface f.Close error so a truncated
JSONL never passes silently as a complete audit dump.
Operator ergonomics
- migrate_cf_data (all 4 subcommands): signal.NotifyContext for SIGINT /
SIGTERM. Ctrl-C mid-Scan now propagates cleanly instead of leaving a
half-converted DynamoDB table.
- ai/ratelimit: doc the Lambda-recycle memory bound to match keylock.Map
so a future reviewer doesn't re-flag the unbounded map.
I/O-changing (user-approved)
- lolschedule daily push auto-prunes subscribers whose Telegram error
matches a terminal marker (blocked / deactivated / chat gone). Transient
errors keep the chat on the list. Subscribe message updated to mention
the auto-cleanup.
- twentyq seed pool grown 50 → 178; repeat-collision threshold moves from
~9 plays to ~17 (birthday paradox).
- util /info flipped Public → Protected — chat/thread/sender IDs are no
longer enumerable by every group member.
- cmd/server WriteTimeout 6min → 75s (cron 60s + 15s slack). No-op on
Lambda; matters only for local non-Lambda runs.
- webhook + cron rejection paths drop response bodies (no fingerprintable
text for internet scanners hitting the public Function URL). Status
codes preserved for CloudWatch metrics; structured log lines carry the
rejection reason for operator triage.
Tests added: TestTruncateRunes, TestRunDailyPush_PrunesDeadSubscribers,
TestIsTerminalSendError, TestInfo_DeniedToNonOwner,
TestInfo_DeniedToChannelMessageNoFrom, plus owner-allowed counterparts.
Telegram routes outgoing messages with no message_thread_id to a forum
supergroup's General topic. Commands sent in a topic were being answered
in General. chathelper.Reply / ReplyHTML now take *models.Message and
forward both ChatID and MessageThreadID; help.go SendMessage and
loldle trySendSticker do the same. Adds regression tests asserting the
field is forwarded for forum topics and omitted for private/regular groups.
The go-telegram/bot v1.20.0 MatchTypeCommand does byte-exact equality
on the full bot_command entity, which in groups is "cmd@botname"
(length 15 for /help@miti99bot) — never equal to the registered name
"cmd". All 27 commands silently missed when used in group chats.
Swap RegisterHandler(..., MatchTypeCommand, ...) for a local matchFunc
that strips the @suffix before comparing. Telegram only routes
/cmd@otherbot to otherbot, so the suffix is safe to drop
unconditionally.
Add a structured `dispatch` log line in the webhook (update_id,
chat_id, chat_type, text preview) so the next silent-drop symptom
shows up in CloudWatch without code archaeology — the existing
[TGBOT] [UPDATE] line only prints struct pointers.
12 new TestMatchCommand sub-tests cover DM, group, mid-text, wrong
command, non-command entity, and bounds edge cases.
Pre-existing 9 errcheck violations introduced by d67517e blocked
golangci-lint on every CI run after the migration toolchain landed.
- Wrap defer Close() in anonymous func to match house style
(cf. internal/modules/lolschedule/api_client.go:162).
- Mark Fprint*/Fprintln to io.Writer with _, _ = (best-effort writes
to os.Stdout / bytes.Buffer; errors not actionable for callers).
No behavior change. go vet, golangci-lint, go test ./... all clean
locally.
Operator-run tooling that moves durable Cloudflare KV data into the live
AWS DynamoDB table, plus a runtime swap of the `value` attribute from
Binary to String so payloads are human-readable in the AWS console.
- cmd/migrate_cf_data: subcommands inventory, kv-import (idempotent via
attribute_not_exists), trading-audit-dump, convert-value-to-string
- internal/migration: policy allowlist, CF KV+D1 REST clients, DynamoDB
writer, report formatter with per-prefix counts + tests
- internal/storage/dynamodb_kv.go: Put writes MemberS, Get reads MemberS;
dropped empty-bytes sentinel (DynamoDB allows empty strings)
Rename:
- Go module github.com/tiennm99/miti99bot-go → github.com/tiennm99/miti99bot
- CloudFormation stack miti99bot-aws-port → miti99bot
- Drop "port", "Cloud Run", "GCP", "cutover", "Phase NN" framing from
active code and docs — project reads as canonical AWS-Lambda from now on.
AWS deploy guide + flow fix:
- New docs/deploy-aws-free-tier-guide.md — Ubuntu 24.04 ARM64 onboarding
with project-local venv (pip awscli + sam-cli), SSM secrets via read -s,
idempotent OIDC provider + role creation, $1 budget alarm.
- Drop sam build from the pipeline — provided.al2023 + makefile builder
expects a Makefile in CodeUri (build/lambda/, the output dir), so the
step always fails. sam deploy --template-file template.yaml now reads
the raw template and zips build/lambda/ directly.
- Rollback section rewritten — use continue-update-rollback /
cancel-update-stack / git-SHA redeploy. Drop the broken
--use-previous-template recipe.
- DynamoDB free-tier row corrected (on-demand is 2.5M read / 1M write
request units, not 25 RCU/WCU).
Updated:
- README.md fully rewritten (drops port/legacy framing, lists modules,
points new users at the free-tier guide).
- aws/README.md retitled "AWS account setup", phase numbers stripped.
- Makefile / .github/workflows/deploy.yml — sam deploy flow.
- samconfig.toml — stack_name = "miti99bot".
- Go comments — Cloud Run → Lambda, Cloud Scheduler → EventBridge
Scheduler, Cloud Logging → CloudWatch Logs.
- Struct field GCPProject → FirestoreProject (env GOOGLE_CLOUD_PROJECT
unchanged).
Plus advisory reports under plans/reports/ from the code-reviewer +
researcher passes that informed the fixes.
Verified: go vet ./..., go build ./..., go test ./... all green.
Removes six modules (loldle-ability/emoji/quote/splash, semantle, doantu)
and prunes the framework deps that were only there to serve them:
- ai.Embedder + Client.Embed + embeddingModel const (semantle only)
- Deps.Embedder + BuildOptions.Embedder
- Deps.Env + Build(env) param + ModuleEnv config field + PHOW2SIM allowlist (doantu only)
- internal/champname package (loldle now owns its lookup helpers directly)
- template.yaml: Phow2simAPIURL parameter + PHOW2SIM_API_URL Lambda env
Active catalog: util, misc, wordle, loldle, lolschedule, twentyq, trading.
go build / vet / test all pass.
- Portfolio, holdings, prices, symbols packages with Telegram command handlers
- Handlers: trade_topup (add credits), trade_buy, trade_sell, trade_convert (coming-soon), trade_stats
- Per-user keylock for concurrency safety
- KBS (VN) as price source; ticker validation regex ^[A-Z0-9]{1,16}$
- Scope-trimmed: no SQL history, no retention cron, no daily refresh, no leaderboard, no forex
(matches upstream miti99bot's current VN-stocks-only state)
- Full unit test coverage for format, portfolio, prices, symbols, handlers
- Extend module.Deps struct with optional Bot field
- Add Bot to registry.BuildOptions and thread through builders
- Pass bot instance from main.go into module factory options
- Enables cron handlers to send messages and access bot state
- Add new modules: doantu, semantle, twentyq, ai (Gemini integration)
- Update module registry with new command registration
- Update tests and documentation for module system
- Update README with new module references
- Implement DynamoDB client wrapper and KV provider
- Full CRUD test coverage for both local and AWS modes
- AWS SDK v2 dependency added to go.mod/go.sum
tiennm99