Commit Graph
267 Commits
Author SHA1 Message Date
tiennm99 0abd33cb42 fix(deploynotify): read commit SHA from Coolify SOURCE_COMMIT runtime env
Baking SOURCE_COMMIT as a Docker build arg never worked: Coolify exposes it
as a runtime env var, not a build arg, so gitSHA was always empty and the owner
DM was skipped. Read SOURCE_COMMIT from the container env at startup (falling
back to the ldflags-baked gitSHA for local builds), forward it via the compose
environment, and drop the dead build-arg baking.
2026-06-28 23:08:17 +07:00
tiennm99 1fa549fa18 feat(modules): log every command with input, sender, chat, result
Emit one structured line per authorized command: input text, sender id +
username, chat type/id/title (DM vs group), and outcome (INFO ok / ERROR with
err). Replaces the prior error-only log; denied commands stay silent.
2026-06-28 22:52:55 +07:00
tiennm99 9d5b774846 refactor(server): delete webhook once on startup, no retry
The Bot API GET reliably clears the webhook, so the retry loop added for the
empty-body POST is no longer needed. One best-effort call; log on failure.
2026-06-28 22:52:55 +07:00
tiennm99 36cb5750e6 fix(telegram): clear webhook via Bot API GET to avoid empty-body POST
go-telegram's DeleteWebhook posts an empty multipart form (DropPendingUpdates
is omitempty); some networks answer that bodyless POST with an empty response,
failing to parse and leaving the webhook active so getUpdates 409s forever.
Replace it with a plain GET /deleteWebhook, which sends no body and parses the
JSON reliably. Keeps pending updates for a lossless cutover.
2026-06-28 22:41:49 +07:00
tiennm99 60a7aa5836 fix(server): retry webhook clear on startup past transient empty-body error
Telegram's edge can return an empty body on the first request after a cold
container start, failing the single best-effort DeleteWebhook with 'unexpected
end of JSON input' and leaving a stale webhook that 409s getUpdates. Retry up
to 5 times with a 1s backoff so long polling is reliably unblocked.
2026-06-28 22:33:34 +07:00
tiennm99 d4f1bbfc83 fix(server): clear Telegram webhook at startup before owner DM
Move DeleteWebhook ahead of the deploynotify owner DM and polling so the
webhook is cleared first thing once the bot is ready, instead of just before
getUpdates. DropPendingUpdates=false keeps the lossless cutover.
2026-06-28 22:18:28 +07:00
tiennm99 8edd0f8c27 build(coolify): use SOURCE_COMMIT build arg for deploynotify
Coolify sets SOURCE_COMMIT automatically; consume it as the build arg
feeding the baked commit SHA instead of a manually-passed GIT_SHA.
2026-06-28 22:15:12 +07:00
tiennm99 cc547136b7 refactor(server): drop HTTP /cron route; run crons in-process only
The in-process scheduler is the sole cron trigger on self-host, so the
/cron/{name} HTTP endpoint and its CRON_SHARED_SECRET are dead surface.
Reduce the HTTP server to GET / (health) and remove the cron secret config,
SSM binding, and now-unused timeout.
2026-06-28 22:15:04 +07:00
tiennm99 15a3239ab1 refactor(storage): replace KVStore with generic typed DocStore[T]
Delete the byte-oriented KVStore/VersionedStore abstraction and the
DynamoDB/memory KV backends. Add a generic typed store (DocStore[T] with
Provider/Collection/Typed) persisting each value as a flattened native
Mongo document (storedDoc[T] via bson inline) — no value envelope.

- MongoDB is the only runtime backend; memory kept for tests/local.
- All modules + deploynotify use typed stores; persisted structs carry
  bson tags == json names (incl. nested lolschedule/wordle types).
- lolschedule wraps its array/scalar values in named structs.
- migrate-dynamo-to-mongo writes the flattened shape via Typed[bson.M]
  with wrap rules; Scan/--dry-run/--verify retained.

Verified: go vet/build clean; full go test green hermetically and
in-container vs real Mongo 7 + DynamoDB Local (storage integration +
migrator e2e).
2026-06-28 18:02:11 +07:00
tiennm99 b4910c2289 docs(plan): native mongo value documents + version CAS plan
Brainstorm decision record (Option A) + validated 2-phase TDD plan for
storing mongo values as native BSON and switching CAS to a version field.
2026-06-28 12:53:20 +07:00
tiennm99 e8a6dbb4de feat(storage): store mongo values as native BSON documents
Implement mongodb_value_codec to encode portfolio/gold/stock values as native
BSON documents instead of JSON-serialized byte strings. Reduces storage size,
improves query efficiency, and eliminates unmarshal overhead. Native BSON
encoding preserves int64 fidelity and enables future native-format queries.
Add NativeValueRepresentation and Int64Fidelity tests validating lossless
encoding. Update migration tool comments. Document native-value behavior
in deploy guide. Update phase-01 plan with implementation notes.
2026-06-28 12:52:48 +07:00
tiennm99 6bc5f334bb refactor(storage): remove legacy firestore backend
Delete Firestore provider implementation and tests (5 files). Extract shared
key helpers to new internal/storage/keys.go module. Remove Firestore case
from server bootstrap and storage factory. Drop cloud.google.com/go/firestore
and related dependencies from go.mod. Remove firestore-emulator test targets
from Makefile. Update README storage section to reflect MongoDB-only backend.
2026-06-28 12:52:40 +07:00
tiennm99 cd82abbb0c refactor(storage): replace value-bytes CAS with version-field optimistic lock
Replace content-addressed storage versioning (CAS) with explicit version fields
on documents. Versions are managed via optimistic locking pattern: Get returns
version, Put increments atomically and fails if version mismatch detected.
Callers (portfolio, lolschedule) updated to use GetVersioned/PutVersioned,
simplifying concurrency handling and eliminating CAS overhead.
2026-06-28 12:52:32 +07:00
tiennm99 ab7a597541 refactor(storage): store mongo value as JSON string, not BinData
Persist the KV value field as a BSON string so it is directly readable in
the Atlas/Compass UI, matching DynamoDB's String storage. Reads still accept
BSON binary as a backward-compat fallback. Non-UTF-8 callers must encode
upstream, same constraint as DynamoDB; all current callers write JSON.
2026-06-28 10:50:41 +07:00
tiennm99 a83582f79d docs(plan): update self-host phases with completion status and implementation details
Mark phases 1–5 (MongoDB, cron, polling, migration, decommission) as completed.
Update phase files with finalized implementation notes and test coverage summaries.
2026-06-28 09:58:41 +07:00
tiennm99 6d1bb5d925 docs(selfhost): add Coolify deploy + AWS decommission guides; retire AWS docs
Add comprehensive Coolify deployment guide with MongoDB Atlas + networking setup.
Add AWS decommission runbook for safe teardown. Update AWS guides to note
deprecation. Update main README with self-hosted quickstart and architecture overview.
2026-06-28 09:58:37 +07:00
tiennm99 4a8466db5b feat(selfhost): containerize for Coolify + decommission AWS deploy
Add multi-stage Dockerfile, docker-compose.yml for local dev, .env.example template,
and Makefile helpers. Disable GitHub Actions AWS deploy workflow. Supports Coolify
container orchestration for self-hosted deployments.
2026-06-28 09:58:32 +07:00
tiennm99 b27d2b3b87 feat(migrate): add DynamoDB-to-MongoDB data migrator
Add standalone migrator tool with scan-and-transform logic for moving persisted
state from DynamoDB to MongoDB. Includes tests and README with usage instructions.
2026-06-28 09:58:28 +07:00
tiennm99 f399717613 refactor(stock): remove FireAnt income-events command
Drop FireAnt-based /income command per AWS decommission plan. Simplifies
stock module and reduces third-party dependencies. Updates telegram-commands.json.
2026-06-28 09:58:25 +07:00
tiennm99 a081427ae5 feat(server): wire MongoDB + cron + polling; rename owner/admin envs
Wire MongoDB client and cron scheduler into server initialization. Rename
TELEGRAM_OWNER to TELEGRAM_ADMIN_ID and SLACK_OWNER to SLACK_ADMIN_ID
for clarity. Enable long-polling as default with cron scheduler for self-hosted.
2026-06-28 09:58:20 +07:00
tiennm99 8d8c563cec feat(telegram): switch to long polling, remove webhook path
Replace webhook-based updates with long-polling GetUpdates loop. Eliminates
external HTTP endpoint requirement and simplifies self-hosted deployments.
Removes webhook.go and associated webhook routing from server.
2026-06-28 09:58:16 +07:00
tiennm99 2432c910dc feat(cron): add in-process cron scheduler
Add robfig/cron-based task scheduler with per-UTC-date idempotency guard. Enables
scheduled jobs (e.g., daily price checks) without external scheduler dependency.
Module interface updated to support cron registration.
2026-06-28 09:58:12 +07:00
tiennm99 bfaecef86c feat(storage): add MongoDB KVProvider backend
Add MongoDB-backed storage provider with connection pooling, key-value operations
(get, set, delete), and comprehensive test coverage. Integrates mongo-driver
dependency for self-hosted deployments.
2026-06-28 09:58:07 +07:00
tiennm99 4bfb961423 docs(selfhost): long polling, minimal env surface, AWS decommission scope
- Telegram transport: long polling only (delete webhook code path)
- Mongo auto-selected from MONGO_URL; in-process cron always-on
  (drop KV_PROVIDER, CRON_MODE, PORT from required env)
- Drop stock/coin/gold API URL overrides (use coded providers);
  remove stock income-events feature + FireAnt token
- Gold VNAppMob key auto-fetched/cached in DB (no env)
- Rename OWNER_ID, ADMIN_IDS; add Phase 5 AWS full decommission
- Final env: TELEGRAM_BOT_TOKEN, MONGO_URL, MONGO_DATABASE, MODULES,
  OWNER_ID, ADMIN_IDS (+ optional GEMINI_API_KEY)
- Red-team (15 findings) + validation (5 sessions) logged
2026-06-27 21:35:12 +07:00
tiennm99 b1171e7cbe docs(selfhost): plan Coolify + MongoDB Atlas self-host with AWS decommission
Add 5-phase plan to self-host on Coolify (docker-compose) with a MongoDB
Atlas backend, an in-process cron scheduler, DynamoDB->Atlas migration,
and full AWS teardown. Red-teamed and validated; decommission scope
verified against live AWS/Cloudflare accounts. Include free-tier audit
and S3-elimination research reports. Ignore wrangler local cache.
2026-06-27 20:46:43 +07:00
tiennm99 c107489f89 refactor(stock): drop symbol cache 2026-06-25 17:30:57 +07:00
tiennm99 8343146c28 docs(stock): add price provider research 2026-06-25 17:17:08 +07:00
tiennm99 412d25b19e fix(stock): drop yahoo price fallback 2026-06-25 17:16:54 +07:00
tiennm99 006ad2b676 fix(stock): expose ssi quote response errors 2026-06-25 16:12:12 +07:00
tiennm99 fca5b8961f docs(stock): record ssi provider research 2026-06-25 15:52:43 +07:00
tiennm99 78e51157d7 fix(stock): use ssi price provider 2026-06-25 15:51:33 +07:00
tiennm99 c0cfcfb8d5 fix(modules): fetch stats prices sequentially to reuse pooled connections
Parallelizing the per-symbol fetches opened N simultaneous TLS handshakes into
an empty connection pool. On the memory-constrained Lambda (256MB ~0.15 vCPU)
those CPU-bound handshakes thrashed and each exceeded the per-fetch timeout, so
every ticker rendered "(no price)". Sequential fetches reuse the price client's
keep-alive connection (one handshake), which is why the code was sequential by
design.

- revert stock and coin stats loops to sequential (keep the reply-budget
  sub-context and 3s per-fetch timeout)
- log per-symbol fetch failures instead of silently swallowing them
2026-06-25 14:28:25 +07:00
tiennm99 958da76d33 fix(modules): reserve reply budget and cap price-fetch timeouts in stats handlers
Stats handlers reused the single bounded update context for both upstream
price fetches and the final Telegram reply, while per-upstream HTTP timeouts
equalled the whole-handler budget. One slow upstream drained the deadline and
the reply failed with "context deadline exceeded" (observed on /stock_stats).

- add chathelper.FetchContext: fetches run under a child context that reserves
  a tail of the deadline for the reply, which is sent on the original context
- cap kbs/coin/gold/vnappmob HTTP timeouts at 3s so one hung upstream cannot
  consume the reply budget
- fetch held stock/coin prices concurrently (bounded) so total latency tracks
  the slowest single fetch instead of their sum; per-symbol failures degrade to
  a "no price" line and the summary still sends
2026-06-25 14:07:59 +07:00
tiennm99 eb8c2e0064 docs(plan): add coin sell message plan 2026-06-20 09:52:35 +07:00
tiennm99 6aeca59b98 fix(coin): improve sell insufficient message 2026-06-20 09:52:23 +07:00
tiennm99 71f3336387 fix(coin): sell command accepts USD amount with dust and zero-price guards
- Change /coin_sell semantics from quantity to USD amount

- Add price-validity and dust-quantity guards to buy/sell

- Improve usage text and error messages

- Add tests for invalid amount, dust, zero price, and insufficient holdings
2026-06-16 17:27:39 +07:00
tiennm99 224a27cfd4 fix(coin): use Binance public market-data endpoint data-api.binance.vision 2026-06-15 14:09:11 +07:00
tiennm99 fd2e84f8fd docs(plans): add VNAppMob key parsing bug-fix plan 2026-06-15 12:20:44 +07:00
tiennm99 b2e8385706 fix(gold): parse VNAppMob JWT object wrapper 2026-06-15 12:20:40 +07:00
tiennm99 e0b0418173 fix(gold): use GET for VNAppMob key refresh endpoint
VNAppMob /api/request_api_key returns 405 for POST and accepts GET.
2026-06-15 11:17:50 +07:00
tiennm99 12dc515047 feat(gold): remove XAU/USD fallback and use VNAppMob SJC only 2026-06-15 11:04:06 +07:00
tiennm99 966e4afd51 chore(plans): add VNAppMob integration plan and completion report 2026-06-15 10:49:19 +07:00
tiennm99 ae92539043 docs(gold): document VNAppMob SJC pricing and env vars 2026-06-15 10:49:19 +07:00
tiennm99 a6bf022e27 feat(gold): integrate VNAppMob SJC gold price with auto-refresh API key 2026-06-15 10:49:19 +07:00
tiennm99 8c2c9a7c84 chore: remove one-time stock-key migration tool
Migration complete and verified in production; the trading->stock
DynamoDB partition copy is no longer needed and the old trading rows
have been deleted.
2026-06-12 15:46:03 +07:00
tiennm99 6d0a30c4c0 fix(migrate): check fmt.Fprintf return to satisfy errcheck lint 2026-06-12 15:12:07 +07:00
tiennm99 32417e4196 refactor(stock): rename trading module to stock
Rename the `trading` module to `stock` across code, config, and docs for
naming consistency with the sibling coin/gold modules.

- Module: internal/modules/trading -> internal/modules/stock (package,
  factory, registry key in cmd/server)
- Commands: trade_* -> stock_* (telegram-commands.json)
- Env/SAM params: TRADING_INCOME_EVENTS_* -> STOCK_INCOME_EVENTS_*
- ModulesCSV updated in template.yaml, samconfig.toml, deploy.yml
- Add cmd/migrate-stock-key: idempotent, non-destructive DynamoDB
  partition copy (pk=trading -> pk=stock) with dry-run default

Production DynamoDB data already migrated (8 keys: 2 portfolios + 6
symbol-cache rows); old trading partition retained as rollback snapshot.
2026-06-12 14:52:12 +07:00
tiennm99 62555dee23 fix(deploy): include coin module in ModulesCSV overrides
coin was registered in the catalog and template default but absent from
the ModulesCSV override in both deploy.yml and samconfig.toml, so the
deployed Lambda never loaded it and all coin_* commands went unregistered.

Add a registration checklist to docs/deploy-aws.md to prevent recurrence.
2026-06-12 11:22:16 +07:00
tiennm99 49a6e7091a fix(coin): accept symbol-first trades 2026-06-12 11:07:27 +07:00
tiennm99 b234b54b2e feat(coin): register telegram commands 2026-06-12 10:53:19 +07:00