Commit Graph
262 Commits
Author SHA1 Message Date
tiennm99 bd7a7fcedf fix(server): clear Telegram webhook at startup before owner DM
Move DeleteWebhook ahead of the deploynotify owner DM and polling so the
webhook is cleared first thing once the bot is ready, instead of just before
getUpdates. DropPendingUpdates=false keeps the lossless cutover.
2026-06-28 22:18:28 +07:00
tiennm99 19a2b2597e build(coolify): use SOURCE_COMMIT build arg for deploynotify
Coolify sets SOURCE_COMMIT automatically; consume it as the build arg
feeding the baked commit SHA instead of a manually-passed GIT_SHA.
2026-06-28 22:15:12 +07:00
tiennm99 2a2416236c refactor(server): drop HTTP /cron route; run crons in-process only
The in-process scheduler is the sole cron trigger on self-host, so the
/cron/{name} HTTP endpoint and its CRON_SHARED_SECRET are dead surface.
Reduce the HTTP server to GET / (health) and remove the cron secret config,
SSM binding, and now-unused timeout.
2026-06-28 22:15:04 +07:00
tiennm99 b81b6501c5 refactor(storage): replace KVStore with generic typed DocStore[T]
Delete the byte-oriented KVStore/VersionedStore abstraction and the
DynamoDB/memory KV backends. Add a generic typed store (DocStore[T] with
Provider/Collection/Typed) persisting each value as a flattened native
Mongo document (storedDoc[T] via bson inline) — no value envelope.

- MongoDB is the only runtime backend; memory kept for tests/local.
- All modules + deploynotify use typed stores; persisted structs carry
  bson tags == json names (incl. nested lolschedule/wordle types).
- lolschedule wraps its array/scalar values in named structs.
- migrate-dynamo-to-mongo writes the flattened shape via Typed[bson.M]
  with wrap rules; Scan/--dry-run/--verify retained.

Verified: go vet/build clean; full go test green hermetically and
in-container vs real Mongo 7 + DynamoDB Local (storage integration +
migrator e2e).
2026-06-28 18:02:11 +07:00
tiennm99 9b39dba929 docs(plan): native mongo value documents + version CAS plan
Brainstorm decision record (Option A) + validated 2-phase TDD plan for
storing mongo values as native BSON and switching CAS to a version field.
2026-06-28 12:53:20 +07:00
tiennm99 a03770cf6b feat(storage): store mongo values as native BSON documents
Implement mongodb_value_codec to encode portfolio/gold/stock values as native
BSON documents instead of JSON-serialized byte strings. Reduces storage size,
improves query efficiency, and eliminates unmarshal overhead. Native BSON
encoding preserves int64 fidelity and enables future native-format queries.
Add NativeValueRepresentation and Int64Fidelity tests validating lossless
encoding. Update migration tool comments. Document native-value behavior
in deploy guide. Update phase-01 plan with implementation notes.
2026-06-28 12:52:48 +07:00
tiennm99 4b15d07d1b refactor(storage): remove legacy firestore backend
Delete Firestore provider implementation and tests (5 files). Extract shared
key helpers to new internal/storage/keys.go module. Remove Firestore case
from server bootstrap and storage factory. Drop cloud.google.com/go/firestore
and related dependencies from go.mod. Remove firestore-emulator test targets
from Makefile. Update README storage section to reflect MongoDB-only backend.
2026-06-28 12:52:40 +07:00
tiennm99 07eddb2733 refactor(storage): replace value-bytes CAS with version-field optimistic lock
Replace content-addressed storage versioning (CAS) with explicit version fields
on documents. Versions are managed via optimistic locking pattern: Get returns
version, Put increments atomically and fails if version mismatch detected.
Callers (portfolio, lolschedule) updated to use GetVersioned/PutVersioned,
simplifying concurrency handling and eliminating CAS overhead.
2026-06-28 12:52:32 +07:00
tiennm99 af0871d45b refactor(storage): store mongo value as JSON string, not BinData
Persist the KV value field as a BSON string so it is directly readable in
the Atlas/Compass UI, matching DynamoDB's String storage. Reads still accept
BSON binary as a backward-compat fallback. Non-UTF-8 callers must encode
upstream, same constraint as DynamoDB; all current callers write JSON.
2026-06-28 10:50:41 +07:00
tiennm99 09c9566541 docs(plan): update self-host phases with completion status and implementation details
Mark phases 1–5 (MongoDB, cron, polling, migration, decommission) as completed.
Update phase files with finalized implementation notes and test coverage summaries.
2026-06-28 09:58:41 +07:00
tiennm99 74643efdb3 docs(selfhost): add Coolify deploy + AWS decommission guides; retire AWS docs
Add comprehensive Coolify deployment guide with MongoDB Atlas + networking setup.
Add AWS decommission runbook for safe teardown. Update AWS guides to note
deprecation. Update main README with self-hosted quickstart and architecture overview.
2026-06-28 09:58:37 +07:00
tiennm99 4b22e48b24 feat(selfhost): containerize for Coolify + decommission AWS deploy
Add multi-stage Dockerfile, docker-compose.yml for local dev, .env.example template,
and Makefile helpers. Disable GitHub Actions AWS deploy workflow. Supports Coolify
container orchestration for self-hosted deployments.
2026-06-28 09:58:32 +07:00
tiennm99 9e914204ce feat(migrate): add DynamoDB-to-MongoDB data migrator
Add standalone migrator tool with scan-and-transform logic for moving persisted
state from DynamoDB to MongoDB. Includes tests and README with usage instructions.
2026-06-28 09:58:28 +07:00
tiennm99 1f6b78f39e refactor(stock): remove FireAnt income-events command
Drop FireAnt-based /income command per AWS decommission plan. Simplifies
stock module and reduces third-party dependencies. Updates telegram-commands.json.
2026-06-28 09:58:25 +07:00
tiennm99 8048be8d8a feat(server): wire MongoDB + cron + polling; rename owner/admin envs
Wire MongoDB client and cron scheduler into server initialization. Rename
TELEGRAM_OWNER to TELEGRAM_ADMIN_ID and SLACK_OWNER to SLACK_ADMIN_ID
for clarity. Enable long-polling as default with cron scheduler for self-hosted.
2026-06-28 09:58:20 +07:00
tiennm99 d8d1ebc0bc feat(telegram): switch to long polling, remove webhook path
Replace webhook-based updates with long-polling GetUpdates loop. Eliminates
external HTTP endpoint requirement and simplifies self-hosted deployments.
Removes webhook.go and associated webhook routing from server.
2026-06-28 09:58:16 +07:00
tiennm99 8e4644b1c2 feat(cron): add in-process cron scheduler
Add robfig/cron-based task scheduler with per-UTC-date idempotency guard. Enables
scheduled jobs (e.g., daily price checks) without external scheduler dependency.
Module interface updated to support cron registration.
2026-06-28 09:58:12 +07:00
tiennm99 1e2a76cee6 feat(storage): add MongoDB KVProvider backend
Add MongoDB-backed storage provider with connection pooling, key-value operations
(get, set, delete), and comprehensive test coverage. Integrates mongo-driver
dependency for self-hosted deployments.
2026-06-28 09:58:07 +07:00
tiennm99 c9c2c7a4ab docs(selfhost): long polling, minimal env surface, AWS decommission scope
- Telegram transport: long polling only (delete webhook code path)
- Mongo auto-selected from MONGO_URL; in-process cron always-on
  (drop KV_PROVIDER, CRON_MODE, PORT from required env)
- Drop stock/coin/gold API URL overrides (use coded providers);
  remove stock income-events feature + FireAnt token
- Gold VNAppMob key auto-fetched/cached in DB (no env)
- Rename OWNER_ID, ADMIN_IDS; add Phase 5 AWS full decommission
- Final env: TELEGRAM_BOT_TOKEN, MONGO_URL, MONGO_DATABASE, MODULES,
  OWNER_ID, ADMIN_IDS (+ optional GEMINI_API_KEY)
- Red-team (15 findings) + validation (5 sessions) logged
2026-06-27 21:35:12 +07:00
tiennm99 0a66fb15bc docs(selfhost): plan Coolify + MongoDB Atlas self-host with AWS decommission
Add 5-phase plan to self-host on Coolify (docker-compose) with a MongoDB
Atlas backend, an in-process cron scheduler, DynamoDB->Atlas migration,
and full AWS teardown. Red-teamed and validated; decommission scope
verified against live AWS/Cloudflare accounts. Include free-tier audit
and S3-elimination research reports. Ignore wrangler local cache.
2026-06-27 20:46:43 +07:00
tiennm99 50f1566494 refactor(stock): drop symbol cache 2026-06-25 17:30:57 +07:00
tiennm99 949bb01427 docs(stock): add price provider research 2026-06-25 17:17:08 +07:00
tiennm99 3800b27954 fix(stock): drop yahoo price fallback 2026-06-25 17:16:54 +07:00
tiennm99 762ccfc836 fix(stock): expose ssi quote response errors 2026-06-25 16:12:12 +07:00
tiennm99 65d02d9aab docs(stock): record ssi provider research 2026-06-25 15:52:43 +07:00
tiennm99 a28a43cdae fix(stock): use ssi price provider 2026-06-25 15:51:33 +07:00
tiennm99 b7838f7347 fix(modules): fetch stats prices sequentially to reuse pooled connections
Parallelizing the per-symbol fetches opened N simultaneous TLS handshakes into
an empty connection pool. On the memory-constrained Lambda (256MB ~0.15 vCPU)
those CPU-bound handshakes thrashed and each exceeded the per-fetch timeout, so
every ticker rendered "(no price)". Sequential fetches reuse the price client's
keep-alive connection (one handshake), which is why the code was sequential by
design.

- revert stock and coin stats loops to sequential (keep the reply-budget
  sub-context and 3s per-fetch timeout)
- log per-symbol fetch failures instead of silently swallowing them
2026-06-25 14:28:25 +07:00
tiennm99 68cfa7abd5 fix(modules): reserve reply budget and cap price-fetch timeouts in stats handlers
Stats handlers reused the single bounded update context for both upstream
price fetches and the final Telegram reply, while per-upstream HTTP timeouts
equalled the whole-handler budget. One slow upstream drained the deadline and
the reply failed with "context deadline exceeded" (observed on /stock_stats).

- add chathelper.FetchContext: fetches run under a child context that reserves
  a tail of the deadline for the reply, which is sent on the original context
- cap kbs/coin/gold/vnappmob HTTP timeouts at 3s so one hung upstream cannot
  consume the reply budget
- fetch held stock/coin prices concurrently (bounded) so total latency tracks
  the slowest single fetch instead of their sum; per-symbol failures degrade to
  a "no price" line and the summary still sends
2026-06-25 14:07:59 +07:00
tiennm99 6a96f09f51 docs(plan): add coin sell message plan 2026-06-20 09:52:35 +07:00
tiennm99 ef1817b43e fix(coin): improve sell insufficient message 2026-06-20 09:52:23 +07:00
tiennm99 27b4c6696d fix(coin): sell command accepts USD amount with dust and zero-price guards
- Change /coin_sell semantics from quantity to USD amount

- Add price-validity and dust-quantity guards to buy/sell

- Improve usage text and error messages

- Add tests for invalid amount, dust, zero price, and insufficient holdings
2026-06-16 17:27:39 +07:00
tiennm99 86c84e360b fix(coin): use Binance public market-data endpoint data-api.binance.vision 2026-06-15 14:09:11 +07:00
tiennm99 a51a60f343 docs(plans): add VNAppMob key parsing bug-fix plan 2026-06-15 12:20:44 +07:00
tiennm99 da9303cd82 fix(gold): parse VNAppMob JWT object wrapper 2026-06-15 12:20:40 +07:00
tiennm99 6007602fb8 fix(gold): use GET for VNAppMob key refresh endpoint
VNAppMob /api/request_api_key returns 405 for POST and accepts GET.
2026-06-15 11:17:50 +07:00
tiennm99 c8fb143532 feat(gold): remove XAU/USD fallback and use VNAppMob SJC only 2026-06-15 11:04:06 +07:00
tiennm99 35c719cd4b chore(plans): add VNAppMob integration plan and completion report 2026-06-15 10:49:19 +07:00
tiennm99 195336e753 docs(gold): document VNAppMob SJC pricing and env vars 2026-06-15 10:49:19 +07:00
tiennm99 a295337281 feat(gold): integrate VNAppMob SJC gold price with auto-refresh API key 2026-06-15 10:49:19 +07:00
tiennm99 73bacf863f chore: remove one-time stock-key migration tool
Migration complete and verified in production; the trading->stock
DynamoDB partition copy is no longer needed and the old trading rows
have been deleted.
2026-06-12 15:46:03 +07:00
tiennm99 a65ff8b60b fix(migrate): check fmt.Fprintf return to satisfy errcheck lint 2026-06-12 15:12:07 +07:00
tiennm99 dce5bcf12c refactor(stock): rename trading module to stock
Rename the `trading` module to `stock` across code, config, and docs for
naming consistency with the sibling coin/gold modules.

- Module: internal/modules/trading -> internal/modules/stock (package,
  factory, registry key in cmd/server)
- Commands: trade_* -> stock_* (telegram-commands.json)
- Env/SAM params: TRADING_INCOME_EVENTS_* -> STOCK_INCOME_EVENTS_*
- ModulesCSV updated in template.yaml, samconfig.toml, deploy.yml
- Add cmd/migrate-stock-key: idempotent, non-destructive DynamoDB
  partition copy (pk=trading -> pk=stock) with dry-run default

Production DynamoDB data already migrated (8 keys: 2 portfolios + 6
symbol-cache rows); old trading partition retained as rollback snapshot.
2026-06-12 14:52:12 +07:00
tiennm99 53578d888b fix(deploy): include coin module in ModulesCSV overrides
coin was registered in the catalog and template default but absent from
the ModulesCSV override in both deploy.yml and samconfig.toml, so the
deployed Lambda never loaded it and all coin_* commands went unregistered.

Add a registration checklist to docs/deploy-aws.md to prevent recurrence.
2026-06-12 11:22:16 +07:00
tiennm99 b70217090a fix(coin): accept symbol-first trades 2026-06-12 11:07:27 +07:00
tiennm99 d5c6751506 feat(coin): register telegram commands 2026-06-12 10:53:19 +07:00
tiennm99 eb7b34d368 feat(coin): add crypto paper trading 2026-06-12 10:47:02 +07:00
tiennm99 7ced30c8c5 feat(gold): add gold commands to Telegram menu 2026-06-11 22:28:24 +07:00
tiennm99 26c5db575b fix(deploy): include gold module in CI ModulesCSV override
CI's --parameter-overrides replaces samconfig.toml values, so every
push reverted the stack to the pre-gold module list.
2026-06-11 22:21:19 +07:00
tiennm99 0e0e3b91fa feat(gold): add multi-provider gold price fallback chain
Replace goldprice.org (403s datacenter IPs) with a provider chain:
gold-api.com -> Swissquote public quotes -> NBP daily fixing (PLN/gram
converted via the shared FX table). Per-provider failures are logged;
a full-chain outage surfaces as a retryable error instead of the
silent no-price reply.
2026-06-11 22:11:30 +07:00
tiennm99 2097a18cb2 fix(deploy): enable gold module in ModulesCSV 2026-06-11 21:57:05 +07:00