From 15343f6fa71cbc9a6f86ac3d35bd0797ff7f777a Mon Sep 17 00:00:00 2001 From: tiennm99 Date: Sun, 28 Jun 2026 09:58:37 +0700 Subject: [PATCH] docs(selfhost): add Coolify deploy + AWS decommission guides; retire AWS docs Add comprehensive Coolify deployment guide with MongoDB Atlas + networking setup. Add AWS decommission runbook for safe teardown. Update AWS guides to note deprecation. Update main README with self-hosted quickstart and architecture overview. --- README.md | 52 ++++++---- docs/aws-decommission-runbook.md | 105 +++++++++++++++++++ docs/deploy-aws-free-tier-guide.md | 5 + docs/deploy-aws.md | 5 + docs/deploy-coolify-selfhosted.md | 156 +++++++++++++++++++++++++++++ 5 files changed, 302 insertions(+), 21 deletions(-) create mode 100644 docs/aws-decommission-runbook.md create mode 100644 docs/deploy-coolify-selfhosted.md diff --git a/README.md b/README.md index 82b5101..ebdfc9c 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # miti99bot -Plug-n-play Telegram bot framework in Go. Runs on AWS Lambda + DynamoDB + EventBridge Scheduler. Strictly free-tier. +Plug-n-play Telegram bot framework in Go. Self-hosted on Coolify + MongoDB Atlas via long polling and an in-process cron scheduler. (Previously ran on AWS Lambda + DynamoDB + EventBridge — now retired; see [`docs/deploy-aws.md`](docs/deploy-aws.md) for history.) ## Modules @@ -22,36 +22,48 @@ Disable any module by editing `MODULES` in `template.yaml`. ## Layout ``` -cmd/server/ entrypoint -internal/server/ HTTP routes (/, /webhook, /cron/{name}) -internal/telegram/ Telegram webhook + bot wrapper -internal/modules/ Module framework, registry, dispatchers, modules -internal/storage/ KVStore interface; memory + dynamodb providers -internal/ai/ Gemini client (used by twentyq) -template.yaml AWS SAM IaC (Lambda + Function URL + DynamoDB + Logs + Budget) -docs/deploy-aws-free-tier-guide.md Full onboarding guide -docs/deploy-aws.md Steady-state operations -aws/README.md One-time AWS account setup +cmd/server/ entrypoint (long polling + in-process cron + HTTP health) +cmd/migrate-dynamo-to-mongo/ one-off DynamoDB → MongoDB Atlas data migrator +internal/server/ HTTP routes (/ health, /cron/{name} manual trigger) +internal/telegram/ Telegram long-polling bot wrapper +internal/cron/ in-process cron scheduler (replaces EventBridge) +internal/modules/ Module framework, registry, dispatchers, modules +internal/storage/ KVProvider interface; memory + firestore + dynamodb + mongodb +internal/ai/ Gemini client (used by twentyq) +docker-compose.yml Coolify self-host stack (single bot service) +docs/deploy-coolify-selfhosted.md Self-host onboarding + cutover runbook +docs/aws-decommission-runbook.md AWS teardown (post-cutover) +template.yaml, aws/ Retired AWS SAM IaC + setup (kept for history) ``` ## Run locally -In-memory KV (no AWS required): +In-memory KV (no database required): ```sh TELEGRAM_BOT_TOKEN=… \ -TELEGRAM_WEBHOOK_SECRET=local \ -PORT=8080 \ MODULES= \ go run ./cmd/server ``` -End-to-end smoke test against a Telegram dev bot needs `ngrok` (local) or a deployed Function URL. The dev bot is created manually; token injected via env vars only. +The bot uses long polling, so a local run talks to Telegram directly — no `ngrok`, no public URL. Ensure the bot's webhook is unset (the server clears it on startup) or `getUpdates` 409s. The dev bot is created manually; token injected via env vars only. -For DynamoDB integration tests: +Persistent MongoDB locally (auto-selected when `MONGO_URL` is set): + +```sh +make mongo-local +TELEGRAM_BOT_TOKEN=… \ +MONGO_URL=mongodb://127.0.0.1:27017 \ +MONGO_DATABASE=miti99bot_dev \ +go run ./cmd/server +``` + +For storage integration tests (each skips when its emulator env var is unset): ```sh make dynamodb-local # docker run amazon/dynamodb-local on :8001 -make test-dynamodb # runs internal/storage tests against DDB Local +make test-dynamodb # internal/storage tests against DynamoDB Local +make mongo-local # docker run mongo:7 on :27017 +make test-mongo # internal/storage tests against local MongoDB ``` ## Test @@ -64,11 +76,9 @@ make test-dynamodb # storage tests against DynamoDB Local (requires Docker) ## Deploy -First-time onboarding: see [`docs/deploy-aws-free-tier-guide.md`](docs/deploy-aws-free-tier-guide.md). +**Self-host (current):** [`docs/deploy-coolify-selfhosted.md`](docs/deploy-coolify-selfhosted.md) — Coolify + MongoDB Atlas (free M0), long polling (no public ingress), in-process cron. Storage auto-selects `mongodb` when `MONGO_URL` is set; the cron scheduler runs by default. Migrate existing data with [`cmd/migrate-dynamo-to-mongo`](cmd/migrate-dynamo-to-mongo/README.md), then tear down AWS via [`docs/aws-decommission-runbook.md`](docs/aws-decommission-runbook.md). -Steady-state operations: [`docs/deploy-aws.md`](docs/deploy-aws.md). - -After the initial setup, every push to `main` triggers `.github/workflows/deploy.yml` (GitHub Actions OIDC → SAM deploy). No long-lived AWS keys. +**AWS (retired):** [`docs/deploy-aws.md`](docs/deploy-aws.md) — kept for history. ## License diff --git a/docs/aws-decommission-runbook.md b/docs/aws-decommission-runbook.md new file mode 100644 index 0000000..3050216 --- /dev/null +++ b/docs/aws-decommission-runbook.md @@ -0,0 +1,105 @@ +# AWS Decommission Runbook + +Delete **everything** `miti99bot` ever deployed to AWS, after the migration + +cutover to Coolify is verified. Run by the operator with the `admin` profile. + +> **Precondition (hard):** run ONLY after the Phase 4 cutover — +> `make migrate-verify` green, bot confirmed live on Coolify via long polling, +> and the EventBridge schedule already disabled at cutover. `sam delete` +> destroys the DynamoDB table, which is the sole copy of prod data until +> migrated. Never run this standalone. + +Account `225603493174`, region `ap-southeast-1` (verified live 2026-06-27). + +## What `sam delete` removes (CloudFormation-managed) + +DynamoDB table `miti99bot-data`, the Lambda + Function URL + invoke +permissions, the Lambda execution role and `SchedulerExecutionRole`, the +`/aws/lambda/miti99bot` log group + metric filter, the `miti99bot-cron-dlq` +SQS queue, the `miti99bot-lolschedule-daily-push` schedule, and the +`miti99bot-monthly` budget (if `AlertEmail` was set). + +## What it does NOT remove (created manually, outside CloudFormation) + +These linger — and the SSM secrets keep your bot token / Gemini key in the +cloud — unless deleted separately: + +- **SSM SecureStrings** (exactly 4): `/miti99bot/prod/telegram-bot-token`, + `/miti99bot/prod/telegram-webhook-secret`, `/miti99bot/prod/gemini-api-key`, + `/miti99bot/prod/cron-shared-secret`. **Deleting these is the security step.** +- **IAM role** `github-deploy-miti99bot` + inline policy `miti99bot-deploy`. +- **IAM OIDC provider** `token.actions.githubusercontent.com` — verified the + account's only OIDC provider and used solely by miti99bot → safe to delete. +- **SAM deploy bucket** `aws-sam-cli-managed-default-samclisourcebucket-ctwpsmoxnwvm` + + bootstrap stack `aws-sam-cli-managed-default` — verified miti99bot is the + sole SAM project → safe to delete. + +## Runbook + +```sh +AWS_PROFILE=admin; REGION=ap-southeast-1; ACCT=225603493174 + +# 1. Safety check — stack still exists (about to be deleted). +aws --profile $AWS_PROFILE cloudformation describe-stacks --stack-name miti99bot \ + --query "Stacks[0].StackStatus" + +# 2. Delete the CloudFormation stack. +aws --profile $AWS_PROFILE sam delete --stack-name miti99bot --region $REGION --no-prompts +aws --profile $AWS_PROFILE cloudformation wait stack-delete-complete --stack-name miti99bot + +# 3. Delete SSM secrets (NOT CFN-managed). List first, then delete. +aws --profile $AWS_PROFILE ssm get-parameters-by-path --path /miti99bot --recursive \ + --query "Parameters[].Name" --output text +for P in telegram-bot-token telegram-webhook-secret gemini-api-key cron-shared-secret; do + aws --profile $AWS_PROFILE ssm delete-parameter --name /miti99bot/prod/$P +done +# delete any extra /miti99bot/* the list revealed + +# 4. Delete the GitHub deploy IAM role (inline policy first). +aws --profile $AWS_PROFILE iam delete-role-policy \ + --role-name github-deploy-miti99bot --policy-name miti99bot-deploy +aws --profile $AWS_PROFILE iam delete-role --role-name github-deploy-miti99bot + +# 5. OIDC provider — re-confirm it's the only one, then delete. +aws --profile $AWS_PROFILE iam list-open-id-connect-providers +aws --profile $AWS_PROFILE iam delete-open-id-connect-provider \ + --open-id-connect-provider-arn arn:aws:iam::$ACCT:oidc-provider/token.actions.githubusercontent.com + +# 6. SAM deploy bucket + bootstrap stack — re-confirm only miti99bot + +# aws-sam-cli-managed-default stacks exist first. +aws --profile $AWS_PROFILE cloudformation list-stacks \ + --query "StackSummaries[?StackStatus!='DELETE_COMPLETE'].StackName" --output text +aws --profile $AWS_PROFILE s3 rb \ + s3://aws-sam-cli-managed-default-samclisourcebucket-ctwpsmoxnwvm --force +aws --profile $AWS_PROFILE cloudformation delete-stack --stack-name aws-sam-cli-managed-default + +# 7. Confirm nothing tagged app=miti99bot remains. +aws --profile $AWS_PROFILE resourcegroupstaggingapi get-resources \ + --tag-filters Key=app,Values=miti99bot --region $REGION +aws --profile $AWS_PROFILE cloudformation list-stacks \ + --query "StackSummaries[?contains(StackName,'miti99bot')].[StackName,StackStatus]" --output table +``` + +Then in the repo: the `.github/workflows/deploy.yml` AWS deploy is disabled on +the `feature/selfhosted` branch (the trigger is removed so a `main` push can't +recreate the stack). + +## Verification checklist + +- [ ] `describe-stacks --stack-name miti99bot` → does not exist. +- [ ] No `/miti99bot/*` SSM parameters remain (secrets purged). +- [ ] `github-deploy-miti99bot` role gone; OIDC provider gone. +- [ ] SAM bucket + bootstrap stack deleted. +- [ ] `resourcegroupstaggingapi` for `app=miti99bot` returns empty. +- [ ] `deploy.yml` no longer recreates the stack on `main`. +- [ ] Cost Explorer shows $0 the following billing period. +- [x] Cloudflare verified clean (2026-06-27): legacy KV/D1 already gone; the 4 + remaining Workers are separate active projects. No action. +- [ ] GCP: no `miti99bot-prod` project (`gcloud projects list | grep miti99bot`). + +## Notes + +- **Secret hygiene (optional):** rotate the Telegram bot token + Gemini key + after teardown — they lived in SSM/CloudWatch under accepted trade-offs. +- **Keep in git:** the `aws/` dir + `template.yaml` history cost nothing and are + useful if AWS is ever revisited. diff --git a/docs/deploy-aws-free-tier-guide.md b/docs/deploy-aws-free-tier-guide.md index 3c66d29..1854592 100644 --- a/docs/deploy-aws-free-tier-guide.md +++ b/docs/deploy-aws-free-tier-guide.md @@ -1,5 +1,10 @@ # Deploy miti99bot to AWS (Free Tier) +> **RETIRED.** `miti99bot` is now self-hosted on Coolify + MongoDB Atlas — see +> [`deploy-coolify-selfhosted.md`](./deploy-coolify-selfhosted.md). The AWS stack +> is decommissioned ([`aws-decommission-runbook.md`](./aws-decommission-runbook.md)). +> Kept for historical reference / if AWS is ever revisited. + End-to-end onboarding guide for deploying `miti99bot` on AWS. Everything below stays inside the AWS free tier in region `ap-southeast-1` (Singapore). Related docs: diff --git a/docs/deploy-aws.md b/docs/deploy-aws.md index e66f7f8..01faccc 100644 --- a/docs/deploy-aws.md +++ b/docs/deploy-aws.md @@ -1,5 +1,10 @@ # Deploy: AWS (Lambda + DynamoDB + EventBridge) +> **RETIRED.** `miti99bot` is now self-hosted on Coolify + MongoDB Atlas — see +> [`deploy-coolify-selfhosted.md`](./deploy-coolify-selfhosted.md). The AWS stack +> is decommissioned ([`aws-decommission-runbook.md`](./aws-decommission-runbook.md)). +> Kept for historical reference / if AWS is ever revisited. + This is the production deploy path for `miti99bot`. Strict free-tier targets, region `ap-southeast-1`. > **First-time setup:** see `aws/README.md`. This doc is for steady-state operations. diff --git a/docs/deploy-coolify-selfhosted.md b/docs/deploy-coolify-selfhosted.md new file mode 100644 index 0000000..fb50828 --- /dev/null +++ b/docs/deploy-coolify-selfhosted.md @@ -0,0 +1,156 @@ +# Deploy: Self-host (Coolify + MongoDB Atlas) + +Run `miti99bot` as a long-lived container on [Coolify](https://coolify.io) with +[MongoDB Atlas](https://www.mongodb.com/atlas) (free M0) for storage. This +replaces the AWS Lambda + DynamoDB + EventBridge path. + +## Architecture + +``` + Telegram <── long poll (getUpdates) ── container (outbound only) + in-process scheduler ───────────────────> module crons + MongoDB Atlas (db / one collection per module) + Coolify env vars (plain secrets) + NO public ingress (polling = outbound only; no domain, no /webhook, no TLS in) +``` + +Same Go binary (`cmd/server`) and module framework as AWS. Three things differ, +all selected automatically from env: + +- **Storage** — `mongodb` auto-selected when `MONGO_URL` is set (no `KV_PROVIDER`). +- **Cron** — an in-process scheduler (`internal/cron`) runs unconditionally and + fires each module cron on its `Schedule` (UTC). No EventBridge. +- **Transport** — long polling (`b.Start`) is the **only** transport. The bot + opens an outbound connection to Telegram and pulls updates, so there is no + public domain, no `/webhook`, and no webhook secret. The container clears any + leftover webhook on startup (`deleteWebhook`) before polling. + +## Required environment + +Copy [`.env.example`](../.env.example) → `.env` (gitignored) and fill in. + +| Var | Required | Notes | +|---|---|---| +| `TELEGRAM_BOT_TOKEN` | ✅ | from @BotFather | +| `MONGO_URL` | ✅ | Atlas SRV string **incl. credentials** — secret, never logged | +| `MONGO_DATABASE` | ✅ | e.g. `miti99bot` | +| `MODULES` | optional | CSV; empty = all modules | +| `OWNER_ID` | optional | owner-only commands (renamed from `BOT_OWNER_ID`) | +| `ADMIN_IDS` | optional | CSV of admin ids (renamed from `ADMIN_USER_IDS`) | +| `GEMINI_API_KEY` | optional | only the `twentyq` module needs it | + +**Leave UNSET on self-host:** all six `*_PARAMETER_NAME` vars (they force an +SSM/AWS lookup that fails with no AWS creds and bricks startup), `KV_PROVIDER`, +`PORT`, `TELEGRAM_WEBHOOK_SECRET`, `CRON_SHARED_SECRET`, `GOLD_VNAPP_API_KEY`, +and the `STOCK/COIN/GOLD *_API_URL` overrides (modules use coded defaults). + +## 1. MongoDB Atlas (M0) + +1. Create a free **M0** cluster (512 MB — ample for the tiny paper-trading KV). +2. **Database user (least privilege):** create a user with role + **`readWrite` on the single app database only** (e.g. `miti99bot`) — never + Atlas admin or cluster-wide. Use a **strong unique password**. +3. **Network access:** add `0.0.0.0/0`. + + > **Accepted trade-off (validated decision).** The Coolify host has no stable + > egress IP, so the Atlas IP allow-list is open to the internet. This widens + > the surface beyond DynamoDB's IAM-gated posture (where the DB was never + > internet-reachable). It is knowingly accepted for self-host. The mandatory + > compensating controls are: (1) strong unique password, (2) least-privilege + > `readWrite`-on-one-db user, (3) the connection string is a secret and is + > never logged (the bot logs only the database name on startup). + +4. Copy the `mongodb+srv://…` connection string into `MONGO_URL` and put the + db name in `MONGO_DATABASE`. + +## 2. Coolify + +1. New resource → from this Git repo (Docker Compose), or a prebuilt image. + The committed [`docker-compose.yml`](../docker-compose.yml) defines the single + `bot` service. +2. Set the env vars above in Coolify. +3. **No public domain / port** is needed — polling is outbound-only. Do not + publish a port or attach a domain. `expose: 8080` keeps the health endpoint + reachable only inside Coolify's network. +4. **Exactly one replica.** Telegram permits only one `getUpdates` consumer per + bot token; a second poller gets HTTP 409, and a second in-process scheduler + double-fires crons. Prefer **stop-first redeploys** so two containers never + overlap near a cron time. +5. **Build arg for deploynotify:** pass `GIT_SHA` (Coolify exposes the commit + SHA) so the owner gets the "new version" DM. Without it, `deploynotify` + stays silent (no crash) — but you lose that notification. +6. **Health check:** use Coolify's HTTP monitor against `GET /` (returns + `text/plain` `miti99bot ok`). Do **not** use a compose `healthcheck` — the + distroless image has no shell/curl and `cmd/server` has no `-healthcheck` + flag. Note: `/` reports healthy even if Mongo is unreachable (the driver + auto-reconnects on the next op); a DB outage will not auto-restart the + container — accepted trade-off. + +## 3. Register the command menu + +Long polling needs no webhook registration — only the command menu: + +```sh +TELEGRAM_BOT_TOKEN=… make telegram-commands-selfhost +``` + +## Cutover runbook + +Zero-loss switch from the live AWS Lambda to the Coolify poller. Coordinate +users to pause activity during the brief window. + +1. **Deploy the Coolify container but keep it stopped/scaled-to-0.** A running + poller would 409 against the live Lambda webhook and its scheduler would + overlap EventBridge. Use a fresh, empty Atlas DB. +2. **Disable/delete the EventBridge schedule** `miti99bot-lolschedule-daily-push` + (it invokes the Lambda directly, independent of transport). The in-process + scheduler's per-UTC-date idempotency guard is the backup. +3. **`deleteWebhook` (mandatory):** + ```sh + TELEGRAM_BOT_TOKEN=… make telegram-deletewebhook-selfhost + ``` + This buffers incoming updates (Telegram retains ~24h) so nothing is lost, and + releases the webhook so the poller won't 409. After this the Lambda stops + receiving updates. +4. **Migrate + verify** (read-only on DynamoDB — see + [`cmd/migrate-dynamo-to-mongo`](../cmd/migrate-dynamo-to-mongo/README.md)): + ```sh + export MONGO_URL=… MONGO_DATABASE=… AWS_PROFILE=miti99bot-migrate + make migrate-dynamo-to-mongo DRY_RUN=1 # review counts + make migrate-dynamo-to-mongo # real run + make migrate-verify # counts must match, exit 0 + ``` + Keep this window short (target minutes). +5. **Start the Coolify container** (1 replica). Its scheduler runs by default + (safe now that EventBridge is off). On startup it `deleteWebhook`s again + (idempotent) and begins polling, draining Telegram's buffered queue. +6. **Confirm:** + ```sh + TELEGRAM_BOT_TOKEN=… make telegram-webhook-info-selfhost + ``` + `url` should be empty and `pending_update_count` should drain toward 0. + Smoke `/ping`, `/stats`, and a coin/stock balance command — migrated state + should be visible from Atlas. +7. **Tear down AWS** once verified — see + [`aws-decommission-runbook.md`](./aws-decommission-runbook.md). + +### Rollback + +The only clean revert is **before `sam delete`**: stop the poller, then +re-`setWebhook` to the Lambda Function URL (the still-deployed Lambda runs its +own webhook code until teardown). Lossless **only until the first post-cutover +Mongo write** — after that, MongoDB/Coolify is the sole system of record. This +short-window RPO was an explicitly accepted decision; no reverse migrator +exists. + +## Local smoke test + +```sh +cp .env.example .env # fill TELEGRAM_BOT_TOKEN, MONGO_URL, MONGO_DATABASE +docker compose up --build +``` + +Boot logs should show `storage backend backend=mongodb database=…` (no +connection string), `cron scheduler started`, and `telegram long polling +started`. `curl localhost:8080/` returns `miti99bot ok`. The bot's webhook must +be unset (the container clears it on startup) or `getUpdates` 409s.