docs(selfhost): add Coolify deploy + AWS decommission guides; retire AWS docs

Add comprehensive Coolify deployment guide with MongoDB Atlas + networking setup. Add AWS decommission runbook for safe teardown. Update AWS guides to note deprecation. Update main README with self-hosted quickstart and architecture overview.
2026-08-24 02:25:37 +00:00 · 2026-06-28 09:58:37 +07:00
parent b0b4b4b4c0
commit 15343f6fa7
5 changed files with 302 additions and 21 deletions
@@ -1,6 +1,6 @@
 # miti99bot

-Plug-n-play Telegram bot framework in Go. Runs on AWS Lambda + DynamoDB + EventBridge Scheduler. Strictly free-tier.
+Plug-n-play Telegram bot framework in Go. Self-hosted on Coolify + MongoDB Atlas via long polling and an in-process cron scheduler. (Previously ran on AWS Lambda + DynamoDB + EventBridge — now retired; see [`docs/deploy-aws.md`](docs/deploy-aws.md) for history.)

 ## Modules

@@ -22,36 +22,48 @@ Disable any module by editing `MODULES` in `template.yaml`.
 ## Layout

 ```
-cmd/server/             entrypoint
-internal/server/        HTTP routes (/, /webhook, /cron/{name})
-internal/telegram/      Telegram webhook + bot wrapper
-internal/modules/       Module framework, registry, dispatchers, modules
-internal/storage/       KVStore interface; memory + dynamodb providers
-internal/ai/            Gemini client (used by twentyq)
-template.yaml           AWS SAM IaC (Lambda + Function URL + DynamoDB + Logs + Budget)
-docs/deploy-aws-free-tier-guide.md   Full onboarding guide
-docs/deploy-aws.md                   Steady-state operations
-aws/README.md                        One-time AWS account setup
+cmd/server/                  entrypoint (long polling + in-process cron + HTTP health)
+cmd/migrate-dynamo-to-mongo/ one-off DynamoDB → MongoDB Atlas data migrator
+internal/server/             HTTP routes (/ health, /cron/{name} manual trigger)
+internal/telegram/           Telegram long-polling bot wrapper
+internal/cron/               in-process cron scheduler (replaces EventBridge)
+internal/modules/            Module framework, registry, dispatchers, modules
+internal/storage/            KVProvider interface; memory + firestore + dynamodb + mongodb
+internal/ai/                 Gemini client (used by twentyq)
+docker-compose.yml           Coolify self-host stack (single bot service)
+docs/deploy-coolify-selfhosted.md    Self-host onboarding + cutover runbook
+docs/aws-decommission-runbook.md     AWS teardown (post-cutover)
+template.yaml, aws/          Retired AWS SAM IaC + setup (kept for history)
 ```

 ## Run locally

-In-memory KV (no AWS required):
+In-memory KV (no database required):

 ```sh
 TELEGRAM_BOT_TOKEN=… \
-TELEGRAM_WEBHOOK_SECRET=local \
-PORT=8080 \
 MODULES= \
 go run ./cmd/server
 ```

-End-to-end smoke test against a Telegram dev bot needs `ngrok` (local) or a deployed Function URL. The dev bot is created manually; token injected via env vars only.
+The bot uses long polling, so a local run talks to Telegram directly — no `ngrok`, no public URL. Ensure the bot's webhook is unset (the server clears it on startup) or `getUpdates` 409s. The dev bot is created manually; token injected via env vars only.

-For DynamoDB integration tests:
+Persistent MongoDB locally (auto-selected when `MONGO_URL` is set):
+
+```sh
+make mongo-local
+TELEGRAM_BOT_TOKEN=… \
+MONGO_URL=mongodb://127.0.0.1:27017 \
+MONGO_DATABASE=miti99bot_dev \
+go run ./cmd/server
+```
+
+For storage integration tests (each skips when its emulator env var is unset):
 ```sh
 make dynamodb-local      # docker run amazon/dynamodb-local on :8001
-make test-dynamodb       # runs internal/storage tests against DDB Local
+make test-dynamodb       # internal/storage tests against DynamoDB Local
+make mongo-local         # docker run mongo:7 on :27017
+make test-mongo          # internal/storage tests against local MongoDB
 ```

 ## Test
@@ -64,11 +76,9 @@ make test-dynamodb    # storage tests against DynamoDB Local (requires Docker)

 ## Deploy

-First-time onboarding: see [`docs/deploy-aws-free-tier-guide.md`](docs/deploy-aws-free-tier-guide.md).
+**Self-host (current):** [`docs/deploy-coolify-selfhosted.md`](docs/deploy-coolify-selfhosted.md) — Coolify + MongoDB Atlas (free M0), long polling (no public ingress), in-process cron. Storage auto-selects `mongodb` when `MONGO_URL` is set; the cron scheduler runs by default. Migrate existing data with [`cmd/migrate-dynamo-to-mongo`](cmd/migrate-dynamo-to-mongo/README.md), then tear down AWS via [`docs/aws-decommission-runbook.md`](docs/aws-decommission-runbook.md).

-Steady-state operations: [`docs/deploy-aws.md`](docs/deploy-aws.md).
-
-After the initial setup, every push to `main` triggers `.github/workflows/deploy.yml` (GitHub Actions OIDC → SAM deploy). No long-lived AWS keys.
+**AWS (retired):** [`docs/deploy-aws.md`](docs/deploy-aws.md) — kept for history.

 ## License

@@ -0,0 +1,105 @@
+# AWS Decommission Runbook
+
+Delete **everything** `miti99bot` ever deployed to AWS, after the migration +
+cutover to Coolify is verified. Run by the operator with the `admin` profile.
+
+> **Precondition (hard):** run ONLY after the Phase 4 cutover —
+> `make migrate-verify` green, bot confirmed live on Coolify via long polling,
+> and the EventBridge schedule already disabled at cutover. `sam delete`
+> destroys the DynamoDB table, which is the sole copy of prod data until
+> migrated. Never run this standalone.
+
+Account `225603493174`, region `ap-southeast-1` (verified live 2026-06-27).
+
+## What `sam delete` removes (CloudFormation-managed)
+
+DynamoDB table `miti99bot-data`, the Lambda + Function URL + invoke
+permissions, the Lambda execution role and `SchedulerExecutionRole`, the
+`/aws/lambda/miti99bot` log group + metric filter, the `miti99bot-cron-dlq`
+SQS queue, the `miti99bot-lolschedule-daily-push` schedule, and the
+`miti99bot-monthly` budget (if `AlertEmail` was set).
+
+## What it does NOT remove (created manually, outside CloudFormation)
+
+These linger — and the SSM secrets keep your bot token / Gemini key in the
+cloud — unless deleted separately:
+
+- **SSM SecureStrings** (exactly 4): `/miti99bot/prod/telegram-bot-token`,
+  `/miti99bot/prod/telegram-webhook-secret`, `/miti99bot/prod/gemini-api-key`,
+  `/miti99bot/prod/cron-shared-secret`. **Deleting these is the security step.**
+- **IAM role** `github-deploy-miti99bot` + inline policy `miti99bot-deploy`.
+- **IAM OIDC provider** `token.actions.githubusercontent.com` — verified the
+  account's only OIDC provider and used solely by miti99bot → safe to delete.
+- **SAM deploy bucket** `aws-sam-cli-managed-default-samclisourcebucket-ctwpsmoxnwvm`
+  + bootstrap stack `aws-sam-cli-managed-default` — verified miti99bot is the
+  sole SAM project → safe to delete.
+
+## Runbook
+
+```sh
+AWS_PROFILE=admin; REGION=ap-southeast-1; ACCT=225603493174
+
+# 1. Safety check — stack still exists (about to be deleted).
+aws --profile $AWS_PROFILE cloudformation describe-stacks --stack-name miti99bot \
+  --query "Stacks[0].StackStatus"
+
+# 2. Delete the CloudFormation stack.
+aws --profile $AWS_PROFILE sam delete --stack-name miti99bot --region $REGION --no-prompts
+aws --profile $AWS_PROFILE cloudformation wait stack-delete-complete --stack-name miti99bot
+
+# 3. Delete SSM secrets (NOT CFN-managed). List first, then delete.
+aws --profile $AWS_PROFILE ssm get-parameters-by-path --path /miti99bot --recursive \
+  --query "Parameters[].Name" --output text
+for P in telegram-bot-token telegram-webhook-secret gemini-api-key cron-shared-secret; do
+  aws --profile $AWS_PROFILE ssm delete-parameter --name /miti99bot/prod/$P
+done
+#   delete any extra /miti99bot/* the list revealed
+
+# 4. Delete the GitHub deploy IAM role (inline policy first).
+aws --profile $AWS_PROFILE iam delete-role-policy \
+  --role-name github-deploy-miti99bot --policy-name miti99bot-deploy
+aws --profile $AWS_PROFILE iam delete-role --role-name github-deploy-miti99bot
+
+# 5. OIDC provider — re-confirm it's the only one, then delete.
+aws --profile $AWS_PROFILE iam list-open-id-connect-providers
+aws --profile $AWS_PROFILE iam delete-open-id-connect-provider \
+  --open-id-connect-provider-arn arn:aws:iam::$ACCT:oidc-provider/token.actions.githubusercontent.com
+
+# 6. SAM deploy bucket + bootstrap stack — re-confirm only miti99bot +
+#    aws-sam-cli-managed-default stacks exist first.
+aws --profile $AWS_PROFILE cloudformation list-stacks \
+  --query "StackSummaries[?StackStatus!='DELETE_COMPLETE'].StackName" --output text
+aws --profile $AWS_PROFILE s3 rb \
+  s3://aws-sam-cli-managed-default-samclisourcebucket-ctwpsmoxnwvm --force
+aws --profile $AWS_PROFILE cloudformation delete-stack --stack-name aws-sam-cli-managed-default
+
+# 7. Confirm nothing tagged app=miti99bot remains.
+aws --profile $AWS_PROFILE resourcegroupstaggingapi get-resources \
+  --tag-filters Key=app,Values=miti99bot --region $REGION
+aws --profile $AWS_PROFILE cloudformation list-stacks \
+  --query "StackSummaries[?contains(StackName,'miti99bot')].[StackName,StackStatus]" --output table
+```
+
+Then in the repo: the `.github/workflows/deploy.yml` AWS deploy is disabled on
+the `feature/selfhosted` branch (the trigger is removed so a `main` push can't
+recreate the stack).
+
+## Verification checklist
+
+- [ ] `describe-stacks --stack-name miti99bot` → does not exist.
+- [ ] No `/miti99bot/*` SSM parameters remain (secrets purged).
+- [ ] `github-deploy-miti99bot` role gone; OIDC provider gone.
+- [ ] SAM bucket + bootstrap stack deleted.
+- [ ] `resourcegroupstaggingapi` for `app=miti99bot` returns empty.
+- [ ] `deploy.yml` no longer recreates the stack on `main`.
+- [ ] Cost Explorer shows $0 the following billing period.
+- [x] Cloudflare verified clean (2026-06-27): legacy KV/D1 already gone; the 4
+      remaining Workers are separate active projects. No action.
+- [ ] GCP: no `miti99bot-prod` project (`gcloud projects list | grep miti99bot`).
+
+## Notes
+
+- **Secret hygiene (optional):** rotate the Telegram bot token + Gemini key
+  after teardown — they lived in SSM/CloudWatch under accepted trade-offs.
+- **Keep in git:** the `aws/` dir + `template.yaml` history cost nothing and are
+  useful if AWS is ever revisited.
@@ -1,5 +1,10 @@
 # Deploy miti99bot to AWS (Free Tier)

+> **RETIRED.** `miti99bot` is now self-hosted on Coolify + MongoDB Atlas — see
+> [`deploy-coolify-selfhosted.md`](./deploy-coolify-selfhosted.md). The AWS stack
+> is decommissioned ([`aws-decommission-runbook.md`](./aws-decommission-runbook.md)).
+> Kept for historical reference / if AWS is ever revisited.
+
 End-to-end onboarding guide for deploying `miti99bot` on AWS. Everything below stays inside the AWS free tier in region `ap-southeast-1` (Singapore).

 Related docs:
@@ -1,5 +1,10 @@
 # Deploy: AWS (Lambda + DynamoDB + EventBridge)

+> **RETIRED.** `miti99bot` is now self-hosted on Coolify + MongoDB Atlas — see
+> [`deploy-coolify-selfhosted.md`](./deploy-coolify-selfhosted.md). The AWS stack
+> is decommissioned ([`aws-decommission-runbook.md`](./aws-decommission-runbook.md)).
+> Kept for historical reference / if AWS is ever revisited.
+
 This is the production deploy path for `miti99bot`. Strict free-tier targets, region `ap-southeast-1`.

 > **First-time setup:** see `aws/README.md`. This doc is for steady-state operations.
@@ -0,0 +1,156 @@
+# Deploy: Self-host (Coolify + MongoDB Atlas)
+
+Run `miti99bot` as a long-lived container on [Coolify](https://coolify.io) with
+[MongoDB Atlas](https://www.mongodb.com/atlas) (free M0) for storage. This
+replaces the AWS Lambda + DynamoDB + EventBridge path.
+
+## Architecture
+
+```
+   Telegram  <── long poll (getUpdates) ──  container  (outbound only)
+   in-process scheduler ───────────────────> module crons
+   MongoDB Atlas (db / one collection per module)
+   Coolify env vars (plain secrets)
+   NO public ingress (polling = outbound only; no domain, no /webhook, no TLS in)
+```
+
+Same Go binary (`cmd/server`) and module framework as AWS. Three things differ,
+all selected automatically from env:
+
+- **Storage** — `mongodb` auto-selected when `MONGO_URL` is set (no `KV_PROVIDER`).
+- **Cron** — an in-process scheduler (`internal/cron`) runs unconditionally and
+  fires each module cron on its `Schedule` (UTC). No EventBridge.
+- **Transport** — long polling (`b.Start`) is the **only** transport. The bot
+  opens an outbound connection to Telegram and pulls updates, so there is no
+  public domain, no `/webhook`, and no webhook secret. The container clears any
+  leftover webhook on startup (`deleteWebhook`) before polling.
+
+## Required environment
+
+Copy [`.env.example`](../.env.example) → `.env` (gitignored) and fill in.
+
+| Var | Required | Notes |
+|---|---|---|
+| `TELEGRAM_BOT_TOKEN` | ✅ | from @BotFather |
+| `MONGO_URL` | ✅ | Atlas SRV string **incl. credentials** — secret, never logged |
+| `MONGO_DATABASE` | ✅ | e.g. `miti99bot` |
+| `MODULES` | optional | CSV; empty = all modules |
+| `OWNER_ID` | optional | owner-only commands (renamed from `BOT_OWNER_ID`) |
+| `ADMIN_IDS` | optional | CSV of admin ids (renamed from `ADMIN_USER_IDS`) |
+| `GEMINI_API_KEY` | optional | only the `twentyq` module needs it |
+
+**Leave UNSET on self-host:** all six `*_PARAMETER_NAME` vars (they force an
+SSM/AWS lookup that fails with no AWS creds and bricks startup), `KV_PROVIDER`,
+`PORT`, `TELEGRAM_WEBHOOK_SECRET`, `CRON_SHARED_SECRET`, `GOLD_VNAPP_API_KEY`,
+and the `STOCK/COIN/GOLD *_API_URL` overrides (modules use coded defaults).
+
+## 1. MongoDB Atlas (M0)
+
+1. Create a free **M0** cluster (512 MB — ample for the tiny paper-trading KV).
+2. **Database user (least privilege):** create a user with role
+   **`readWrite` on the single app database only** (e.g. `miti99bot`) — never
+   Atlas admin or cluster-wide. Use a **strong unique password**.
+3. **Network access:** add `0.0.0.0/0`.
+
+   > **Accepted trade-off (validated decision).** The Coolify host has no stable
+   > egress IP, so the Atlas IP allow-list is open to the internet. This widens
+   > the surface beyond DynamoDB's IAM-gated posture (where the DB was never
+   > internet-reachable). It is knowingly accepted for self-host. The mandatory
+   > compensating controls are: (1) strong unique password, (2) least-privilege
+   > `readWrite`-on-one-db user, (3) the connection string is a secret and is
+   > never logged (the bot logs only the database name on startup).
+
+4. Copy the `mongodb+srv://…` connection string into `MONGO_URL` and put the
+   db name in `MONGO_DATABASE`.
+
+## 2. Coolify
+
+1. New resource → from this Git repo (Docker Compose), or a prebuilt image.
+   The committed [`docker-compose.yml`](../docker-compose.yml) defines the single
+   `bot` service.
+2. Set the env vars above in Coolify.
+3. **No public domain / port** is needed — polling is outbound-only. Do not
+   publish a port or attach a domain. `expose: 8080` keeps the health endpoint
+   reachable only inside Coolify's network.
+4. **Exactly one replica.** Telegram permits only one `getUpdates` consumer per
+   bot token; a second poller gets HTTP 409, and a second in-process scheduler
+   double-fires crons. Prefer **stop-first redeploys** so two containers never
+   overlap near a cron time.
+5. **Build arg for deploynotify:** pass `GIT_SHA` (Coolify exposes the commit
+   SHA) so the owner gets the "new version" DM. Without it, `deploynotify`
+   stays silent (no crash) — but you lose that notification.
+6. **Health check:** use Coolify's HTTP monitor against `GET /` (returns
+   `text/plain` `miti99bot ok`). Do **not** use a compose `healthcheck` — the
+   distroless image has no shell/curl and `cmd/server` has no `-healthcheck`
+   flag. Note: `/` reports healthy even if Mongo is unreachable (the driver
+   auto-reconnects on the next op); a DB outage will not auto-restart the
+   container — accepted trade-off.
+
+## 3. Register the command menu
+
+Long polling needs no webhook registration — only the command menu:
+
+```sh
+TELEGRAM_BOT_TOKEN=… make telegram-commands-selfhost
+```
+
+## Cutover runbook
+
+Zero-loss switch from the live AWS Lambda to the Coolify poller. Coordinate
+users to pause activity during the brief window.
+
+1. **Deploy the Coolify container but keep it stopped/scaled-to-0.** A running
+   poller would 409 against the live Lambda webhook and its scheduler would
+   overlap EventBridge. Use a fresh, empty Atlas DB.
+2. **Disable/delete the EventBridge schedule** `miti99bot-lolschedule-daily-push`
+   (it invokes the Lambda directly, independent of transport). The in-process
+   scheduler's per-UTC-date idempotency guard is the backup.
+3. **`deleteWebhook` (mandatory):**
+   ```sh
+   TELEGRAM_BOT_TOKEN=… make telegram-deletewebhook-selfhost
+   ```
+   This buffers incoming updates (Telegram retains ~24h) so nothing is lost, and
+   releases the webhook so the poller won't 409. After this the Lambda stops
+   receiving updates.
+4. **Migrate + verify** (read-only on DynamoDB — see
+   [`cmd/migrate-dynamo-to-mongo`](../cmd/migrate-dynamo-to-mongo/README.md)):
+   ```sh
+   export MONGO_URL=… MONGO_DATABASE=… AWS_PROFILE=miti99bot-migrate
+   make migrate-dynamo-to-mongo DRY_RUN=1   # review counts
+   make migrate-dynamo-to-mongo             # real run
+   make migrate-verify                      # counts must match, exit 0
+   ```
+   Keep this window short (target minutes).
+5. **Start the Coolify container** (1 replica). Its scheduler runs by default
+   (safe now that EventBridge is off). On startup it `deleteWebhook`s again
+   (idempotent) and begins polling, draining Telegram's buffered queue.
+6. **Confirm:**
+   ```sh
+   TELEGRAM_BOT_TOKEN=… make telegram-webhook-info-selfhost
+   ```
+   `url` should be empty and `pending_update_count` should drain toward 0.
+   Smoke `/ping`, `/stats`, and a coin/stock balance command — migrated state
+   should be visible from Atlas.
+7. **Tear down AWS** once verified — see
+   [`aws-decommission-runbook.md`](./aws-decommission-runbook.md).
+
+### Rollback
+
+The only clean revert is **before `sam delete`**: stop the poller, then
+re-`setWebhook` to the Lambda Function URL (the still-deployed Lambda runs its
+own webhook code until teardown). Lossless **only until the first post-cutover
+Mongo write** — after that, MongoDB/Coolify is the sole system of record. This
+short-window RPO was an explicitly accepted decision; no reverse migrator
+exists.
+
+## Local smoke test
+
+```sh
+cp .env.example .env   # fill TELEGRAM_BOT_TOKEN, MONGO_URL, MONGO_DATABASE
+docker compose up --build
+```
+
+Boot logs should show `storage backend backend=mongodb database=…` (no
+connection string), `cron scheduler started`, and `telegram long polling
+started`. `curl localhost:8080/` returns `miti99bot ok`. The bot's webhook must
+be unset (the container clears it on startup) or `getUpdates` 409s.