Commit Graph
20 Commits
Author SHA1 Message Date
tiennm99 bdb8757dbe feat: rename lol schedule module 2026-07-01 10:38:49 +07:00
tiennm99 6eeb7ef447 feat(stats): use queryable Mongo usage records 2026-07-01 10:22:03 +07:00
tiennm99 6e35c222d3 refactor(deploy): remove retired aws support 2026-06-29 00:11:42 +07:00
tiennm99 3e8d6bf2a7 refactor(storage): replace KVStore with generic typed DocStore[T]
Delete the byte-oriented KVStore/VersionedStore abstraction and the
DynamoDB/memory KV backends. Add a generic typed store (DocStore[T] with
Provider/Collection/Typed) persisting each value as a flattened native
Mongo document (storedDoc[T] via bson inline) — no value envelope.

- MongoDB is the only runtime backend; memory kept for tests/local.
- All modules + deploynotify use typed stores; persisted structs carry
  bson tags == json names (incl. nested lolschedule/wordle types).
- lolschedule wraps its array/scalar values in named structs.
- migrate-dynamo-to-mongo writes the flattened shape via Typed[bson.M]
  with wrap rules; Scan/--dry-run/--verify retained.

Verified: go vet/build clean; full go test green hermetically and
in-container vs real Mongo 7 + DynamoDB Local (storage integration +
migrator e2e).
2026-06-28 18:02:11 +07:00
tiennm99 e75d9bc378 feat(storage): store mongo values as native BSON documents
Implement mongodb_value_codec to encode portfolio/gold/stock values as native
BSON documents instead of JSON-serialized byte strings. Reduces storage size,
improves query efficiency, and eliminates unmarshal overhead. Native BSON
encoding preserves int64 fidelity and enables future native-format queries.
Add NativeValueRepresentation and Int64Fidelity tests validating lossless
encoding. Update migration tool comments. Document native-value behavior
in deploy guide. Update phase-01 plan with implementation notes.
2026-06-28 12:52:48 +07:00
tiennm99 f383533b07 refactor(storage): remove legacy firestore backend
Delete Firestore provider implementation and tests (5 files). Extract shared
key helpers to new internal/storage/keys.go module. Remove Firestore case
from server bootstrap and storage factory. Drop cloud.google.com/go/firestore
and related dependencies from go.mod. Remove firestore-emulator test targets
from Makefile. Update README storage section to reflect MongoDB-only backend.
2026-06-28 12:52:40 +07:00
tiennm99 79e438546f refactor(storage): replace value-bytes CAS with version-field optimistic lock
Replace content-addressed storage versioning (CAS) with explicit version fields
on documents. Versions are managed via optimistic locking pattern: Get returns
version, Put increments atomically and fails if version mismatch detected.
Callers (portfolio, lolschedule) updated to use GetVersioned/PutVersioned,
simplifying concurrency handling and eliminating CAS overhead.
2026-06-28 12:52:32 +07:00
tiennm99 2090bfc911 refactor(storage): store mongo value as JSON string, not BinData
Persist the KV value field as a BSON string so it is directly readable in
the Atlas/Compass UI, matching DynamoDB's String storage. Reads still accept
BSON binary as a backward-compat fallback. Non-UTF-8 callers must encode
upstream, same constraint as DynamoDB; all current callers write JSON.
2026-06-28 10:50:41 +07:00
tiennm99 a4c6d2f95c feat(storage): add MongoDB KVProvider backend
Add MongoDB-backed storage provider with connection pooling, key-value operations
(get, set, delete), and comprehensive test coverage. Integrates mongo-driver
dependency for self-hosted deployments.
2026-06-28 09:58:07 +07:00
tiennm99 3cad105a81 fix(storage): report missing CAS support as unsupported instead of conflict
- prefix.go: return errors.ErrUnsupported when inner store lacks CompareAndSwap, enabling fail-fast instead of infinite retries
- Adds comprehensive CAS semantics test coverage for all KV backends:
  * memory_kv_test.go: new tests for basic operations and CAS failures
  * prefix_test.go: tests for wrapped CAS errors and unsupported operations
  * dynamodb_kv_test.go, firestore_kv_test.go: CAS failure scenarios
- portfolio_test.go: test retry exhaustion, business-error short-circuit, concurrent updates, and fail-fast on unsupported CAS
2026-06-11 21:40:16 +07:00
tiennm99 0a795b1436 feat(gold): add compare-and-swap portfolio updates with multi-backend support
Implement atomic UpdatePortfolio with retry pattern for concurrent-safe updates.
Add CAS support to DynamoDB, Firestore, Memory, and prefix stores.
2026-06-11 21:12:58 +07:00
tiennm99 d67517e1fb feat(migration): cf→aws migration toolchain + DynamoDB value as String
Operator-run tooling that moves durable Cloudflare KV data into the live
AWS DynamoDB table, plus a runtime swap of the `value` attribute from
Binary to String so payloads are human-readable in the AWS console.

- cmd/migrate_cf_data: subcommands inventory, kv-import (idempotent via
  attribute_not_exists), trading-audit-dump, convert-value-to-string
- internal/migration: policy allowlist, CF KV+D1 REST clients, DynamoDB
  writer, report formatter with per-prefix counts + tests
- internal/storage/dynamodb_kv.go: Put writes MemberS, Get reads MemberS;
  dropped empty-bytes sentinel (DynamoDB allows empty strings)
2026-05-16 10:33:35 +07:00
tiennm99 9c9bc65db8 fix(ci): clear sam lint + golangci-lint findings
- template.yaml: drop redundant DependsOn (W3005) — Ref in LoggingConfig
  already implies BotFunctionLogGroup ordering.
- dynamodb_kv.go: remove unused errIsTableMissing and orphaned errors import.
- trading/prices.go: wrap deferred resp.Body.Close to satisfy errcheck.
2026-05-14 09:57:12 +07:00
tiennm99 f3b9891a54 refactor: rename module to miti99bot, canonicalize AWS deploy path
Rename:
- Go module github.com/tiennm99/miti99bot-go → github.com/tiennm99/miti99bot
- CloudFormation stack miti99bot-aws-port → miti99bot
- Drop "port", "Cloud Run", "GCP", "cutover", "Phase NN" framing from
  active code and docs — project reads as canonical AWS-Lambda from now on.

AWS deploy guide + flow fix:
- New docs/deploy-aws-free-tier-guide.md — Ubuntu 24.04 ARM64 onboarding
  with project-local venv (pip awscli + sam-cli), SSM secrets via read -s,
  idempotent OIDC provider + role creation, $1 budget alarm.
- Drop sam build from the pipeline — provided.al2023 + makefile builder
  expects a Makefile in CodeUri (build/lambda/, the output dir), so the
  step always fails. sam deploy --template-file template.yaml now reads
  the raw template and zips build/lambda/ directly.
- Rollback section rewritten — use continue-update-rollback /
  cancel-update-stack / git-SHA redeploy. Drop the broken
  --use-previous-template recipe.
- DynamoDB free-tier row corrected (on-demand is 2.5M read / 1M write
  request units, not 25 RCU/WCU).

Updated:
- README.md fully rewritten (drops port/legacy framing, lists modules,
  points new users at the free-tier guide).
- aws/README.md retitled "AWS account setup", phase numbers stripped.
- Makefile / .github/workflows/deploy.yml — sam deploy flow.
- samconfig.toml — stack_name = "miti99bot".
- Go comments — Cloud Run → Lambda, Cloud Scheduler → EventBridge
  Scheduler, Cloud Logging → CloudWatch Logs.
- Struct field GCPProject → FirestoreProject (env GOOGLE_CLOUD_PROJECT
  unchanged).

Plus advisory reports under plans/reports/ from the code-reviewer +
researcher passes that informed the fixes.

Verified: go vet ./..., go build ./..., go test ./... all green.
2026-05-13 22:05:38 +07:00
tiennm99 f632c68956 refactor(modules): drop loldle variants, semantle, doantu and dead framework surface
Removes six modules (loldle-ability/emoji/quote/splash, semantle, doantu)
and prunes the framework deps that were only there to serve them:

  - ai.Embedder + Client.Embed + embeddingModel const (semantle only)
  - Deps.Embedder + BuildOptions.Embedder
  - Deps.Env + Build(env) param + ModuleEnv config field + PHOW2SIM allowlist (doantu only)
  - internal/champname package (loldle now owns its lookup helpers directly)
  - template.yaml: Phow2simAPIURL parameter + PHOW2SIM_API_URL Lambda env

Active catalog: util, misc, wordle, loldle, lolschedule, twentyq, trading.

go build / vet / test all pass.
2026-05-11 16:02:06 +07:00
tiennm99 cd8c0df84b chore: gofmt dynamodb_kv.go 2026-05-10 03:06:27 +07:00
tiennm99 91b27783cc feat(storage): add DynamoDB KV provider sibling to Firestore
- Implement DynamoDB client wrapper and KV provider
- Full CRUD test coverage for both local and AWS modes
- AWS SDK v2 dependency added to go.mod/go.sum
2026-05-10 02:29:39 +07:00
tiennm99 84f660d9d9 chore(tooling): golangci-lint + govulncheck + defensive guards
Phase 6 of the 2026-05-09 review remediation plan. Bundle of small
hygiene fixes — none individually urgent but better folded together
than scattered across follow-ups.

- .golangci.yml: enable errcheck/govet/gosec/staticcheck/unused/
  ineffassign/gocyclo/misspell/revive. Tuned to the codebase style
  (no universal exported-doc requirement, gocyclo cap at 20 to
  accommodate handler dispatch). 0 issues across the tree.
- ci.yml: add golangci-lint job + govulncheck (informational).
- Defensive guards:
  - registry.go: Module.Name mismatch now errors at Build instead of
    silently overwriting (TestBuild_RejectsFactoryNameMismatch).
  - cmd/server/main.go: PORT env validated numerically + 0..65535.
  - firestore_provider.go: For() re-validates module name; invalid
    names return an invalidStore whose every op errors with
    ErrInvalidModuleName.
- Dead code removal:
  - wordle: gameTTLSeconds const + pickDaily/hashDJB2/todayUTC
    helpers + their tests deleted (pickDaily was unused;
    daily.go renamed pick_random.go).
- Dependency: golang.org/x/net v0.52.0 -> v0.54.0 (resolves
  GO-2026-4918 HTTP/2 infinite-loop CVE).
- Deferred from the original phase plan: Docker digest pinning
  (Dependabot handles), per-handler file splits (largest file 279 LOC;
  splits would churn for marginal gain).

go test -race -count=1 ./... clean (15 packages); golangci-lint run
clean (0 issues).
2026-05-09 16:33:21 +07:00
tiennm99 28a9676690 feat(storage): Firestore KVStore + KVProvider abstraction
Phase 04 of go-port-cloud-run plan. Introduces KVProvider abstraction
with memory backend (via Prefixed wrapper) and Firestore backend (via
collection-per-module isolation). Backend selection gated by env vars:
GOOGLE_CLOUD_PROJECT or FIRESTORE_EMULATOR_HOST → Firestore, else memory.
Emulator-gated tests via `make test-emulator`. Security hardened: emulator
fallback project ID, prefix validation on List, length-in-bytes docs.
2026-05-08 23:51:24 +07:00
tiennm99 25a5f37d3d feat(server,modules): bootstrap server and module framework
Implements Phases 02 (partial) and 03 of the go-port-cloud-run plan.
Introduces module framework with per-module KV prefix isolation,
health check endpoint, request timeout protection, and comprehensive
test coverage. Cloud Run deployment deferred to Phase 01.

Security hardening: constant-time secret comparison, cron auth bridge,
and secrets stripped from dependency environment exports. Includes
Dockerfile, GitHub CI workflow (vet + race + build), and integration
tests for module lifecycle.
2026-05-08 23:27:12 +07:00