From 9f50237a3c2c33d1de6e1ba68ad178fa09b0bfa8 Mon Sep 17 00:00:00 2001 From: tiennm99 Date: Mon, 11 May 2026 15:59:50 +0700 Subject: [PATCH] docs: purge remaining Upstash references and sync to DO storage Drop /admin/migrate-from-upstash from public API docs, replace deprecated UPSTASH_* env vars with ALLOWED_ORIGINS/ENVIRONMENT, update Worker conventions to reflect DO bindings (env.CANVAS_ROOM) instead of Redis client patterns, and rewrite system-architecture security/data-flow sections to cover cookie+IP identity, transactionSync atomicity, WS Origin allowlist, per-identity WS cap, and broadcast sequence numbers. --- .env.example | 7 +++-- README.md | 31 +++++++++++--------- docs/code-standards.md | 7 +++-- docs/deployment-guide.md | 46 +++++------------------------ docs/system-architecture.md | 58 +++++++++++++++++++------------------ 5 files changed, 63 insertions(+), 86 deletions(-) diff --git a/.env.example b/.env.example index 2dfe98d..4fcda91 100644 --- a/.env.example +++ b/.env.example @@ -1,2 +1,5 @@ -UPSTASH_REDIS_REST_URL= -UPSTASH_REDIS_REST_TOKEN= +# No external secrets required. +# Canvas + cooldown state live inside the CanvasRoom Durable Object (SQLite). +# Configurable Worker vars (set via wrangler.json or `wrangler secret put`): +# ALLOWED_ORIGINS - comma-separated WebSocket origin allowlist (empty = allow all) +# ENVIRONMENT - "production" enables fail-closed identity resolution diff --git a/README.md b/README.md index 4dc0579..a297654 100644 --- a/README.md +++ b/README.md @@ -81,22 +81,28 @@ npm run deploy # Builds frontend + deploys worker to Cloudflare ``` src/ ├── worker.js # Hono entry — thin proxy + edge validation -├── admin/ -│ └── migrate-from-upstash.js # One-shot Upstash → DO importer (token-gated) ├── durable-objects/ │ ├── canvas-room.js # DO: storage + cooldown + WS hub │ └── lib/ │ ├── schema.js # Idempotent CREATE TABLE -│ ├── chunk-storage.js # BLOB chunk read/write/import +│ ├── chunk-storage.js # BLOB chunk read/write │ └── cooldown-store.js # Rate-limit acquire + lazy GC ├── lib/ │ ├── constants.js # CANVAS_WIDTH/HEIGHT, CHUNK_BYTES, palette │ ├── canvas-decoder.js # Raw bytes → RGBA (client-side) -│ ├── canvas-storage.js # Legacy Upstash reader (used by migration only) -│ ├── redis-client.js # Legacy Upstash REST helpers (migration only) -│ ├── rate-limiter.js # Legacy Upstash cooldown (orphaned, awaits removal) +│ ├── cookie.js # parseCookie + formatSetCookie +│ ├── get-user-id.js # Cookie+IP rate-limit identity +│ ├── pixel-buffer.js # Pending-stroke buffer (undo/redo) │ ├── image-uploader.js # Browser-side batched uploader -│ └── get-user-id.js # IP-based identity +│ ├── image-pipeline.js # Image-to-canvas processing +│ ├── image-pipeline-client.js # Client-side queue +│ ├── image-pipeline-worker.js # Web Worker handler +│ ├── image-job-storage.js # IndexedDB job persistence +│ ├── image-resize.js # Resampling +│ ├── image-transform.js # Rotation / flip +│ ├── image-to-palette.js # Palette quantization +│ ├── image-color-correction.js # Brightness / contrast / saturation +│ └── dither-kernels.js # Dithering algorithms ├── client/ │ ├── main.js # Svelte mount │ ├── App.svelte # Root + WebSocket @@ -106,6 +112,7 @@ src/ │ ├── ColorPicker.svelte # Favorites + 256-color grid + custom picker │ ├── CanvasControls.svelte # Zoom buttons + coordinates │ ├── DrawToolbar.svelte # Paint / submit / undo / redo +│ ├── HelpOverlay.svelte # Keyboard shortcut help │ └── ImageImporter.svelte # Image-to-canvas uploader └── index.html # Vite entry ``` @@ -140,15 +147,11 @@ Place pixels on the canvas. WebSocket for real-time pixel updates. Messages are JSON: ```json -{ "type": "pixels", "pixels": [{ "x": 100, "y": 200, "color": 27 }] } +{ "type": "pixels", "seq": 42, "pixels": [{ "x": 100, "y": 200, "color": 27 }] } ``` -### `POST /admin/migrate-from-upstash` (transitional) - -Token-gated one-shot endpoint that pulls the canvas from a legacy Upstash -Redis instance and imports it into the Durable Object. Slated for removal -after the production migration completes (Phase 4 of -[`plans/260509-2309-canvas-on-do-storage`](plans/260509-2309-canvas-on-do-storage)). +`seq` is a monotonic broadcast counter; the client uses it to detect missed +frames and refetch the canvas to resync. ## Configuration diff --git a/docs/code-standards.md b/docs/code-standards.md index ffd5ecb..142b792 100644 --- a/docs/code-standards.md +++ b/docs/code-standards.md @@ -26,10 +26,11 @@ src/ ### Worker (src/worker.js, src/lib/*) -- Functions receive `env` parameter for Cloudflare bindings (Redis credentials, DO bindings) +- Functions receive `env` parameter for Cloudflare bindings (DO bindings, vars) - No global state — Workers are stateless between requests -- Use `@upstash/redis/cloudflare` (REST-based, not TCP) -- Bitfield operations use builder pattern: `redis.bitfield(key).set().exec()` +- Durable Object access via `env.CANVAS_ROOM.get(env.CANVAS_ROOM.idFromName('main'))` +- Identity flows through `resolveIdentity(request, env)` (cookie-first, IP fallback) +- Edge handlers validate input before forwarding to the DO; the DO re-validates at the trust boundary ### Client (src/client/*) diff --git a/docs/deployment-guide.md b/docs/deployment-guide.md index 36ecd9f..fa357c7 100644 --- a/docs/deployment-guide.md +++ b/docs/deployment-guide.md @@ -31,39 +31,6 @@ The first deploy applies the `wrangler.json` migration that registers 5. `curl -I https://your-url/api/canvas` should report `cf-cache-status: HIT` after a couple of warm requests (10 s edge cache). -## (Optional) One-Shot Migration from Upstash - -Only if you have an existing Upstash-backed deployment to import. - -```bash -# 1. Set credentials for the legacy Upstash instance -npx wrangler secret put UPSTASH_REDIS_REST_URL -npx wrangler secret put UPSTASH_REDIS_REST_TOKEN - -# 2. Generate and set a migration token -npx wrangler secret put MIGRATION_TOKEN -# Paste a random 32-byte hex value - -# 3. Deploy -npm run deploy - -# 4. Run the import once -curl -X POST -H "Authorization: Bearer $TOKEN" \ - https://your-worker.workers.dev/admin/migrate-from-upstash -# Expect: {"ok":true,"bytes_imported":16777216,"samples_checked":N,"mismatches":[]} - -# 5. Verify in browser; wait 7 days as rollback safety -# 6. Run Phase 4 cleanup (see plans/260509-2309-canvas-on-do-storage) -``` - -After Phase 4 cleanup deletes the migration code, also delete the -secrets: -```bash -npx wrangler secret delete UPSTASH_REDIS_REST_URL -npx wrangler secret delete UPSTASH_REDIS_REST_TOKEN -npx wrangler secret delete MIGRATION_TOKEN -``` - ## Custom Domain ```bash @@ -97,18 +64,19 @@ sessions stay well under 100K/day. ## Troubleshooting - **Canvas loads empty**: expected on first deploy — DO `canvas_chunks` - table is empty until pixels are placed (or migration runs). + table is empty until pixels are placed. - **WebSocket not connecting**: verify the wrangler migration applied via `wrangler tail` — should see no errors on DO instantiation. - **`cf-cache-status` shows MISS**: edge caching may need an extra `caches.default.put` wrap if `Cache-Control` headers aren't honored through the worker → DO → response chain. Verify with two consecutive `curl -I` requests; second should HIT. -- **Migration import fails with `size_mismatch`**: the legacy Upstash - data isn't 16 MB. Resize CHUNK constants or delete the partial Upstash - data and start fresh. -- **`already_populated` from migration endpoint**: pass `?force=1` to - overwrite. Use only when you're certain. +- **`forbidden_origin` on `/api/ws`**: production sets `ALLOWED_ORIGINS` + in `wrangler.json` `vars`. Add the requesting origin or empty the var + for dev. Empty allowlist accepts all origins. +- **`no_identity` 500 on `/api/place` or `/api/canvas`**: production + fails closed when neither the `rplace_id` cookie nor `cf-connecting-ip` + is present. Indicates a proxy misconfiguration in front of the Worker. - **Storage billing meter ticking up**: per-account 5 GB free cap. A 16 MB canvas is harmless; the worry only appears if you stand up many rooms or hit a runaway insert. diff --git a/docs/system-architecture.md b/docs/system-architecture.md index 678fbff..04badac 100644 --- a/docs/system-architecture.md +++ b/docs/system-architecture.md @@ -30,20 +30,24 @@ Cloudflare Worker (Hono — thin proxy) ``` 1. User draws on canvas → optimistic render into a pending buffer. 2. User hits Submit → POST /api/place { pixels: [{x, y, color}, ...] }. -3. Worker validates input (bounds, types, batch ≤ 2048, body ≤ 128 KB). -4. Worker resolves userId from CF-Connecting-IP and forwards to DO /place. +3. Worker validates input (bounds, types, batch ≤ 2048, Content-Length). +4. Worker resolves identity (cookie or IP fallback) and forwards to DO /place. 5. DO atomically: a. cooldowns.tryAcquire(userId, 1s) — UPDATE expired or INSERT new row. On conflict (active claim), responds 429. - b. chunk_storage.writePixels — group pixels by chunk_id, read each - touched chunk's BLOB, modify in memory, INSERT OR REPLACE. - c. Broadcast `{type:'pixels', pixels}` to all hibernating WebSockets. + b. state.storage.transactionSync(() => chunk_storage.writePixels(...)) + — group pixels by chunk_id, read each touched chunk's BLOB, modify + in memory, INSERT OR REPLACE. Transaction makes a multi-chunk batch + all-or-nothing. + c. On write failure, cooldown is refunded so transient errors don't + soft-DOS the user. + d. Broadcast `{type:'pixels', seq, pixels}` to all hibernating WebSockets. 6. Response: { ok: true } (or { error, retryAfter } on rate limit). ``` -The DO is single-threaded; the entire 5a-c sequence runs without -preemption, so cooldown check + write + broadcast are effectively atomic -without explicit transactions. +The DO is single-threaded; the cooldown + write + broadcast sequence runs +without preemption. The transactionSync wrapper provides atomicity across +multiple BLOB chunks. ### Canvas Loading @@ -91,7 +95,7 @@ without explicit transactions. | Column | Type | Notes | |---|---|---| -| `user_id` | TEXT PRIMARY KEY | Hashed CF-Connecting-IP | +| `user_id` | TEXT PRIMARY KEY | `cookie:` or `ip:` | | `expires_at` | INTEGER NOT NULL | ms epoch; row becomes stale past this point | - `idx_cooldowns_expires` keeps lazy GC sweeps cheap. @@ -112,17 +116,6 @@ UPDATE cooldowns SET expires_at = ? WHERE user_id = ? AND expires_at <= ? Hardness comes from the DO single-threaded model; no SQL-level locking needed. -## Migration Endpoint (transitional) - -`POST /admin/migrate-from-upstash` (token-gated): one-shot importer that -pulls the legacy Upstash canvas via `lib/canvas-storage.js` (4 chunked -GETRANGE calls) and posts the raw 16 MB bytes to DO `/import`. The DO -splits into CHUNK_COUNT BLOB rows in a single sync transaction, then -the worker round-trips a sample-byte verification. - -Removed in Phase 4 of `plans/260509-2309-canvas-on-do-storage` along with -`@upstash/redis` and `ioredis` dependencies. - ## Free-tier Footprint (CF, 2026) | Resource | Quota | rplace usage at hobby scale | Headroom | @@ -140,16 +133,25 @@ the worker handler with the Cache API to enforce caching. ## Security -- **Rate limiting**: race-safe at the actor (single-threaded DO). -- **Identity**: CF-Connecting-IP hashed to userId — unspoofable. +- **Rate limiting**: race-safe at the actor (single-threaded DO); writes + wrapped in `state.storage.transactionSync` so a multi-chunk batch is + all-or-nothing. +- **Identity**: opaque `rplace_id` cookie (HttpOnly, Secure, SameSite=Lax, + 1y) issued on first `/api/canvas`. Falls back to a SHA-256 prefix of + `cf-connecting-ip` when no cookie is present. In production a request + with neither returns 500 `no_identity`. +- **Cooldown refund**: failed writes refund the cooldown row so a + transient storage error doesn't soft-DOS the user for 1 s. - **Input validation**: strict bounds + type checks at the worker edge, - re-validated at the DO trust boundary. + re-validated at the DO trust boundary. `Content-Length` required on + POST `/api/place` (411 if missing/zero, 413 if above cap). - **Body cap**: ~128 KB per `/api/place` (2048 pixels × ~64 B JSON each). -- **Migration endpoint**: Bearer-token gated. Token-comparison is direct - string equality — adequate at hobby scale; consider constant-time - comparison if exposure profile changes. -- **DO isolation**: `/canvas`, `/place`, `/import`, `/ws` are intra-DO - paths only — not internet-reachable except via the worker. +- **WS Origin allowlist**: `ALLOWED_ORIGINS` env var (comma-separated) + rejects upgrades from disallowed origins; empty = allow all (dev). +- **Per-identity WS cap**: 5 concurrent sockets per identity prevents + broadcast amplification; further upgrades return 429. +- **DO isolation**: `/canvas`, `/place`, `/ws` are intra-DO paths only — + not internet-reachable except via the worker. ## Operational Notes