Both plans shipped to production:
- 260509-2309-canvas-on-do-storage (Upstash → DO migration, deployed
2026-05-10, Upstash fully removed in a977adc).
- 260510-0232-fix-do-migration-followups (post-migration P1 fixes,
docs cleanup, full DO test coverage — all completed 2026-05-11).
Git history retains the full content if needed; plans/reports/ kept
as-is for the original research/review artifacts.
Drop /admin/migrate-from-upstash from public API docs, replace deprecated
UPSTASH_* env vars with ALLOWED_ORIGINS/ENVIRONMENT, update Worker
conventions to reflect DO bindings (env.CANVAS_ROOM) instead of Redis
client patterns, and rewrite system-architecture security/data-flow
sections to cover cookie+IP identity, transactionSync atomicity, WS
Origin allowlist, per-identity WS cap, and broadcast sequence numbers.
Server:
- Origin allowlist on /api/ws (env.ALLOWED_ORIGINS, comma-separated; empty
= allow all for dev/preview)
- per-identity WS connection cap (MAX_WS_PER_IDENTITY = 5) using
acceptWebSocket(socket, [identity]) tagging; 6th upgrade returns 429
- ws.send 'ping' triggers a {type:'pong'} reply so dead connections fire
onclose promptly instead of waiting on TCP keepalive
Client:
- buffer WS pixels arriving during the initial canvas fetch and replay
them after committedColors is replaced; previously the post-fetch
Uint8Array assignment silently overwrote any pixels broadcast in the
fetch window (the documented C2 race)
- 30s ping / 60s pong watchdog closes the socket if pong stops arriving,
routing through the existing exponential-backoff reconnect path
Tests:
- four /api/ws cases: missing upgrade, disallowed origin, allowed origin,
empty allowlist (dev default). Sentinel uses status 200 because undici
rejects 101 in Node-side Response constructors.
Plan: plans/260510-0232-fix-do-migration-followups/phase-03-ws-hardening-client-race.md
- resolveIdentity prefers an opaque rplace_id cookie; falls back to a
cf-connecting-ip hash; in production a request with neither now returns
500 no_identity instead of bucketing all such traffic together
- /api/canvas issues Set-Cookie when no cookie is present so subsequent
requests escape NAT-shared IP buckets (mobile/CGNAT users)
- DO maintains an in-memory monotonic broadcast counter; broadcast frames
carry { seq } so the client can detect missed pixels and refetch
- client tracks lastSeq, refetches on gap, resets on every (re)connect
NAT/CGNAT users previously shared a single 1Hz bucket per egress IP. With
cookie identity they each get their own bucket. Cookie is HttpOnly, Secure,
SameSite=Lax, 1y Max-Age. Stripped/cleared cookies fall through to IP.
The seq counter resets on DO hibernation rehydrate; client always refetches
on reconnect, so a reset is indistinguishable from a fresh connect.
Plan: plans/260510-0232-fix-do-migration-followups/phase-02-cookie-ip-identity.md
- wrap writePixels in state.storage.transactionSync so a partial multi-chunk
failure doesn't leave the canvas half-written or out of sync with the WS
broadcast
- size new chunk buffer against chunkSize(chunkId) instead of the persisted
blob's length so writes after a canvas-grow no longer silently drop OOB
bytes in the formerly-last short chunk
- refund the cooldown row when writePixels throws so transient storage
errors stop soft-DOSing the user (and halving image-uploader throughput)
- bound readAllChunks by chunk_id < CHUNK_COUNT and trim oversized blobs so
orphan rows from a future shrink no longer crash GET /api/canvas
- require a positive Content-Length on /api/place (411) and reject above the
pre-parse cap (413); previously a missing or zero header bypassed the cap
- drop String(err) from the 500 response body
- drain the INSERT cursor symmetrically with the UPDATE branch in tryAcquire
- assert CHUNK_BYTES <= 2 MB at module load (DO SQLite per-cell BLOB cap)
- correct the inverted webSocketClose comment and guard the re-close call
- add tests for missing / zero / oversized Content-Length
Plan: plans/260510-0232-fix-do-migration-followups/phase-01-do-storage-atomicity.md
Fixes from code review of canvas-on-do migration (commit c3f7c02):
- worker.js /api/ws: rewrite request URL to '/ws' so the DO pathname
switch dispatches correctly. The original c.req.raw kept '/api/ws'
which the DO never matched → 404 on every WS upgrade.
- migrate-from-upstash.js pickSampleOffsets: use TOTAL_PIXELS - 1 for the
last byte instead of CANVAS_WIDTH * CANVAS_WIDTH (only correct when
the canvas is square; constants explicitly invite non-square).
- chunk-storage.js writePixels: clarify atomicity comment — the loop is
atomic *because it has no awaits*, not because of any implicit DO
transaction. Added guidance for future maintainers.
- cooldown-store.js tryAcquire: GC sweep wrapped in try/catch so a
transient failure can't drop the user's allowed: true response.
Docs:
- README.md: drop Upstash from tech stack, redraw architecture,
document new project layout (durable-objects/lib, admin/), add
CHUNK_BYTES to configuration table.
- docs/system-architecture.md: full rewrite for DO-storage data flow,
document SQLite schema, race-safe rate-limit pattern, free-tier table.
- docs/deployment-guide.md: drop Upstash setup, add optional one-shot
migration runbook, update free-tier table to actual May 2026 limits.
Tests: 112 pass, 6 skipped (pending Phase 4 rewrite via
@cloudflare/vitest-pool-workers). Bundle dry-run clean.
Local wrangler dev smoke test was attempted but the sandboxed env
hangs HTTP requests at the workerd layer (TCP connects, no response).
Routing fix verified by code inspection; user must verify in their
own dev or production.
Earlier docs assumed a 1 GB per-DO storage cap. Actual May 2026 free-tier
limits per the CF DO docs:
- 10 GB per Durable Object
- 5 GB per account (the real Free-tier ceiling)
- 2 MB per BLOB row
- 1,000 req/sec soft cap per DO
Practical effect: canvas can grow to ~70,000 × 70,000 on Free (was previously
documented as ~32K × 32K). Existing 64 KB CHUNK_BYTES is well-suited:
clustered writes (single pixels + image patches) minimize read-modify-write
amplification; bigger chunks would only help uniform-random writes which
isn't rplace's workload.
No code change. Just doc accuracy.
Single GETRANGE of the 16 MiB canvas base64-encodes to ~22 MB, exceeding
Upstash's 10 MB per-request limit on the Free plan and causing 500s on
/api/canvas. Split into 4x4 MiB parallel ranges (~5.3 MB base64 each).
Replace Canvas2D render path with pixi.js v8. Canvas stays authored
as CPU RGBA on an OffscreenCanvas; pixi uses that as a texture source
and uploads on demand. Pan/zoom become pure GPU transforms on a
Container — no CPU work per frame. Overlay is a second sprite in the
same container so it tracks pan/zoom for free.
Key points:
- app.ticker.stop() + manual app.renderer.render() on rAF: idle frames
cost 0, active frames cost one GPU draw.
- mainTexture.source.update() called only when imageDataDirty (same
gating as before). Writes stay cheap.
- resolution: Math.min(devicePixelRatio, 2). iPhone 3x → 2x = 44%
fewer fragments; pixel art gains nothing from 3x.
- Public API preserved: applyUpdates, undo/redo, setOverlay, panBy,
gotoPoint, getCommittedColor, getPendingPixels, commitPending,
refetchCanvas, cancelStrokeIfAny all unchanged.
- Overlay sprite rebuilt on setOverlay() with destroy({texture,
textureSource}) to avoid GPU leaks.
Bundle: +200KB main, +~100KB pixi chunks, ~70KB gz total. Tradeoff
accepted for mobile perf parity with wplace.
Mobile lag root cause: every pan/zoom tick ran putImageData(imageData, 0, 0),
a ~67MB copy into the offscreen. On a phone, that dominates the frame budget.
- CanvasRenderer: split render() into a deferred scheduler + doRender().
requestAnimationFrame coalesces redundant calls within one frame. An
imageDataDirty flag gates putImageData — only re-upload when pixels
actually changed (setPixelRgba / loadCanvas). Pan+zoom renders are now
pure compositor work.
- Zoom clamp: minZoom = min(viewportW/CW, viewportH/CH). Computed in
App.svelte from a resize-tracked viewport, passed to CanvasRenderer
(wheel + pinch use it) and CanvasControls (zoom-out button disables at
bound). Replaces the hardcoded 0.25 that let phones see empty void.
- Pan clamp: clampPan() runs before every paint. When canvas >= viewport,
canvas always covers the screen. When canvas < viewport (never happens
at minZoom, but safe), canvas stays inside.
Ship four UX gaps identified from wplace comparison + direct review.
- Eyedropper: new 'eyedrop' mode button in DrawToolbar, sampled on
mouseUp-without-drag. Alt+click samples in any mode (Photoshop
convention). 'I' key toggles the mode; after sampling, mode snaps
back to 'paint'.
- HelpOverlay.svelte: modal listing all keybindings. Triggered by '?'
/ '/' keys, a '?' button in CanvasControls, or Esc to close.
- Connection indicator: WS readyState reflected as a colored dot in
CanvasControls (green=open, amber pulse=connecting/reconnecting,
red=closed). No more silent disconnects.
- First-visit hint: one-line banner explaining paint→submit flow and
core shortcuts. localStorage-gated (rplace:hint:v1), auto-dismisses
on first successful commit or explicit X.
- scripts/: remove upload-colors.js (broken — imported removed constants)
and image-to-colors.js (orphan — flow moved fully client-side to
ImageImporter). Drop sharp from devDependencies.
- constants.js: drop unused BITS_PER_PIXEL export.
- dither-kernels.js: make KERNEL_* consts internal; only the
ERROR_DIFFUSION_KERNELS map is consumed externally.
- image-pipeline.js: drop unused pipeline.source() accessor.
- image-to-palette.js: drop `options.dither` legacy boolean shim
(no external caller uses it) and its test.
- canvas-storage.js, App.svelte: drop comments that restated names.
Replaces the synchronous reactive pipeline with a debounced worker RPC.
Slider drags and file loads no longer block the main thread.
- image-pipeline.js: pure staged pipeline (transform → resize →
color-correction → quantize). Each upstream stage keeps one cached
slot keyed on the inputs it depends on, so a single-slider change
reruns only the downstream stages.
- image-pipeline-worker.js: hosts the pipeline inside a module Worker.
Accepts set-source (transferable) and run(params). Returns indices
+ preview RGBA as transferable ArrayBuffers.
- image-pipeline-client.js: main-thread RPC with request-id tracking
and stale-response dropping.
- ImageImporter.svelte: two-tier scheduler — throttled quick request
(<=384px) for the preview panel while inputs churn, debounced full
request (180ms) for overlay/buildPixels/opaque-count on settle.
Source buffer cloned and transferred to the worker on load/resume.
Main bundle shrinks ~4KB (pipeline code now in its own worker chunk).
Resolves browser freeze when importing large images.
- image-to-palette.js: add lazy 32^3 Uint8Array LUT (32 KB) for
nearest-color lookup. Replaces 256-iteration linear scan per pixel
in quantizeNearest, error-diffusion, and ordered-dither paths.
- ImageImporter.svelte: cap initial resizeW/resizeH to canvas bounds
so a full-res photo does not run the pipeline at source size before
the user can touch Fit.
Canvas:
- CANVAS_W/H = 4096, total 16,777,216 pixels
- BITS_PER_PIXEL = 8 (byte-aligned) — raw Redis bytes are palette indices
- Canvas-decoder becomes an identity wrap/copy
- Storage BITFIELD uses u8; offset = y*W + x
- Redis key versioned to rplace:canvas:v2 so old 32-color/2048^2 data is
orphaned (operators can DEL the old key to reclaim memory)
Palette:
- 256 entries, generated deterministically:
- 0..15 = 16-step grayscale ramp (pure black -> pure white)
- 16..255 = 240 HSL wheel (4 lightness rings x 60 hues @ 82% saturation)
- nearestPaletteIndex(r,g,b) helper for custom-color snapping
UI:
- ColorPicker: 16-swatch favorites strip (grays + 8 accents) + current-color
swatch + expand toggle for the full 16x16 grid + "Custom..." button that
opens the native <input type="color"> and snaps to nearest palette entry
- Default selected color bumped to index 0 (black)
Tests + docs:
- canvas-decoder tests rewritten for identity u8 decode
- canvas-storage tests updated for u8 offsets
- image-to-palette tests anchored to PALETTE_BLACK=0 / PALETTE_WHITE=15 and
COLORS_RGBA[i] probes (no more hardcoded old 32-color indices)
- integration test uses u8 BITFIELD and canvas-aware bounds
- README, system-architecture, deployment-guide updated (storage math,
migration note for orphaned old key)
* feat(ui): pick-position, cooldown badge, goto, shortcuts, progressive skip, resume
- Pick-position mode: "Pick on canvas" replaces the broken "Use cursor"
button. Next left-click (no drag) sets origin; Esc or second press cancels.
- Visible cooldown on Submit: disables button + shows "Wait Xs" countdown
after a successful place or 429 response.
- Goto-coordinates: "x,y" input in CanvasControls; Enter centers the
viewport on that canvas point.
- Keyboard shortcuts: Q/E zoom, WASD pan (40px), Esc cancels pick or stroke.
Guarded against firing in focused inputs.
- Progressive skip-already-matching: uploader re-filters remaining queue
every 8 batches via an optional shouldSkip predicate.
- Auto-save + resume: new image-job-storage module (localStorage + PNG
dataURL); uploader emits onCheckpoint after each batch; importer shows
a Resume/Discard banner when an unfinished job is found on mount.
* docs(reports): wplace-autobot logic + wplace.live design research
* refactor(rate-limit): switch to 1 req/sec cooldown, batch size up to 2048
Replace per-pixel credit/token-bucket model with a simple per-user cooldown
(SET NX EX 1). Batch size is now independent of the rate limit and capped
at MAX_BATCH_SIZE = 2048.
- rate-limiter: SET NX EX replaces Lua credit script
- worker: response shape { ok: true } (no credits field)
- client: drop credit state/timer/UserInfo; uploader paces by cooldown
- tests: mock checkRateLimit; integration test exercises SET NX EX
- docs: README, system-architecture, code-standards, deployment-guide
* chore(plans): remove implemented plan directories
rplace-implementation (base build), review-fixes, and
image-importer-enhancements are all shipped. Keep plans/reports/ as
historical code-review and research references.
Two quick quality-of-life options for logo and background-heavy imports:
- Skip-white: near-white pixels (r,g,b >= threshold, default 230) are
marked transparent in the output so they don't consume credits. Useful
when the source has a white background you don't want to paint.
- Paint-transparent: fully-transparent source pixels are treated as opaque
white before quantization. Useful if the canvas background is white and
you want empty regions of a logo to blend in.
Both toggles compose with every existing dither method; paint-transparent
runs first, so combining the two leaves originally-transparent pixels as
skip (since they're synthesized white, then the skip-white pass drops
them).
UI adds the two checkboxes and a threshold slider shown only when
skip-white is on. CLI gains --skip-white, --white-threshold,
--paint-transparent. 4 new unit tests pin the behavior including the
paint+skip interaction.
Brightness, contrast, saturation, and gamma adjustments before palette
quantization. Pure-function applyColorCorrection in src/lib keeps the CLI
and browser importer in sync. Inserted post-resize so slider moves stay
responsive on large sources; precision loss is negligible through a
32-color palette.
UI is a collapsible "Color correction" section with four sliders, live
values, a "Reset" button, and an "on" badge when any value differs from
default. CLI gains --brightness, --contrast, --saturation, --gamma
(all -100..+100 except gamma which is 0.1..3.0).
9 unit tests pin down identity, saturate-to-0-or-255 clamping, greyscale
at saturation=-100, gamma midtone direction, and alpha preservation.
Replace the single Floyd-Steinberg toggle with a method dropdown offering
none, floyd, atkinson, jarvis, burkes, sierra, sierra-lite (error diffusion)
and bayer-2 / bayer-4 / bayer-8 (ordered). Kernels live in
src/lib/dither-kernels.js as plain data; rgbaToPalette dispatches via a
shared runErrorDiffusion runner and a runOrderedDither for Bayer.
The legacy `dither: true` option keeps working (maps to `method: 'floyd'`),
so existing CLI invocations and tests stay green. CLI gains
`--dither-method <name>` which takes precedence over `--dither`.
13 unit tests cover exact-palette hits, transparent pixels, all-method
smoke, kernel weight sums (Atkinson=0.75, others=1.0), and verify Bayer
actually scatters on mid-grey where plain nearest would produce one color.
Pure-function transformRgba in src/lib/image-transform.js supports flip H,
flip V, and rotation in 90° steps (fixed application order: rotate → flipH
→ flipV). Wired into the importer pipeline before resize with buttons for
each op; ±90° rotations auto-swap the resize dims so output aspect tracks
the rotation. CLI gains --flip-h, --flip-v, --rotate flags using the same
shared module.
8 new unit tests cover identity, each flip, 90/180/270 rotation, the
flipH+flipV ≡ 180° identity, and invalid rotation rejection.
CanvasRenderer.setOverlay draws the palette-converted import preview on top
of committed + pending layers at (x, y) with a configurable alpha. Image
importer gains an "Overlay" toggle and opacity slider; the overlay
reactively follows resize/dither/origin changes and clears on panel close,
toggle-off, or image swap. Lets the user align the upload before spending
credits.
Add W/H inputs, aspect-ratio lock, resampling method dropdown (nearest /
bilinear / box), "Fit to canvas" (respects current origin) and "1:1" reset
buttons to the import panel. Pipeline now runs resize before palette
quantization so dither/skip-matching operate on the final dimensions.
Shared src/lib/image-resize.js is reused by scripts/image-to-colors.js,
which gains --width, --height, --method flags with aspect-preserve when
only one dimension is provided.
Six phases spanning resize, on-canvas overlay preview, transforms, more
dithering algorithms, color correction, and skip-white/paint-transparent
toggles. Based on WPlace-AutoBOT's image-processor.js feature set, scoped
to what fits our 32-color canvas.
New floating "Import Image" panel decodes PNG/JPG/WebP in the browser, shows
a palette-matched preview, and uploads pixels at a user-chosen origin with
pause/resume/cancel and live progress. Options:
- Skip-matching: compare against CanvasRenderer.getCommittedColor so pixels
already correct don't waste credits.
- Dither (Floyd-Steinberg): toggle between nearest-color and error-diffusion
quantization; reactively re-quantizes the cached source on change.
Uploader (src/lib/image-uploader.js) tracks credits locally with regen and
backs off on 429 responses so it can run alongside manual drawing.
Convert PNG/JPG/WebP to rplace palette JSON (nearest-color or Floyd-Steinberg
dither) and upload at a target (x, y) with client-side credit throttling and
429 backoff. Shared palette conversion lives in src/lib/image-to-palette.js so
the browser importer can reuse it.
Manual CompressionStream + Content-Encoding: gzip caused the browser
to receive un-decoded gzipped bytes through the wrangler-dev + vite
proxy path, surfacing as "Canvas buffer truncated: got 3598 bytes"
in the new decoder bounds check.
Cloudflare's edge auto-compresses compressible content already, so the
manual gzip layer was redundant in prod and broken in dev.
Also wraps getFullCanvas in try/catch and returns a JSON error envelope
on failure instead of letting Hono's default handler return a 500 with
text body the client decoder would mis-parse.
Three review reports (backend, frontend, security/scale) and a
follow-up re-review after pull. Plan documents which findings were
addressed in this sweep and which were intentionally deferred.
- Add CanvasRoom broadcast/close/error tests with mock WebSockets
- Remove @cloudflare/vitest-pool-workers (incompatible with Vitest 4)
- Clean up vitest config (single config, no integration workspace)
- 71 tests across 7 test files, all passing
Use path-based REST API with Upstash-Encoding: base64 header for
GETRANGE to prevent binary data corruption through JSON text encoding.
Verified all 32 colors round-trip correctly.
- Use Upstash REST API directly for BITFIELD (SDK builder broken in v1.37)
- Add redisRaw() helper for raw command execution
- Wrap setPixels in try-catch, return JSON errors instead of 500 text
- Client handles non-JSON server responses gracefully
- Toolbar: larger 44px touch targets, separators, better contrast
Replace per-pixel immediate placement with local buffer system:
- Paint mode (click) and Draw mode (drag) for pixel placement
- Undo/Redo strokes (Ctrl+Z/Y) before submitting
- Submit button sends batch to server for storage + broadcast
- Right-click drag to pan in draw mode
- Increase MAX_BATCH_SIZE to 512 for larger batches
- Handle base64 encoding from Upstash GETRANGE (atob fallback)
- Add optimistic credit deduction on pixel placement
- Handle non-ok API responses in placePixel
- Enable WebSocket proxy in Vite dev config
- Remove dead CANVAS_WIDTH/HEIGHT vars from wrangler.json
- Suppress favicon 404 with empty data URI
tiennm99
