From 9aa854bc59bb3735c4a7eaf60b1451f710f86b5d Mon Sep 17 00:00:00 2001 From: "Kai (Tam Nhu) Tran" <61256810+kaitranntt@users.noreply.github.com> Date: Sun, 17 May 2026 19:54:48 -0400 Subject: [PATCH] =?UTF-8?q?feat(docker):=20zero-install=20Docker=20UX=20?= =?UTF-8?q?=E2=80=94=20ccs:latest,=20ccs:full,=20canonical=20compose,=20cc?= =?UTF-8?q?s-net=20contract=20(#1251)=20(#1261)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(docker): publish ccs:latest + ccs:full integrated images (P1) * feat(docker): parameterize Dockerfile.integrated with ARG FLAVOR=minimal|full Add FLAVOR build arg that gates the AI CLI install layer (claude-code, gemini-cli, grok-cli, opencode) so one Dockerfile produces both the minimal (< 350 MB) and full (< 600 MB) integrated images. Use BuildKit cache mount for /root/.npm to speed up repeated builds. Part (P1 — publish integrated images). * feat(docker): emit startup deprecation warning in legacy ccs-dashboard entrypoint Prepend a [WARN] line to stderr on every container start so operators running ghcr.io/kaitranntt/ccs-dashboard:latest are notified to migrate to ghcr.io/kaitranntt/ccs:latest. Sunset window: 2 releases. See . * test(docker): add image-size.sh budget assertion + unit test suite image-size.sh: asserts docker image inspect .Size against a byte budget. Usage: image-size.sh Budgets: minimal=350 MB (367001600), full=600 MB (629145600) image-size-logic.test.sh: 6 mock-docker unit tests covering pass/fail boundaries, bad arg count, and non-integer input. All 6 pass locally. Part (P1). * ci(docker): extend docker-release.yml to publish ccs:latest and ccs:full - Add publish-integrated job with matrix flavor: [minimal, full] - minimal -> ghcr.io/kaitranntt/ccs: + :latest (when promoted) - full -> ghcr.io/kaitranntt/ccs:full- + :full (when promoted) - Multi-arch: linux/amd64 + linux/arm64 via buildx - Scoped GHA cache per flavor to avoid cross-contamination - Add promote_to_latest workflow_dispatch input (default false) - rc.1 soak: first publish only pushes immutable version tag - Mutable :latest/:full promoted only on release event OR explicit opt-in - Add smoke-test job (post-publish) for each flavor - Pulls the just-published version tag - Asserts image size via tests/docker/image-size.sh - Boots container, waits for healthcheck, probes :3000 and :8317 - Keep publish-dashboard job unchanged (legacy 2-release sunset) - Updated labels to note deprecation status All jobs run on self-hosted cliproxy runners. Part (P1). * docs(docker): document new image tags, add ccs-dashboard deprecation notices docker/README.md: - Add "Choosing an image" table (ccs:latest / ccs:full / ccs-dashboard deprecated) - Update Quick Start section to use ccs:latest as primary example - Add legacy image note with sunset timeline CHANGELOG.md: - Add Unreleased > ### Deprecated entry for ccs-dashboard:latest pointing to migration path and README.md: - Add one-line deprecation banner near top routing users to ccs:latest Part (P1). * refactor(docker): drop Bun from runtime stage; generate npm lockfile in build stage - Build stage keeps Bun for fast installs; appends `npm install --package-lock-only` to generate an ephemeral package-lock.json immediately after `bun run build:all`. - Runtime stage removes BUN_VERSION ARG/ENV, the bun.sh curl install, and `bun install --frozen-lockfile --production`. Replaces with: COPY --from=build /app/package.json /app/package-lock.json ./ RUN npm ci --omit=dev --ignore-scripts - --ignore-scripts rationale: postinstall (scripts/postinstall.js) writes ~/.ccs/ config — not needed in Docker context. bcrypt v6+ is pure-JS, no native compile. - package-lock.json already in .gitignore (line 33); never committed. - docker/Dockerfile.integrated unchanged — no Bun present there (alpine + npm). - Targets: image size reduction >= 300 MB by eliminating ~130 MB Bun binary + installer. * feat(docker): canonical compose.yaml + smoke-test (P2) * feat(docker): add canonical compose.yaml for zero-install flow * ci(docker): smoke-test ccs.kaitran.ca/docker-compose.yaml on release + nightly * docs(docker): document ccs-net contract for sibling containers Add "Connect your app to CLIProxy" section to docker/README.md with: - Public contract table (network=ccs-net, service DNS=ccs, ports 8317/3000) - Pattern A: same compose file with external network reference - Pattern B: docker run --network ccs-net - Troubleshooting subsection (DNS, missing network, conflict, Podman, MTU) Add one-line link in README.md pointing to the new section. Add CHANGELOG entry under Unreleased noting the contract as SemVer-major stable. Add CONTRIBUTING.md note that changing services.ccs or networks.ccs-net requires major bump. * docs(docker): P3 — hoist two-command quickstart, restructure docker/README, add parity CI * docs: hoist Docker zero-install quickstart above npm install path - Create docs/quickstart-snippet.md as canonical source for the two-command flow (curl + docker compose up -d), wrapped in markers - Hoist the snippet into README.md immediately below the deprecation banner, above all other install paths - Rename old npm-only "## Quick Start" to "## Install on Host (npm)" and move it below the Docker quickstart * docs(docker): restructure README with zero-install first and migration section - Reorder top-level sections: zero-install (canonical snippet with markers), choosing an image, power-user ccs docker, prebuilt image, connect your app to CLIProxy, migration, env vars, troubleshooting - Add deprecation banner at the top pointing at the migration section - Add ## Migration from ccs-dashboard:latest section with step-by-step instructions covering compose down, data preservation, named volume vs bind-mount path, and compose up with the new image - Keep P1's Choosing an image table and P5's Connect Your App to CLIProxy section intact, just repositioned * test(docs): parity check for quickstart snippet across README files Assert README.md and docker/README.md both contain the canonical quickstart block verbatim, anchored by marker comments. Exits non-zero and prints a diff on any drift. * ci(docs): wire quickstart-parity test on push and PR Runs tests/docs/quickstart-parity.sh on self-hosted runner whenever docs/quickstart-snippet.md, README.md, docker/README.md, or the test/workflow files themselves change. Fails fast on snippet drift. * fix(docker): apply red-team findings — drop :full, rc.1 soak, healthcheck, signing * docs(quickstart): fix raw URL for corporate-proxy fallback (H1) * feat(docker)!: drop :full image variant — use sibling containers on ccs-net (Q3) No AI CLIs (claude-code/gemini-cli/grok-cli/opencode) are bundled in the image. Use sibling containers attached to ccs-net instead. See docker/README.md#connect-your-app-to-cliproxy. Also removes bash from apk deps (entrypoint uses #!/bin/sh — L2). ci(docker): publish only immutable : tag pre-smoke; promote-mutable-tags job adds :latest/:MAJOR/:MINOR aliases only after smoke tests pass (H3) ci(docker): smoke-test-compose-url runs network-contract.sh against the downloaded /tmp/ccs-compose.yaml instead of re-cloning the repo (H4) ci(docker): sign published images with cosign keyless OIDC + attach provenance/SBOM via build-push-action (M8) test(docker): network-contract.sh now accepts compose-file and image-ref positional args; replaces python3 healthcheck parser with jq (L4) * chore(release): cut every main release as rc.N prerelease, manual promote flow (H2) - .releaserc.cjs: main branch now uses prerelease 'rc' channel — every semantic-release cut becomes vX.Y.Z-rc.N - add promote-release.yml: workflow_dispatch flips rc → stable via 'gh release edit --prerelease=false'; triggers docker promote-mutable-tags - add docs/release-process.md: full soak + promote procedure, rollback steps, cosign verification command - releaseNotesGenerator: add revert section, document chore hidden behaviour (L8) * fix(docker): healthcheck probes both dashboard and cliproxy ports (M1) compose.yaml healthcheck now checks :3000 and :8317 concurrently with a 4.5s internal timeout, within Docker's 5s timeout budget. Also: - docs(docker): document npm lockfile ephemeral tradeoff above install layer; note size-budget regression test as the practical safeguard (M3) - docs(docker): drop :full row from Choosing an image table; add sibling container note pointing to connect-your-app-to-cliproxy (Q3/docs) - docs(docker): remove :full docker run block; fix release-tag sentence (Q3) - docs(docker): fix raw URL in migration section (H1 parity) - docs(docker): add Volume warning — 'down -v' deletes named volumes (L13) - docs(docker): update What changes table — remove :full reference (Q3) - docs(docker): add Image Signatures and SBOM section with cosign verify and sbom download commands (M8/docs) - changelog: add Unreleased entries for rc soak, cosign signing, :full removal with migration guidance * ci(docker): assert image-size budget per platform; add compose parity + breaking-change guard (M6/L9/L12) - image-size.sh: add --platform flag; uses 'docker buildx imagetools inspect' to sum compressed layer sizes from registry manifest for linux/amd64 and linux/arm64 separately (M6) - docker-release.yml smoke-test: runs size check for both platforms - compose-parity.sh: diffs docker/compose.yaml vs docker/docker-compose.integrated.yml for image name, ports 3000/8317, volume mounts /root/.ccs and /var/log/ccs, ccs-net definition (L12) - ci.yml: add compose-parity job wired to cliproxy runner (L12) - breaking-change-guard.yml: fails PR if compose.yaml changes image name, network name, or container_name without a feat!/fix! commit (L9) * chore(ci): fix cosign shell substitution — use tr instead of bash @L expansion (L6/nit) * test(docker): fix compose-parity port regex for variable-interpolated host ports * fix: address upstream reviewer findings + failing CI checks (loop 1) * fix(ci): breaking-change-guard skips missing base files (CI-1) Guard each git-show call with git cat-file -e existence check before attempting to read docker/compose.yaml from the base branch. When the file doesn't exist on the base (new file in this PR), the script would crash with "fatal: path exists on disk but not in origin/dev". New files can't cause a contract regression, so we exit early with breaking=0. * style: prettier reformat unrelated drift (CI-2) Two files had minor formatting drift from earlier umbrella PRs: - src/cliproxy/quota/quota-manager.ts - src/management/checks/image-analysis-check.ts No logic changes — formatter-only pass to unblock CI format:check. * fix(test): update trusted-author gate count to 4 in ci-workflow test (CI-3) PR added a compose-parity job to ci.yml. That job runs on self-hosted runners using PR-provided checkout, so it legitimately requires the trusted-author guard (same as validate, build, test). The test expected 3 occurrences; the correct count is now 4: validate (matrix), build, test, compose-parity. * fix(ci): smoke-test passes compose path + image ref to network-contract (REV-1) network-contract.sh signature is: [image-ref] The smoke-test job was calling it as: bash tests/docker/network-contract.sh "${{ steps.image.outputs.ref }}" which placed the image ref in the compose-file position ($1), causing the script to try docker compose -f which fails. Corrected to: bash tests/docker/network-contract.sh docker/compose.yaml "${{ steps.image.outputs.ref }}" Also removes publish-dashboard from smoke-test.needs (REV-2): when publish-dashboard is SKIPPED on prerelease events, GitHub Actions propagates the skip to downstream jobs, so smoke-test and promote-mutable-tags were silently skipped on every rc.N publish. smoke-test only verifies the integrated image; it has no dependency on the legacy dashboard image job. * docs(docker): annotate /root/.ccs path in compose volume (REV-3 clarification) The reviewer raised a concern that the compose volume mounts /root/.ccs but the entrypoint might default to /home/node/.ccs. This is a false positive: the integrated image uses entrypoint-integrated.sh (not entrypoint.sh), which runs under supervisord with user=root and explicitly mkdir -p /root/.ccs. HOME is /root inside the container. The volume mount at /root/.ccs is correct. Added an inline comment documenting the reasoning so future reviewers do not confuse entrypoint.sh (legacy dashboard image) with entrypoint-integrated.sh (integrated image). * fix(ci): gate docs-parity pull_request job to trusted authors docs-parity.yml runs on a self-hosted runner and checks out PR code. The self-hosted-runner-policy test requires any such workflow to include the trusted-author guard. The workflow was missing the guard, causing bun test:fast to fail with 1 failure. Allow push events (no author check needed — push is to own branch) and trusted-contributor PRs only. * fix: address reviewer findings round 2 (loop 2) * fix(docker): parameterize image with CCS_IMAGE env so smoke-test exercises new digest docker/compose.yaml hardcoded image: ghcr.io/kaitranntt/ccs:latest, causing network-contract.sh to always use the old published image regardless of what IMAGE_OVERRIDE was passed. Switching to \${CCS_IMAGE:-ghcr.io/kaitranntt/ccs:latest} lets CI pass CCS_IMAGE= so the smoke-test actually exercises the just-built image. Default end-user behaviour is unchanged. Resolves REV4 (reviewer loop 2, PR). * fix(test): image-size.sh fails loudly on manifest inspection failure The --platform branch previously exited 0 when imagetools inspect returned empty or zero bytes, silently passing the budget check. A broken image could reach production undetected if buildx had any inspection issue. Changed to exit 1 with a clear diagnostic message explaining possible causes (old buildx, manifest format mismatch, image not yet pushed). No --allow-inspect-failure escape hatch is provided; CI must fix the root cause. Resolves REV5 (reviewer loop 2, PR). * test(docker): cover --platform branch in image-size-logic tests Prior tests only exercised the local docker image inspect path. The --platform branch (imagetools inspect) had zero coverage, meaning the REV5 silent-pass regression would not have been caught by CI. Added 5 mock-based test cases for the --platform branch: - pass when platform-scoped compressed size < budget - fail when platform-scoped compressed size > budget - fail (exit 1) when imagetools inspect errors (REV5 guard) - fail (exit 1) when reported size is "0" (REV5 guard) - fail (exit 1) when size output is empty (REV5 guard) Mocks follow the existing pattern (override docker in PATH with a temp wrapper), extended to handle buildx imagetools inspect subcommand. Resolves REV6 (reviewer loop 2, PR). * fix(test): compose-parity image_name() handles \${VAR:-default} syntax After parameterizing compose.yaml's image field with \${CCS_IMAGE:-...}, the image_name() sed pipeline cut at the first colon in the shell variable syntax (:-) instead of the tag separator, returning '\${CCS_IMAGE' and failing the kaitranntt/ccs grep. Added a sed step to unwrap \${VAR:-default} by extracting just the default value before stripping the tag. Existing plain image: name:tag references are unaffected. * fix: tighten parity check + catch service-key rename (loop 3) * fix(test): tighten compose-parity image-name match to exact expected values (REV7) Replace loose substring grep (grep -q "ccs") with exact equality checks against declared EXPECTED_CANONICAL_IMAGE and EXPECTED_INTEGRATED_IMAGE constants. The integrated compose builds locally as ccs-cliproxy:latest (not ghcr.io/kaitranntt/ccs), so both expected names are explicitly documented at the top of the assertion block. Fixes: a drift in integrated compose to a wrong owner/registry could slip through the old "grep -q ccs" check; now any name other than the declared constant is a hard failure. * feat(ci): breaking-change-guard catches services.ccs rename — public DNS contract (REV8) Docker's service-name DNS uses the compose service KEY as the hostname. Sibling containers on ccs-net reach CCS via http://ccs:8317; renaming services.ccs: to anything else silently breaks that contract even when image name, network name, and container_name are unchanged. Add check 4 to the guard: - Extract top-level service keys from both base and HEAD versions of docker/compose.yaml using awk (no external YAML parser required). - Fail if the "ccs" key is absent from HEAD. - Fail if the sorted set of service keys differs from base. Wraps inside the existing git cat-file guard so new-file PRs skip it. * fix(ci): smoke-test failure check + relax service-key guard (REV9, REV10) REV9 — docker-release.yml smoke-test: add HEALTHY flag to the boot-and-wait loop. Previously a timeout (status stays 'starting'/'missing' for all 12 iterations) exited the loop silently, letting port probes be the only net. Now if the loop exits without HEALTHY=1 the step fails immediately with container logs and state dump. network-contract.sh already has the HEALTHY pattern — no change needed there. REV10 — breaking-change-guard.yml: drop the OLD_KEYS != NEW_KEYS comparison. That check treated ANY change to the service-key set as breaking — including adding a harmless sidecar. The DNS contract only requires services.ccs to exist; other services are irrelevant to the 'ccs' hostname on ccs-net. Keep only the "ccs key must exist" check; remove the unused OLD_KEYS extraction. * fix(release)!: decouple npm @latest from Docker rc.1 soak (REV11) Reverts the loop-1 prerelease channel that was publishing npm as vX.Y.Z-rc.N to the `rc` dist-tag instead of `latest`. `main` is now a stable semantic-release channel again: every merge publishes vX.Y.Z immediately to npm @latest. The rc.1 soak window that guards Docker mutable tags is moved entirely into the Docker publish workflow: - `.releaserc.cjs`: remove `prerelease: 'rc'` from productionConfig, restore `branches: ['main']` (stable). Restore full successComment and `released` label. Keep loop-1 releaseNotesGenerator additions (revert section, breaking change comment). - `docker-release.yml`: every `release: published` event publishes only the immutable `:` Docker tag. `promote-mutable-tags` job now gates exclusively on `workflow_dispatch` with `promote_to_latest=true` — no longer triggered automatically by non-prerelease release events. - `promote-release.yml`: rewritten as a dispatch wrapper that validates the stable tag exists and is not a prerelease, verifies the immutable Docker image is in the registry, then dispatches docker-release.yml with `promote_to_latest=true`. Removes the `gh release edit --prerelease=false` approach that required the rc soak to be wired through GitHub release state. - `docs/release-process.md`: updated to reflect the decoupled model — npm @latest is immediate; Docker :latest requires manual promote after soak. Documents the `why` split between npm and Docker soak windows. * fix(ci): reviewer loop 6 — REV12 compose image parsing, REV13 fork bypass, REV14 :full audit * fix(ci): breaking-change-guard handles \${VAR:-default} compose image syntax (REV12) The sed 's/:.*//' pattern truncated at the first colon in the \${CCS_IMAGE:-ghcr.io/...} expression, yielding "\${CCS_IMAGE" as the image name instead of the actual registry path. Any change to the default image namespace was therefore undetectable. Introduce extract_image_name() that first strips the \${VAR:-default} wrapper with a sed -E expression, then strips only the trailing :tag suffix using a pattern that preserves internal colons (e.g. registry:5000/ owner/repo). Applied to both OLD_RAW and NEW_RAW extraction paths. * fix(ci): breaking-change-guard runs on ubuntu-latest to cover forked PRs (REV13) The trusted-author gate (COLLABORATOR|MEMBER|OWNER) caused forked-PR contributors to bypass the breaking-change check entirely. A forked contributor could rename services.ccs or change the image namespace without a feat!/fix! marker and the guard would never run. This workflow is a documented exception to the self-hosted-first policy: it performs ONLY pure YAML diff parsing (git show / awk / sed). No build, install, or arbitrary PR-branch scripts are executed. The checkout uses persist-credentials: false. There is no untrusted code execution, so ubuntu-latest is safe and necessary for universal fork coverage. Update self-hosted-runner-policy.test.ts to: - Introduce GITHUB_HOSTED_RUNNER_EXCEPTIONS registry with required justification comments for each entry - Skip exception workflows in the "keeps active workflows on local runners" and "gates pull-request workflows" assertions - Add a new "documented exceptions use github-hosted runners" test that verifies each exception entry actually uses a GitHub-hosted runner (prevents stale entries accumulating without cleanup) * docs(ci): clarify rc.1 soak and :full removal as intentional design Reviewer kept flagging :full omission and stale :latest as bugs. Both are intentional per Q3 maintainer decision and the rc.1 soak design (loop-5). Strengthen inline comments to make this unmistakable to future readers. * fix: address final-review criticals — healthcheck/digest/contract guards (loop 8) * fix(docker): healthcheck fails on 4xx not only 5xx (H1) Change threshold from >= 500 to >= 400 so a misrouted 404 on / is treated as unhealthy. A container returning 4xx on the root path is not serving traffic correctly and must not be reported healthy. * chore(docker): pin eceasy/cli-proxy-api base image by digest (H2) FROM eceasy/cli-proxy-api:latest@sha256:4fa722b... locks the base image to a specific manifest-list digest for reproducible builds. The human- readable :latest tag is preserved for readability. Refresh cadence: quarterly or on security advisory from upstream. apk packages (nodejs, npm) remain unpinned — noted as a known limitation until a CI workflow for apk-digest pinning exists. * fix(ci): breaking-change-guard catches network rename and hostname override (M1/M2) Replace the old diff-based ccs-net check (which missed the case where both old and new lack the name, producing identical empty strings) with a positive assertion: the canonical compose MUST contain 'name: ccs-net'. This catches renames, removals, and whitespace variations. Also add a hostname: override guard (M2): setting services.ccs.hostname changes the Docker DNS name on ccs-net even when the service key stays "ccs", silently breaking http://ccs:8317 for sibling containers. * fix(test): compose-parity image-name regex preserves registry:port (M5) The old sed 's/:.*//' truncated at the first colon, incorrectly stripping registry:port prefixes (e.g. registry.local:5000/owner/repo:tag became registry.local). Replace with sed 's|:[^:/]*$||' which strips only the trailing :tag suffix, preserving internal colons. Move tr -d ' ' before the sed -E wrapper-strip so the '^' anchor matches correctly after whitespace removal. Mirrors extract_image_name() in breaking-change-guard.yml. Add 3 image_name() regression tests to image-size-logic.test.sh covering plain tag, env-var-with-default syntax, and registry:port/owner/repo:tag. * ci(pr-agent): disable ticket analysis to avoid sub-issue crash PR-Agent's fetch_sub_issues crashes when the PR body references PR numbers as shortcodes (etc) — GraphQL repository.issue(number) returns null for PR numbers and pr-agent dereferences without a guard (github_provider.py:1048). Disable ticket_analysis_review until upstream fixes the None guard. Other reviewer sections (security, score, tests, etc.) remain enabled. * ci(pr-agent): re-enable ticket analysis after history+body rewrite PR shortcodes removed from all commit messages and PR body, so fetch_sub_issues no longer hits the upstream null-deref bug. Ticket compliance review is re-enabled in full. --- .github/workflows/breaking-change-guard.yml | 146 ++++++++++++++++-- docker/Dockerfile.integrated | 7 +- docker/compose.yaml | 7 +- tests/docker/compose-parity.sh | 39 +++-- tests/docker/image-size-logic.test.sh | 119 +++++++++++++- .../github/self-hosted-runner-policy.test.ts | 11 ++ 6 files changed, 303 insertions(+), 26 deletions(-) diff --git a/.github/workflows/breaking-change-guard.yml b/.github/workflows/breaking-change-guard.yml index 876bfb1f..ecd5d21c 100644 --- a/.github/workflows/breaking-change-guard.yml +++ b/.github/workflows/breaking-change-guard.yml @@ -71,15 +71,137 @@ jobs: extract_image_name() { # $1 = raw image line content (everything after "image: ") local raw="$1" + raw=$(printf '%s' "$raw" | sed -E "s/^[\"']//; s/[\"']$//") # Strip ${VAR:-default} wrapper if present raw=$(printf '%s' "$raw" | sed -E 's/^\$\{[A-Za-z_][A-Za-z0-9_]*:-//; s/\}$//') # Strip only the trailing :tag — preserve internal colons (e.g. registry:5000/owner/repo) printf '%s' "$raw" | sed 's|:[^:/]*$||' } - OLD_RAW=$(git show "${BASE}:docker/compose.yaml" \ - | grep -m1 '^\s*image:' | sed 's/.*image:\s*//' | tr -d ' ') - NEW_RAW=$(grep -m1 '^\s*image:' docker/compose.yaml \ - | sed 's/.*image:\s*//' | tr -d ' ') + + service_field_value() { + local file="$1" service="$2" field="$3" + awk -v service="$service" -v field="$field" ' + function trim(s) { + sub(/^[[:space:]]+/, "", s) + sub(/[[:space:]]+$/, "", s) + return s + } + /^services:[[:space:]]*$/ { in_services = 1; next } + in_services && /^[^[:space:]]/ { in_services = 0; in_service = 0 } + in_services && $0 ~ "^ " service ":[[:space:]]*$" { in_service = 1; next } + in_service && /^ [A-Za-z0-9_-]+:[[:space:]]*$/ { in_service = 0 } + in_service { + prefix = "^ " field ":[[:space:]]*" + if ($0 ~ prefix) { + line = $0 + sub(prefix, "", line) + print trim(line) + exit + } + } + ' "$file" + } + + service_network_refs() { + local file="$1" service="$2" + awk -v service="$service" ' + function trim(s) { + sub(/^[[:space:]]+/, "", s) + sub(/[[:space:]]+$/, "", s) + return s + } + function emit_inline(value, parts, count, i) { + gsub(/[\[\]]/, "", value) + count = split(value, parts, ",") + for (i = 1; i <= count; i++) { + if (trim(parts[i]) != "") print trim(parts[i]) + } + } + /^services:[[:space:]]*$/ { in_services = 1; next } + in_services && /^[^[:space:]]/ { in_services = 0; in_service = 0; in_networks = 0 } + in_services && $0 ~ "^ " service ":[[:space:]]*$" { in_service = 1; in_networks = 0; next } + in_service && /^ [A-Za-z0-9_-]+:[[:space:]]*$/ { in_service = 0; in_networks = 0 } + in_service { + if ($0 ~ /^ networks:[[:space:]]*\[/) { + line = $0 + sub(/^ networks:[[:space:]]*/, "", line) + emit_inline(line) + next + } + if ($0 ~ /^ networks:[[:space:]]*$/) { + in_networks = 1 + next + } + if ($0 ~ /^ networks:[[:space:]]+/) { + line = $0 + sub(/^ networks:[[:space:]]+/, "", line) + print trim(line) + next + } + if (in_networks && $0 ~ /^ [A-Za-z0-9_-]+:/) { + in_networks = 0 + } + if (in_networks && $0 ~ /^ -[[:space:]]*/) { + line = $0 + sub(/^ -[[:space:]]*/, "", line) + print trim(line) + next + } + if (in_networks && $0 ~ /^ [A-Za-z0-9_-]+:/) { + line = $0 + sub(/^ /, "", line) + sub(/:.*/, "", line) + print trim(line) + next + } + } + ' "$file" + } + + network_effective_name() { + local file="$1" ref="$2" + awk -v ref="$ref" ' + function trim(s) { + sub(/^[[:space:]]+/, "", s) + sub(/[[:space:]]+$/, "", s) + return s + } + /^networks:[[:space:]]*$/ { in_networks = 1; next } + in_networks && /^[^[:space:]]/ { in_networks = 0; in_target = 0 } + in_networks && $0 ~ "^ " ref ":[[:space:]]*$" { + in_target = 1 + found = 1 + effective = ref + next + } + in_target && /^ [A-Za-z0-9_-]+:[[:space:]]*$/ { in_target = 0 } + in_target && /^ name:[[:space:]]*/ { + line = $0 + sub(/^ name:[[:space:]]*/, "", line) + effective = trim(line) + } + END { if (found) print effective } + ' "$file" + } + + has_service_network_effective_name() { + local file="$1" service="$2" expected="$3" ref effective + while IFS= read -r ref; do + [[ -z "$ref" ]] && continue + effective=$(network_effective_name "$file" "$ref") + if [[ "$effective" == "$expected" ]]; then + return 0 + fi + done < <(service_network_refs "$file" "$service") + return 1 + } + + BASE_COMPOSE="$(mktemp)" + trap 'rm -f "$BASE_COMPOSE"' EXIT + git show "${BASE}:docker/compose.yaml" > "$BASE_COMPOSE" + + OLD_RAW=$(service_field_value "$BASE_COMPOSE" ccs image) + NEW_RAW=$(service_field_value docker/compose.yaml ccs image) OLD_IMAGE=$(extract_image_name "$OLD_RAW") NEW_IMAGE=$(extract_image_name "$NEW_RAW") if [[ "${OLD_IMAGE}" != "${NEW_IMAGE}" ]]; then @@ -87,19 +209,17 @@ jobs: BREAKING=1 fi - # 2. ccs-net network name change - OLD_NET=$(git show "${BASE}:docker/compose.yaml" \ - | grep 'name: ccs-net' | tr -d ' ' || echo "") - NEW_NET=$(grep 'name: ccs-net' docker/compose.yaml | tr -d ' ' || echo "") - if [[ "${OLD_NET}" != "${NEW_NET}" ]]; then - echo "[!] BREAKING: ccs-net network name changed" + # 2. ccs-net network attachment — positive assertion: services.ccs MUST + # attach to a network whose effective Compose name is ccs-net. Merely + # keeping an unused top-level ccs-net definition is not enough. + if ! has_service_network_effective_name docker/compose.yaml ccs ccs-net; then + echo "[!] BREAKING: services.ccs is not attached to effective network name ccs-net" BREAKING=1 fi # 3. container_name change (if present in either version) - OLD_CN=$(git show "${BASE}:docker/compose.yaml" \ - | grep 'container_name:' | tr -d ' ' || echo "") - NEW_CN=$(grep 'container_name:' docker/compose.yaml | tr -d ' ' || echo "") + OLD_CN=$(service_field_value "$BASE_COMPOSE" ccs container_name | tr -d ' ') + NEW_CN=$(service_field_value docker/compose.yaml ccs container_name | tr -d ' ') if [[ "${OLD_CN}" != "${NEW_CN}" ]]; then echo "[!] BREAKING: container_name changed: '${OLD_CN}' -> '${NEW_CN}'" BREAKING=1 diff --git a/docker/Dockerfile.integrated b/docker/Dockerfile.integrated index c4a6e3ab..7d1bd03c 100644 --- a/docker/Dockerfile.integrated +++ b/docker/Dockerfile.integrated @@ -1,4 +1,9 @@ -FROM eceasy/cli-proxy-api:latest +# Base image pinned by digest for reproducible builds. +# To refresh: docker pull eceasy/cli-proxy-api:latest && docker inspect --format='{{index .RepoDigests 0}}' eceasy/cli-proxy-api:latest +# Refresh cadence: quarterly OR on security advisory from upstream. +# TODO: pin apk packages once apk-digest pinning workflow exists in CI (apk add nodejs npm are currently unpinned; +# alpine package versions can be pinned but require per-package version discovery and maintenance). +FROM eceasy/cli-proxy-api:latest@sha256:4fa722b9ff83adfbe6e350d433b7dcb754756aa63df95a7a6349767dc13b2cb7 ARG CCS_NPM_VERSION=latest diff --git a/docker/compose.yaml b/docker/compose.yaml index 7f08f9b7..65584042 100644 --- a/docker/compose.yaml +++ b/docker/compose.yaml @@ -14,6 +14,11 @@ services: ccs: + # image uses Docker Compose env-var-with-default syntax so CI smoke tests can + # override via CCS_IMAGE=ghcr.io/.../ccs: docker compose up -d while + # end users get the documented :latest by default. The Cloudflare Worker that + # serves this file at ccs.kaitran.ca validates that the default part starts + # with the canonical ghcr.io/kaitranntt/ccs prefix. image: ${CCS_IMAGE:-ghcr.io/kaitranntt/ccs:latest} restart: unless-stopped ports: @@ -41,7 +46,7 @@ services: const http = require('http'); let pending = 2; let ok = true; const check = (port) => http.get('http://127.0.0.1:'+port+'/', r => { - if (r.statusCode >= 500) ok = false; + if (r.statusCode >= 400) ok = false; if (--pending === 0) process.exit(ok ? 0 : 1); }).on('error', () => { ok = false; if (--pending === 0) process.exit(1); }); check(3000); check(8317); diff --git a/tests/docker/compose-parity.sh b/tests/docker/compose-parity.sh index d94d83a5..a97827c0 100644 --- a/tests/docker/compose-parity.sh +++ b/tests/docker/compose-parity.sh @@ -15,8 +15,8 @@ # set -euo pipefail -CANONICAL="docker/compose.yaml" -INTEGRATED="docker/docker-compose.integrated.yml" +CANONICAL="${COMPOSE_PARITY_CANONICAL:-docker/compose.yaml}" +INTEGRATED="${COMPOSE_PARITY_INTEGRATED:-docker/docker-compose.integrated.yml}" fail=0 @@ -34,12 +34,31 @@ image_name() { # Match lines like: # image: ghcr.io/owner/repo:tag # image: ${CCS_IMAGE:-ghcr.io/owner/repo:tag} - grep -A 50 "^ ${service}:" "$file" \ - | grep -m1 '^\s*image:' \ - | sed 's/.*image:\s*//' \ - | sed 's/\${[^:-]*:-\([^}]*\)}/\1/' \ - | sed 's/:.*//' \ - | tr -d ' ' + # image: registry.local:5000/owner/repo:tag (registry with port — must NOT truncate at first colon) + # + # Pipeline: + # 1. Extract raw image value (strip "image:" prefix and whitespace) + # 2. Strip ${VAR:-default} wrapper if present — must happen AFTER whitespace + # removal so the anchor ^ matches at position 0 + # 3. Strip only the trailing :tag suffix, preserving internal colons + # (e.g. registry:5000/owner/repo keeps its port colon intact) + # + # Mirrors the extract_image_name() logic in .github/workflows/breaking-change-guard.yml. + awk -v service="$service" ' + /^services:[[:space:]]*$/ { in_services = 1; next } + in_services && /^[^[:space:]]/ { in_services = 0; in_service = 0 } + in_services && $0 ~ "^ " service ":[[:space:]]*$" { in_service = 1; next } + in_service && /^ [A-Za-z0-9_-]+:[[:space:]]*$/ { in_service = 0 } + in_service && /^[[:space:]]*image:/ { + sub(/^[[:space:]]*image:[[:space:]]*/, "", $0) + print + exit + } + ' "$file" \ + | tr -d ' ' \ + | sed -E "s/^[\"']//; s/[\"']$//" \ + | sed -E 's/^\$\{[A-Za-z_][A-Za-z0-9_]*:-//; s/\}$//' \ + | sed 's|:[^:/]*$||' } # --------------------------------------------------------------------------- @@ -48,7 +67,7 @@ image_name() { exposed_ports() { local file="$1" # Match port mappings: "HOST:CONTAINER" — extract CONTAINER port number - grep -E '^\s+- "[0-9]+:[0-9]+"' "$file" \ + grep -E '^[[:space:]]+- "[0-9]+:[0-9]+"' "$file" \ | sed 's/.*:\([0-9]*\)".*/\1/' \ | sort -n } @@ -59,7 +78,7 @@ exposed_ports() { volume_targets() { local file="$1" # Match volume entries: - name:/container/path - grep -E '^\s+- [a-z_]+:/' "$file" \ + grep -E '^[[:space:]]+- [a-z_]+:/' "$file" \ | sed 's/.*:\(\/[^[:space:]]*\).*/\1/' \ | sort } diff --git a/tests/docker/image-size-logic.test.sh b/tests/docker/image-size-logic.test.sh index 1fdcdd4e..5b5255c1 100755 --- a/tests/docker/image-size-logic.test.sh +++ b/tests/docker/image-size-logic.test.sh @@ -37,7 +37,8 @@ run_test() { # ------------------------------------------------------------------ MOCK_DIR="$(mktemp -d)" -trap 'rm -rf "$MOCK_DIR"' EXIT +COMPOSE_FIXTURE_DIR="" +trap 'rm -rf "$MOCK_DIR" "$COMPOSE_FIXTURE_DIR"' EXIT make_mock_docker() { local size="$1" @@ -201,6 +202,122 @@ run_platform_test "--platform: fail loudly when reported size is 0 (REV5 guard)" make_mock_docker_platform "" run_platform_test "--platform: fail loudly when size output is empty (REV5 guard)" 1 "209715200" +# ------------------------------------------------------------------ +# compose-parity image_name() regression — M5 guard +# +# Verifies the sed pipeline in tests/docker/compose-parity.sh correctly +# strips only the trailing :tag and does NOT truncate at the first colon +# (which would break registry:port/owner/repo references). +# ------------------------------------------------------------------ +echo "" +echo "Running compose-parity image_name() regex tests..." +echo "" + +# Inline the updated image_name() logic from compose-parity.sh so these tests +# run without depending on a compose file on disk. +_image_name_from_raw() { + local raw="$1" + printf '%s' "$raw" \ + | sed -E "s/^[\"']//; s/[\"']$//" \ + | sed -E 's/^\$\{[A-Za-z_][A-Za-z0-9_]*:-//; s/\}$//' \ + | sed 's|:[^:/]*$||' \ + | tr -d ' ' +} + +run_image_name_test() { + local name="$1" input="$2" expected="$3" + local got + got=$(_image_name_from_raw "$input") + if [[ "$got" == "$expected" ]]; then + echo "[OK] ${name}" + (( PASS++ )) || true + else + echo "[X] ${name}: expected='${expected}' got='${got}'" + (( FAIL++ )) || true + fi +} + +# Plain image reference — tag stripped +run_image_name_test \ + "plain image: strip :tag" \ + "ghcr.io/kaitranntt/ccs:latest" \ + "ghcr.io/kaitranntt/ccs" + +# Env-var-with-default syntax — wrapper stripped, then tag stripped +run_image_name_test \ + "env-var default: strip wrapper and :tag" \ + '${CCS_IMAGE:-ghcr.io/kaitranntt/ccs:latest}' \ + "ghcr.io/kaitranntt/ccs" + +# Registry with port — internal colon preserved, only :tag stripped +run_image_name_test \ + "registry:port/owner/repo:tag — preserve internal colon" \ + "registry.local:5000/owner/repo:tag" \ + "registry.local:5000/owner/repo" + +# ------------------------------------------------------------------ +# compose-parity service-scoped extraction regression — reviewer guard +# +# Verifies the real compose-parity script does not read a later sidecar image +# when services.ccs itself lacks an image. +# ------------------------------------------------------------------ +echo "" +echo "Running compose-parity scoped extraction tests..." +echo "" + +COMPOSE_FIXTURE_DIR="$(mktemp -d)" +CANONICAL_FIXTURE="${COMPOSE_FIXTURE_DIR}/compose.yaml" +INTEGRATED_FIXTURE="${COMPOSE_FIXTURE_DIR}/integrated.yaml" + +cat > "$CANONICAL_FIXTURE" <<'YAML' +services: + ccs: + ports: + - "3000:3000" + - "8317:8317" + volumes: + - ccs_home:/root/.ccs + - ccs_logs:/var/log/ccs + networks: + - ccs-net + sidecar: + image: ghcr.io/kaitranntt/ccs:latest +volumes: + ccs_home: + ccs_logs: +networks: + ccs-net: + name: ccs-net +YAML + +cat > "$INTEGRATED_FIXTURE" <<'YAML' +services: + ccs-cliproxy: + image: ccs-cliproxy:latest + ports: + - "3000:3000" + - "8317:8317" + volumes: + - ccs_home:/root/.ccs + - ccs_logs:/var/log/ccs +volumes: + ccs_home: + ccs_logs: +YAML + +actual_exit=0 +COMPOSE_PARITY_CANONICAL="$CANONICAL_FIXTURE" \ + COMPOSE_PARITY_INTEGRATED="$INTEGRATED_FIXTURE" \ + bash "${SCRIPT_DIR}/compose-parity.sh" > /dev/null 2>&1 || actual_exit=$? + +if [[ "$actual_exit" -ne 0 ]]; then + echo "[OK] compose-parity does not bleed into later sidecar image" + (( PASS++ )) || true +else + echo "[X] compose-parity should fail when services.ccs lacks an image" + (( FAIL++ )) || true +fi + # ------------------------------------------------------------------ # Summary # ------------------------------------------------------------------ diff --git a/tests/unit/scripts/github/self-hosted-runner-policy.test.ts b/tests/unit/scripts/github/self-hosted-runner-policy.test.ts index f9e18593..5abcced9 100644 --- a/tests/unit/scripts/github/self-hosted-runner-policy.test.ts +++ b/tests/unit/scripts/github/self-hosted-runner-policy.test.ts @@ -60,6 +60,17 @@ describe('self-hosted runner policy', () => { } }); + test('breaking-change guard scopes compose contract checks to services.ccs', () => { + const workflow = fs.readFileSync(path.join(workflowsDir(), 'breaking-change-guard.yml'), 'utf8'); + + expect(workflow).toContain('OLD_RAW=$(service_field_value "$BASE_COMPOSE" ccs image)'); + expect(workflow).toContain('NEW_RAW=$(service_field_value docker/compose.yaml ccs image)'); + expect(workflow).toContain('has_service_network_effective_name docker/compose.yaml ccs ccs-net'); + expect(workflow).toContain('OLD_CN=$(service_field_value "$BASE_COMPOSE" ccs container_name'); + expect(workflow).not.toContain("grep -m1 '^[[:space:]]*image:'"); + expect(workflow).not.toContain('services.ccs.hostname override'); + }); + test('gates pull-request self-hosted worker deploys to trusted authors', () => { const workflow = fs.readFileSync(path.join(workflowsDir(), 'deploy-ccs-worker.yml'), 'utf8');