# GitHub Binary Installer

Install CLI tools directly from GitHub Releases at runtime. Covers Go, Rust,
shell, and other binary-distributed tools not available via `apk` / `pip` / `npm`.

Closes [#741](https://github.com/nextlevelbuilder/goclaw/issues/741).

## Install Syntax

```
github:owner/repo[@tag]
```

Examples:
- `github:cli/cli` → latest release
- `github:jesseduffield/lazygit@v0.42.0` → specific version
- `github:sharkdp/fd@v9.0.0` → specific version with dot separator

## How It Works

1. Fetches release metadata from the GitHub API
2. Auto-selects asset matching `linux` + current arch (amd64 / arm64)
3. Streams download to a temp file, enforcing a max size cap
4. Verifies SHA256 if the publisher ships `checksums.txt` / `SHA256SUMS`
5. Validates ELF magic bytes + 64-bit class + machine matches runtime arch
6. Extracts archive safely (tar.gz / zip / raw binary) with path-traversal + zip-bomb guards
7. Installs to `{runtimeDir}/bin/` (prepended to `$PATH`)
8. Persists a manifest for later listing + uninstall

## Usage

### Web UI

1. Admin Settings → Packages page
2. Scroll to **GitHub Binaries** section
3. Enter `owner/repo[@tag]` or click **Browse releases** to pick a version
4. Click **Install**

### HTTP API

```bash
# Install
curl -X POST http://gateway/v1/packages/install \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"package": "github:jesseduffield/lazygit@v0.42.0"}'

# List installed (includes pip/npm/system + github)
curl http://gateway/v1/packages -H "Authorization: Bearer $ADMIN_TOKEN"

# Browse releases (picker UI uses this)
curl 'http://gateway/v1/packages/github-releases?repo=cli/cli&limit=10' \
  -H "Authorization: Bearer $VIEWER_TOKEN"

# Uninstall
curl -X POST http://gateway/v1/packages/uninstall \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"package": "github:lazygit"}'
```

## Admin Configuration

All configuration is driven by environment variables — **never** place the
token in `config.json`.

| Env var | Default | Notes |
|---------|---------|-------|
| `GOCLAW_PACKAGES_GITHUB_TOKEN` | `""` | Optional PAT: rate 60/hr → 5000/hr + private repo access |
| `GOCLAW_PACKAGES_MAX_ASSET_SIZE_MB` | `200` | Applies to both download cap and 2× uncompressed cap |
| `GOCLAW_PACKAGES_GITHUB_ALLOWED_ORGS` | `""` | Comma-separated allowlist (empty = all orgs allowed) |
| `GOCLAW_PACKAGES_GITHUB_BIN_DIR` | `{runtimeDir}/bin` | Where extracted binaries land |
| `GOCLAW_PACKAGES_GITHUB_MANIFEST` | `{bin_dir}/../github-packages.json` | Manifest path |

`packages.scratch_dir` in `config.json` is optional. If it is empty or cannot
be created, updates use `{runtimeDir}/tmp` so bare-metal services do not depend
on root-owned release directories such as `/opt/goclaw/tmp`.

Token scopes:
- public-only repos: no scopes required
- private repos: `repo`
- org-SSO-enforced repos: must be SSO-authorized PAT

## Security

- HTTPS-only downloads with SSRF host allowlist:
  `github.com`, `api.github.com`, `objects.githubusercontent.com`,
  `release-assets.githubusercontent.com`, `codeload.github.com`
- Every redirect hop re-validated (blocks redirect-based host escape)
- Literal IP hostnames (v4 / v6) always rejected (blocks cloud-metadata access)
- SHA256 verification when publisher ships `checksums.txt` / `SHA256SUMS` (constant-time compare)
- ELF magic + 64-bit class + machine-arch validation before `chmod +x`
- Path-traversal prevention in archive extraction (rejects `..`, absolute, Windows drive, null byte)
- Zip-bomb guard (cumulative uncompressed bytes capped at 2× max asset size)
- Symlink / hardlink entries skipped, never written
- Admin-only API + master-scope guard on install/uninstall
- Picker endpoint `/v1/packages/github-releases` throttled per user (30 req/min, burst 10) to protect GitHub API quota; anonymous fallback keyed by remote IP. Response is `429 Too Many Requests` with `Retry-After: 60` when tripped.
- Token never logged (startup log prints `token_set=bool`)

## Troubleshooting

### "glibc not found" / segfault on execution

GoClaw runs on Alpine Linux (musl libc). Many Go/Rust binaries target glibc.

**Fix:** pick a musl-compatible release asset. Look for names containing:
- `*-musl.tar.gz` (explicit musl)
- `*-linux-static*` (fully static)
- Go binaries with `CGO_ENABLED=0` typically work out of the box

Known-good musl releases:
- `ripgrep`: `ripgrep-*-x86_64-unknown-linux-musl.tar.gz`
- `starship`: `starship-x86_64-unknown-linux-musl.tar.gz`
- `gh`: `gh_*_linux_amd64.tar.gz` (static)

### "no matching asset found"

Asset naming doesn't fit the heuristic. Open the release page and confirm assets
exist for `linux` + your arch. Workaround: file an upstream issue asking for
standard `linux_amd64` / `linux_arm64` naming.

### "arch mismatch"

Binary is `amd64` but runtime is `arm64` (or vice versa). Pick a release asset
matching the host arch — the release picker UI filters automatically.

### "rate limit exceeded"

Anonymous GitHub API is capped at 60 req/hr. Set
`GOCLAW_PACKAGES_GITHUB_TOKEN` to bump to 5000/hr.

### "checksum mismatch"

Hard-fail. Indicates tampered download or publisher re-signing without updating
the release. Do not force-install; report upstream.

## Limitations (Phase 1)

- Linux-only (Lite/Desktop editions not yet supported)
- Docker and bare-metal gateway editions (default runtime dir resolves to `/app/data/.runtime/bin` in Docker or `/var/lib/goclaw/data/.runtime/bin` on bare-metal Linux)
- Installs all top-level executables in an archive (no interactive picker if
  archive contains multiple binaries)
- No version history / rollback — re-installing replaces in place
- Global manifest (not per-tenant)

## Updating Installed Packages

Update flow is **Phase 1 GitHub-only** (pip/npm/apk deferred to Phase 2).

### UI

The Runtime & Packages page renders a summary bar above the GitHub Binaries
section when updates are available:

```
┌─────────────────────────────────────────────────────────┐
│ 🟡 3 updates available                                   │
│    Last checked 5m ago   [Refresh]  [Update All]        │
└─────────────────────────────────────────────────────────┘
```

Per-row `[Update]` buttons appear next to each package with a newer release.
Clicking applies the update via atomic `.bak` swap with automatic rollback on
failure.

### API

All write endpoints require **master-scope admin** (tenant admins are denied):

| Endpoint | Purpose |
|---|---|
| `GET /v1/packages/updates` | Cache snapshot + `{stale, ageSeconds, ttlSeconds}` (operator+) |
| `POST /v1/packages/updates/refresh` | Force sync CheckAll — fetch from GitHub |
| `POST /v1/packages/update` | Apply one: body `{"package":"github:lazygit","toVersion":"v0.44.5"}` |
| `POST /v1/packages/updates/apply-all` | Sequential apply; body `{"packages":[...]}` (empty = all). Always returns 200 — inspect `failed[]` |

### Behaviour

- **Stale-while-revalidate**: `GET /updates` returns the cached snapshot
  immediately and triggers a background refresh if the cache is older than
  `packages.updates_check_ttl` (default `1h`).
- **ETag**: responses use `If-None-Match`, so repeated checks cost zero
  rate-limit budget (304 responses don't count against 60/hr).
- **Pre-releases**: if your current tag matches `(-alpha|-beta|-rc|-pre|-preview|-dev|-nightly)`,
  the checker polls both `/releases/latest` and `/releases?per_page=5` and
  picks the newest via `golang.org/x/mod/semver.Compare`. This correctly
  handles the `v1.0.0-rc.1 → v1.0.0` stable transition.
- **Non-semver tags** (e.g. `2024-01-15`): string-compare fallback. Never
  downgrades — if the candidate string is lexically less than current, the
  update is suppressed.
- **Atomic swap**: two-phase rename. Phase A renames ALL current binaries to
  `{name}.bak.{unixNano}`; Phase B renames the new binaries in place. On any
  failure during Phase B, Phase A's renames are rolled back. Manifest is
  persisted AFTER all swaps succeed, with retries (100ms/500ms/1s).

### WebSocket events

Owner clients receive (non-owner master admins use the HTTP API directly):

```
package.update.checked    {count, checked_at}
package.update.started    {source, name, from_version, to_version}
package.update.succeeded  {source, name, from_version, to_version, duration_ms}
package.update.failed     {source, name, reason}
```

### Troubleshooting Updates

#### "Binary updated but manifest save failed" (manifestDesynced=true)

The `.bak` files are deleted but the manifest didn't record the new version.
Next update attempt will re-apply the same version. Manual recovery is not
required — just run the update again OR restart the gateway (which re-reads
the manifest). No data loss.

#### Corrupt updates cache

Symptom: UI shows no updates available despite newer releases.

Recovery: delete `/app/data/.runtime/updates-cache.json`, click `[Refresh]`.

#### Rate-limit exhaustion

Symptom: `Refresh` returns 429 or check returns partial results.

Check response header `X-RateLimit-Reset` (Unix epoch). Wait or set
`packages.github_token` in config (Phase 2 auth — unwired in Phase 1).

#### Scratch dir leftover after crash

Path: `{runtimeDir}/tmp/{name}-{tag}-{nanos}/` unless
`packages.scratch_dir` points to another writable directory.

Safe to remove any `{name}-*-*` directory under tmp after ensuring no active
update is in flight. Phase 2 will add startup GC.

#### Mid-swap process crash

Phase 1 leaves `.bak.{nanos}` files on disk. Manual recovery:
1. Check `{BinDir}` for `*.bak.*` files.
2. If the main binary is MISSING, rename the `.bak.{nanos}` back to the
   original name.
3. If the main binary EXISTS but is the new version you wanted, delete the
   `.bak.{nanos}`.
4. Re-run the update via UI — idempotent.

## See Also

- [`docs/packages-pip-npm.md`](./packages-pip-npm.md) — pip + npm package updates (Phase 2a)
- [`docs/14-skills-runtime.md`](./14-skills-runtime.md) — Overview of the runtime packages system
- Issue [#741](https://github.com/nextlevelbuilder/goclaw/issues/741) — Original feature request