coolify/dev/coold-dev.md

# coold Dev Environment Notes

This file documents the current local v5/coold dev setup in Coolify.

## Roles

- `scripts/dev.sh` owns local developer convenience.
- Lima VMs act like real deployment servers.
- `coolify init bootstrap` owns host wiring:
  - WireGuard
  - Podman mesh networks
  - Corrosion config/schema/service
  - coold install/service
  - builder install/config
  - default-deny firewall service
- Coolify/Laravel dev owns Flux-specific wiring:
  - starts Flux inside the Coolify container
  - mints dev host JWTs
  - installs `/etc/coolify/host-jwt` into each VM
  - adds a `coold.service` systemd drop-in with `COOLIFY_COOLD_FLUX_URL`

## Main commands

```bash
scripts/dev.sh up
scripts/dev.sh down
scripts/dev.sh clean-vms
scripts/dev.sh list
```

`clean-vms` deletes the Lima VM instances and VM-local state, including disks,
containers, WireGuard keys, Corrosion DB, installed binaries, and firewall state.
It does not delete the Coolify repo.

## coolify helper

```bash
scripts/dev.sh coolify install
scripts/dev.sh coolify path
scripts/dev.sh coolify bootstrap-command
scripts/dev.sh coolify run <args>
```

On macOS, the helper builds `coolify` from the local coold repo because the
nightly release currently publishes Linux binaries. On Linux, it downloads the
nightly release artifact.

The generated bootstrap command uses Lima's forwarded SSH ports and dev
WireGuard endpoint overrides, for example:

```bash
.dev/bin/coolify init bootstrap \
  --nodes "127.0.0.1:<ssh1>,127.0.0.1:<ssh2>" \
  --ssh-key "$HOME/.lima/_config/user" \
  --ssh-user "$USER" \
  --wg-listen-port-overrides "127.0.0.1:<ssh1>=51821,127.0.0.1:<ssh2>=51822" \
  --wg-endpoint-overrides "127.0.0.1:<ssh1>=host.lima.internal:51821,127.0.0.1:<ssh2>=host.lima.internal:51822" \
  --coold-version "nightly" \
  --corrosion-version "v1.0.0" \
  --yes
```

Lima does not allow direct root SSH by default, so dev uses the normal Lima user
with passwordless sudo. `coolify` wraps remote commands in `sudo -n bash -lc`
when the SSH user is not `root`.

## Default dev topology

After `coolify init bootstrap`, defaults are:

| VM | WireGuard IP | WireGuard endpoint | Podman subnet | Gateway |
| --- | --- | --- | --- | --- |
| `coold-dev` | `100.64.0.1` | `host.lima.internal:51821` | `10.210.0.0/24` | `10.210.0.1` |
| `coold-dev-2` | `100.64.0.2` | `host.lima.internal:51822` | `10.210.1.0/24` | `10.210.1.1` |

## Checking state

```bash
scripts/dev.sh corrosion check
scripts/dev.sh corrosion containers
scripts/dev.sh corrosion config
scripts/dev.sh corrosion logs 1
scripts/dev.sh corrosion logs 2
```

`corrosion containers` shows both Corrosion `service_endpoints` and rootful /
rootless Podman containers.

## Example nginx containers

```bash
scripts/dev.sh example-nginx up
scripts/dev.sh example-nginx check-dns
scripts/dev.sh example-nginx down
```

The example containers are started with coold DNS:

```bash
--dns <local-mesh-gateway>
--dns-search default.coolify.internal
```

Expected service discovery format:

```text
<container-name>.default.coolify.internal
```

Example:

```text
coolify-example-nginx-2.default.coolify.internal -> 10.210.1.x
```

## Firewall behavior

The mesh firewall is default-deny for inter-container traffic. Host-to-host
WireGuard traffic can work while container-to-container traffic is blocked.

Allow traffic:

```bash
scripts/dev.sh firewall allow 10.210.0.2 10.210.1.2 tcp 80
scripts/dev.sh firewall list
```

Revoke traffic:

```bash
scripts/dev.sh firewall revoke
scripts/dev.sh firewall revoke <rule-id>
```

### Why dev adds allow rules to both hosts

`business traffic` from container A on host 1 to container B on host 2 crosses
forwarding/firewall logic on both sides:

```text
10.210.0.2
  -> source host bridge/firewall
  -> coold-dev wg0
  -> WireGuard
  -> coold-dev-2 wg0
  -> destination host bridge/firewall
  -> 10.210.1.2
```

The default-deny hooks can drop the packet on either the source or destination
host. For the two-node dev setup, `scripts/dev.sh firewall allow` writes the
same allow tuple to every coold VM so the path works reliably.

Production should become topology-aware instead of blindly writing to every
host:

- cross-host traffic: write allow rules to the source and destination hosts
- same-host traffic: write the allow rule only to that host
- larger clusters: do not install unrelated allow rules on unaffected hosts

## Manual connectivity checks

Before allow rule, this should time out:

```bash
scripts/dev.sh shell 1
sudo podman exec coolify-example-nginx wget -T 3 -qO- http://10.210.1.2
```

After allow rule, this should return nginx HTML:

```bash
scripts/dev.sh firewall allow 10.210.0.2 10.210.1.2 tcp 80
scripts/dev.sh shell 1
sudo podman exec coolify-example-nginx wget -T 5 -qO- http://10.210.1.2 | head -n 1
```

Expected:

```html
<!DOCTYPE html>
```