feat(v5): add server reconciliation and canvas APIs

Split V5 dashboard behavior into domain controllers and policies,
add agent token rotation/revocation, status reconciliation jobs,
ingress firewall syncing, and canvas connection APIs.

Add migrations for V5 status tracking, server capabilities, resource
connection aliases, and revoked agent tokens.
This commit is contained in:
Andras Bacsai
2026-07-06 17:40:37 +02:00
parent df854feee8
commit 6ae45684f9
147 changed files with 22444 additions and 10407 deletions
+64
View File
@@ -0,0 +1,64 @@
# V5 Architecture Fix Plan
Source: /Users/heyandras/.claude/plans/what-do-you-think-soft-firefly.md
## Wave 1 (parallel) — DONE
- [x] 1. Split DashboardController into domain controllers + Laravel policies (denyAsNotFound), dedupe cluster serializer
- [x] 6. Frontend: extract Dashboard.tsx components, useCallback/memo, unified optimistic rollback, use-pending-ids reuse, mid-drag snap-back fix, types.ts drift, env-scoped merge
- [x] 5. Hot-path index migration (wireguard_management_ip, node_address, host, runtime_container_id, last_seen_at)
## Wave 2 (parallel, after wave 1) — DONE
- [x] 2. Status enums (ApplicationStatus/ServerStatus/IngressStatus/ContainerState) + observed_at ordered ingestion
- [x] 3. Reconcile + prune scheduled jobs (V5ReconcileServersJob every 5m + per-server V5ReconcileServerStateJob, 24h container-status prune)
- [x] 4. Job uniqueness (ShouldBeUnique deploy+bootstrap) + queued broadcasts (ShouldBroadcast, afterCommit, null-safe payloads)
- [x] 7. Laravel↔coold verb handshake: UnsupportedCooldVerb detection (flux 501), graceful ingress degradation, coold_version persisted
## Wave 3 (everything else) — DONE
- [x] Morph map (v5.application alias) + uuid collision retry + drop per-insert Schema::hasColumn + defaults dedup
- [x] v5_servers.uuid non-null; capabilities → indexed has_coold/is_ingress booleans (wire format preserved)
- [x] Firewall vs DB atomicity (DB=desired state, flux converge, compensating rollback; revoke-first destroy)
- [x] Deploy failure compensation (stop+force-remove orphaned container, original error preserved)
- [x] Caddyfile hostname/port validation + ValidHostname newline-bypass fix
- [x] Ambiguous host_id resolution warning
## Wave 4 — DONE
- [x] Full V5 suite: 262 passed (1901 assertions); tsc clean; npm build ok; pint clean
## Wave 5 (deep dives)
- [ ] Clusters.tsx + remaining frontend audit
- [ ] coold/flux Rust internals + security audit
- [ ] V5 test quality/coverage audit
## Skipped (product decisions, documented)
- Soft deletes on infra rows (changes cascade semantics — needs product call)
- TLS in v5 ingress (feature, not fix)
- config coold.php/flux.php merge (cosmetic)
## Wave 5 (deep dives) — DONE
- [x] coold/flux Rust audit → findings reported (NOT fixed — separate repo, see session recap: no-TLS gRPC, wildcard cap profiles, lost status updates on outage, exec exit_code always 0, mount-allowlist gaps, unauthenticated Corrosion gossip)
- [x] Frontend audit → all MUST/SHOULD-FIX applied (stale connections on env switch, deleteCluster shadow null-deref, persistSelection ok-guard, useTeamChannel extraction, apiRequest timeouts in Clusters, echo logging gated)
- [x] Test-quality audit → all applied (shared V5TestSchema helper killed schema drift, DashboardTest 174-test monolith split into 12 files, substring tests quarantined in V5FrontendSourceContractTest, +20 new tests: policies, RemoveBootstrapMarker, broadcast payloads, channel auth)
## Wave 6 (audit fixes) — DONE
- [x] v4/v5 currentTeam session cross-contamination (full Team model, write-on-change only)
- [x] flux_url preflight 422 before bootstrap dispatch
- [x] Bootstrap marker/coold_version ordering
- [x] Enum literals sweep (jobs + StopCaddyIngress)
- [x] ManagesConnectionFirewallRules + SerializesResourceConnections → app/Support/V5 classes
## Final state
289 V5 tests passed (2005 assertions) + 333 v4 unit slice green; tsc clean; npm build ok; pint clean. Nothing committed.
## Wave 7 (security + JWT, cut off by session limit, then recovered) — DONE
- [x] JWT: mint explicit 21-primitive caps (config flux.host_capabilities), NOT the host-agent:default wildcard that flux treats as authorize-all; escape-hatch profile config; jti claim + persisted agent_token_jti; kid header; TTL 24h→1h (config); RevokedAgentToken model + migration + isRevoked API; inbound bearer array (laravel_api_tokens) for rotation
- [x] Authz: V5 policies role-gate mutations via isAdminOfTeam (403), keep denyAsNotFound (404) for cross-team; ClusterController::store authorize
- [x] Input: ValidServerIp rejects private/reserved ranges behind config('coold.allow_private_server_ips'); error-detail leak → generic messages + Log::warning; throttle:v5 limiter (RouteServiceProvider)
- [x] Stability: reconcile+refresh honor/advance status_observed_at (shared StatusObservation); Configured + full podman states in enums; deploy persists runtime_container_id after create; reconcile jobs on v5-reconcile queue; status_message churn fixed
- [x] Team-delete teardown: Team::deleting → V5TeardownTeamJob (best-effort per-server container/ingress/marker teardown, self-contained payload)
## Wave 7 recovery fix (post-cutoff)
- [x] FATAL: V5ReconcileServersJob + V5ReconcileServerStateJob redeclared `public $queue = 'v5-reconcile'` — incompatible with Queueable trait's `public $queue;` on PHP 8.5 → hard fatal crashing BOTH pest suite and `php artisan test` bootstrap (job discovery). Moved queue assignment to onQueue() in constructor.
- [x] Stale test: ResourceConnectionControllerTest asserted old snapshot-fail detail; scenario hits the restore path → updated to "The previous rules were restored." (correct behavior)
## Final state (Wave 7)
322 V5 tests passed (2124 assertions) via BOTH vendor/bin/pest AND php artisan test; v4 slice 308 passed; tsc clean; npm build ok; pint clean.
@@ -2,8 +2,12 @@
namespace App\Actions\V5\Application;
use App\Enums\V5\ApplicationStatus;
use App\Enums\V5\ContainerState;
use App\Enums\V5\ServerStatus;
use App\Models\V5\Application;
use App\Services\Flux\FluxClient;
use Illuminate\Support\Facades\Log;
use Lorisleiva\Actions\Concerns\AsAction;
class DeployNginxApplication
@@ -21,34 +25,91 @@ class DeployNginxApplication
return $this->markFailed($application, 'No server is attached to this application.');
}
$hostId = $server->wireguard_management_ip ?: $server->node_address ?: $server->host;
if ($server->status !== ServerStatus::Installed->value || $server->last_bootstrapped_at === null) {
return $this->markFailed($application, "Bootstrap server {$server->name} before deploying to it.");
}
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
return $this->markFailed($application, 'No Flux host ID is available for this server.');
}
$containerId = null;
try {
$this->fluxClient->pullImage($hostId, $application->image);
$containerId = $this->fluxClient->createContainer($hostId, $this->containerSpec($application));
// Persist the runtime id the instant the container exists, before
// start/inspect. A worker SIGKILL at the job timeout would otherwise
// orphan a created container whose id only lived in this local var,
// leaving failed()/reconcile unable to find and clean it by id.
$application->update([
'status' => ApplicationStatus::Created->value,
'status_message' => 'Container created.',
'runtime_container_id' => $containerId,
]);
$this->fluxClient->startContainer($hostId, $containerId);
$inspect = $this->fluxClient->inspectContainer($hostId, $containerId);
if (! $this->isContainerRunning($inspect)) {
$this->cleanUpContainer($application, $hostId, $containerId);
return $this->markFailed($application, 'Container did not stay running.');
}
$application->update([
'status' => 'running',
'status' => ApplicationStatus::Running->value,
'status_message' => 'Container started.',
'runtime_container_id' => $containerId,
]);
return $application->refresh()->load('server');
} catch (\Throwable $e) {
if (is_string($containerId) && $containerId !== '') {
$this->cleanUpContainer($application, $hostId, $containerId);
}
return $this->markFailed($application, $e->getMessage());
}
}
/**
* Best-effort compensation for a failed deploy: stop and force-remove the
* container this run created so it is never left orphaned on the node, then
* null the runtime id we persisted right after create so a cleaned-up
* failure never leaves a dangling id that reconcile would try to reap.
* Cleanup failures only log a warning and never mask the original error.
*/
private function cleanUpContainer(Application $application, string $hostId, string $containerId): void
{
try {
$this->fluxClient->stopContainer($hostId, $containerId);
} catch (\Throwable $e) {
Log::warning('Could not stop the container created by a failed v5 deploy.', [
'application_id' => $application->getKey(),
'container_id' => $containerId,
'error' => $e->getMessage(),
]);
}
try {
$this->fluxClient->removeContainer($hostId, $containerId, force: true);
} catch (\Throwable $e) {
Log::warning('Could not remove the container created by a failed v5 deploy.', [
'application_id' => $application->getKey(),
'container_id' => $containerId,
'error' => $e->getMessage(),
]);
}
if ($application->runtime_container_id === $containerId) {
$application->update(['runtime_container_id' => null]);
}
}
/**
* @return array<string, mixed>
*/
@@ -92,13 +153,13 @@ class DeployNginxApplication
return true;
}
return is_string($inspect['state'] ?? null) && $inspect['state'] === 'running';
return is_string($inspect['state'] ?? null) && $inspect['state'] === ContainerState::Running->value;
}
private function markFailed(Application $application, string $message): Application
{
$application->update([
'status' => 'failed',
'status' => ApplicationStatus::Failed->value,
'status_message' => str($message)->limit(10000)->toString(),
]);
@@ -2,10 +2,18 @@
namespace App\Actions\V5\Flux;
use App\Enums\V5\ApplicationStatus;
use App\Enums\V5\ContainerState;
use App\Enums\V5\IngressStatus;
use App\Enums\V5\ServerStatus;
use App\Models\V5\Application as V5Application;
use App\Models\V5\ContainerStatus;
use App\Models\V5\Server as V5Server;
use App\Support\V5\StatusObservation;
use Carbon\CarbonImmutable;
use Carbon\CarbonInterface;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\Log;
use Lorisleiva\Actions\Concerns\AsAction;
class ApplyFluxResourceStatusUpdate
@@ -37,7 +45,7 @@ class ApplyFluxResourceStatusUpdate
*/
private function upsertContainerStatus(array $payload): ?ContainerStatus
{
$status = $this->status($payload);
$status = $this->status($payload, ContainerState::class);
$containerId = $this->stringValue($payload, 'container_id') ?? $this->stringValue($payload, 'runtime_container_id');
$server = $this->findServer($payload);
@@ -45,17 +53,36 @@ class ApplyFluxResourceStatusUpdate
return null;
}
ContainerStatus::query()->updateOrCreate([
$observedAt = $this->observedAt($payload);
$existing = ContainerStatus::query()
->where('server_id', $server->id)
->where('container_id', $containerId)
->first();
if ($this->isStaleObservation($observedAt, $existing?->status_observed_at, 'container status', [
'server_id' => $server->id,
'container_id' => $containerId,
], [
])) {
return $existing;
}
$attributes = [
'team_id' => $server->team_id,
'container_name' => $this->stringValue($payload, 'container_name') ?? $this->stringValue($payload, 'name'),
'image' => $this->stringValue($payload, 'image'),
'status' => $status,
'status_message' => $this->statusMessage($payload, 'Container state received from coold.'),
'last_seen_at' => now(),
]);
];
if ($observedAt !== null) {
$attributes['status_observed_at'] = $observedAt;
}
ContainerStatus::query()->updateOrCreate([
'server_id' => $server->id,
'container_id' => $containerId,
], $attributes);
return ContainerStatus::query()
->where('server_id', $server->id)
@@ -68,7 +95,7 @@ class ApplyFluxResourceStatusUpdate
*/
private function updateApplication(array $payload): ?V5Application
{
$status = $this->status($payload);
$status = $this->status($payload, ApplicationStatus::class);
if ($status === null) {
return null;
@@ -80,13 +107,39 @@ class ApplyFluxResourceStatusUpdate
return null;
}
$application->update([
$observedAt = $this->observedAt($payload);
if ($this->isStaleObservation($observedAt, $application->status_observed_at, 'application status', [
'application_id' => $application->id,
])) {
return $application;
}
$payloadContainerId = $this->stringValue($payload, 'runtime_container_id')
?? $this->stringValue($payload, 'container_id');
// Payloads may carry no timestamp, so the container id remains an
// ordering signal as a second layer: an update for a superseded
// container is stale and must not overwrite the current one's state.
if (
$payloadContainerId !== null
&& $application->runtime_container_id !== null
&& $payloadContainerId !== $application->runtime_container_id
) {
return $application;
}
$attributes = [
'status' => $status,
'status_message' => $this->statusMessage($payload, 'Status updated by flux.'),
'runtime_container_id' => $this->stringValue($payload, 'runtime_container_id')
?? $this->stringValue($payload, 'container_id')
?? $application->runtime_container_id,
]);
'runtime_container_id' => $payloadContainerId ?? $application->runtime_container_id,
];
if ($observedAt !== null) {
$attributes['status_observed_at'] = $observedAt;
}
$application->update($attributes);
return $application->refresh();
}
@@ -96,7 +149,7 @@ class ApplyFluxResourceStatusUpdate
*/
private function updateServer(array $payload): ?V5Server
{
$status = $this->status($payload);
$status = $this->status($payload, ServerStatus::class);
if ($status === null) {
return null;
@@ -108,22 +161,40 @@ class ApplyFluxResourceStatusUpdate
return null;
}
$server->update([
$observedAt = $this->observedAt($payload);
if ($this->isStaleObservation($observedAt, $server->status_observed_at, 'server status', [
'server_id' => $server->id,
])) {
return $server;
}
$attributes = [
'status' => $status,
'last_status_check' => 'flux',
'last_status_output' => $this->statusMessage($payload, 'Status updated by flux.'),
'last_status_checked_at' => now(),
]);
];
if ($observedAt !== null) {
$attributes['status_observed_at'] = $observedAt;
}
$server->update($attributes);
return $server->refresh();
}
/**
* The ingress state shares the server row but describes a different
* resource, so it deliberately does not read or write the server's
* `status_observed_at` watermark.
*
* @param array<string, mixed> $payload
*/
private function updateCaddyIngress(array $payload): ?V5Server
{
$status = $this->status($payload);
$status = $this->status($payload, IngressStatus::class);
if ($status === null) {
return null;
@@ -151,13 +222,17 @@ class ApplyFluxResourceStatusUpdate
*/
private function findApplication(array $payload): ?V5Application
{
$query = V5Application::query()->with('server');
$server = $this->findServer($payload);
if ($server instanceof V5Server) {
$query->where('server_id', $server->id);
if (! $server instanceof V5Server) {
return null;
}
$query = V5Application::query()
->with('server')
->where('server_id', $server->id)
->where('team_id', $server->team_id);
$applicationUuid = $this->stringValue($payload, 'application_uuid') ?? $this->stringValue($payload, 'resource_uuid');
if ($applicationUuid !== null) {
@@ -211,21 +286,62 @@ class ApplyFluxResourceStatusUpdate
return null;
}
return V5Server::query()
->where('wireguard_management_ip', $hostId)
->orWhere('node_address', $hostId)
->orWhere('host', $hostId)
->first();
$matches = V5Server::query()
->where('uuid', $hostId)
->limit(2)
->get();
if ($matches->count() > 1) {
Log::warning('Dropping flux resource status update: host id matches multiple v5 servers.', [
'host_id' => $hostId,
'server_ids' => $matches->pluck('id')->all(),
]);
return null;
}
return $matches->first();
}
/**
* Map the raw payload status onto the given status enum. Unknown values
* are never written to the database: they fall back to the enum's
* Unknown case and are logged.
*
* @param array<string, mixed> $payload
* @param class-string<ApplicationStatus|ContainerState|IngressStatus|ServerStatus> $enumClass
*/
private function status(array $payload, string $enumClass): ?string
{
$raw = $this->stringValue($payload, 'status') ?? $this->stringValue($payload, 'state');
return StatusObservation::normalize($raw, $enumClass);
}
/**
* @param array<string, mixed> $payload
*/
private function status(array $payload): ?string
private function observedAt(array $payload): ?CarbonInterface
{
$status = $this->stringValue($payload, 'status') ?? $this->stringValue($payload, 'state');
$observedAt = $this->stringValue($payload, 'observed_at');
return $status === null ? null : strtolower($status);
if ($observedAt === null) {
return null;
}
return rescue(fn (): CarbonImmutable => CarbonImmutable::parse($observedAt), null, false);
}
/**
* A payload that carries an observation timestamp older than the one
* already persisted is stale (delivered out of order) and must not
* clobber the newer state.
*
* @param array<string, mixed> $logContext
*/
private function isStaleObservation(?CarbonInterface $observedAt, ?CarbonInterface $currentObservedAt, string $context, array $logContext): bool
{
return StatusObservation::isStale($observedAt, $currentObservedAt, $context, $logContext);
}
/**
@@ -247,18 +363,4 @@ class ApplyFluxResourceStatusUpdate
return is_string($value) && $value !== '' ? $value : null;
}
/**
* @param array<string, mixed> $payload
*/
private function intValue(array $payload, string $key): ?int
{
$value = data_get($payload, $key);
if (is_int($value)) {
return $value;
}
return is_string($value) && ctype_digit($value) ? (int) $value : null;
}
}
@@ -5,6 +5,7 @@ namespace App\Actions\V5\Proxy;
use App\Models\V5\Application;
use App\Models\V5\ApplicationDomain;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Log;
use Lorisleiva\Actions\Concerns\AsAction;
use Symfony\Component\Yaml\Yaml;
@@ -12,6 +13,14 @@ class GenerateCaddyIngressConfiguration
{
use AsAction;
/**
* Strict RFC 1123 hostname: dot-separated alphanumeric labels with inner
* hyphens, max 253 characters. Anchored with \A/\z (never $) so values
* containing newlines, braces, quotes, whitespace, or control characters
* can never inject extra directives into the generated Caddyfile.
*/
private const HOSTNAME_PATTERN = '/\A(?=.{1,253}\z)[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?(?:\.[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?)*\z/i';
/**
* @param Collection<int, Application>|null $applications
* @return array{compose: string, caddyfile: string, apps: array<int, array{name: string, caddyfile: string}>}
@@ -92,12 +101,42 @@ CADDY;
private function applicationRoute(Application $application, ApplicationDomain $domain): ?string
{
if ($domain->domain === '') {
if ($domain->domain === null || $domain->domain === '') {
return null;
}
$namespace = $application->mesh_namespace ?: 'default';
$upstream = "{$application->container_name}.{$namespace}.coolify.internal:{$application->internal_port}";
$internalPort = (int) $application->internal_port;
if (! $this->isSafeHostname($domain->domain)) {
Log::warning('Skipping a caddy ingress route with an unsafe domain.', [
'application_id' => $application->getKey(),
'domain' => $domain->domain,
]);
return null;
}
if (! $this->isSafeHostname($application->container_name) || ! $this->isSafeHostname($namespace)) {
Log::warning('Skipping a caddy ingress route with an unsafe container name or namespace.', [
'application_id' => $application->getKey(),
'container_name' => $application->container_name,
'namespace' => $namespace,
]);
return null;
}
if ($internalPort < 1 || $internalPort > 65535) {
Log::warning('Skipping a caddy ingress route with an out-of-range internal port.', [
'application_id' => $application->getKey(),
'internal_port' => $application->internal_port,
]);
return null;
}
$upstream = "{$application->container_name}.{$namespace}.coolify.internal:{$internalPort}";
return implode("\n", [
"http://{$domain->domain} {",
@@ -106,6 +145,11 @@ CADDY;
]);
}
private function isSafeHostname(mixed $value): bool
{
return is_string($value) && preg_match(self::HOSTNAME_PATTERN, $value) === 1;
}
private function appFileName(Application $application): string
{
return 'app_'.$application->getKey();
+31 -11
View File
@@ -2,17 +2,19 @@
namespace App\Actions\V5\Proxy;
use App\Exceptions\V5\UnsupportedCooldVerb;
use App\Models\V5\Application;
use App\Models\V5\Server;
use App\Services\Flux\FluxClient;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Log;
use Lorisleiva\Actions\Concerns\AsAction;
class StartCaddyIngress
{
use AsAction;
private const FIREWALL_PORTS = [80, 443];
private const FIREWALL_PORTS = [80];
public function __construct(private readonly FluxClient $fluxClient) {}
@@ -22,30 +24,48 @@ class StartCaddyIngress
return 'Server is not an ingress server.';
}
$hostId = $server->wireguard_management_ip ?: $server->node_address;
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
return 'Server is missing its Flux host id.';
throw new \RuntimeException('Server is missing its Flux host id.');
}
$configuration = GenerateCaddyIngressConfiguration::run($this->applications($server));
$output = $this->fluxClient->applyIngress($hostId, 'caddy', $configuration['caddyfile'], $this->ingressApps($configuration['apps']));
$firewallWarning = null;
foreach (self::FIREWALL_PORTS as $port) {
$this->fluxClient->applyFirewallRule($hostId, [
'id' => "v5-caddy-ingress:{$port}",
'namespace' => 'default',
'src' => '0.0.0.0/0',
'dst' => 'coolify-v5-caddy',
'proto' => 'tcp',
'port' => $port,
]);
try {
$this->fluxClient->applyFirewallRule($hostId, [
'id' => "v5-caddy-ingress:{$port}",
'namespace' => 'default',
'src' => '0.0.0.0/0',
'dst' => 'coolify-v5-caddy',
'proto' => 'tcp',
'port' => $port,
]);
} catch (UnsupportedCooldVerb $exception) {
$firewallWarning = "Caddy ingress is running, but this node's coold does not support {$exception->verb}, so the managed firewall was not updated for port {$port}.";
Log::warning('V5 caddy ingress firewall rule skipped: coold verb unsupported', [
'server_id' => $server->id,
'port' => $port,
'verb' => $exception->verb,
'message' => $exception->getMessage(),
]);
break;
}
}
if ($server->exists) {
$server->update([
'ingress_type' => 'caddy',
'ingress_status' => 'running',
...($firewallWarning === null ? [] : [
'last_status_check' => 'flux',
'last_status_output' => $firewallWarning,
]),
]);
}
+36 -8
View File
@@ -2,36 +2,64 @@
namespace App\Actions\V5\Proxy;
use App\Enums\V5\IngressStatus;
use App\Exceptions\V5\UnsupportedCooldVerb;
use App\Models\V5\Server;
use App\Services\Flux\FluxClient;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
use Lorisleiva\Actions\Concerns\AsAction;
class StopCaddyIngress
{
use AsAction;
private const FIREWALL_PORTS = [80, 443];
private const FIREWALL_PORTS = [80];
public function __construct(private readonly FluxClient $fluxClient) {}
public function handle(Server $server): string
{
$hostId = $server->wireguard_management_ip ?: $server->node_address;
if (! $server->isIngress() && $server->ingress_type === null) {
return 'Server is not an ingress server.';
}
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
return 'Server is missing its Flux host id.';
throw new \RuntimeException('Server is missing its Flux host id.');
}
// Revoke first: if stopping the container fails the allow rules must not
// stay orphaned on the host.
foreach (self::FIREWALL_PORTS as $port) {
$this->revokeFirewallRuleIfPresent($hostId, "v5-caddy-ingress:{$port}");
}
$output = $this->fluxClient->stopIngress($hostId, 'caddy');
foreach (self::FIREWALL_PORTS as $port) {
$this->fluxClient->revokeFirewallRule($hostId, "v5-caddy-ingress:{$port}");
}
if ($server->exists) {
$server->update(['ingress_status' => 'exited']);
$server->update(['ingress_status' => IngressStatus::Exited->value]);
}
return $output;
}
private function revokeFirewallRuleIfPresent(string $hostId, string $ruleId): void
{
try {
$this->fluxClient->revokeFirewallRule($hostId, $ruleId);
} catch (UnsupportedCooldVerb $exception) {
Log::warning('V5 caddy ingress firewall revoke skipped: coold verb unsupported', [
'host_id' => $hostId,
'rule_id' => $ruleId,
'verb' => $exception->verb,
'message' => $exception->getMessage(),
]);
} catch (\RuntimeException $exception) {
if (! str_contains(Str::lower($exception->getMessage()), 'not found')) {
throw $exception;
}
}
}
}
@@ -0,0 +1,101 @@
<?php
namespace App\Actions\V5\Server;
use App\Models\PrivateKey;
use App\Models\V5\Server;
use Illuminate\Support\Facades\Process;
use Lorisleiva\Actions\Concerns\AsAction;
class PushHostAgentToken
{
use AsAction;
/**
* Best-effort SSH push of a freshly minted host JWT to the on-host jwt path.
*
* coold re-reads the JWT file on every reconnect and flux drops the stream
* at the token's exp, so overwriting the file in place is enough for the
* next reconnect to pick up the new token coold is intentionally NOT
* restarted here (a restart would force an unnecessary disconnect of a
* stream that is still valid on the current token).
*
* Mirrors V5BootstrapServerJob::enrollCooldIntoFlux for the write mechanics
* (printf %s <token> | sudo tee <path>; chmod 600) and RemoveBootstrapMarker
* for the SSH/temp-key mechanics. Returns whether the write succeeded;
* every failure path (missing key, SSH error, exception) resolves to false
* and always cleans up the temporary key file.
*/
public function handle(Server $server, string $token): bool
{
$server->loadMissing('privateKey');
if (! $server->privateKey instanceof PrivateKey) {
return false;
}
$jwtPath = trim((string) config('coold.flux_host_jwt_path', '/etc/coolify/host-jwt'));
if ($jwtPath === '') {
$jwtPath = '/etc/coolify/host-jwt';
}
$jwtPath = str_replace(["\r", "\n"], '', $jwtPath);
$token = str_replace(["\r", "\n"], '', $token);
$keyDirectory = storage_path('app/ssh/keys');
if (! is_dir($keyDirectory)) {
mkdir($keyDirectory, 0700, true);
}
$keyLocation = tempnam($keyDirectory, 'v5_ssh_key_');
if ($keyLocation === false) {
return false;
}
file_put_contents($keyLocation, $server->privateKey->private_key);
chmod($keyLocation, 0600);
$tokenArgument = escapeshellarg($token);
$jwtPathArgument = $this->shellPathArg($jwtPath);
$script = <<<SH
set -e
SUDO=''
if [ "\$(id -u)" != "0" ]; then SUDO='sudo'; fi
\$SUDO mkdir -p /etc/coolify
printf %s {$tokenArgument} | \$SUDO tee {$jwtPathArgument} >/dev/null
\$SUDO chmod 600 {$jwtPathArgument}
SH;
try {
$result = Process::timeout(30)->run([
'ssh',
'-o', 'BatchMode=yes',
'-o', 'LogLevel=ERROR',
'-o', 'StrictHostKeyChecking=no',
'-o', 'UserKnownHostsFile=/dev/null',
'-o', 'ConnectTimeout=10',
'-o', 'IdentitiesOnly=yes',
'-i', $keyLocation,
'-p', (string) $server->ssh_port,
"{$server->ssh_user}@{$server->host}",
$script,
]);
return $result->successful();
} catch (\Throwable) {
return false;
} finally {
@unlink($keyLocation);
}
}
private function shellPathArg(string $value): string
{
if (preg_match('/^[A-Za-z0-9_\/:.,@%+=-]+$/', $value) === 1) {
return $value;
}
return escapeshellarg($value);
}
}
@@ -0,0 +1,75 @@
<?php
namespace App\Actions\V5\Server;
use App\Models\PrivateKey;
use App\Models\V5\Server;
use App\Services\Flux\AgentTokenIssuer;
use Illuminate\Support\Facades\Process;
use Lorisleiva\Actions\Concerns\AsAction;
class RemoveBootstrapMarker
{
use AsAction;
/**
* Best-effort removal of the on-host bootstrap identity (marker, host JWT and
* Flux drop-in) so a re-added server can never silently adopt stale state.
*
* The host token jti is revoked first (a local DB write plus a best-effort
* push to the flux revocation store) so a captured or pre-copied token is
* recorded revoked even when the host is unreachable see
* AgentTokenIssuer::revoke.
*/
public function handle(Server $server): bool
{
app(AgentTokenIssuer::class)->revoke($server);
$server->loadMissing('privateKey');
if (! $server->privateKey instanceof PrivateKey) {
return false;
}
$keyDirectory = storage_path('app/ssh/keys');
if (! is_dir($keyDirectory)) {
mkdir($keyDirectory, 0700, true);
}
$keyLocation = tempnam($keyDirectory, 'v5_ssh_key_');
if ($keyLocation === false) {
return false;
}
file_put_contents($keyLocation, $server->privateKey->private_key);
chmod($keyLocation, 0600);
$script = implode("\n", [
"SUDO=''",
'if [ "$(id -u)" != "0" ]; then SUDO=\'sudo\'; fi',
'$SUDO rm -f /etc/coolify/v5-node.json /etc/coolify/host-jwt /etc/systemd/system/coold.service.d/10-flux.conf',
]);
try {
$result = Process::timeout(15)->run([
'ssh',
'-o', 'BatchMode=yes',
'-o', 'LogLevel=ERROR',
'-o', 'StrictHostKeyChecking=no',
'-o', 'UserKnownHostsFile=/dev/null',
'-o', 'ConnectTimeout=10',
'-o', 'IdentitiesOnly=yes',
'-i', $keyLocation,
'-p', (string) $server->ssh_port,
"{$server->ssh_user}@{$server->host}",
$script,
]);
return $result->successful();
} catch (\Throwable) {
return false;
} finally {
@unlink($keyLocation);
}
}
}
+12 -4
View File
@@ -2,6 +2,7 @@
namespace App\Actions\V5\Server;
use App\Enums\V5\ServerStatus;
use App\Models\PrivateKey;
use App\Models\Team;
use App\Models\User;
@@ -9,6 +10,14 @@ use App\Models\V5\Cluster;
use App\Models\V5\Server;
use Lorisleiva\Actions\Concerns\AsAction;
/**
* Registers local Lima development VMs (provisioned by scripts/dev.sh) as
* cluster servers. They are intentionally seeded as Installed with
* last_bootstrapped_at already set but has_coold=false, so they skip the real
* bootstrap flow by design: V5BootstrapServerJob early-returns on a non-null
* last_bootstrapped_at, and V5ReconcileServersJob ignores them until
* something marks has_coold=true.
*/
class SyncDevLimaServers
{
use AsAction;
@@ -39,8 +48,6 @@ class SyncDevLimaServers
'description' => 'Local Lima development cluster managed by scripts/dev.sh.',
]);
$capabilities = [];
foreach ($servers as $server) {
$wireguardManagementIp = $server['wireguard_management_ip'] ?? null;
$values = [
@@ -49,8 +56,9 @@ class SyncDevLimaServers
'host' => $server['host'],
'ssh_user' => $server['ssh_user'],
'ssh_port' => $server['ssh_port'],
'status' => 'installed',
'capabilities' => $capabilities,
'status' => ServerStatus::Installed->value,
'has_coold' => false,
'is_ingress' => false,
'builder_enabled' => false,
'builder_capacity' => 0,
'node_address' => $wireguardManagementIp ?: $server['host'],
+135
View File
@@ -0,0 +1,135 @@
<?php
namespace App\Console\Commands;
use App\Services\Flux\AgentTokenIssuer;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\File;
/**
* Generates the ES256 (EC P-256) keypair used to authorize coold host agents
* against flux. Laravel signs the per-host JWT with the private key
* (config('flux.jwt_private_key_path')); flux verifies it with the matching
* public key (config('flux.jwt_public_key_path')). Without this keypair a fresh
* install cannot mint host tokens, so this command is a bootstrap prerequisite.
*
* @see AgentTokenIssuer
*/
class V5FluxGenerateKeys extends Command
{
protected $signature = 'v5:flux-generate-keys
{--force : Overwrite an existing private key instead of refusing}
{--show-public : Print the generated public key PEM so it can be provisioned to flux}';
protected $description = 'Generate the ES256 keypair Flux uses to sign and verify coold host agent JWTs.';
public function handle(AgentTokenIssuer $agentTokenIssuer): int
{
$privateKeyPath = (string) config('flux.jwt_private_key_path');
$publicKeyPath = (string) config('flux.jwt_public_key_path');
if ($privateKeyPath === '' || $publicKeyPath === '') {
$this->error('Flux JWT key paths are not configured (flux.jwt_private_key_path / flux.jwt_public_key_path).');
return self::FAILURE;
}
// Idempotent by default: re-running during provisioning must not clobber
// a live key (which would instantly invalidate every host token on
// disk). Refuse unless --force is passed, and exit SUCCESS so a
// provisioning script can call this unconditionally on every deploy.
if (File::exists($privateKeyPath) && ! $this->option('force')) {
$this->warn("A Flux JWT private key already exists at {$privateKeyPath}.");
$this->line('Refusing to overwrite it. Re-run with --force to replace it (this invalidates every host token currently on disk).');
return self::SUCCESS;
}
// curve_name drives the actual EC key (P-256). private_key_bits is
// still validated by PHP's generic length check (>= 384) even though it
// is irrelevant to EC, so it must be present or openssl_pkey_new fails
// with "Private key length must be at least 384 bits, configured to 0".
$keyPair = openssl_pkey_new([
'private_key_type' => OPENSSL_KEYTYPE_EC,
'curve_name' => 'prime256v1',
'private_key_bits' => 384,
]);
if ($keyPair === false) {
$this->error('Failed to generate an EC P-256 keypair: '.openssl_error_string());
return self::FAILURE;
}
$privatePem = '';
if (! openssl_pkey_export($keyPair, $privatePem)) {
$this->error('Failed to export the private key PEM: '.openssl_error_string());
return self::FAILURE;
}
$details = openssl_pkey_get_details($keyPair);
if ($details === false || ! isset($details['key'])) {
$this->error('Failed to read the generated public key PEM.');
return self::FAILURE;
}
$publicPem = (string) $details['key'];
$this->writeKeyFile($privateKeyPath, $privatePem, 0600);
$this->writeKeyFile($publicKeyPath, $publicPem, 0644);
// Self-check: the whole point of this command is that AgentTokenIssuer
// can mint with the key we just wrote. If the format were wrong (e.g.
// not a PEM EC private key Firebase\JWT accepts for ES256) this fails
// loudly here instead of silently at the first real host bootstrap.
try {
$token = $agentTokenIssuer->issue('flux-keygen-selfcheck');
} catch (\Throwable $exception) {
$this->error('Generated a keypair but AgentTokenIssuer could not mint a token with it: '.$exception->getMessage());
return self::FAILURE;
}
if (substr_count($token, '.') !== 2) {
$this->error('Generated key produced a malformed JWT (expected 3 segments).');
return self::FAILURE;
}
$this->info('Generated a fresh ES256 (EC P-256) Flux keypair.');
$this->line(" Private key (0600): {$privateKeyPath}");
$this->line(" Public key (0644): {$publicKeyPath}");
$this->newLine();
$this->line('Provision the PUBLIC key to flux — flux verifies every host JWT with it.');
$this->line('Keep the PRIVATE key secret and on the Laravel host only.');
if ($this->option('show-public')) {
$this->newLine();
$this->line(rtrim($publicPem));
}
return self::SUCCESS;
}
/**
* Write a key file with exact permissions, creating the parent directory at
* 0700 if missing. chmod is applied after the write because umask can
* loosen both the mkdir mode and the created file mode.
*/
private function writeKeyFile(string $path, string $contents, int $mode): void
{
$directory = dirname($path);
if (! is_dir($directory)) {
File::makeDirectory($directory, 0700, true);
@chmod($directory, 0700);
}
File::put($path, $contents);
@chmod($path, $mode);
}
}
+10
View File
@@ -15,6 +15,8 @@ use App\Jobs\RegenerateSslCertJob;
use App\Jobs\ScheduledJobManager;
use App\Jobs\ServerManagerJob;
use App\Jobs\UpdateCoolifyJob;
use App\Jobs\V5ReconcileServersJob;
use App\Jobs\V5RotateAgentTokensJob;
use App\Models\InstanceSettings;
use Illuminate\Console\Scheduling\Schedule;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;
@@ -49,6 +51,14 @@ class Kernel extends ConsoleKernel
$this->scheduleInstance->command('sanctum:prune-expired --hours=1')->hourly()->onOneServer();
$this->scheduleInstance->job(new ApiTokenExpirationWarningJob)->hourly()->onOneServer();
// V5 reconciliation loop: pull-based safety net for the push-only
// coold -> flux -> webhook status pipeline, plus container status pruning.
$this->scheduleInstance->job(new V5ReconcileServersJob)->everyFiveMinutes()->withoutOverlapping()->onOneServer();
// V5 host JWT rotation: re-mints and SSH-pushes a fresh host token
// before the on-disk token expires so coold reconnects stay authorized.
$this->scheduleInstance->job(new V5RotateAgentTokensJob)->hourly()->withoutOverlapping()->onOneServer();
if (isDev()) {
// Instance Jobs
$this->scheduleInstance->command('horizon:snapshot')->everyMinute();
+28
View File
@@ -0,0 +1,28 @@
<?php
namespace App\Enums\V5;
/**
* Lifecycle states persisted on `v5_applications.status`.
*
* Besides Coolify's own states (creating, failed, unknown), the column also
* receives raw container runtime states reported by coold, so the Docker and
* Podman container states are part of the catalog.
*/
enum ApplicationStatus: string
{
case Creating = 'creating';
case Configured = 'configured';
case Created = 'created';
case Starting = 'starting';
case Running = 'running';
case Restarting = 'restarting';
case Paused = 'paused';
case Removing = 'removing';
case Stopping = 'stopping';
case Stopped = 'stopped';
case Exited = 'exited';
case Dead = 'dead';
case Failed = 'failed';
case Unknown = 'unknown';
}
+26
View File
@@ -0,0 +1,26 @@
<?php
namespace App\Enums\V5;
/**
* Runtime states persisted on `v5_container_statuses.status`.
*
* Named ContainerState (not ContainerStatus) to avoid clashing with the
* App\Models\V5\ContainerStatus Eloquent model. Covers the Docker and Podman
* container states reported by coold.
*/
enum ContainerState: string
{
case Configured = 'configured';
case Created = 'created';
case Starting = 'starting';
case Running = 'running';
case Restarting = 'restarting';
case Paused = 'paused';
case Removing = 'removing';
case Stopping = 'stopping';
case Stopped = 'stopped';
case Exited = 'exited';
case Dead = 'dead';
case Unknown = 'unknown';
}
+25
View File
@@ -0,0 +1,25 @@
<?php
namespace App\Enums\V5;
/**
* States persisted on `v5_servers.ingress_status`.
*
* The value mirrors the ingress proxy container's runtime state as reported
* by coold, so the Docker and Podman container states are part of the catalog.
*/
enum IngressStatus: string
{
case Configured = 'configured';
case Created = 'created';
case Starting = 'starting';
case Running = 'running';
case Restarting = 'restarting';
case Paused = 'paused';
case Removing = 'removing';
case Stopping = 'stopping';
case Stopped = 'stopped';
case Exited = 'exited';
case Dead = 'dead';
case Unknown = 'unknown';
}
+15
View File
@@ -0,0 +1,15 @@
<?php
namespace App\Enums\V5;
/**
* Lifecycle states persisted on `v5_servers.status`.
*/
enum ServerStatus: string
{
case Added = 'added';
case Installed = 'installed';
case Failed = 'failed';
case Unreachable = 'unreachable';
case Unknown = 'unknown';
}
+13 -69
View File
@@ -4,16 +4,23 @@ namespace App\Events;
use App\Models\V5\Application as V5Application;
use App\Models\V5\Server as V5Server;
use App\Support\V5\CanvasResourceSerializer;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcastNow;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Queue\SerializesModels;
class V5CanvasResourceUpdated implements ShouldBroadcastNow
class V5CanvasResourceUpdated implements ShouldBroadcast
{
use Dispatchable, InteractsWithSockets, SerializesModels;
/**
* Push the queued broadcast job only after the dispatching database
* transaction commits, so workers never serialize pre-commit state.
*/
public bool $afterCommit = true;
public function __construct(
public int $teamId,
public ?int $applicationId = null,
@@ -38,6 +45,7 @@ class V5CanvasResourceUpdated implements ShouldBroadcastNow
*/
public function broadcastWith(): array
{
$serializer = app(CanvasResourceSerializer::class);
$application = $this->applicationId !== null
? V5Application::query()->with(['server', 'domains'])->find($this->applicationId)
: null;
@@ -52,78 +60,14 @@ class V5CanvasResourceUpdated implements ShouldBroadcastNow
: null;
return [
'application' => $application instanceof V5Application ? $this->serializeApplication($application) : null,
'application' => $application instanceof V5Application ? $serializer->serializeApplication($application) : null,
'applications' => $applications
->map(fn (V5Application $application) => $this->serializeApplication($application))
->map(fn (V5Application $application) => $serializer->serializeApplication($application))
->values()
->all(),
'caddyIngress' => $caddyIngress instanceof V5Server && $caddyIngress->isIngress()
? $this->serializeCaddyIngress($caddyIngress)
? $serializer->serializeCaddyIngress($caddyIngress)
: null,
];
}
/**
* @return array<string, mixed>
*/
private function serializeApplication(V5Application $application): array
{
$server = $application->server;
$isServerReachable = ! $server instanceof V5Server || $this->isServerReachable($server);
return [
'id' => (string) $application->id,
'name' => $application->name,
'image' => $application->image,
'containerName' => $application->container_name,
'status' => $application->status,
'statusMessage' => $application->status_message,
'effectiveStatus' => $isServerReachable ? $application->status : 'unknown',
'effectiveStatusMessage' => $isServerReachable
? $application->status_message
: $this->serverStatusMessage($server),
'runtimeContainerId' => $application->runtime_container_id,
'serverName' => $server?->name,
'serverStatus' => $server?->status,
'serverStatusMessage' => $server instanceof V5Server ? $this->serverStatusMessage($server) : null,
'isServerReachable' => $isServerReachable,
'serverIngressEnabled' => (bool) $server?->isIngress(),
'meshNamespace' => $application->mesh_namespace,
'ingressEnabled' => $application->ingress_enabled,
'internalPort' => $application->internal_port,
'domains' => $application->domains->pluck('domain')->values()->all(),
'meshFqdn' => $application->container_name.'.'.($application->mesh_namespace ?: 'default').'.coolify.internal',
'canvasX' => $application->canvas_x,
'canvasY' => $application->canvas_y,
];
}
private function isServerReachable(V5Server $server): bool
{
return $server->status !== 'unreachable';
}
private function serverStatusMessage(?V5Server $server): ?string
{
return $server?->last_status_output ?: null;
}
/**
* @return array<string, mixed>
*/
private function serializeCaddyIngress(V5Server $server): array
{
$isServerReachable = $this->isServerReachable($server);
return [
'id' => (string) $server->id,
'name' => $server->name,
'host' => $server->host,
'type' => $server->ingressType(),
'status' => $isServerReachable ? $server->ingressStatus() : 'unreachable',
'statusMessage' => $isServerReachable ? null : $this->serverStatusMessage($server),
'canvasX' => $server->canvas_x ?? -352,
'canvasY' => $server->canvas_y ?? 0,
];
}
}
+10 -60
View File
@@ -3,17 +3,23 @@
namespace App\Events;
use App\Models\V5\Cluster as V5Cluster;
use App\Models\V5\Server as V5Server;
use App\Support\V5\ClusterSerializer;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcastNow;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Queue\SerializesModels;
class V5ClusterUpdated implements ShouldBroadcastNow
class V5ClusterUpdated implements ShouldBroadcast
{
use Dispatchable, InteractsWithSockets, SerializesModels;
/**
* Push the queued broadcast job only after the dispatching database
* transaction commits, so workers never serialize pre-commit state.
*/
public bool $afterCommit = true;
public function __construct(public int $teamId, public int $clusterId) {}
public function broadcastOn(): array
@@ -42,63 +48,7 @@ class V5ClusterUpdated implements ShouldBroadcastNow
->find($this->clusterId);
return [
'cluster' => $cluster instanceof V5Cluster ? $this->serializeCluster($cluster) : null,
];
}
/**
* @return array<string, mixed>
*/
private function serializeCluster(V5Cluster $cluster): array
{
return [
'id' => (string) $cluster->id,
'name' => $cluster->name,
'description' => $cluster->description,
'wireguardInterface' => $cluster->wireguard_interface,
'wireguardManagementPool' => $cluster->wireguard_management_pool,
'wireguardListenPort' => $cluster->wireguard_listen_port,
'containerNetworkPool' => $cluster->container_network_pool,
'containerNetworkPrefix' => $cluster->container_network_prefix,
'namespaces' => $cluster->namespaces ?? V5Cluster::DEFAULT_NAMESPACES,
'defaultDenyContainers' => $cluster->default_deny_containers,
'cooldVersion' => $cluster->coold_version,
'corrosionVersion' => $cluster->corrosion_version,
'corrosionGossipPort' => $cluster->corrosion_gossip_port,
'corrosionApiPort' => $cluster->corrosion_api_port,
'builderEnabled' => $cluster->builder_enabled,
'builderCapacity' => $cluster->builder_capacity,
'builderCpuQuota' => $cluster->builder_cpu_quota,
'builderMemoryMax' => $cluster->builder_memory_max,
'builderTimeoutSecs' => $cluster->builder_timeout_secs,
'lastCliAction' => $cluster->last_cli_action,
'lastCliStatus' => $cluster->last_cli_status,
'lastCliSummary' => $cluster->last_cli_summary,
'lastCliRanAt' => $cluster->last_cli_ran_at?->toJSON(),
'serversCount' => $cluster->servers_count ?? $cluster->servers->count(),
'servers' => $cluster->servers->map(fn (V5Server $server) => [
'id' => (string) $server->id,
'name' => $server->name,
'host' => $server->host,
'status' => $server->status,
'capabilities' => $server->capabilities ?? [],
'builderEnabled' => $server->builder_enabled,
'builderCapacity' => $server->builder_capacity,
'builderCpuQuota' => $server->builder_cpu_quota,
'uuid' => $server->uuid,
'nodeAddress' => $server->node_address,
'wireguardListenPortOverride' => $server->wireguard_listen_port_override,
'wireguardEndpointOverride' => $server->wireguard_endpoint_override,
'wireguardManagementIp' => $server->wireguard_management_ip,
'wireguardPublicKey' => $server->wireguard_public_key,
'containerSubnets' => $server->container_subnets ?? [],
'privateKeyName' => $server->privateKey?->name,
'lastBootstrappedAt' => $server->last_bootstrapped_at?->toJSON(),
'lastBootstrapAction' => $server->last_bootstrap_action,
'lastBootstrapStatus' => $server->last_bootstrap_status,
'lastBootstrapOutput' => $server->last_bootstrap_output,
'lastBootstrapRanAt' => $server->last_bootstrap_ran_at?->toJSON(),
])->all(),
'cluster' => $cluster instanceof V5Cluster ? app(ClusterSerializer::class)->serialize($cluster) : null,
];
}
}
+3 -2
View File
@@ -69,8 +69,9 @@ class Handler extends ExceptionHandler
*/
public function render($request, Throwable $e)
{
// Handle authorization exceptions for API routes
if ($e instanceof AuthorizationException) {
// Handle authorization exceptions for API routes. Exceptions carrying
// an explicit status (e.g. denyAsNotFound) keep it via parent::render.
if ($e instanceof AuthorizationException && ! $e->hasStatus()) {
if ($request->is('api/*') || $request->expectsJson()) {
if ($request->is('api/*')) {
auditLog('api.auth.policy_denied', [
@@ -0,0 +1,18 @@
<?php
namespace App\Exceptions\V5;
use RuntimeException;
/**
* The per-node coold agent does not implement the dispatched verb. Flux
* rejects these before they reach the node, so callers can degrade
* gracefully instead of treating the miss as an operational failure.
*/
class UnsupportedCooldVerb extends RuntimeException
{
public function __construct(public readonly string $verb, string $message = '')
{
parent::__construct($message !== '' ? $message : "The node's coold agent does not support the {$verb} verb.");
}
}
@@ -12,9 +12,7 @@ class FluxResourceStatusController extends Controller
{
public function __invoke(Request $request): JsonResponse
{
$token = config('flux.laravel_api_token');
if (! is_string($token) || $token === '' || ! hash_equals($token, (string) $request->bearerToken())) {
if (! $this->authorizedBearer($request)) {
abort(401);
}
@@ -40,6 +38,7 @@ class FluxResourceStatusController extends Controller
'state' => ['required_without:status', 'string', 'max:64'],
'status_message' => ['nullable', 'string', 'max:1000'],
'message' => ['nullable', 'string', 'max:1000'],
'observed_at' => ['nullable', 'string', 'date'],
])->validate();
$resource = ApplyFluxResourceStatusUpdate::run($validated);
@@ -60,4 +59,53 @@ class FluxResourceStatusController extends Controller
'message' => 'Resource status updated.',
]);
}
/**
* Constant-time match the presented bearer token against every accepted
* inbound token. Accepting an array (config('flux.laravel_api_tokens'),
* falling back to the single config('flux.laravel_api_token')) lets an
* operator rotate by serving old+new tokens simultaneously.
*
* SECURITY: this is still a shared global secret every flux instance
* presents the same token, so it cannot be scoped or revoked per-flux, and
* a leak forces a fleet-wide rotation. The target design is per-flux,
* individually rotatable tokens; until then the array support above is the
* mitigation that makes rotation possible without downtime.
*/
private function authorizedBearer(Request $request): bool
{
$presented = (string) $request->bearerToken();
if ($presented === '') {
return false;
}
foreach ($this->acceptedTokens() as $token) {
if (hash_equals($token, $presented)) {
return true;
}
}
return false;
}
/**
* @return array<int, string>
*/
private function acceptedTokens(): array
{
$tokens = config('flux.laravel_api_tokens', []);
$tokens = is_array($tokens) ? $tokens : [];
$single = config('flux.laravel_api_token');
if (is_string($single) && $single !== '') {
$tokens[] = $single;
}
return array_values(array_filter(
array_map(fn ($token): string => is_string($token) ? $token : '', $tokens),
fn (string $token): bool => $token !== ''
));
}
}
@@ -0,0 +1,635 @@
<?php
namespace App\Http\Controllers\V5;
use App\Actions\V5\Application\DestroyNginxApplication;
use App\Actions\V5\Proxy\StartCaddyIngress;
use App\Enums\V5\ApplicationStatus;
use App\Enums\V5\IngressStatus;
use App\Enums\V5\ServerStatus;
use App\Exceptions\V5\UnsupportedCooldVerb;
use App\Http\Controllers\Controller;
use App\Http\Controllers\V5\Concerns\HandlesIngressSyncErrors;
use App\Http\Controllers\V5\Concerns\ResolvesCurrentTeam;
use App\Http\Controllers\V5\Concerns\ResolvesProjectSelection;
use App\Http\Controllers\V5\Concerns\SerializesCanvasResources;
use App\Jobs\V5DeployApplicationJob;
use App\Models\Environment;
use App\Models\Project;
use App\Models\Team;
use App\Models\V5\Application as V5Application;
use App\Models\V5\ApplicationDomain as V5ApplicationDomain;
use App\Models\V5\ResourceConnection;
use App\Models\V5\Server as V5Server;
use App\Rules\ValidHostname;
use App\Services\Flux\FluxClient;
use App\Support\V5\CanvasResourceSerializer;
use App\Support\V5\ConnectionFirewallSync;
use App\Support\V5\StatusObservation;
use Carbon\CarbonImmutable;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
use Illuminate\Http\Response;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
use Illuminate\Validation\Rule;
class ApplicationController extends Controller
{
use HandlesIngressSyncErrors;
use ResolvesCurrentTeam;
use ResolvesProjectSelection;
use SerializesCanvasResources;
private const DEFAULT_NGINX_IMAGE = 'docker.io/library/nginx:alpine';
public function __construct(private readonly ConnectionFirewallSync $firewallSync) {}
public function store(Request $request): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$projects = $this->projects($currentTeam);
[$selectedProject, $selectedEnvironment] = $this->selectedProjectAndEnvironment($request, $projects);
if ($selectedProject === null || $selectedEnvironment === null) {
return response()->json([
'message' => 'Select a project and environment before deploying nginx.',
], 422);
}
$project = $this->projectQuery($currentTeam)
->where('uuid', $selectedProject['uuid'])
->first();
if (! $project instanceof Project) {
abort(403);
}
$environment = $this->selectedEnvironment($project, $selectedEnvironment['uuid']);
if (! $environment instanceof Environment) {
abort(403);
}
$validated = $request->validate([
'server_uuid' => ['nullable', 'string', 'max:255'],
'image' => ['nullable', 'string', 'max:255', 'regex:/^[a-zA-Z0-9][a-zA-Z0-9._\/:@-]*$/'],
]);
$image = trim($validated['image'] ?? '') ?: self::DEFAULT_NGINX_IMAGE;
$server = V5Server::query()
->where('team_id', $currentTeam->id)
->when(
isset($validated['server_uuid']),
fn (Builder $query) => $query->where('uuid', $validated['server_uuid']),
fn (Builder $query) => $query
->orderByRaw('last_bootstrapped_at is null')
->orderBy('name')
)
->first();
if (! $server instanceof V5Server) {
return response()->json([
'message' => 'Add a v5 server before deploying nginx.',
], 422);
}
if ($server->status !== ServerStatus::Installed->value || $server->last_bootstrapped_at === null) {
return response()->json([
'message' => "Bootstrap server {$server->name} before deploying to it.",
], 422);
}
$canvasPosition = $this->nextApplicationCanvasPosition($currentTeam, $project, $environment);
$application = V5Application::query()->create([
'team_id' => $currentTeam->id,
'project_id' => $project->id,
'environment_id' => $environment->id,
'server_id' => $server->id,
'created_by_user_id' => $request->user()->id,
'name' => 'nginx-test',
'image' => $image,
'container_name' => 'coolify-v5-nginx-'.strtolower((string) Str::ulid()),
'status' => ApplicationStatus::Creating->value,
'status_message' => 'Starting nginx container.',
'mesh_namespace' => 'default',
'canvas_x' => $canvasPosition['canvas_x'],
'canvas_y' => $canvasPosition['canvas_y'],
]);
V5DeployApplicationJob::dispatch($application->id);
return response()->json([
'application' => $this->serializeApplication($application),
], 202);
}
public function refresh(Request $request, FluxClient $fluxClient): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$projects = $this->projects($currentTeam);
[$selectedProject, $selectedEnvironment] = $this->selectedProjectAndEnvironment($request, $projects);
if ($selectedProject === null || $selectedEnvironment === null) {
return response()->json([
'message' => 'Select a project and environment before refreshing applications.',
], 422);
}
$applications = $this->applicationQuery($currentTeam, $selectedProject, $selectedEnvironment)
->with('server')
->get();
$errors = [];
$applications
->groupBy('server_id')
->each(function (Collection $serverApplications) use ($fluxClient, &$errors): void {
/** @var V5Application|null $firstApplication */
$firstApplication = $serverApplications->first();
$server = $firstApplication?->server;
$hostId = $server?->fluxHostId();
if (! $server instanceof V5Server || ! is_string($hostId) || $hostId === '') {
$errors[] = 'A server is missing its Flux host id.';
return;
}
// The moment we query coold is the observation time for the rows
// this refresh writes, so a fresher webhook always wins the
// status_observed_at watermark and is never clobbered.
$observedAt = CarbonImmutable::now();
try {
$containers = collect($fluxClient->listContainers($hostId));
} catch (\Throwable $e) {
$errors[] = $e->getMessage();
return;
}
$serverApplications->each(function (V5Application $application) use ($containers, $observedAt): void {
$container = $containers->first(function (array $container) use ($application): bool {
return ($application->runtime_container_id !== null && ($container['id'] ?? null) === $application->runtime_container_id)
|| ($container['name'] ?? null) === $application->container_name;
});
if (! is_array($container)) {
// A creating application without a container id simply has
// not materialized yet; the deploy job will settle it.
if ($application->status === ApplicationStatus::Creating->value && $application->runtime_container_id === null) {
return;
}
if (StatusObservation::isStale($observedAt, $application->status_observed_at, 'application status', ['application_id' => $application->id])) {
return;
}
$application->update([
'status' => ApplicationStatus::Exited->value,
'status_message' => 'Container not found on server.',
'status_observed_at' => $observedAt,
]);
return;
}
if (StatusObservation::isStale($observedAt, $application->status_observed_at, 'application status', ['application_id' => $application->id])) {
return;
}
$rawState = is_string($container['state'] ?? null) && $container['state'] !== '' ? $container['state'] : null;
$application->update([
'status' => StatusObservation::normalize($rawState, ApplicationStatus::class) ?? ApplicationStatus::Unknown->value,
'status_message' => 'Container state refreshed from coold.',
'status_observed_at' => $observedAt,
'runtime_container_id' => is_string($container['id'] ?? null) ? $container['id'] : $application->runtime_container_id,
]);
});
});
V5Server::query()
->where('team_id', $currentTeam->id)
->orderBy('name')
->get()
->filter(fn (V5Server $server) => $server->isIngress())
->each(function (V5Server $server) use ($fluxClient, &$errors): void {
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
$errors[] = "Caddy ingress server {$server->name} is missing its Flux host id.";
return;
}
try {
$containers = collect($fluxClient->listContainers($hostId));
} catch (\Throwable $e) {
$errors[] = $e->getMessage();
return;
}
$container = $containers->first(fn (array $container) => ($container['name'] ?? null) === 'coolify-v5-caddy');
$rawState = is_array($container) && is_string($container['state'] ?? null) && $container['state'] !== '' ? $container['state'] : null;
$state = $rawState !== null
? (StatusObservation::normalize($rawState, IngressStatus::class) ?? IngressStatus::Unknown->value)
: IngressStatus::Exited->value;
$server->update([
'ingress_type' => 'caddy',
'ingress_status' => $state,
'last_status_check' => 'flux',
'last_status_output' => 'Caddy ingress state refreshed from coold.',
'last_status_checked_at' => now(),
]);
});
return response()->json([
'applications' => $this->applicationQuery($currentTeam, $selectedProject, $selectedEnvironment)
->with('server')
->orderBy('created_at')
->get()
->map(fn (V5Application $application) => $this->serializeApplication($application))
->all(),
'caddyIngresses' => $this->caddyIngresses($currentTeam),
'errors' => $errors,
]);
}
public function logs(Request $request, V5Application $application): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('view', [$application, $currentTeam]);
$application->loadMissing('server');
$server = $application->server;
$hostId = $server?->fluxHostId();
$containerId = $application->runtime_container_id;
$logs = null;
$logsError = null;
// A container id only appears once the deploy actually created one; a
// deploy that failed before that (e.g. host not connected) has none, so
// there is nothing to fetch and the frontend just shows the status.
if (is_string($containerId) && $containerId !== '' && $server instanceof V5Server && $server->status !== ServerStatus::Unreachable->value && is_string($hostId) && $hostId !== '') {
try {
$logs = app(FluxClient::class)->containerLogs($hostId, $containerId);
} catch (UnsupportedCooldVerb $exception) {
$logsError = "This node's coold does not support container logs.";
} catch (\RuntimeException $exception) {
Log::warning('V5 application container logs request failed', [
'application_id' => $application->id,
'message' => $exception->getMessage(),
]);
$logsError = 'Could not fetch container logs through Flux. Check the Flux and coold status, then try again.';
}
}
return response()->json([
'status' => $application->status,
'statusMessage' => $application->status_message,
'containerId' => $containerId,
'logs' => $logs,
'logsError' => $logsError,
]);
}
public function updatePosition(Request $request, V5Application $application): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('update', [$application, $currentTeam]);
$validated = $request->validate([
'canvas_x' => ['required', 'integer', 'min:-100000', 'max:100000'],
'canvas_y' => ['required', 'integer', 'min:-100000', 'max:100000'],
]);
$application->update([
'canvas_x' => $validated['canvas_x'],
'canvas_y' => $validated['canvas_y'],
]);
return response()->json([
'application' => $this->serializeApplication($application->refresh()->load('server')),
]);
}
public function updateIngress(Request $request, V5Application $application): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('updateIngress', [$application, $currentTeam]);
$validated = $request->validate([
'ingress_enabled' => ['required', 'boolean'],
'internal_port' => ['nullable', 'integer', 'min:1', 'max:65535'],
'domains' => [Rule::requiredIf(fn () => $request->boolean('ingress_enabled')), 'array', 'min:1'],
'domains.*' => ['required', 'string', 'max:255', 'distinct:ignore_case', new ValidHostname],
]);
$application->loadMissing('server');
if ($validated['ingress_enabled'] && ! $application->server?->isIngress()) {
return response()->json([
'message' => 'Enable ingress on the server before enabling app ingress.',
], 422);
}
if ($validated['ingress_enabled'] && array_key_exists('domains', $validated)) {
$conflict = $this->conflictingApplicationDomain($application, $validated['domains']);
if ($conflict instanceof V5ApplicationDomain) {
return response()->json([
'message' => "The domain {$conflict->domain} is already used by application \"{$conflict->application?->name}\" on this server.",
], 422);
}
}
$originalAttributes = $application->only(['ingress_enabled', 'internal_port']);
$originalDomains = $application->domains()->pluck('domain')->all();
DB::transaction(function () use ($application, $validated): void {
$application->update([
'ingress_enabled' => $validated['ingress_enabled'],
'internal_port' => $validated['internal_port'] ?? null,
]);
if (array_key_exists('domains', $validated)) {
$application->domains()->delete();
collect($validated['domains'])
->map(fn (string $domain) => trim($domain))
->filter()
->unique()
->each(fn (string $domain) => V5ApplicationDomain::query()->create([
'application_id' => $application->id,
'domain' => $domain,
]));
}
});
$application->refresh()->load(['server', 'domains']);
if ($application->server?->isIngress() && $application->server->status === ServerStatus::Installed->value) {
try {
StartCaddyIngress::run($application->server);
} catch (\RuntimeException $exception) {
$this->restoreApplicationIngress($application, $originalAttributes, $originalDomains);
return $this->ingressSyncErrorResponse($exception);
}
}
return response()->json([
'application' => $this->serializeApplication($application),
]);
}
public function updateCaddyIngressPosition(Request $request, V5Server $server): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('updateCanvasPosition', [$server, $currentTeam]);
$validated = $request->validate([
'canvas_x' => ['required', 'integer', 'min:-100000', 'max:100000'],
'canvas_y' => ['required', 'integer', 'min:-100000', 'max:100000'],
]);
$server->update([
'canvas_x' => $validated['canvas_x'],
'canvas_y' => $validated['canvas_y'],
]);
return response()->json([
'caddyIngress' => $this->serializeCaddyIngress($server->refresh()),
]);
}
public function destroy(Request $request, V5Application $application, FluxClient $fluxClient): Response|JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('delete', [$application, $currentTeam]);
$application->loadMissing(['server', 'domains']);
$server = $application->server;
$connections = $this->applicationResourceConnections($application);
if ($request->boolean('delete_locally')) {
$this->deleteApplicationLocally($application, $connections);
return response()->noContent();
}
$oldFirewallRules = $connections
->flatMap(function (ResourceConnection $connection): Collection {
// Deletion must never be blocked by an endpoint that already lost
// its server; those rules can no longer be revoked anyway.
try {
return $this->firewallSync->rulesFor($connection->load('rules'));
} catch (\RuntimeException $exception) {
report($exception);
return collect();
}
});
$originalIngressAttributes = null;
$originalIngressDomains = [];
$ingressConfigurationChanged = false;
try {
$this->firewallSync->sync($fluxClient, $oldFirewallRules, collect());
} catch (\RuntimeException $exception) {
report($exception);
return response()->json([
'message' => 'Could not sync firewall rules through Flux.',
'detail' => $exception->getMessage(),
], 502);
}
if ($server instanceof V5Server && $server->isIngress() && $server->status === ServerStatus::Installed->value && $application->ingress_enabled) {
$originalIngressAttributes = $application->only(['ingress_enabled', 'internal_port']);
$originalIngressDomains = $application->domains()->pluck('domain')->all();
DB::transaction(function () use ($application): void {
$application->update([
'ingress_enabled' => false,
'internal_port' => null,
]);
$application->domains()->delete();
});
try {
StartCaddyIngress::run($server);
$ingressConfigurationChanged = true;
} catch (\RuntimeException $exception) {
$this->restoreApplicationIngress($application, $originalIngressAttributes, $originalIngressDomains);
return $this->ingressSyncErrorResponse($exception);
}
}
$error = DestroyNginxApplication::run($application);
if ($error !== null) {
if ($originalIngressAttributes !== null) {
$this->restoreApplicationIngress($application, $originalIngressAttributes, $originalIngressDomains);
if ($ingressConfigurationChanged && $server instanceof V5Server) {
try {
StartCaddyIngress::run($server);
} catch (\RuntimeException $exception) {
report($exception);
}
}
}
try {
$this->firewallSync->sync($fluxClient, collect(), $oldFirewallRules);
} catch (\RuntimeException $exception) {
report($exception);
}
return response()->json([
'message' => $error,
'can_delete_locally' => true,
], 422);
}
$this->deleteApplicationLocally($application, $connections);
return response()->noContent();
}
/**
* @param Collection<int, ResourceConnection> $connections
*/
private function deleteApplicationLocally(V5Application $application, Collection $connections): void
{
DB::transaction(function () use ($application, $connections): void {
$connections->each(function (ResourceConnection $connection): void {
$connection->rules()->delete();
$connection->delete();
});
$application->delete();
});
}
/**
* @return Collection<int, ResourceConnection>
*/
private function applicationResourceConnections(V5Application $application): Collection
{
return ResourceConnection::query()
->where('team_id', $application->team_id)
->where(function (Builder $query) use ($application): void {
$query
->where(function (Builder $query) use ($application): void {
$query
->where('resource_one_type', $application->getMorphClass())
->where('resource_one_id', $application->id);
})
->orWhere(function (Builder $query) use ($application): void {
$query
->where('resource_two_type', $application->getMorphClass())
->where('resource_two_id', $application->id);
});
})
->with('rules')
->get();
}
/**
* @return array{canvas_x: int, canvas_y: int}
*/
private function nextApplicationCanvasPosition(Team $currentTeam, Project $project, Environment $environment): array
{
$existingApplications = V5Application::query()
->where('team_id', $currentTeam->id)
->where('project_id', $project->id)
->where('environment_id', $environment->id)
->get(['canvas_x', 'canvas_y']);
$horizontalStep = CanvasResourceSerializer::CARD_WIDTH + CanvasResourceSerializer::CARD_GAP;
$verticalStep = CanvasResourceSerializer::CARD_HEIGHT + CanvasResourceSerializer::CARD_GAP;
for ($row = 0; $row < 100; $row++) {
for ($column = 0; $column < 100; $column++) {
$candidate = [
'canvas_x' => $column * $horizontalStep,
'canvas_y' => $row * $verticalStep,
];
if (! $this->canvasPositionCollides($candidate, $existingApplications)) {
return $candidate;
}
}
}
return [
'canvas_x' => $existingApplications->max('canvas_x') + $horizontalStep,
'canvas_y' => 0,
];
}
/**
* @param array{canvas_x: int, canvas_y: int} $candidate
* @param Collection<int, V5Application> $existingApplications
*/
private function canvasPositionCollides(array $candidate, Collection $existingApplications): bool
{
return $existingApplications->contains(function (V5Application $application) use ($candidate) {
return abs($candidate['canvas_x'] - $application->canvas_x) < CanvasResourceSerializer::CARD_WIDTH + CanvasResourceSerializer::CARD_GAP
&& abs($candidate['canvas_y'] - $application->canvas_y) < CanvasResourceSerializer::CARD_HEIGHT + CanvasResourceSerializer::CARD_GAP;
});
}
/**
* @param array<int, string> $domains
*/
private function conflictingApplicationDomain(V5Application $application, array $domains): ?V5ApplicationDomain
{
$normalizedDomains = collect($domains)
->map(fn (string $domain) => Str::lower(trim($domain)))
->filter()
->values();
if ($normalizedDomains->isEmpty()) {
return null;
}
return V5ApplicationDomain::query()
->whereIn(DB::raw('LOWER(domain)'), $normalizedDomains->all())
->whereHas('application', fn (Builder $query) => $query
->where('server_id', $application->server_id)
->whereKeyNot($application->id)
->where('ingress_enabled', true))
->with('application:id,name')
->first();
}
/**
* @param array<string, mixed> $attributes
* @param array<int, string> $domains
*/
private function restoreApplicationIngress(V5Application $application, array $attributes, array $domains): void
{
DB::transaction(function () use ($application, $attributes, $domains): void {
$application->update($attributes);
$application->domains()->delete();
foreach ($domains as $domain) {
V5ApplicationDomain::query()->create([
'application_id' => $application->id,
'domain' => $domain,
]);
}
});
}
}
@@ -0,0 +1,207 @@
<?php
namespace App\Http\Controllers\V5;
use App\Http\Controllers\Controller;
use App\Http\Controllers\V5\Concerns\ResolvesCurrentTeam;
use App\Http\Controllers\V5\Concerns\ResolvesProjectSelection;
use App\Http\Controllers\V5\Concerns\ValidatesBuilderConfiguration;
use App\Models\PrivateKey;
use App\Models\Team;
use App\Models\V5\Cluster as V5Cluster;
use App\Services\Flux\FluxHealth;
use App\Support\V5\ClusterSerializer;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
use Illuminate\Validation\Rule;
use Inertia\Inertia;
use Inertia\Response;
class ClusterController extends Controller
{
use ResolvesCurrentTeam;
use ResolvesProjectSelection;
use ValidatesBuilderConfiguration;
public function index(Request $request, FluxHealth $fluxHealth): Response
{
$currentTeam = $request->attributes->get('v5.currentTeam');
$projects = $this->projects($currentTeam);
[$selectedProject, $selectedEnvironment] = $this->selectedProjectAndEnvironment($request, $projects);
return Inertia::render('Clusters', [
'currentTeam' => $this->serializeCurrentTeam($currentTeam),
'flux' => $fluxHealth->check(),
'clusters' => $this->clusters($currentTeam),
'privateKeys' => $this->privateKeys($currentTeam),
'projects' => $projects,
'selectedProjectUuid' => $selectedProject['uuid'] ?? null,
'selectedEnvironmentUuid' => $selectedEnvironment['uuid'] ?? null,
]);
}
public function show(Request $request, V5Cluster $cluster): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('view', [$cluster, $currentTeam]);
return response()->json([
'cluster' => app(ClusterSerializer::class)->serializeFresh($cluster),
]);
}
public function store(Request $request): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('create', [V5Cluster::class, $currentTeam]);
$validated = $request->validate([
'name' => [
'required',
'string',
'max:255',
Rule::unique('v5_clusters', 'name')->where('team_id', $currentTeam->id),
],
'description' => ['nullable', 'string', 'max:1000'],
'wireguard_interface' => ['sometimes', 'string', 'max:32', 'regex:/^[a-zA-Z0-9_.-]+$/'],
'wireguard_management_pool' => ['sometimes', 'string', 'max:64', $this->ipv4CidrRule()],
'wireguard_listen_port' => ['sometimes', 'integer', 'min:1', 'max:65535'],
'container_network_pool' => ['sometimes', 'string', 'max:64', $this->ipv4CidrRule()],
'container_network_prefix' => ['sometimes', 'integer', 'min:1', 'max:32'],
'namespaces' => ['sometimes', 'array', 'min:1'],
'namespaces.*' => ['string', 'distinct', 'regex:/^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$/'],
'default_deny_containers' => ['sometimes', 'boolean'],
'coold_version' => ['sometimes', 'string', 'max:64'],
'corrosion_version' => ['sometimes', 'string', 'max:64'],
'corrosion_gossip_port' => ['sometimes', 'integer', 'min:1', 'max:65535'],
'corrosion_api_port' => ['sometimes', 'integer', 'min:1', 'max:65535'],
'builder_enabled' => ['sometimes', 'boolean'],
'builder_capacity' => $this->builderCapacityRules(
$this->requestedBuilderEnabled($request, true)
),
'builder_cpu_quota' => ['sometimes', 'string', 'max:32'],
'builder_memory_max' => ['sometimes', 'string', 'max:32'],
'builder_timeout_secs' => ['sometimes', 'integer', 'min:1', 'max:86400'],
]);
$cluster = V5Cluster::query()->create([
...$this->defaultClusterConfiguration(),
...collect($validated)->except(['name', 'description'])->all(),
'team_id' => $currentTeam->id,
'created_by_user_id' => $request->user()->id,
'name' => $validated['name'],
'description' => $validated['description'] ?? null,
]);
return response()->json([
'cluster' => app(ClusterSerializer::class)->serializeFresh($cluster),
], 201);
}
public function destroy(Request $request, V5Cluster $cluster): \Illuminate\Http\Response|JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('delete', [$cluster, $currentTeam]);
if ($cluster->servers()->exists()) {
return response()->json([
'message' => 'Only empty clusters can be deleted.',
], 422);
}
$cluster->delete();
return response()->noContent();
}
/**
* @return array<int, array<string, mixed>>
*/
private function clusters(mixed $currentTeam): array
{
if (! $currentTeam instanceof Team) {
return [];
}
$serializer = app(ClusterSerializer::class);
return V5Cluster::query()
->where('team_id', $currentTeam->id)
->with(['servers' => fn ($query) => $query
->with('privateKey')
->orderBy('name')])
->withCount('servers')
->orderBy('name')
->get()
->map(fn (V5Cluster $cluster) => $serializer->serialize($cluster))
->all();
}
/**
* @return array<int, array{id: string, name: string}>
*/
private function privateKeys(mixed $currentTeam): array
{
if (! $currentTeam instanceof Team) {
return [];
}
return PrivateKey::query()
->where('team_id', $currentTeam->id)
->where('is_git_related', false)
->orderBy('name')
->get(['id', 'uuid', 'name'])
->map(fn (PrivateKey $privateKey) => [
'id' => $privateKey->uuid,
'name' => $privateKey->name,
])
->all();
}
/**
* @return array<string, mixed>
*/
private function defaultClusterConfiguration(): array
{
return [
'wireguard_interface' => V5Cluster::DEFAULT_WIREGUARD_INTERFACE,
'wireguard_management_pool' => V5Cluster::DEFAULT_WIREGUARD_MANAGEMENT_POOL,
'wireguard_listen_port' => V5Cluster::DEFAULT_WIREGUARD_LISTEN_PORT,
'container_network_pool' => V5Cluster::DEFAULT_CONTAINER_NETWORK_POOL,
'container_network_prefix' => V5Cluster::DEFAULT_CONTAINER_NETWORK_PREFIX,
'namespaces' => V5Cluster::DEFAULT_NAMESPACES,
'default_deny_containers' => true,
'coold_version' => V5Cluster::DEFAULT_COOLD_VERSION,
'corrosion_version' => V5Cluster::DEFAULT_CORROSION_VERSION,
'corrosion_gossip_port' => V5Cluster::DEFAULT_CORROSION_GOSSIP_PORT,
'corrosion_api_port' => V5Cluster::DEFAULT_CORROSION_API_PORT,
'builder_enabled' => true,
'builder_capacity' => V5Cluster::DEFAULT_BUILDER_CAPACITY,
'builder_cpu_quota' => V5Cluster::DEFAULT_BUILDER_CPU_QUOTA,
'builder_memory_max' => V5Cluster::DEFAULT_BUILDER_MEMORY_MAX,
'builder_timeout_secs' => V5Cluster::DEFAULT_BUILDER_TIMEOUT_SECS,
];
}
private function ipv4CidrRule(): \Closure
{
return function (string $attribute, mixed $value, \Closure $fail): void {
if (! is_string($value) || ! str_contains($value, '/')) {
$fail('The :attribute must be a valid IPv4 CIDR range.');
return;
}
[$ip, $prefix] = explode('/', $value, 2);
if (
filter_var($ip, FILTER_VALIDATE_IP, FILTER_FLAG_IPV4) === false
|| ! ctype_digit($prefix)
|| (int) $prefix < 0
|| (int) $prefix > 32
) {
$fail('The :attribute must be a valid IPv4 CIDR range.');
}
};
}
}
@@ -0,0 +1,40 @@
<?php
namespace App\Http\Controllers\V5\Concerns;
use Illuminate\Http\JsonResponse;
use Illuminate\Support\Str;
trait HandlesIngressSyncErrors
{
protected function ingressSyncErrorResponse(\RuntimeException $exception): JsonResponse
{
return response()->json([
'message' => $this->friendlyIngressSyncError($exception->getMessage()),
'detail' => $exception->getMessage(),
], 502);
}
protected function friendlyIngressSyncError(string $message): string
{
$normalized = Str::lower($message);
if (str_contains($normalized, 'invalid http response') || str_contains($normalized, 'could not talk to flux')) {
return 'Could not reach Flux. Check that Flux is running in the Coolify container and try again.';
}
if (str_contains($normalized, 'dispatch timeout') || str_contains($normalized, 'timed out')) {
return 'coold did not respond in time. Check that the server agent is running and connected to Flux.';
}
if (str_contains($normalized, 'validate caddyfile')) {
return 'Caddy rejected the generated ingress configuration. Check the domains and internal port, then try again.';
}
if (str_contains($normalized, 'start caddy ingress') || str_contains($normalized, 'reload caddy ingress')) {
return 'Could not start Caddy ingress on the server. Check that Podman is running and port 80 is available.';
}
return 'Could not update ingress. Check Flux and coold logs, then try again.';
}
}
@@ -0,0 +1,22 @@
<?php
namespace App\Http\Controllers\V5\Concerns;
use App\Models\Team;
use Illuminate\Http\Request;
trait ResolvesCurrentTeam
{
/**
* Resolve the current team set by the EnsureCurrentTeam middleware, or
* abort with a 404 so resources outside the team stay invisible.
*/
protected function currentTeamOrFail(Request $request): Team
{
$currentTeam = $request->attributes->get('v5.currentTeam');
abort_unless($currentTeam instanceof Team, 404);
return $currentTeam;
}
}
@@ -0,0 +1,115 @@
<?php
namespace App\Http\Controllers\V5\Concerns;
use App\Models\Environment;
use App\Models\Project;
use App\Models\Team;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Http\Request;
trait ResolvesProjectSelection
{
protected const SELECTED_PROJECT_SESSION_KEY = 'v5.selectedProjectUuid';
protected const SELECTED_ENVIRONMENT_SESSION_KEY = 'v5.selectedEnvironmentUuid';
/**
* @return array{id: int}|null
*/
protected function serializeCurrentTeam(mixed $currentTeam): ?array
{
if (! $currentTeam instanceof Team) {
return null;
}
return [
'id' => $currentTeam->id,
];
}
/**
* @param array<int, array{uuid: string, name: string, environments: array<int, array{uuid: string, name: string}>}> $projects
* @return array{0: array{uuid: string, name: string, environments: array<int, array{uuid: string, name: string}>}|null, 1: array{uuid: string, name: string}|null}
*/
protected function selectedProjectAndEnvironment(Request $request, array $projects): array
{
$selectedProjectUuid = $request->session()->get(self::SELECTED_PROJECT_SESSION_KEY);
$selectedEnvironmentUuid = $request->session()->get(self::SELECTED_ENVIRONMENT_SESSION_KEY);
$selectedProject = null;
foreach ($projects as $project) {
if ($project['uuid'] === $selectedProjectUuid) {
$selectedProject = $project;
break;
}
}
$selectedProject ??= $projects[0] ?? null;
$selectedEnvironment = null;
foreach ($selectedProject['environments'] ?? [] as $environment) {
if ($environment['uuid'] === $selectedEnvironmentUuid) {
$selectedEnvironment = $environment;
break;
}
}
$selectedEnvironment ??= $selectedProject['environments'][0] ?? null;
return [$selectedProject, $selectedEnvironment];
}
protected function selectedEnvironment(Project $project, ?string $environmentUuid): ?Environment
{
if ($environmentUuid === null) {
return $project->environments->first();
}
$environment = $project->environments->firstWhere('uuid', $environmentUuid);
if (! $environment instanceof Environment) {
abort(422, 'The selected environment is not available for the selected project.');
}
return $environment;
}
/**
* @return array<int, array{uuid: string, name: string, environments: array<int, array{uuid: string, name: string}>}>
*/
protected function projects(mixed $currentTeam): array
{
if (! $currentTeam instanceof Team) {
return [];
}
return $this->projectQuery($currentTeam)
->get()
->map(fn (Project $project) => [
'uuid' => $project->uuid,
'name' => $project->name,
'environments' => $project->environments
->map(fn ($environment) => [
'uuid' => $environment->uuid,
'name' => $environment->name,
])
->all(),
])
->all();
}
protected function projectQuery(Team $currentTeam): Builder
{
return Project::query()
->select(['id', 'uuid', 'name', 'team_id'])
->where('team_id', $currentTeam->id)
->with(['environments' => fn ($query) => $query
->select(['id', 'uuid', 'name', 'project_id'])
->orderByRaw("CASE WHEN LOWER(name) = 'production' THEN 0 ELSE 1 END")
->orderByRaw('LOWER(name)')])
->orderByRaw('LOWER(name)');
}
}
@@ -0,0 +1,63 @@
<?php
namespace App\Http\Controllers\V5\Concerns;
use App\Models\Team;
use App\Models\V5\Application as V5Application;
use App\Models\V5\Server as V5Server;
use App\Support\V5\CanvasResourceSerializer;
use Illuminate\Database\Eloquent\Builder;
trait SerializesCanvasResources
{
/**
* @return array<string, mixed>
*/
protected function serializeApplication(V5Application $application): array
{
return app(CanvasResourceSerializer::class)->serializeApplication($application);
}
/**
* @return array<string, mixed>
*/
protected function serializeCaddyIngress(V5Server $server, int $index = 0): array
{
return app(CanvasResourceSerializer::class)->serializeCaddyIngress($server, $index);
}
/**
* @return array<int, array<string, mixed>>
*/
protected function caddyIngresses(mixed $currentTeam): array
{
if (! $currentTeam instanceof Team) {
return [];
}
return V5Server::query()
->where('team_id', $currentTeam->id)
->orderBy('name')
->get()
->filter(fn (V5Server $server) => $server->isIngress())
->values()
->map(fn (V5Server $server, int $index) => $this->serializeCaddyIngress($server, $index))
->all();
}
/**
* @param array{uuid: string} $selectedProject
* @param array{uuid: string} $selectedEnvironment
* @return Builder<V5Application>
*/
protected function applicationQuery(Team $currentTeam, array $selectedProject, array $selectedEnvironment): Builder
{
return V5Application::query()
->where('team_id', $currentTeam->id)
->whereHas('project', fn (Builder $query) => $query
->where('team_id', $currentTeam->id)
->where('uuid', $selectedProject['uuid']))
->whereHas('environment', fn (Builder $query) => $query
->where('uuid', $selectedEnvironment['uuid']));
}
}
@@ -0,0 +1,30 @@
<?php
namespace App\Http\Controllers\V5\Concerns;
use Illuminate\Http\Request;
trait ValidatesBuilderConfiguration
{
/**
* @return array<int, string>
*/
protected function builderCapacityRules(bool $builderEnabled, bool $required = false): array
{
return [
$required ? 'required' : 'sometimes',
'integer',
$builderEnabled ? 'min:1' : 'min:0',
'max:1000',
];
}
protected function requestedBuilderEnabled(Request $request, bool $default): bool
{
if (! $request->has('builder_enabled')) {
return $default;
}
return $request->boolean('builder_enabled');
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,330 @@
<?php
namespace App\Http\Controllers\V5;
use App\Http\Controllers\Controller;
use App\Http\Controllers\V5\Concerns\ResolvesCurrentTeam;
use App\Http\Controllers\V5\Concerns\ResolvesProjectSelection;
use App\Models\Environment;
use App\Models\Project;
use App\Models\Team;
use App\Models\V5\Application as V5Application;
use App\Models\V5\ResourceConnection;
use App\Services\Flux\FluxClient;
use App\Support\V5\ConnectionFirewallSync;
use App\Support\V5\ResourceConnectionSerializer;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
use Illuminate\Http\Response;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Illuminate\Validation\Rule;
class ResourceConnectionController extends Controller
{
use ResolvesCurrentTeam;
use ResolvesProjectSelection;
public function __construct(
private readonly ConnectionFirewallSync $firewallSync,
private readonly ResourceConnectionSerializer $connectionSerializer,
) {}
public function store(Request $request): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$projects = $this->projects($currentTeam);
[$selectedProject, $selectedEnvironment] = $this->selectedProjectAndEnvironment($request, $projects);
if ($selectedProject === null || $selectedEnvironment === null) {
return response()->json([
'message' => 'Select a project and environment before connecting resources.',
], 422);
}
$project = $this->projectQuery($currentTeam)
->where('uuid', $selectedProject['uuid'])
->first();
if (! $project instanceof Project) {
abort(403);
}
$environment = $this->selectedEnvironment($project, $selectedEnvironment['uuid']);
if (! $environment instanceof Environment) {
abort(403);
}
$validated = $request->validate([
'resource_one' => ['required', 'array'],
'resource_one.type' => ['required', 'string', Rule::in(['application'])],
'resource_one.uuid' => ['required', 'string', 'max:255'],
'resource_two' => ['required', 'array'],
'resource_two.type' => ['required', 'string', Rule::in(['application'])],
'resource_two.uuid' => ['required', 'string', 'max:255'],
]);
$resourceOne = $this->resolveConnectableResource($currentTeam, $project, $environment, $validated['resource_one']);
$resourceTwo = $this->resolveConnectableResource($currentTeam, $project, $environment, $validated['resource_two']);
if ($this->resourceIdentity($resourceOne) === $this->resourceIdentity($resourceTwo)) {
return response()->json([
'message' => 'A resource cannot connect to itself.',
], 422);
}
$connection = ResourceConnection::query()->firstOrCreate(
[
'team_id' => $currentTeam->id,
'resource_pair_key' => $this->resourcePairKey($resourceOne, $resourceTwo),
],
[
'project_id' => $project->id,
'environment_id' => $environment->id,
'resource_one_type' => $resourceOne->getMorphClass(),
'resource_one_id' => $resourceOne->getKey(),
'resource_two_type' => $resourceTwo->getMorphClass(),
'resource_two_id' => $resourceTwo->getKey(),
'created_by_user_id' => $request->user()->id,
],
);
return response()->json([
'connection' => $this->connectionSerializer->serialize($connection->load('rules')),
], $connection->wasRecentlyCreated ? 201 : 200);
}
/**
* Update the connection's rules, then converge the node firewalls.
*
* Ordering & failure semantics:
* 1. Snapshot the current DB rules and their firewall representation; abort
* with 502 before mutating anything when the snapshot cannot be built.
* 2. Commit the requested rules in a DB transaction the DB always holds
* the desired state.
* 3. Converge the node firewalls through Flux. Nodes whose coold lacks the
* firewall verbs (UnsupportedCooldVerb) are tolerated: the committed
* rules are kept and the request succeeds.
* 4. On a real Flux failure the previous rules are restored in a second DB
* transaction, the node firewalls are rolled back to the restored rules
* best-effort (warning-logged when that also fails deterministic rule
* ids keep a later re-sync idempotent), and the original error surfaces
* to the caller as a 502 {message, detail} response.
*/
public function update(Request $request, ResourceConnection $connection, FluxClient $fluxClient): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('update', [$connection, $currentTeam]);
$validated = $request->validate([
'ports_by_direction' => ['present', 'array'],
'ports_by_direction.*' => ['array'],
'ports_by_direction.*.*' => ['integer', 'min:1', 'max:65535', 'distinct'],
]);
$connection->load('rules');
$oldRulePayloads = $connection->rules
->map(fn ($rule): array => [
'source_resource_type' => $rule->source_resource_type,
'source_resource_id' => $rule->source_resource_id,
'target_resource_type' => $rule->target_resource_type,
'target_resource_id' => $rule->target_resource_id,
'protocol' => $rule->protocol,
'port' => $rule->port,
])
->all();
try {
$oldFirewallRules = $this->firewallSync->rulesFor($connection);
} catch (\RuntimeException $exception) {
report($exception);
Log::warning('V5 resource connection firewall snapshot failed', [
'connection_id' => $connection->id,
'message' => $exception->getMessage(),
]);
return response()->json([
'message' => 'Could not sync firewall rules through Flux.',
'detail' => 'The connection was left unchanged. Check the server diagnostics and try again.',
], 502);
}
DB::transaction(function () use ($connection, $validated): void {
$connection->rules()->delete();
$resourcesByUuid = $this->connectionSerializer->applicationsByUuid($connection);
foreach ($validated['ports_by_direction'] as $direction => $ports) {
[$sourceResourceUuid, $targetResourceUuid] = array_pad(explode('->', (string) $direction, 2), 2, null);
$sourceResource = is_string($sourceResourceUuid) ? $resourcesByUuid->get($sourceResourceUuid) : null;
$targetResource = is_string($targetResourceUuid) ? $resourcesByUuid->get($targetResourceUuid) : null;
if (! $sourceResource instanceof V5Application || ! $targetResource instanceof V5Application) {
continue;
}
foreach (array_unique($ports) as $port) {
$connection->rules()->create([
'source_resource_type' => $this->resourceTypeForConnectionUuid($connection, $sourceResource->uuid),
'source_resource_id' => $sourceResource->id,
'target_resource_type' => $this->resourceTypeForConnectionUuid($connection, $targetResource->uuid),
'target_resource_id' => $targetResource->id,
'protocol' => 'tcp',
'port' => (int) $port,
]);
}
}
});
$connection->refresh()->load('rules');
$newFirewallRules = null;
try {
$newFirewallRules = $this->firewallSync->rulesFor($connection);
$this->firewallSync->sync($fluxClient, $oldFirewallRules, $newFirewallRules);
} catch (\RuntimeException $exception) {
$this->restoreConnectionRules($connection, $oldRulePayloads);
$this->rollBackFirewallRules($fluxClient, $connection, $newFirewallRules, $oldFirewallRules);
report($exception);
Log::warning('V5 resource connection firewall sync failed', [
'connection_id' => $connection->id,
'message' => $exception->getMessage(),
]);
return response()->json([
'message' => 'Could not sync firewall rules through Flux.',
'detail' => 'The previous rules were restored. Check the server diagnostics and try again.',
], 502);
}
return response()->json([
'connection' => $this->connectionSerializer->serialize($connection),
]);
}
/**
* Delete the connection using revoke-first ordering.
*
* The node firewall rules are revoked before any DB rows are removed. When
* a revoke fails with a real error the delete is aborted with a 502
* {message, detail} response so the DB never loses track of rules that may
* still be open on a reachable node; UnsupportedCooldVerb and
* already-missing rules are tolerated. When the firewall snapshot cannot
* be built (an endpoint lost its server host id) the node cannot be
* addressed at all, so the failure is reported and the delete proceeds.
* Deterministic rule ids make a retried delete revoke the same node-side
* rules idempotently.
*/
public function destroy(Request $request, ResourceConnection $connection, FluxClient $fluxClient): Response|JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('delete', [$connection, $currentTeam]);
try {
$oldFirewallRules = $this->firewallSync->rulesFor($connection->load('rules'));
} catch (\RuntimeException $exception) {
report($exception);
$oldFirewallRules = collect();
}
try {
$this->firewallSync->sync($fluxClient, $oldFirewallRules, collect());
} catch (\RuntimeException $exception) {
report($exception);
Log::warning('V5 resource connection firewall revoke failed', [
'connection_id' => $connection->id,
'message' => $exception->getMessage(),
]);
return response()->json([
'message' => 'Could not sync firewall rules through Flux.',
'detail' => 'The connection was not deleted. Check the server diagnostics and try again.',
], 502);
}
$connection->delete();
return response()->noContent();
}
/**
* @param array{type: string, uuid: string} $resource
*/
private function resolveConnectableResource(Team $team, Project $project, Environment $environment, array $resource): Model
{
return match ($resource['type']) {
'application' => V5Application::query()
->where('team_id', $team->id)
->where('project_id', $project->id)
->where('environment_id', $environment->id)
->where('uuid', $resource['uuid'])
->firstOrFail(),
};
}
private function resourcePairKey(Model $resourceOne, Model $resourceTwo): string
{
return collect([
$this->resourceIdentity($resourceOne),
$this->resourceIdentity($resourceTwo),
])->sort()->implode('|');
}
private function resourceIdentity(Model $resource): string
{
return $resource->getMorphClass().':'.$resource->getKey();
}
private function resourceTypeForConnectionUuid(ResourceConnection $connection, string $resourceUuid): string
{
$resourcesByUuid = $this->connectionSerializer->applicationsByUuid($connection);
$resource = $resourcesByUuid->get($resourceUuid);
return $resource instanceof V5Application && (int) $connection->resource_one_id === $resource->id
? $connection->resource_one_type
: $connection->resource_two_type;
}
/**
* @param array<int, array<string, mixed>> $rulePayloads
*/
private function restoreConnectionRules(ResourceConnection $connection, array $rulePayloads): void
{
DB::transaction(function () use ($connection, $rulePayloads): void {
$connection->rules()->delete();
foreach ($rulePayloads as $rulePayload) {
$connection->rules()->create($rulePayload);
}
});
}
/**
* Best-effort roll back of a partially converged node firewall to the
* restored rules after a failed forward sync. Skipped when the forward
* sync never started (the node was not touched). Failures are only logged
* because the DB already holds the restored, authoritative rules and the
* deterministic rule ids keep a later re-sync idempotent.
*
* @param Collection<int, array{id: string, hostId: string, rule: array{id: string, namespace: string, src: string, dst: string, proto: string, port: int}}>|null $attemptedFirewallRules
* @param Collection<int, array{id: string, hostId: string, rule: array{id: string, namespace: string, src: string, dst: string, proto: string, port: int}}> $restoredFirewallRules
*/
private function rollBackFirewallRules(FluxClient $fluxClient, ResourceConnection $connection, ?Collection $attemptedFirewallRules, Collection $restoredFirewallRules): void
{
if (! $attemptedFirewallRules instanceof Collection) {
return;
}
try {
$this->firewallSync->sync($fluxClient, $attemptedFirewallRules, $restoredFirewallRules);
} catch (\RuntimeException $exception) {
Log::warning('V5 resource connection firewall rollback failed; node firewall may diverge from the restored rules until the next sync', [
'connection_id' => $connection->id,
'message' => $exception->getMessage(),
]);
}
}
}
@@ -0,0 +1,761 @@
<?php
namespace App\Http\Controllers\V5;
use App\Actions\V5\Proxy\StartCaddyIngress;
use App\Actions\V5\Proxy\StopCaddyIngress;
use App\Actions\V5\Server\RemoveBootstrapMarker;
use App\Enums\V5\ServerStatus;
use App\Events\V5ClusterUpdated;
use App\Http\Controllers\Controller;
use App\Http\Controllers\V5\Concerns\HandlesIngressSyncErrors;
use App\Http\Controllers\V5\Concerns\ResolvesCurrentTeam;
use App\Http\Controllers\V5\Concerns\ValidatesBuilderConfiguration;
use App\Jobs\V5BootstrapServerJob;
use App\Models\PrivateKey;
use App\Models\V5\Application as V5Application;
use App\Models\V5\Cluster as V5Cluster;
use App\Models\V5\Server as V5Server;
use App\Rules\ValidServerIp;
use App\Services\Flux\FluxClient;
use App\Support\V5\ClusterSerializer;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
use Illuminate\Http\Response;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Process;
use Illuminate\Validation\Rule;
class ServerController extends Controller
{
use HandlesIngressSyncErrors;
use ResolvesCurrentTeam;
use ValidatesBuilderConfiguration;
public function store(Request $request, V5Cluster $cluster): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('create', [V5Server::class, $currentTeam, $cluster]);
$validated = $request->validate([
'name' => ['required', 'string', 'max:255'],
'host' => [
'required',
'string',
'max:255',
$this->noControlCharactersRule(),
new ValidServerIp,
Rule::unique('v5_servers', 'host')
->where('team_id', $currentTeam->id)
->where('ssh_port', (int) $request->input('ssh_port', 22)),
],
'ssh_user' => ['required', 'string', 'max:255', 'regex:/^[A-Za-z0-9._-]+$/', $this->noControlCharactersRule()],
'ssh_port' => ['required', 'integer', 'min:1', 'max:65535'],
'private_key_uuid' => [
'required',
'string',
Rule::exists('private_keys', 'uuid')->where('team_id', $currentTeam->id),
],
'node_address' => [
'nullable',
'string',
'max:255',
$this->noControlCharactersRule(),
new ValidServerIp,
Rule::unique('v5_servers', 'node_address')->where('team_id', $currentTeam->id),
],
'builder_enabled' => ['sometimes', 'boolean'],
'builder_capacity' => $this->builderCapacityRules(
$this->requestedBuilderEnabled($request, $cluster->builder_enabled)
),
'builder_cpu_quota' => ['sometimes', 'string', 'max:32'],
'wireguard_listen_port_override' => ['nullable', 'integer', 'min:1', 'max:65535'],
'wireguard_endpoint_override' => [
'nullable',
'string',
'max:255',
$this->noControlCharactersRule(),
$this->hostPortRule(),
Rule::unique('v5_servers', 'wireguard_endpoint_override')->where('cluster_id', $cluster->id),
],
'ingress_enabled' => ['sometimes', 'boolean'],
'ingress_type' => [
Rule::requiredIf(fn () => $request->boolean('ingress_enabled')),
'nullable',
'string',
Rule::in(['caddy']),
],
]);
$capacity = $this->clusterServerCapacity($cluster);
if ($capacity !== null && $cluster->servers()->count() >= $capacity) {
return response()->json([
'message' => "This cluster's network pools are full ({$capacity} server(s) max). Grow the pools or remove a server first.",
], 422);
}
$builderEnabled = (bool) ($validated['builder_enabled'] ?? $cluster->builder_enabled);
$ingressEnabled = (bool) ($validated['ingress_enabled'] ?? false);
$ingressType = $ingressEnabled ? $validated['ingress_type'] : null;
$builderCapacity = (int) ($validated['builder_capacity'] ?? $cluster->builder_capacity);
$builderCpuQuota = $validated['builder_cpu_quota'] ?? $cluster->builder_cpu_quota;
$devWireguardOverrides = $this->devLimaWireguardOverrides($validated['host'], (int) $validated['ssh_port']);
$privateKey = PrivateKey::query()
->where('team_id', $currentTeam->id)
->where('uuid', $validated['private_key_uuid'])
->firstOrFail();
V5Server::query()->create([
'team_id' => $currentTeam->id,
'cluster_id' => $cluster->id,
'created_by_user_id' => $request->user()->id,
'name' => $validated['name'],
'host' => $validated['host'],
'ssh_user' => $validated['ssh_user'],
'ssh_port' => $validated['ssh_port'],
'private_key_id' => $privateKey->id,
'status' => ServerStatus::Added->value,
'ingress_type' => $ingressType,
'is_ingress' => $ingressEnabled,
'builder_enabled' => $builderEnabled,
'builder_capacity' => $builderCapacity,
'builder_cpu_quota' => $builderCpuQuota,
'node_address' => $validated['node_address'] ?? $validated['host'],
'wireguard_listen_port_override' => $validated['wireguard_listen_port_override'] ?? $devWireguardOverrides['listen_port'],
'wireguard_endpoint_override' => $validated['wireguard_endpoint_override'] ?? $devWireguardOverrides['endpoint'],
]);
return response()->json([
'cluster' => app(ClusterSerializer::class)->serializeFresh($cluster),
], 201);
}
public function update(Request $request, V5Cluster $cluster, V5Server $server): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('update', [$server, $currentTeam, $cluster]);
$validated = $request->validate([
'builder_enabled' => ['required', 'boolean'],
'builder_capacity' => $this->builderCapacityRules(
$request->boolean('builder_enabled'),
required: true
),
'builder_cpu_quota' => ['required', 'string', 'max:32'],
'ingress_enabled' => ['sometimes', 'boolean'],
'ingress_type' => [
Rule::requiredIf(fn () => $request->boolean('ingress_enabled')),
'nullable',
'string',
Rule::in(['caddy']),
],
]);
$wasIngress = $server->isIngress();
$builderEnabled = (bool) $validated['builder_enabled'];
$ingressEnabled = (bool) ($validated['ingress_enabled'] ?? $wasIngress);
$ingressType = $ingressEnabled ? ($validated['ingress_type'] ?? $server->ingress_type ?? 'caddy') : null;
$originalServerAttributes = $server->only([
'is_ingress',
'ingress_type',
'ingress_status',
'builder_enabled',
'builder_capacity',
'builder_cpu_quota',
]);
// Stop the ingress before persisting the change: StopCaddyIngress needs
// the server's current ingress state, and a failed stop must leave the
// capability untouched.
if ($wasIngress && ! $ingressEnabled && $server->status === ServerStatus::Installed->value) {
try {
StopCaddyIngress::run($server);
} catch (\RuntimeException $exception) {
return $this->ingressSyncErrorResponse($exception);
}
}
$server->update([
'is_ingress' => $ingressEnabled,
'ingress_type' => $ingressType,
'builder_enabled' => $builderEnabled,
'builder_capacity' => (int) $validated['builder_capacity'],
'builder_cpu_quota' => $validated['builder_cpu_quota'],
]);
$server->refresh();
if (! $wasIngress && $ingressEnabled && $server->status === ServerStatus::Installed->value) {
try {
StartCaddyIngress::run($server);
} catch (\RuntimeException $exception) {
$server->update($originalServerAttributes);
return $this->ingressSyncErrorResponse($exception);
}
}
return response()->json([
'cluster' => app(ClusterSerializer::class)->serializeFresh($cluster),
]);
}
public function check(Request $request, V5Cluster $cluster, V5Server $server): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('check', [$server, $currentTeam, $cluster]);
if (! $server->privateKey instanceof PrivateKey) {
return response()->json([
'status' => 'failed',
'output' => 'No private key is attached to this server.',
'checkedAt' => now()->toJSON(),
]);
}
$keyDirectory = storage_path('app/ssh/keys');
if (! is_dir($keyDirectory)) {
mkdir($keyDirectory, 0700, true);
}
$keyLocation = tempnam($keyDirectory, 'v5_ssh_key_');
if ($keyLocation === false) {
return response()->json([
'status' => 'failed',
'output' => 'Could not create a temporary SSH key file.',
'checkedAt' => now()->toJSON(),
]);
}
file_put_contents($keyLocation, $server->privateKey->private_key);
chmod($keyLocation, 0600);
$target = "{$server->ssh_user}@{$server->host}";
$command = [
'ssh',
'-o',
'BatchMode=yes',
'-o',
'LogLevel=ERROR',
'-o',
'StrictHostKeyChecking=no',
'-o',
'UserKnownHostsFile=/dev/null',
'-o',
'ConnectTimeout=10',
'-o',
'IdentitiesOnly=yes',
'-i',
$keyLocation,
'-p',
(string) $server->ssh_port,
$target,
"printf 'SSH connection OK\n'; hostname; uname -srm; command -v docker || true; command -v podman || true",
];
try {
$result = Process::timeout(15)->run($command);
$output = trim($result->output()."\n".$result->errorOutput());
$status = $result->successful() ? 'reachable' : 'failed';
} catch (\Throwable $e) {
$output = $e->getMessage();
$status = 'failed';
} finally {
@unlink($keyLocation);
}
return response()->json([
'status' => $status,
'output' => str($output !== '' ? $output : 'No output returned.')->limit(10000)->toString(),
'checkedAt' => now()->toJSON(),
]);
}
public function cooldLogs(Request $request, V5Cluster $cluster, V5Server $server, FluxClient $fluxClient): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('viewDiagnostics', [$server, $currentTeam, $cluster]);
$validated = $request->validate([
'tail' => ['sometimes', 'integer', 'min:1', 'max:1000'],
]);
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
return response()->json([
'message' => 'This server is missing its Flux host id.',
], 422);
}
try {
$output = $fluxClient->cooldLogs($hostId, (int) ($validated['tail'] ?? 200));
} catch (\Throwable $e) {
Log::warning('V5 coold logs request failed', [
'server_id' => $server->id,
'message' => $e->getMessage(),
]);
if ($server->privateKey instanceof PrivateKey) {
try {
return response()->json([
'output' => $this->cooldLogsOverSsh($server, (int) ($validated['tail'] ?? 200)),
'source' => 'ssh',
'fetchedAt' => now()->toJSON(),
]);
} catch (\Throwable $sshException) {
Log::warning('V5 coold logs SSH fallback failed', [
'server_id' => $server->id,
'message' => $sshException->getMessage(),
]);
}
}
return response()->json([
'message' => 'Could not fetch coold logs through Flux. Check the Flux and coold status, then try again.',
], 502);
}
return response()->json([
'output' => $output,
'source' => 'flux',
'fetchedAt' => now()->toJSON(),
]);
}
private function cooldLogsOverSsh(V5Server $server, int $tail): string
{
return $this->runServerSshCommand(
$server,
'sudo -n journalctl -u coold -n '.max(1, min($tail, 1000)).' --no-pager -q || journalctl -u coold -n '.max(1, min($tail, 1000)).' --no-pager -q',
'SSH coold log command failed.',
);
}
private function runServerSshCommand(V5Server $server, string $remoteCommand, string $failureMessage): string
{
$keyDirectory = storage_path('app/ssh/keys');
if (! is_dir($keyDirectory)) {
mkdir($keyDirectory, 0700, true);
}
$keyLocation = tempnam($keyDirectory, 'v5_ssh_key_');
if ($keyLocation === false) {
throw new \RuntimeException('Could not create a temporary SSH key file.');
}
file_put_contents($keyLocation, $server->privateKey->private_key);
chmod($keyLocation, 0600);
try {
$result = Process::timeout(15)->run([
'ssh',
'-o',
'BatchMode=yes',
'-o',
'LogLevel=ERROR',
'-o',
'StrictHostKeyChecking=no',
'-o',
'UserKnownHostsFile=/dev/null',
'-o',
'ConnectTimeout=10',
'-o',
'IdentitiesOnly=yes',
'-i',
$keyLocation,
'-p',
(string) $server->ssh_port,
"{$server->ssh_user}@{$server->host}",
$remoteCommand,
]);
$output = trim($result->output()."\n".$result->errorOutput());
if (! $result->successful()) {
throw new \RuntimeException($output !== '' ? $output : $failureMessage);
}
return str($output)->limit(10000)->toString();
} finally {
@unlink($keyLocation);
}
}
public function corrosionTables(Request $request, V5Cluster $cluster, V5Server $server, FluxClient $fluxClient): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('viewDiagnostics', [$server, $currentTeam, $cluster]);
$validated = $request->validate([
'limit' => ['sometimes', 'integer', 'min:1', 'max:1000'],
]);
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
return response()->json([
'message' => 'This server is missing its Flux host id.',
], 422);
}
try {
$output = $fluxClient->corrosionTables($hostId, (int) ($validated['limit'] ?? 200));
} catch (\Throwable $e) {
Log::warning('V5 corrosion tables request failed', [
'server_id' => $server->id,
'message' => $e->getMessage(),
]);
if ($server->privateKey instanceof PrivateKey) {
try {
return response()->json([
'output' => $this->corrosionTablesOverSsh($server, $cluster, (int) ($validated['limit'] ?? 200)),
'source' => 'ssh',
'fetchedAt' => now()->toJSON(),
]);
} catch (\Throwable $sshException) {
Log::warning('V5 corrosion tables SSH fallback failed', [
'server_id' => $server->id,
'message' => $sshException->getMessage(),
]);
}
}
return response()->json([
'message' => 'Could not fetch corrosion tables through Flux. Check the Flux and coold status, then try again.',
], 502);
}
return response()->json([
'output' => $output,
'source' => 'flux',
'fetchedAt' => now()->toJSON(),
]);
}
private function corrosionTablesOverSsh(V5Server $server, V5Cluster $cluster, int $limit): string
{
$limit = max(1, min($limit, 1000));
$script = <<<'PYTHON'
python3 - <<'PY'
import json
import urllib.request
limit = __LIMIT__
url = "http://127.0.0.1:__PORT__/v1/queries"
def query(sql):
request = urllib.request.Request(
url,
data=json.dumps([sql, []]).encode(),
headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(request, timeout=10) as response:
return json.loads(response.read().decode())
def quote_identifier(value):
return '"' + value.replace('"', '""') + '"'
tables = []
for row in query("SELECT name FROM sqlite_schema WHERE type = 'table' AND name NOT LIKE 'sqlite_%' ORDER BY name"):
name = row[0] if row else None
if not isinstance(name, str):
continue
identifier = quote_identifier(name)
columns = [column[1] for column in query(f"PRAGMA table_info({identifier})") if len(column) > 1]
rows = query(f"SELECT * FROM {identifier} LIMIT {limit}")
tables.append({"name": name, "columns": columns, "rows": rows})
print(json.dumps({"limit": limit, "tables": tables}, separators=(",", ":")))
PY
PYTHON;
return $this->runServerSshCommand($server, str_replace(
['__LIMIT__', '__PORT__'],
[(string) $limit, (string) $cluster->corrosion_api_port],
$script,
), 'SSH corrosion table command failed.');
}
public function firewallRules(Request $request, V5Cluster $cluster, V5Server $server, FluxClient $fluxClient): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('viewDiagnostics', [$server, $currentTeam, $cluster]);
$validated = $request->validate([
'namespace' => ['sometimes', 'string', 'max:63'],
]);
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
return response()->json([
'message' => 'This server is missing its Flux host id.',
], 422);
}
try {
$rules = $fluxClient->listFirewallRules($hostId, (string) ($validated['namespace'] ?? ''));
} catch (\Throwable $e) {
Log::warning('V5 firewall rules request failed', [
'server_id' => $server->id,
'message' => $e->getMessage(),
]);
return response()->json([
'message' => 'Could not fetch firewall rules through Flux. Check the Flux and coold status, then try again.',
], 502);
}
return response()->json([
'rules' => $rules,
'source' => 'flux',
'fetchedAt' => now()->toJSON(),
]);
}
public function bootstrap(Request $request, V5Cluster $cluster, V5Server $server): JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('bootstrap', [$server, $currentTeam, $cluster]);
// Fail fast: the bootstrap job hard-fails on Flux enrollment (after
// the WireGuard mesh is already built) when no Flux URL is configured.
if (trim((string) config('coold.flux_url', '')) === '') {
return response()->json([
'message' => 'COOLIFY_COOLD_FLUX_URL is not configured, so bootstrapped servers cannot be enrolled into Flux. Set it and retry the bootstrap.',
], 422);
}
if ($server->last_bootstrapped_at !== null) {
return response()->json([
'message' => 'This server is already bootstrapped.',
], 409);
}
$installedServers = $cluster->servers()
->with('privateKey')
->whereNotNull('last_bootstrapped_at')
->orderBy('name')
->get();
$server->load('privateKey');
$servers = $installedServers->toBase()
->push($server)
->unique('id')
->values();
if ($servers->contains(fn (V5Server $server) => ! $server->privateKey instanceof PrivateKey)) {
return response()->json([
'message' => 'The new server and every already-bootstrapped server in this cluster must have a private key before extending the cluster.',
], 422);
}
$claim = DB::transaction(function () use ($cluster, $server, $installedServers): array {
$clusterServers = $cluster->servers()->lockForUpdate()->get();
$activeServer = $clusterServers->first(fn (V5Server $candidate): bool => $this->hasActiveBootstrapClaim($candidate));
if ($activeServer instanceof V5Server) {
return ['claimed' => false, 'active_server_id' => $activeServer->id];
}
// Sweep provably dead claims (lost job or killed worker) so retries
// are possible and the UI reflects reality.
$clusterServers
->filter(fn (V5Server $candidate): bool => in_array($candidate->last_bootstrap_status, ['queued', 'running'], true))
->each(fn (V5Server $candidate) => $candidate->update([
'last_bootstrap_status' => 'failed',
'last_bootstrap_output' => 'The previous bootstrap attempt timed out or its worker died. Retry the bootstrap.',
]));
$server->update([
'last_bootstrap_action' => $installedServers->isEmpty() ? 'bootstrap' : 'extend',
'last_bootstrap_status' => 'queued',
'last_bootstrap_output' => "Queued Coolify bootstrap for {$server->name}.",
'last_bootstrap_ran_at' => now(),
]);
return ['claimed' => true, 'active_server_id' => null];
});
if (! $claim['claimed']) {
return response()->json([
'cluster' => app(ClusterSerializer::class)->serializeFresh($cluster),
'message' => $claim['active_server_id'] === $server->id
? 'Bootstrap is already queued or running for this server.'
: 'Another server bootstrap is already queued or running for this cluster.',
], 409);
}
V5ClusterUpdated::dispatch($currentTeam->id, $cluster->id);
V5BootstrapServerJob::dispatch($cluster->id, $server->id);
return response()->json([
'cluster' => app(ClusterSerializer::class)->serializeFresh($cluster),
'message' => 'Bootstrap queued.',
], 202);
}
public function destroy(Request $request, V5Cluster $cluster, V5Server $server): Response|JsonResponse
{
$currentTeam = $this->currentTeamOrFail($request);
$this->authorize('delete', [$server, $currentTeam, $cluster]);
if (V5Application::query()->where('server_id', $server->id)->exists()) {
return response()->json([
'message' => 'Delete or move applications from this server before deleting it.',
], 422);
}
$warning = null;
if ($server->last_bootstrapped_at !== null) {
if ($server->isIngress() && $server->status === ServerStatus::Installed->value) {
try {
StopCaddyIngress::run($server);
} catch (\Throwable $exception) {
report($exception);
$warning = 'Could not stop the Caddy ingress on the server before deleting it.';
}
}
if (! RemoveBootstrapMarker::run($server)) {
$warning = 'Could not clean up the server over SSH. Remove /etc/coolify/v5-node.json manually before re-adding this server.';
}
}
$server->delete();
return response()->json(array_filter([
'cluster' => app(ClusterSerializer::class)->serializeFresh($cluster),
'warning' => $warning,
]));
}
/**
* A queued claim is active while the job could still pick it up; a running
* claim is active until the job timeout (plus margin) has passed. Anything
* older is provably dead because the job runs with $tries = 1.
*/
private function hasActiveBootstrapClaim(V5Server $server): bool
{
$ranAt = $server->last_bootstrap_ran_at;
return match ($server->last_bootstrap_status) {
'queued' => $ranAt !== null && $ranAt->gt(now()->subMinutes(15)),
'running' => $ranAt !== null && $ranAt->gt(now()->subSeconds(V5BootstrapServerJob::TIMEOUT_SECONDS + 300)),
default => false,
};
}
/**
* @return array{listen_port: int|null, endpoint: string|null}
*/
private function devLimaWireguardOverrides(string $host, int $sshPort): array
{
if (! app()->environment(['local', 'development', 'testing']) || $host !== 'host.docker.internal') {
return ['listen_port' => null, 'endpoint' => null];
}
if ($sshPort < 60001 || $sshPort > 60009) {
return ['listen_port' => null, 'endpoint' => null];
}
$wireguardPort = $sshPort - 8180;
return [
'listen_port' => $wireguardPort,
'endpoint' => "host.lima.internal:{$wireguardPort}",
];
}
private function clusterServerCapacity(V5Cluster $cluster): ?int
{
$namespaceCount = max(1, count($cluster->namespaces ?? V5Cluster::DEFAULT_NAMESPACES));
[, $poolPrefix] = array_pad(explode('/', (string) $cluster->container_network_pool, 2), 2, null);
$containerPrefix = (int) $cluster->container_network_prefix;
if (! is_string($poolPrefix) || ! ctype_digit($poolPrefix) || $containerPrefix < (int) $poolPrefix || $containerPrefix > 32) {
return null;
}
$containerCapacity = intdiv(2 ** ($containerPrefix - (int) $poolPrefix), $namespaceCount);
[, $managementPrefix] = array_pad(explode('/', (string) $cluster->wireguard_management_pool, 2), 2, null);
$managementCapacity = is_string($managementPrefix) && ctype_digit($managementPrefix) && (int) $managementPrefix <= 30
? (2 ** (32 - (int) $managementPrefix)) - 2
: null;
return $managementCapacity === null ? $containerCapacity : min($containerCapacity, $managementCapacity);
}
private function noControlCharactersRule(): \Closure
{
return function (string $attribute, mixed $value, \Closure $fail): void {
if (! is_string($value)) {
return;
}
if (preg_match('/[\x00-\x1F\x7F]/', $value) === 1) {
$fail('The :attribute contains invalid control characters.');
}
};
}
private function hostPortRule(): \Closure
{
return function (string $attribute, mixed $value, \Closure $fail): void {
if ($value === null || $value === '') {
return;
}
if (! is_string($value)) {
$fail('The :attribute must be in host:port format.');
return;
}
$value = trim($value);
if (preg_match('/^\[(?<host>.+)]:(?<port>\d+)$/', $value, $matches) === 1) {
$host = trim((string) $matches['host']);
$port = trim((string) $matches['port']);
} else {
$separatorPosition = strrpos($value, ':');
if ($separatorPosition === false) {
$fail('The :attribute must be in host:port format.');
return;
}
$host = trim(substr($value, 0, $separatorPosition));
$port = trim(substr($value, $separatorPosition + 1));
if (str_contains($host, ':')) {
$fail('The :attribute must use [ipv6]:port format for IPv6 addresses.');
return;
}
}
if ($host === '' || $port === '' || ! ctype_digit($port) || (int) $port < 1 || (int) $port > 65535) {
$fail('The :attribute must be in host:port format.');
return;
}
$failed = false;
(new ValidServerIp)->validate($attribute, $host, function () use (&$failed): void {
$failed = true;
});
if ($failed) {
$fail('The :attribute must be in host:port format.');
}
};
}
}
+1
View File
@@ -92,6 +92,7 @@ class Kernel extends HttpKernel
'v5.authenticated' => [
'auth',
'verified',
'throttle:v5',
V5EnsureCurrentTeam::class,
],
+7 -3
View File
@@ -25,7 +25,13 @@ class EnsureCurrentTeam
abort(403, 'No team available for this user.');
}
session(['currentTeam' => $currentTeam]);
// The v4 UI stores a full Team model under the same session key and
// reads arbitrary columns off it, so only rewrite the session when the
// resolved team actually changed — and always store the full model.
if (data_get(session('currentTeam'), 'id') !== $currentTeam->id) {
session(['currentTeam' => $currentTeam]);
}
$request->attributes->set('v5.currentTeam', $currentTeam);
return $next($request);
@@ -37,7 +43,6 @@ class EnsureCurrentTeam
if ($sessionTeamId) {
$sessionTeam = $user->teams()
->select('teams.id', 'teams.name', 'teams.description', 'teams.personal_team')
->whereKey($sessionTeamId)
->first();
@@ -47,7 +52,6 @@ class EnsureCurrentTeam
}
return $user->teams()
->select('teams.id', 'teams.name', 'teams.description', 'teams.personal_team')
->orderBy('teams.id')
->first();
}
+435 -32
View File
@@ -3,39 +3,47 @@
namespace App\Jobs;
use App\Actions\V5\Proxy\StartCaddyIngress;
use App\Enums\V5\ServerStatus;
use App\Events\V5ClusterUpdated;
use App\Models\PrivateKey;
use App\Models\V5\Cluster as V5Cluster;
use App\Models\V5\Server as V5Server;
use App\Services\Flux\AgentTokenIssuer;
use App\Services\Flux\FluxClient;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeEncrypted;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\Middleware\WithoutOverlapping;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Process;
class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldQueue
class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldBeUnique, ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
private const BOOTSTRAP_MARKER_PATH = '/etc/coolify/v5-node.json';
public const TIMEOUT_SECONDS = 7200;
public int $tries = 1;
public int $timeout = 7200;
public int $timeout = self::TIMEOUT_SECONDS;
/**
* Second idempotency layer on top of the controller's DB bootstrap claim,
* aligned with its running-claim window (TIMEOUT_SECONDS plus margin).
*/
public int $uniqueFor = self::TIMEOUT_SECONDS + 300;
public function __construct(public int $clusterId, public int $serverId) {}
/**
* @return array<int, object>
*/
public function middleware(): array
public function uniqueId(): string
{
return [(new WithoutOverlapping("v5-bootstrap-server-{$this->serverId}"))->expireAfter(7200)->dontRelease()];
return (string) $this->serverId;
}
public function handle(): void
@@ -58,18 +66,28 @@ class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldQueue
->unique('id')
->values();
$started = V5Server::query()
->whereKey($server->id)
->where('last_bootstrap_status', 'queued')
->update([
'last_bootstrap_action' => $action,
'last_bootstrap_status' => 'running',
'last_bootstrap_output' => "Starting Coolify CLI {$action} for {$server->name}...",
'last_bootstrap_ran_at' => now(),
]);
if ($started === 0) {
return;
}
$server->refresh();
if ($servers->contains(fn (V5Server $server) => ! $server->privateKey instanceof PrivateKey)) {
$this->markFailed($server, $action, 'The new server and every already-bootstrapped server in this cluster must have a private key before extending the cluster.');
return;
}
$server->update([
'last_bootstrap_action' => $action,
'last_bootstrap_status' => 'running',
'last_bootstrap_output' => "Starting Coolify CLI {$action} for {$server->name}...",
'last_bootstrap_ran_at' => now(),
]);
$this->broadcastClusterUpdated($server);
$keyDirectory = storage_path('app/ssh/keys');
@@ -89,18 +107,23 @@ class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldQueue
$existingBootstrap = $this->detectExistingBootstrap($server, $sshConfigLocation);
if (($existingBootstrap['cluster_id'] ?? null) !== null) {
if ((string) $existingBootstrap['cluster_id'] !== (string) $cluster->id) {
$markerClusterUuid = $existingBootstrap['cluster_uuid'] ?? null;
if (
(string) $existingBootstrap['cluster_id'] !== (string) $cluster->id
|| (is_string($markerClusterUuid) && $markerClusterUuid !== $cluster->uuid)
) {
$this->markFailed($server, $action, 'This server is already bootstrapped for another cluster. Reset the host bootstrap state before joining this cluster.');
return;
}
$this->adoptExistingBootstrap($server, $existingBootstrap);
$this->adoptExistingBootstrap($cluster, $server, $existingBootstrap, $sshConfigLocation);
return;
}
$result = Process::timeout(300)
$result = Process::timeout(7200)
->run($this->bootstrapCommand($cluster, $servers, $server, $sshConfigLocation, $action));
$output = trim($result->output()."\n".$result->errorOutput());
$successful = $result->successful();
@@ -117,22 +140,26 @@ class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldQueue
return;
}
$capabilities = collect($server->capabilities ?? [])
->push('coold')
->when($server->isIngress(), fn ($capabilities) => $capabilities->push('ingress'))
->unique()
->values()
->all();
$this->persistBootstrapAssignments($cluster, $server, $result->output(), $sshConfigLocation);
$server->refresh();
// Resolve the coold version once so the on-host marker and the
// database row always agree.
$cooldVersion = $this->bootstrappedCooldVersion($cluster, $result->output());
$this->writeBootstrapMarker($cluster, $server, $sshConfigLocation, $cooldVersion);
$this->enrollCooldIntoFlux($server, $sshConfigLocation);
$this->waitForFluxHostConnection($server);
$server->update([
'status' => 'installed',
'capabilities' => $capabilities,
'status' => ServerStatus::Installed->value,
'has_coold' => true,
'coold_version' => $cooldVersion,
'last_bootstrapped_at' => now(),
]);
$this->broadcastClusterUpdated($server);
$this->writeBootstrapMarker($cluster, $server, $sshConfigLocation);
if ($server->isIngress()) {
StartCaddyIngress::run($server->fresh('privateKey'));
}
@@ -183,11 +210,13 @@ class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldQueue
'init',
$action,
'--format',
'table',
'json',
'--nodes',
$servers->map(fn (V5Server $server) => $this->bootstrapNode($server))->implode(','),
'--ssh-config',
$sshConfigLocation,
'--ssh-user',
$newServer->ssh_user,
'--namespaces',
implode(',', $cluster->namespaces ?? V5Cluster::DEFAULT_NAMESPACES),
'--container-pool',
@@ -308,14 +337,17 @@ class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldQueue
/**
* @param array<string, mixed> $marker
*/
private function adoptExistingBootstrap(V5Server $server, array $marker): void
private function adoptExistingBootstrap(V5Cluster $cluster, V5Server $server, array $marker, string $sshConfigLocation): void
{
$bootstrapNode = $this->bootstrapNode($server);
$serverUuid = is_string($marker['server_uuid'] ?? null) ? $marker['server_uuid'] : null;
$updates = [
'wireguard_management_ip' => is_string($marker['wireguard_management_ip'] ?? null) ? $marker['wireguard_management_ip'] : $server->wireguard_management_ip,
'wireguard_public_key' => is_string($marker['wireguard_public_key'] ?? null) ? $marker['wireguard_public_key'] : $server->wireguard_public_key,
'coold_version' => is_string($marker['coold_version'] ?? null) && trim($marker['coold_version']) !== '' ? trim($marker['coold_version']) : $cluster->coold_version,
'container_subnets' => is_array($marker['container_subnets'] ?? null) ? $marker['container_subnets'] : $server->container_subnets,
'status' => 'installed',
'has_coold' => true,
'status' => ServerStatus::Installed->value,
'last_bootstrap_status' => 'succeeded',
'last_bootstrap_output' => 'Adopted existing Coolify bootstrap state for this cluster.',
'last_bootstrap_ran_at' => now(),
@@ -329,28 +361,399 @@ class V5BootstrapServerJob implements ShouldBeEncrypted, ShouldQueue
$server->update($updates);
$this->broadcastClusterUpdated($server);
$this->enrollCooldIntoFlux($server->fresh(), $sshConfigLocation, $bootstrapNode);
$this->waitForFluxHostConnection($server->fresh());
if ($server->isIngress()) {
StartCaddyIngress::run($server->fresh('privateKey'));
}
}
private function writeBootstrapMarker(V5Cluster $cluster, V5Server $server, string $sshConfigLocation): void
private function persistBootstrapAssignments(V5Cluster $cluster, V5Server $server, string $output, string $sshConfigLocation): void
{
$verifiedNode = $this->verifiedBootstrapNode($output, $server);
$wireguardManagementIp = is_array($verifiedNode) && is_string($verifiedNode['wireguard_ip'] ?? null)
? $verifiedNode['wireguard_ip']
: null;
$warnings = [];
if (! is_string($wireguardManagementIp) || $wireguardManagementIp === '') {
$wireguardManagementIp = $this->readWireguardManagementIp($cluster, $server, $sshConfigLocation, $warnings);
}
$wireguardPublicKey = $this->readWireguardPublicKey($cluster, $server, $sshConfigLocation, $warnings);
$containerSubnets = $this->readContainerSubnets($cluster, $server, $sshConfigLocation, $warnings);
$updates = [];
if ($wireguardManagementIp !== null && $wireguardManagementIp !== '') {
$updates['wireguard_management_ip'] = $wireguardManagementIp;
if (! is_string($server->node_address) || $server->node_address === '' || $server->node_address === $server->host) {
$updates['node_address'] = $wireguardManagementIp;
}
} else {
$warnings[] = 'Warning: could not determine the WireGuard management IP from the CLI output.';
}
if ($wireguardPublicKey !== null && $wireguardPublicKey !== '') {
$updates['wireguard_public_key'] = $wireguardPublicKey;
}
if ($containerSubnets !== []) {
$updates['container_subnets'] = $containerSubnets;
}
if ($warnings !== []) {
$updates['last_bootstrap_output'] = str(trim($server->last_bootstrap_output."\n".implode("\n", $warnings)))
->limit(20000)
->toString();
}
if ($updates !== []) {
$server->update($updates);
}
}
/**
* @param array<int, string> $warnings
*/
private function readWireguardManagementIp(V5Cluster $cluster, V5Server $server, string $sshConfigLocation, array &$warnings): ?string
{
$interface = escapeshellarg($cluster->wireguard_interface);
$script = implode("\n", [
"SUDO=''",
'if [ "$(id -u)" != "0" ]; then SUDO=\'sudo\'; fi',
"\$SUDO ip -4 -o addr show dev {$interface} | awk '{print \$4}' | cut -d/ -f1 | head -n1",
]);
$result = Process::timeout(15)->run([
'ssh',
'-F',
$sshConfigLocation,
$this->bootstrapNode($server),
$script,
]);
$ipAddress = trim($result->output());
if (! $result->successful() || filter_var($ipAddress, FILTER_VALIDATE_IP, FILTER_FLAG_IPV4) === false) {
$warnings[] = 'Warning: could not read the WireGuard management IP from the server.';
return null;
}
return $ipAddress;
}
/**
* @return array<string, mixed>|null
*/
private function verifiedBootstrapNode(string $output, V5Server $server): ?array
{
$decoded = $this->decodedBootstrapOutput($output);
if (! is_array($decoded)) {
return null;
}
$verifiedNodes = data_get($decoded, 'verified');
if (! is_array($verifiedNodes)) {
return null;
}
$bootstrapNode = $this->bootstrapNode($server);
foreach ($verifiedNodes as $verifiedNode) {
if (! is_array($verifiedNode)) {
continue;
}
$host = $verifiedNode['host'] ?? $verifiedNode['node'] ?? $verifiedNode['name'] ?? null;
if ($host === $bootstrapNode || $host === $server->uuid || $host === $server->name || $host === $server->host) {
return $verifiedNode;
}
}
return null;
}
/**
* @return array<string, mixed>|null
*/
private function decodedBootstrapOutput(string $output): ?array
{
$output = trim($output);
if ($output === '' || ! str_starts_with($output, '{')) {
return null;
}
try {
$decoded = json_decode($output, true, flags: JSON_THROW_ON_ERROR);
} catch (\JsonException) {
return null;
}
return is_array($decoded) ? $decoded : null;
}
/**
* The CLI init JSON output does not currently report the installed coold
* version, so fall back to the version the cluster asked the CLI to
* install (`--coold-version`). If a future CLI adds a `coold_version` key
* to its JSON output, prefer that.
*/
private function bootstrappedCooldVersion(V5Cluster $cluster, string $output): ?string
{
$reported = data_get($this->decodedBootstrapOutput($output), 'coold_version');
if (is_string($reported) && trim($reported) !== '') {
return trim($reported);
}
return $cluster->coold_version;
}
/**
* @param array<int, string> $warnings
*/
private function readWireguardPublicKey(V5Cluster $cluster, V5Server $server, string $sshConfigLocation, array &$warnings): ?string
{
$interface = escapeshellarg($cluster->wireguard_interface);
$script = implode("\n", [
"SUDO=''",
'if [ "$(id -u)" != "0" ]; then SUDO=\'sudo\'; fi',
"\$SUDO wg show {$interface} public-key",
]);
$result = Process::timeout(15)->run([
'ssh',
'-F',
$sshConfigLocation,
$this->bootstrapNode($server),
$script,
]);
$publicKey = trim($result->output());
if (! $result->successful() || $publicKey === '') {
$warnings[] = 'Warning: could not read the WireGuard public key from the server.';
return null;
}
return $publicKey;
}
/**
* The container subnets are allocated by the coolify CLI on the host; the podman
* networks it creates are the source of truth, so read them back instead of
* re-deriving the allocation locally.
*
* @param array<int, string> $warnings
* @return array<string, string>
*/
private function readContainerSubnets(V5Cluster $cluster, V5Server $server, string $sshConfigLocation, array &$warnings): array
{
$namespaces = $cluster->namespaces ?? V5Cluster::DEFAULT_NAMESPACES;
if ($namespaces === []) {
return [];
}
$namespaceArguments = collect($namespaces)
->map(fn (string $namespace): string => escapeshellarg($namespace))
->implode(' ');
$script = implode("\n", [
"SUDO=''",
'if [ "$(id -u)" != "0" ]; then SUDO=\'sudo\'; fi',
"for ns in {$namespaceArguments}; do",
' printf \'%s=\' "$ns"',
' $SUDO podman network inspect "coolify-${ns}-mesh" --format \'{{range .Subnets}}{{.Subnet}}{{end}}\' 2>/dev/null || true',
' printf \'\n\'',
'done',
]);
$result = Process::timeout(30)->run([
'ssh',
'-F',
$sshConfigLocation,
$this->bootstrapNode($server),
$script,
]);
if (! $result->successful()) {
$warnings[] = 'Warning: could not read the container subnets from the server.';
return [];
}
$subnets = [];
foreach (preg_split('/\r?\n/', trim($result->output())) ?: [] as $line) {
[$namespace, $subnet] = array_pad(explode('=', trim($line), 2), 2, null);
if (! is_string($namespace) || ! in_array($namespace, $namespaces, true) || ! $this->isIpv4Cidr($subnet)) {
continue;
}
$subnets[$namespace] = $subnet;
}
if (count($subnets) !== count($namespaces)) {
$warnings[] = 'Warning: could not read every container subnet from the server; the stored subnets may be incomplete.';
}
return $subnets;
}
private function isIpv4Cidr(?string $value): bool
{
if (! is_string($value) || ! str_contains($value, '/')) {
return false;
}
[$ip, $prefix] = explode('/', $value, 2);
return filter_var($ip, FILTER_VALIDATE_IP, FILTER_FLAG_IPV4) !== false
&& ctype_digit($prefix)
&& (int) $prefix <= 32;
}
private function enrollCooldIntoFlux(V5Server $server, string $sshConfigLocation, ?string $bootstrapNode = null): void
{
$fluxUrl = trim((string) config('coold.flux_url', ''));
if ($fluxUrl === '') {
throw new \RuntimeException('COOLIFY_COOLD_FLUX_URL is not configured, so the server cannot be enrolled into Flux. Set it and retry the bootstrap.');
}
$jwtPath = trim((string) config('coold.flux_host_jwt_path', '/etc/coolify/host-jwt'));
if ($jwtPath === '') {
$jwtPath = '/etc/coolify/host-jwt';
}
$fluxUrl = str_replace(["\r", "\n"], '', $fluxUrl);
$jwtPath = str_replace(["\r", "\n"], '', $jwtPath);
$hostId = $server->fluxHostId();
$token = app(AgentTokenIssuer::class)->issueForServer($server);
$tokenArgument = $this->shellArg($token);
$hostId = str_replace(["\r", "\n"], '', $hostId);
$jwtPathArgument = $this->shellPathArg($jwtPath);
$dropInDirectory = '/etc/systemd/system/coold.service.d';
$dropInPath = "{$dropInDirectory}/10-flux.conf";
$script = <<<SH
set -e
SUDO=''
if [ "\$(id -u)" != "0" ]; then SUDO='sudo'; fi
\$SUDO mkdir -p /etc/coolify {$dropInDirectory}
printf %s {$tokenArgument} | \$SUDO tee {$jwtPathArgument} >/dev/null
\$SUDO chmod 600 {$jwtPathArgument}
cat <<'COOLIFY_FLUX_ENV' | \$SUDO tee {$dropInPath} >/dev/null
[Service]
Environment=COOLIFY_COOLD_FLUX_URL={$fluxUrl}
Environment=COOLIFY_COOLD_HOST_ID={$hostId}
Environment=COOLIFY_COOLD_HOST_JWT_PATH={$jwtPath}
COOLIFY_FLUX_ENV
\$SUDO systemctl daemon-reload
\$SUDO systemctl restart coold.service
SH;
$result = Process::timeout(60)->run([
'ssh',
'-F',
$sshConfigLocation,
$bootstrapNode ?? $this->bootstrapNode($server),
$script,
]);
if (! $result->successful()) {
$output = trim($result->output()."\n".$result->errorOutput());
throw new \RuntimeException(
($output !== '' ? $output : 'Could not enroll coold into Flux.')
."\nThe WireGuard mesh was created successfully; retrying this bootstrap is safe and will resume from Flux enrollment."
);
}
}
private function waitForFluxHostConnection(V5Server $server): void
{
$timeoutSeconds = (int) config('flux.bootstrap_host_connection_timeout_seconds', 30);
if ($timeoutSeconds <= 0) {
return;
}
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
throw new \RuntimeException('Server is missing its Flux host id after bootstrap.');
}
$deadline = time() + $timeoutSeconds;
$lastError = null;
do {
try {
app(FluxClient::class)->cooldLogs($hostId, 1);
return;
} catch (\Throwable $exception) {
$lastError = $exception->getMessage();
sleep(1);
}
} while (time() < $deadline);
throw new \RuntimeException(
'The server was bootstrapped, but coold did not connect to Flux in time. '
.'Wait a moment and retry the bootstrap before deploying applications.'
.($lastError !== null ? " Last Flux error: {$lastError}" : '')
);
}
private function shellArg(string $value): string
{
return escapeshellarg($value);
}
private function shellPathArg(string $value): string
{
if (preg_match('/^[A-Za-z0-9_\/:.,@%+=-]+$/', $value) === 1) {
return $value;
}
return $this->shellArg($value);
}
private function writeBootstrapMarker(V5Cluster $cluster, V5Server $server, string $sshConfigLocation, ?string $cooldVersion = null): void
{
$payload = base64_encode(json_encode([
'cluster_id' => $cluster->id,
'cluster_uuid' => $cluster->uuid,
'server_uuid' => $server->uuid,
'wireguard_management_ip' => $server->wireguard_management_ip,
'wireguard_public_key' => $server->wireguard_public_key,
'coold_version' => $cooldVersion ?? $server->coold_version ?? $cluster->coold_version,
'container_subnets' => $server->container_subnets ?? [],
], JSON_THROW_ON_ERROR));
Process::timeout(15)->run([
$result = Process::timeout(15)->run([
'ssh',
'-F',
$sshConfigLocation,
$this->bootstrapNode($server),
"payload='{$payload}'; if [ \"$(id -u)\" = \"0\" ]; then mkdir -p /etc/coolify && printf %s \"$payload\" | base64 -d > ".escapeshellarg(self::BOOTSTRAP_MARKER_PATH)."; else sudo mkdir -p /etc/coolify && printf %s \"$payload\" | base64 -d | sudo tee ".escapeshellarg(self::BOOTSTRAP_MARKER_PATH).' >/dev/null; fi',
]);
if (! $result->successful()) {
$output = trim($result->output()."\n".$result->errorOutput());
throw new \RuntimeException('Could not write the bootstrap marker to the server: '.($output !== '' ? $output : 'the SSH command failed.'));
}
}
/**
+53
View File
@@ -0,0 +1,53 @@
<?php
namespace App\Jobs;
use App\Actions\V5\Application\DeployNginxApplication;
use App\Models\V5\Application as V5Application;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
class V5DeployApplicationJob implements ShouldBeUnique, ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public int $tries = 1;
public int $timeout = 300;
/**
* Job timeout plus a safety margin so a lost lock can never block
* redeploys of the same application forever.
*/
public int $uniqueFor = 360;
public function __construct(public int $applicationId) {}
public function uniqueId(): string
{
return (string) $this->applicationId;
}
public function handle(): void
{
$application = V5Application::query()->find($this->applicationId);
if (! $application instanceof V5Application) {
return;
}
DeployNginxApplication::run($application);
}
public function failed(?\Throwable $exception): void
{
V5Application::query()->find($this->applicationId)?->update([
'status' => 'failed',
'status_message' => str($exception?->getMessage() ?? 'The deploy job failed.')->limit(10000)->toString(),
]);
}
}
+255
View File
@@ -0,0 +1,255 @@
<?php
namespace App\Jobs;
use App\Enums\V5\ApplicationStatus;
use App\Enums\V5\ContainerState;
use App\Enums\V5\ServerStatus;
use App\Models\V5\Application as V5Application;
use App\Models\V5\ContainerStatus;
use App\Models\V5\Server as V5Server;
use App\Services\Flux\FluxClient;
use App\Support\V5\StatusObservation;
use Carbon\CarbonImmutable;
use Carbon\CarbonInterface;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Log;
/**
* Actively reconciles one v5 server against the containers coold actually
* reports. V5 status is normally push-only (coold -> flux -> webhook), so a
* dropped webhook leaves rows stale forever; this job is the pull-based
* safety net scheduled via V5ReconcileServersJob.
*/
class V5ReconcileServerStateJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Reconcile runs on its own queue so the 5-minute fleet fan-out (one
* blocking flux call per server) can never starve user-triggered deploys
* and bootstraps sharing the default queue. Set via onQueue() in the
* constructor rather than a `$queue` property redeclaration, which the
* Queueable trait already defines (redeclaring with a default is an
* incompatible property composition and fatals on PHP 8.5).
*/
public int $tries = 1;
public int $timeout = 120;
public function __construct(public int $serverId)
{
$this->onQueue('v5-reconcile');
}
public function handle(FluxClient $fluxClient): void
{
$server = V5Server::query()->find($this->serverId);
if (! $server instanceof V5Server) {
return;
}
$hostId = $server->fluxHostId();
if ($hostId === '') {
Log::warning('V5 reconcile skipped: server is missing a Flux host id.', ['server_id' => $server->id]);
return;
}
// The moment we query coold is the observation time for every row this
// pass writes; a webhook that lands with a newer observation while this
// (possibly delayed) snapshot is processed must win the watermark.
$observedAt = CarbonImmutable::now();
try {
$containers = collect($fluxClient->listContainers($hostId));
} catch (\Throwable $exception) {
$this->markServerUnreachable($server, $exception, $observedAt);
return;
}
$this->markServerReachable($server, $containers->count(), $observedAt);
$this->refreshContainerStatuses($server, $containers, $observedAt);
$this->reconcileApplications($server, $containers, $observedAt);
}
private function markServerUnreachable(V5Server $server, \Throwable $exception, CarbonInterface $observedAt): void
{
Log::warning('V5 reconcile could not reach the server via flux.', [
'server_id' => $server->id,
'error' => $exception->getMessage(),
]);
$attributes = [
'last_status_check' => 'reconcile',
'last_status_output' => str($exception->getMessage())->limit(1000)->toString(),
'last_status_checked_at' => now(),
];
if (! StatusObservation::isStale($observedAt, $server->status_observed_at, 'server status', ['server_id' => $server->id])) {
// Only an installed server can degrade to unreachable; added or
// failed servers keep their bootstrap-driven status.
$attributes['status'] = $server->status === ServerStatus::Installed->value
? ServerStatus::Unreachable->value
: $server->status;
$attributes['status_observed_at'] = $observedAt;
}
$server->update($attributes);
}
private function markServerReachable(V5Server $server, int $containerCount, CarbonInterface $observedAt): void
{
$attributes = [
'last_status_check' => 'reconcile',
'last_status_output' => "Reconciled {$containerCount} containers from coold.",
'last_status_checked_at' => now(),
];
if (! StatusObservation::isStale($observedAt, $server->status_observed_at, 'server status', ['server_id' => $server->id])) {
$attributes['status'] = $server->status === ServerStatus::Unreachable->value
? ServerStatus::Installed->value
: $server->status;
$attributes['status_observed_at'] = $observedAt;
}
$server->update($attributes);
}
/**
* @param Collection<int, mixed> $containers
*/
private function refreshContainerStatuses(V5Server $server, Collection $containers, CarbonInterface $observedAt): void
{
$containers->each(function (mixed $container) use ($server, $observedAt): void {
if (! is_array($container) || ! is_string($container['id'] ?? null) || $container['id'] === '') {
return;
}
$existing = ContainerStatus::query()
->where('server_id', $server->id)
->where('container_id', $container['id'])
->first();
if (StatusObservation::isStale($observedAt, $existing?->status_observed_at, 'container status', [
'server_id' => $server->id,
'container_id' => $container['id'],
])) {
return;
}
ContainerStatus::query()->updateOrCreate([
'server_id' => $server->id,
'container_id' => $container['id'],
], [
'team_id' => $server->team_id,
'container_name' => is_string($container['name'] ?? null) ? $container['name'] : null,
'image' => is_string($container['image'] ?? null) ? $container['image'] : null,
'status' => $this->containerState($container, ContainerState::class),
'status_message' => 'Container state reconciled from coold.',
'status_observed_at' => $observedAt,
'last_seen_at' => now(),
]);
});
}
/**
* @param Collection<int, mixed> $containers
*/
private function reconcileApplications(V5Server $server, Collection $containers, CarbonInterface $observedAt): void
{
V5Application::query()
->where('server_id', $server->id)
->get()
->each(function (V5Application $application) use ($containers, $observedAt): void {
try {
$this->reconcileApplication($application, $containers, $observedAt);
} catch (\Throwable $exception) {
Log::warning('V5 reconcile failed for an application.', [
'application_id' => $application->id,
'error' => $exception->getMessage(),
]);
}
});
}
/**
* @param Collection<int, mixed> $containers
*/
private function reconcileApplication(V5Application $application, Collection $containers, CarbonInterface $observedAt): void
{
$container = $containers->first(function (mixed $container) use ($application): bool {
return is_array($container)
&& (($application->runtime_container_id !== null && ($container['id'] ?? null) === $application->runtime_container_id)
|| ($container['name'] ?? null) === $application->container_name);
});
if (! is_array($container)) {
// A creating application without a container id simply has not
// materialized yet; the deploy job will settle it.
if ($application->status === ApplicationStatus::Creating->value && $application->runtime_container_id === null) {
return;
}
if (StatusObservation::isStale($observedAt, $application->status_observed_at, 'application status', ['application_id' => $application->id])) {
return;
}
$attributes = [
'status' => ApplicationStatus::Exited->value,
'status_observed_at' => $observedAt,
];
if ($application->status !== ApplicationStatus::Exited->value) {
$attributes['status_message'] = 'Container not found on server during reconcile.';
}
$application->update($attributes);
return;
}
if (StatusObservation::isStale($observedAt, $application->status_observed_at, 'application status', ['application_id' => $application->id])) {
return;
}
$status = $this->containerState($container, ApplicationStatus::class);
$attributes = [
'status' => $status,
'status_observed_at' => $observedAt,
'runtime_container_id' => is_string($container['id'] ?? null) && $container['id'] !== ''
? $container['id']
: $application->runtime_container_id,
];
// Only write status_message when the status actually changes: the
// status column is what a viewer cares about, and a constant message
// would otherwise fire a broadcast + full re-serialization every cycle.
if ($status !== $application->status) {
$attributes['status_message'] = 'Container state reconciled from coold.';
}
$application->update($attributes);
}
/**
* @param array<string, mixed> $container
* @param class-string<ApplicationStatus|ContainerState> $enumClass
*/
private function containerState(array $container, string $enumClass): string
{
$state = $container['state'] ?? null;
$raw = is_string($state) && $state !== '' ? $state : null;
return StatusObservation::normalize($raw, $enumClass) ?? $enumClass::Unknown->value;
}
}
+89
View File
@@ -0,0 +1,89 @@
<?php
namespace App\Jobs;
use App\Enums\V5\ServerStatus;
use App\Models\V5\Application as V5Application;
use App\Models\V5\ContainerStatus;
use App\Models\V5\Server as V5Server;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;
/**
* Scheduled fan-out for the v5 reconciliation loop: dispatches one
* V5ReconcileServerStateJob per managed server and prunes container status
* rows that no webhook has refreshed within the TTL.
*/
class V5ReconcileServersJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public const CONTAINER_STATUS_TTL_HOURS = 24;
public int $tries = 1;
public int $timeout = 60;
/**
* Reconcile runs on its own queue so the 5-minute fleet fan-out can never
* starve user-triggered deploys and bootstraps sharing the default queue.
* Set via onQueue() rather than a `$queue` property redeclaration, which
* the Queueable trait already defines (redeclaring with a default is an
* incompatible property composition and fatals on PHP 8.5).
*/
public function __construct()
{
$this->onQueue('v5-reconcile');
}
public function handle(): void
{
$this->dispatchReconcileJobs();
$this->pruneContainerStatuses();
}
private function dispatchReconcileJobs(): void
{
V5Server::query()
// Unreachable servers stay in the loop so a recovered node is
// restored to installed by its next successful reconcile.
->whereIn('status', [ServerStatus::Installed->value, ServerStatus::Unreachable->value])
->where('has_coold', true)
->get()
->each(function (V5Server $server): void {
try {
V5ReconcileServerStateJob::dispatch($server->id);
} catch (\Throwable $exception) {
Log::warning('V5 reconcile dispatch failed for a server.', [
'server_id' => $server->id,
'error' => $exception->getMessage(),
]);
}
});
}
private function pruneContainerStatuses(): void
{
$cutoff = now()->subHours(self::CONTAINER_STATUS_TTL_HOURS);
$liveContainerIds = V5Application::query()
->whereNotNull('runtime_container_id')
->pluck('runtime_container_id')
->all();
ContainerStatus::query()
->where(function ($query) use ($cutoff): void {
$query
->where('last_seen_at', '<', $cutoff)
->orWhere(function ($query) use ($cutoff): void {
$query->whereNull('last_seen_at')->where('created_at', '<', $cutoff);
})
->orWhereNotIn('server_id', V5Server::query()->select('id'));
})
->when($liveContainerIds !== [], fn ($query) => $query->whereNotIn('container_id', $liveContainerIds))
->delete();
}
}
+150
View File
@@ -0,0 +1,150 @@
<?php
namespace App\Jobs;
use App\Actions\V5\Server\PushHostAgentToken;
use App\Enums\V5\ServerStatus;
use App\Models\V5\Server as V5Server;
use App\Services\Flux\AgentTokenIssuer;
use App\Services\Flux\FluxClient;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
/**
* Re-mints and delivers a fresh host JWT for one managed server before its
* on-disk token expires.
*
* RPC-FIRST, SSH-FALLBACK: the new token is delivered over the live coold RPC
* stream by default (Laravel -> flux UDS -> coold's `host.jwt.set` command),
* because that reuses the already authenticated flux<->coold channel and works
* while the CURRENT token is still valid which is exactly when rotation runs
* (at ~12h remaining, well before the 24h exp). Only if the RPC push fails (the
* host's stream is down because its token already lapsed, flux rejects the verb,
* a timeout, etc.) do we fall back to the SSH push, which recovers a node whose
* token already expired and whose stream is therefore gone.
*
* PUSH-THEN-PERSIST: the new token is delivered to the host FIRST, and the
* server's jti/expires_at are only advanced AFTER a successful delivery via
* EITHER path. If both delivery paths fail the DB is left untouched, so the old
* expires_at keeps the server inside the dispatcher's rotation window and the
* next cycle simply retries we never advance the watermark on a token the
* host never received (which would strand the host on the expiring old token
* until it fully lapsed).
*
* NO-REVOKE-ON-ROTATION: the previously issued jti is intentionally NOT revoked
* here. The old token is still legitimately valid until its own exp and coold
* may still be connected on it; revoking it would risk cutting the live stream.
* Revocation belongs to teardown/re-home (RemoveBootstrapMarker), not routine
* rotation the old token simply ages out on its own exp.
*/
class V5RotateAgentTokenJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public int $tries = 3;
public int $timeout = 60;
/**
* Rotation shares the reconcile queue so the hourly fleet fan-out can never
* starve user-triggered deploys and bootstraps on the default queue. Set via
* onQueue() rather than a `$queue` property redeclaration, which the
* Queueable trait already defines (redeclaring with a default is an
* incompatible property composition and fatals on PHP 8.5).
*/
public function __construct(public int $serverId)
{
$this->onQueue('v5-reconcile');
}
public function handle(): void
{
$server = V5Server::query()->with('privateKey')->find($this->serverId);
if (! $server instanceof V5Server) {
return;
}
if (! $this->isEligible($server)) {
return;
}
$hostId = $server->fluxHostId();
if ($hostId === '') {
Log::warning('V5 token rotation skipped: server is missing a Flux host id.', ['server_id' => $server->id]);
return;
}
$ttl = (int) config('flux.host_token_ttl');
$jti = (string) Str::uuid();
$token = app(AgentTokenIssuer::class)->issue($hostId, null, $ttl, [
'jti' => $jti,
'team_id' => (string) $server->team_id,
'cluster_id' => (string) $server->cluster_id,
'server_id' => $hostId,
'wireguard_management_ip' => (string) $server->wireguard_management_ip,
]);
$delivery = $this->deliverToken($server, $hostId, $token);
if ($delivery === null) {
Log::warning('V5 token rotation could not deliver the new host token; leaving the existing token in place.', [
'server_id' => $server->id,
'host' => $server->host,
]);
return;
}
$server->update([
'agent_token_jti' => $jti,
'agent_token_expires_at' => now()->addSeconds($ttl),
]);
Log::debug('V5 token rotation delivered a fresh host token.', [
'server_id' => $server->id,
'delivery' => $delivery,
]);
}
/**
* Deliver the freshly minted token to the host, preferring the live coold
* RPC stream and falling back to the SSH push on any RPC failure.
*
* @return 'rpc'|'ssh'|null The path that succeeded, or null if both failed.
*/
private function deliverToken(V5Server $server, string $hostId, string $token): ?string
{
try {
app(FluxClient::class)->pushHostToken($hostId, $token);
return 'rpc';
} catch (\Throwable $exception) {
Log::info('V5 token rotation RPC push failed; falling back to SSH.', [
'server_id' => $server->id,
'error' => $exception->getMessage(),
]);
}
if (PushHostAgentToken::run($server, $token)) {
return 'ssh';
}
return null;
}
private function isEligible(V5Server $server): bool
{
return $server->status === ServerStatus::Installed->value
&& (bool) $server->has_coold
&& $server->last_bootstrapped_at !== null;
}
}
+65
View File
@@ -0,0 +1,65 @@
<?php
namespace App\Jobs;
use App\Enums\V5\ServerStatus;
use App\Models\V5\Server as V5Server;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;
/**
* Scheduled fan-out for host JWT rotation: dispatches one V5RotateAgentTokenJob
* per managed server whose on-disk token is missing or within the configured
* refresh threshold of expiry, so a fresh token is always on disk before the
* current one lapses.
*/
class V5RotateAgentTokensJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public int $tries = 1;
public int $timeout = 60;
/**
* Rotation shares the reconcile queue so the hourly fleet fan-out can never
* starve user-triggered deploys and bootstraps on the default queue. Set via
* onQueue() rather than a `$queue` property redeclaration, which the
* Queueable trait already defines (redeclaring with a default is an
* incompatible property composition and fatals on PHP 8.5).
*/
public function __construct()
{
$this->onQueue('v5-reconcile');
}
public function handle(): void
{
$threshold = now()->addSeconds((int) config('flux.host_token_refresh_threshold'));
V5Server::query()
->where('status', ServerStatus::Installed->value)
->where('has_coold', true)
->whereNotNull('last_bootstrapped_at')
->where(function ($query) use ($threshold): void {
$query
->whereNull('agent_token_expires_at')
->orWhere('agent_token_expires_at', '<', $threshold);
})
->get()
->each(function (V5Server $server): void {
try {
V5RotateAgentTokenJob::dispatch($server->id);
} catch (\Throwable $exception) {
Log::warning('V5 token rotation dispatch failed for a server.', [
'server_id' => $server->id,
'error' => $exception->getMessage(),
]);
}
});
}
}
+325
View File
@@ -0,0 +1,325 @@
<?php
namespace App\Jobs;
use App\Actions\V5\Application\DestroyNginxApplication;
use App\Actions\V5\Proxy\StopCaddyIngress;
use App\Actions\V5\Server\RemoveBootstrapMarker;
use App\Enums\V5\ServerStatus;
use App\Models\PrivateKey;
use App\Models\Team;
use App\Models\V5\Application as V5Application;
use App\Models\V5\Server as V5Server;
use App\Services\Flux\AgentTokenIssuer;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Schema;
/**
* Best-effort, on-host teardown for a team that is being deleted.
*
* Deleting a v4 Team DB-cascades every v5_servers / v5_applications /
* v5_container_statuses / v5_resource_connections row (see the
* cascadeOnDelete() foreign keys in the v5 migrations) WITHOUT running the
* app-level teardown that the per-resource destroy flows use. That would leave
* orphaned podman containers, a running Caddy ingress, the WireGuard mesh and
* coold itself alive on every host with no DB record left to reach them.
*
* This job mirrors the ServerController::destroy / ApplicationController::destroy
* teardown sequence for every v5 server owned by the team:
* 1. remove each application's container (DestroyNginxApplication),
* 2. stop the Caddy ingress on ingress servers (StopCaddyIngress),
* 3. remove the on-host bootstrap identity marker, host-jwt and Flux
* drop-in (RemoveBootstrapMarker).
*
* Because the cascade deletes the servers, applications and private keys the
* moment the team is gone, the payload is captured at dispatch time (from the
* Team `deleting` hook, which fires BEFORE the rows vanish) as plain arrays,
* including the SSH private-key material needed to reach each host. The job
* rebuilds in-memory, non-persisted models from that payload so it can reuse
* the exact same actions without touching the (now missing) DB rows.
*
* BEST-EFFORT / LIMITATIONS: teardown is best-effort. Each host and each action
* is guarded so a single unreachable host can never abort teardown of the other
* hosts, and the team deletion itself never fails because of teardown. The
* payload is fully self-contained (host, SSH creds/private key, applications,
* token jti + expiry), so a framework-level queue retry is safe every step is
* idempotent (podman rm -f / ingress stop / rm -f are all no-ops when the target
* is already gone).
*
* RESIDUAL LIMITATION: a host that is unreachable at team-deletion time orphans
* its containers, ingress and mesh PERMANENTLY the DB rows the reconcilers key
* off are gone, so there is no later reconciliation. The single operator-facing
* signal is the `Log::error` emitted at the end of handle() listing the host
* ids/hosts that could not be torn down; grep for "v5 team teardown incomplete"
* to find them.
*/
class V5TeardownTeamJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* A small retry budget: the payload is self-contained and every teardown
* step is idempotent, so retrying an unreachable host is safe. There is no
* point retrying forever the host may simply be gone.
*/
public int $tries = 3;
public int $timeout = 300;
/**
* @param array<int, array<string, mixed>> $servers Self-contained per-server teardown payload captured before the cascade.
*/
public function __construct(
public int $teamId,
public array $servers,
) {}
/**
* Collect the team's v5 servers (with their applications and SSH key
* material) into a self-contained payload and dispatch the teardown job.
*
* Must be called from the Team `deleting` hook, while the rows still exist.
* Returns without dispatching when the team owns no v5 servers.
*/
public static function dispatchForTeam(Team $team): void
{
// Guard against contexts where the v5 tables do not exist (e.g. v4-only
// schemas) so team deletion is never broken by this teardown.
if (! Schema::hasTable('v5_servers')) {
return;
}
$servers = V5Server::query()
->where('team_id', $team->id)
->with('privateKey')
->get();
if ($servers->isEmpty()) {
return;
}
$applicationsByServer = V5Application::query()
->where('team_id', $team->id)
->whereNotNull('server_id')
->get()
->groupBy('server_id');
$payload = $servers->map(function (V5Server $server) use ($applicationsByServer): array {
return [
'id' => $server->id,
'uuid' => $server->uuid,
'name' => $server->name,
'host' => $server->host,
'ssh_user' => $server->ssh_user,
'ssh_port' => (int) $server->ssh_port,
'node_address' => $server->node_address,
'wireguard_management_ip' => $server->wireguard_management_ip,
'is_ingress' => (bool) $server->is_ingress,
'ingress_type' => $server->ingress_type,
'status' => $server->status,
'last_bootstrapped_at' => $server->last_bootstrapped_at?->toISOString(),
// Captured before the cascade removes the row so the job can
// revoke the host token after the DB rows are gone.
'agent_token_jti' => $server->agent_token_jti,
'agent_token_expires_at' => $server->agent_token_expires_at?->toISOString(),
// Encrypted at rest on the model; needed to SSH into the host.
'private_key' => $server->privateKey instanceof PrivateKey ? $server->privateKey->private_key : null,
'applications' => ($applicationsByServer[$server->id] ?? collect())
->map(fn (V5Application $application): array => [
'id' => $application->id,
'container_name' => $application->container_name,
'runtime_container_id' => $application->runtime_container_id,
])
->values()
->all(),
];
})->all();
self::dispatch($team->id, $payload);
}
public function handle(): void
{
$incompleteHosts = [];
foreach ($this->servers as $serverPayload) {
if (! $this->teardownServer($serverPayload)) {
$incompleteHosts[] = [
'server_id' => $serverPayload['id'] ?? null,
'host' => $serverPayload['host'] ?? null,
];
}
}
// Teardown is best-effort and never fails the job (an unreachable host
// must not abort the others), so this is the single operator-facing
// signal that some hosts could not be reached and may now hold orphaned
// containers/mesh with no DB row left to reconcile them.
if ($incompleteHosts !== []) {
Log::error('v5 team teardown incomplete — '.count($incompleteHosts).' host(s) may have orphaned containers/mesh', [
'team_id' => $this->teamId,
'hosts' => $incompleteHosts,
]);
}
}
/**
* Tear down a single host. Returns false when any on-host teardown step
* (container removal, ingress stop, bootstrap-marker removal) failed, so the
* caller can surface the host as potentially orphaned. Never throws: a
* single unreachable host must not abort teardown of the other hosts.
*
* @param array<string, mixed> $serverPayload
*/
private function teardownServer(array $serverPayload): bool
{
$server = $this->reconstructServer($serverPayload);
$serverId = $serverPayload['id'] ?? null;
$host = $serverPayload['host'] ?? null;
$succeeded = true;
foreach ($serverPayload['applications'] ?? [] as $applicationPayload) {
try {
$application = $this->reconstructApplication($applicationPayload, $server);
DestroyNginxApplication::run($application);
} catch (\Throwable $exception) {
$succeeded = false;
Log::warning('V5 team teardown: failed to remove application container', [
'team_id' => $this->teamId,
'server_id' => $serverId,
'host' => $host,
'container_name' => $applicationPayload['container_name'] ?? null,
'error' => $exception->getMessage(),
]);
}
}
if ($server->isIngress() && $server->status === ServerStatus::Installed->value) {
try {
StopCaddyIngress::run($server);
} catch (\Throwable $exception) {
$succeeded = false;
Log::warning('V5 team teardown: failed to stop Caddy ingress', [
'team_id' => $this->teamId,
'server_id' => $serverId,
'host' => $host,
'error' => $exception->getMessage(),
]);
}
}
if (($serverPayload['last_bootstrapped_at'] ?? null) !== null) {
try {
if (! RemoveBootstrapMarker::run($server)) {
$succeeded = false;
Log::warning('V5 team teardown: could not remove on-host bootstrap identity over SSH', [
'team_id' => $this->teamId,
'server_id' => $serverId,
'host' => $host,
]);
}
} catch (\Throwable $exception) {
$succeeded = false;
Log::warning('V5 team teardown: bootstrap marker removal threw', [
'team_id' => $this->teamId,
'server_id' => $serverId,
'host' => $host,
'error' => $exception->getMessage(),
]);
}
}
// Revocation is best-effort and independent of the on-host cleanup: a
// failed flux push does not mean the host is orphaned, so it never flips
// $succeeded (it is logged separately inside AgentTokenIssuer::revoke).
$this->revokeAgentTokenIfSupported($server, $serverId, $host);
return $succeeded;
}
/**
* Reconstruct a non-persisted V5Server (with its private key relation
* pre-set) so the teardown actions never hit the deleted DB rows.
*
* @param array<string, mixed> $serverPayload
*/
private function reconstructServer(array $serverPayload): V5Server
{
$server = new V5Server;
$server->forceFill([
'id' => $serverPayload['id'] ?? null,
'uuid' => $serverPayload['uuid'] ?? null,
'name' => $serverPayload['name'] ?? null,
'host' => $serverPayload['host'] ?? null,
'ssh_user' => $serverPayload['ssh_user'] ?? null,
'ssh_port' => $serverPayload['ssh_port'] ?? 22,
'node_address' => $serverPayload['node_address'] ?? null,
'wireguard_management_ip' => $serverPayload['wireguard_management_ip'] ?? null,
'is_ingress' => (bool) ($serverPayload['is_ingress'] ?? false),
'ingress_type' => $serverPayload['ingress_type'] ?? null,
'status' => $serverPayload['status'] ?? null,
'agent_token_jti' => $serverPayload['agent_token_jti'] ?? null,
'agent_token_expires_at' => $serverPayload['agent_token_expires_at'] ?? null,
]);
// Non-persisted: StopCaddyIngress / the actions must not try to update a
// row that the cascade already removed.
$server->exists = false;
$privateKeyMaterial = $serverPayload['private_key'] ?? null;
if (is_string($privateKeyMaterial) && $privateKeyMaterial !== '') {
$privateKey = new PrivateKey;
$privateKey->forceFill(['private_key' => $privateKeyMaterial]);
$server->setRelation('privateKey', $privateKey);
} else {
$server->setRelation('privateKey', null);
}
return $server;
}
/**
* @param array<string, mixed> $applicationPayload
*/
private function reconstructApplication(array $applicationPayload, V5Server $server): V5Application
{
$application = new V5Application;
$application->forceFill([
'id' => $applicationPayload['id'] ?? null,
'container_name' => $applicationPayload['container_name'] ?? null,
'runtime_container_id' => $applicationPayload['runtime_container_id'] ?? null,
'server_id' => $server->id,
]);
$application->exists = false;
$application->setRelation('server', $server);
return $application;
}
/**
* If a coold-side agent-token revocation ever lands on AgentTokenIssuer,
* call it best-effort. Guarded so this job never hard-depends on a method
* that may not exist yet.
*/
private function revokeAgentTokenIfSupported(V5Server $server, mixed $serverId, mixed $host): void
{
if (! method_exists(AgentTokenIssuer::class, 'revokeForServer')) {
return;
}
try {
app(AgentTokenIssuer::class)->revokeForServer($server);
} catch (\Throwable $exception) {
Log::warning('V5 team teardown: agent token revocation failed', [
'team_id' => $this->teamId,
'server_id' => $serverId,
'host' => $host,
'error' => $exception->getMessage(),
]);
}
}
}
+12
View File
@@ -4,6 +4,7 @@ namespace App\Models;
use App\Actions\User\RevokeUserTeamTokens;
use App\Events\ServerReachabilityChanged;
use App\Jobs\V5TeardownTeamJob;
use App\Notifications\Channels\SendsDiscord;
use App\Notifications\Channels\SendsEmail;
use App\Notifications\Channels\SendsPushover;
@@ -75,6 +76,17 @@ class Team extends Model implements SendsDiscord, SendsEmail, SendsPushover, Sen
});
static::deleting(function (Team $team) {
// Best-effort on-host teardown of this team's v5 resources BEFORE the
// DB cascade removes the servers/applications/private keys. Captured
// synchronously into a queued job so an unreachable host cannot block
// or fail the team deletion (see V5TeardownTeamJob). Guarded so a v5
// teardown problem never breaks v4 team deletion.
try {
V5TeardownTeamJob::dispatchForTeam($team);
} catch (\Throwable $exception) {
report($exception);
}
RevokeUserTeamTokens::forTeam($team->id);
foreach ($team->privateKeys as $key) {
+6 -2
View File
@@ -2,6 +2,7 @@
namespace App\Models\V5;
use App\Enums\V5\ApplicationStatus;
use App\Events\V5CanvasResourceUpdated;
use App\Models\Environment;
use App\Models\Project;
@@ -9,6 +10,7 @@ use App\Models\Team;
use App\Models\User;
use Illuminate\Database\Eloquent\Relations\BelongsTo;
use Illuminate\Database\Eloquent\Relations\HasMany;
use Illuminate\Support\Facades\DB;
class Application extends V5Model
{
@@ -26,6 +28,7 @@ class Application extends V5Model
'container_name',
'status',
'status_message',
'status_observed_at',
'runtime_container_id',
'mesh_namespace',
'ingress_enabled',
@@ -35,7 +38,7 @@ class Application extends V5Model
];
protected $attributes = [
'status' => 'creating',
'status' => ApplicationStatus::Creating->value,
'mesh_namespace' => 'default',
'ingress_enabled' => false,
'canvas_x' => 0,
@@ -46,7 +49,7 @@ class Application extends V5Model
{
static::updated(function (self $application): void {
if ($application->wasChanged(['status', 'status_message', 'runtime_container_id'])) {
V5CanvasResourceUpdated::dispatch($application->team_id, $application->id);
DB::afterCommit(fn () => V5CanvasResourceUpdated::dispatch($application->team_id, $application->id));
}
});
}
@@ -54,6 +57,7 @@ class Application extends V5Model
protected function casts(): array
{
return [
'status_observed_at' => 'datetime',
'ingress_enabled' => 'boolean',
'internal_port' => 'integer',
'canvas_x' => 'integer',
+2
View File
@@ -8,6 +8,8 @@ class ApplicationDomain extends V5Model
{
protected $table = 'v5_application_domains';
protected bool $hasUuidColumn = false;
protected $fillable = [
'application_id',
'domain',
+6
View File
@@ -11,6 +11,12 @@ class Cluster extends V5Model
{
protected $table = 'v5_clusters';
/**
* Single source of truth for cluster defaults: `$attributes` below is
* built from these consts, and the column defaults in
* 2026_06_16_130649_v5_create_clusters_table mirror them (kept there for
* historical rows only update both when changing a default).
*/
public const DEFAULT_WIREGUARD_INTERFACE = 'wg0';
public const DEFAULT_WIREGUARD_MANAGEMENT_POOL = '100.64.0.0/16';
+4
View File
@@ -9,6 +9,8 @@ class ContainerStatus extends V5Model
{
protected $table = 'v5_container_statuses';
protected bool $hasUuidColumn = false;
protected $fillable = [
'team_id',
'server_id',
@@ -17,12 +19,14 @@ class ContainerStatus extends V5Model
'image',
'status',
'status_message',
'status_observed_at',
'last_seen_at',
];
protected function casts(): array
{
return [
'status_observed_at' => 'datetime',
'last_seen_at' => 'datetime',
];
}
+2
View File
@@ -9,6 +9,8 @@ class ResourceConnectionRule extends V5Model
{
protected $table = 'v5_resource_connection_rules';
protected bool $hasUuidColumn = false;
protected $fillable = [
'connection_id',
'source_resource_type',
+45
View File
@@ -0,0 +1,45 @@
<?php
namespace App\Models\V5;
use Illuminate\Database\Eloquent\Relations\BelongsTo;
/**
* A host-agent JWT that has been revoked (typically on server destroy/re-home).
*
* flux does not yet consult this list (see AgentTokenIssuer::revoke docblock);
* it exists so the Laravel side owns the data and API needed for revocation the
* moment flux gains a revocation check.
*/
class RevokedAgentToken extends V5Model
{
protected bool $hasUuidColumn = false;
protected $table = 'v5_revoked_agent_tokens';
protected $fillable = [
'jti',
'server_id',
'revoked_at',
'expires_at',
];
/**
* @return array<string, string>
*/
protected function casts(): array
{
return [
'revoked_at' => 'datetime',
'expires_at' => 'datetime',
];
}
/**
* @return BelongsTo<Server, $this>
*/
public function server(): BelongsTo
{
return $this->belongsTo(Server::class);
}
}
+55 -14
View File
@@ -2,12 +2,15 @@
namespace App\Models\V5;
use App\Enums\V5\IngressStatus;
use App\Events\V5CanvasResourceUpdated;
use App\Events\V5ClusterUpdated;
use App\Models\PrivateKey;
use App\Models\Team;
use App\Models\User;
use Illuminate\Database\Eloquent\Casts\Attribute;
use Illuminate\Database\Eloquent\Relations\BelongsTo;
use Illuminate\Support\Facades\DB;
class Server extends V5Model
{
@@ -24,9 +27,12 @@ class Server extends V5Model
'ssh_user',
'ssh_port',
'status',
'status_observed_at',
'ingress_type',
'ingress_status',
'capabilities',
'has_coold',
'is_ingress',
'builder_enabled',
'builder_capacity',
'builder_cpu_quota',
@@ -35,6 +41,9 @@ class Server extends V5Model
'wireguard_endpoint_override',
'wireguard_management_ip',
'wireguard_public_key',
'coold_version',
'agent_token_jti',
'agent_token_expires_at',
'container_subnets',
'canvas_x',
'canvas_y',
@@ -60,22 +69,22 @@ class Server extends V5Model
}
if ($server->wasChanged('status') && $server->cluster_id !== null) {
V5ClusterUpdated::dispatch($server->team_id, $server->cluster_id);
DB::afterCommit(fn () => V5ClusterUpdated::dispatch($server->team_id, $server->cluster_id));
}
if ($server->wasChanged('status')) {
V5CanvasResourceUpdated::dispatch(
DB::afterCommit(fn () => V5CanvasResourceUpdated::dispatch(
$server->team_id,
null,
$server->isIngress() ? $server->id : null,
$server->id,
);
));
return;
}
if ($server->isIngress()) {
V5CanvasResourceUpdated::dispatch($server->team_id, null, $server->id);
DB::afterCommit(fn () => V5CanvasResourceUpdated::dispatch($server->team_id, null, $server->id));
}
});
}
@@ -83,20 +92,56 @@ class Server extends V5Model
protected function casts(): array
{
return [
'capabilities' => 'array',
'has_coold' => 'boolean',
'is_ingress' => 'boolean',
'builder_enabled' => 'boolean',
'container_subnets' => 'array',
'canvas_x' => 'integer',
'canvas_y' => 'integer',
'status_observed_at' => 'datetime',
'agent_token_expires_at' => 'datetime',
'last_bootstrapped_at' => 'datetime',
'last_bootstrap_ran_at' => 'datetime',
'last_status_checked_at' => 'datetime',
];
}
public function fluxHostId(): string
{
return (string) $this->uuid;
}
/**
* Virtual attribute kept for wire-format compatibility: capabilities are
* stored as the indexed has_coold / is_ingress booleans, but reads and
* writes of `capabilities` keep working with the historical string array.
* Unknown capability names are dropped on write.
*
* The dropped `capabilities` column intentionally stays in `$fillable`:
* call sites still mass-assign it, and this mutator maps those writes
* onto the boolean columns.
*/
protected function capabilities(): Attribute
{
return Attribute::make(
get: fn () => array_values(array_filter([
$this->has_coold ? 'coold' : null,
$this->is_ingress ? 'ingress' : null,
])),
set: fn (?array $capabilities) => [
'has_coold' => in_array('coold', $capabilities ?? [], true),
'is_ingress' => in_array('ingress', $capabilities ?? [], true),
],
);
}
public function hasCapability(string $capability): bool
{
return in_array($capability, $this->capabilities ?? [], true);
return match ($capability) {
'coold' => (bool) $this->has_coold,
'ingress' => (bool) $this->is_ingress,
default => false,
};
}
/**
@@ -104,7 +149,7 @@ class Server extends V5Model
*/
public function withCapability(string $capability): array
{
return collect($this->capabilities ?? [])
return collect($this->capabilities)
->push($capability)
->unique()
->values()
@@ -116,7 +161,7 @@ class Server extends V5Model
*/
public function withoutCapability(string $capability): array
{
return collect($this->capabilities ?? [])
return collect($this->capabilities)
->reject(fn (string $existingCapability) => $existingCapability === $capability)
->values()
->all();
@@ -124,16 +169,12 @@ class Server extends V5Model
public function isIngress(): bool
{
return $this->hasCapability('ingress');
return (bool) $this->is_ingress;
}
public function ingressStatus(): string
{
if ($this->ingress_status !== null) {
return $this->ingress_status;
}
return $this->status === 'installed' ? 'running' : 'unknown';
return $this->ingress_status ?? IngressStatus::Unknown->value;
}
public function ingressType(): string
+37 -8
View File
@@ -3,26 +3,55 @@
namespace App\Models\V5;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\Schema;
abstract class V5Model extends Model
{
/**
* Whether the model's table has a `uuid` column. Models without one (set
* this to false there) skip public-id generation and route on the primary
* key instead.
*/
protected bool $hasUuidColumn = true;
public function getRouteKeyName(): string
{
return 'uuid';
return $this->hasUuidColumn ? 'uuid' : $this->getKeyName();
}
protected static function boot(): void
{
parent::boot();
static::creating(function (Model $model): void {
if (
Schema::hasColumn($model->getTable(), 'uuid')
&& ! $model->getAttribute('uuid')
) {
$model->setAttribute('uuid', new_public_id());
static::creating(function (self $model): void {
if ($model->hasUuidColumn && ! $model->getAttribute('uuid')) {
$model->setAttribute('uuid', $model->newUniquePublicId());
}
});
}
/**
* Generate a public id, regenerating (up to three candidates) when one is
* already taken. A concurrent insert between this exists() check and our
* own insert can still collide; the unique index then rejects the insert,
* which is an acceptable residual race for these cheap, retryable writes.
*/
protected function newUniquePublicId(): string
{
$attempts = 0;
do {
$candidate = $this->newPublicIdCandidate();
$attempts++;
} while (
$attempts < 3
&& $this->newModelQuery()->where('uuid', $candidate)->exists()
);
return $candidate;
}
protected function newPublicIdCandidate(): string
{
return new_public_id();
}
}
+74
View File
@@ -0,0 +1,74 @@
<?php
namespace App\Policies\V5;
use App\Models\Team;
use App\Models\User;
use App\Models\V5\Application;
use Illuminate\Auth\Access\Response;
class ApplicationPolicy
{
/**
* Determine whether the user can view the application within the current team.
*
* Read-only diagnostics (deploy status, container logs) are available to any
* member of the owning team; apps from other teams stay hidden as a 404.
*/
public function view(User $user, Application $application, Team $team): Response
{
return $this->belongsToTeam($application, $team);
}
/**
* Determine whether the user can update the application within the current team.
*/
public function update(User $user, Application $application, Team $team): Response
{
return $this->allowIfAdminAndScoped($user, $application, $team);
}
/**
* Determine whether the user can update the application's ingress configuration.
*/
public function updateIngress(User $user, Application $application, Team $team): Response
{
return $this->allowIfAdminAndScoped($user, $application, $team);
}
/**
* Determine whether the user can delete the application within the current team.
*/
public function delete(User $user, Application $application, Team $team): Response
{
return $this->allowIfAdminAndScoped($user, $application, $team);
}
/**
* Run the team scoping check first (mismatch stays hidden as a 404) and
* only then the role check (403 for members on their own team's app).
*/
private function allowIfAdminAndScoped(User $user, Application $application, Team $team): Response
{
$scope = $this->belongsToTeam($application, $team);
if ($scope->denied()) {
return $scope;
}
return $user->isAdminOfTeam($team->id)
? Response::allow()
: Response::deny('You do not have permission to manage applications in this team.');
}
/**
* Applications outside the current team must stay invisible, so
* mismatches deny as not found instead of forbidden.
*/
private function belongsToTeam(Application $application, Team $team): Response
{
return $application->team_id === $team->id
? Response::allow()
: Response::denyAsNotFound();
}
}
+64
View File
@@ -0,0 +1,64 @@
<?php
namespace App\Policies\V5;
use App\Models\Team;
use App\Models\User;
use App\Models\V5\Cluster;
use Illuminate\Auth\Access\Response;
class ClusterPolicy
{
/**
* Determine whether the user can view the cluster within the current team.
* Read-only, so gated on team membership alone.
*/
public function view(User $user, Cluster $cluster, Team $team): Response
{
return $this->belongsToTeam($cluster, $team);
}
/**
* Determine whether the user can create a cluster in the current team.
* There is no model to scope yet, so this is a pure role gate.
*/
public function create(User $user, Team $team): Response
{
return $this->allowIfAdmin($user, $team);
}
/**
* Determine whether the user can delete the cluster within the current team.
*/
public function delete(User $user, Cluster $cluster, Team $team): Response
{
$scope = $this->belongsToTeam($cluster, $team);
if ($scope->denied()) {
return $scope;
}
return $this->allowIfAdmin($user, $team);
}
/**
* Members may read but not mutate; only admins/owners of the team pass.
*/
private function allowIfAdmin(User $user, Team $team): Response
{
return $user->isAdminOfTeam($team->id)
? Response::allow()
: Response::deny('You do not have permission to manage clusters in this team.');
}
/**
* Clusters outside the current team must stay invisible, so mismatches
* deny as not found instead of forbidden.
*/
private function belongsToTeam(Cluster $cluster, Team $team): Response
{
return $cluster->team_id === $team->id
? Response::allow()
: Response::denyAsNotFound();
}
}
@@ -0,0 +1,55 @@
<?php
namespace App\Policies\V5;
use App\Models\Team;
use App\Models\User;
use App\Models\V5\ResourceConnection;
use Illuminate\Auth\Access\Response;
class ResourceConnectionPolicy
{
/**
* Determine whether the user can update the connection within the current team.
*/
public function update(User $user, ResourceConnection $connection, Team $team): Response
{
return $this->allowIfAdminAndScoped($user, $connection, $team);
}
/**
* Determine whether the user can delete the connection within the current team.
*/
public function delete(User $user, ResourceConnection $connection, Team $team): Response
{
return $this->allowIfAdminAndScoped($user, $connection, $team);
}
/**
* Run the team scoping check first (mismatch stays hidden as a 404) and
* only then the role check (403 for members on their own team's connection).
*/
private function allowIfAdminAndScoped(User $user, ResourceConnection $connection, Team $team): Response
{
$scope = $this->belongsToTeam($connection, $team);
if ($scope->denied()) {
return $scope;
}
return $user->isAdminOfTeam($team->id)
? Response::allow()
: Response::deny('You do not have permission to manage resource connections in this team.');
}
/**
* Connections outside the current team must stay invisible, so
* mismatches deny as not found instead of forbidden.
*/
private function belongsToTeam(ResourceConnection $connection, Team $team): Response
{
return $connection->team_id === $team->id
? Response::allow()
: Response::denyAsNotFound();
}
}
+121
View File
@@ -0,0 +1,121 @@
<?php
namespace App\Policies\V5;
use App\Models\Team;
use App\Models\User;
use App\Models\V5\Cluster;
use App\Models\V5\Server;
use Illuminate\Auth\Access\Response;
class ServerPolicy
{
/**
* Determine whether the user can add a server to the cluster within the
* current team. Denies as forbidden (not "not found") to preserve the
* historical 403 on cluster/team mismatch, and requires an admin/owner
* role to mutate cluster infrastructure.
*/
public function create(User $user, Team $team, Cluster $cluster): Response
{
if ($cluster->team_id !== $team->id) {
return Response::deny();
}
return $this->allowIfAdmin($user, $team);
}
/**
* Determine whether the user can update the server within the current team.
*/
public function update(User $user, Server $server, Team $team, Cluster $cluster): Response
{
return $this->allowIfAdminAndScoped($user, $server, $team, $cluster);
}
/**
* Determine whether the user can delete the server within the current team.
*/
public function delete(User $user, Server $server, Team $team, Cluster $cluster): Response
{
return $this->allowIfAdminAndScoped($user, $server, $team, $cluster);
}
/**
* Determine whether the user can run a connectivity check against the server.
*/
public function check(User $user, Server $server, Team $team, Cluster $cluster): Response
{
return $this->allowIfAdminAndScoped($user, $server, $team, $cluster);
}
/**
* Determine whether the user can bootstrap the server.
*/
public function bootstrap(User $user, Server $server, Team $team, Cluster $cluster): Response
{
return $this->allowIfAdminAndScoped($user, $server, $team, $cluster);
}
/**
* Determine whether the user can view server diagnostics (coold logs,
* corrosion tables, firewall rules). Read-only, so gated on team
* membership alone.
*/
public function viewDiagnostics(User $user, Server $server, Team $team, Cluster $cluster): Response
{
return $this->belongsToClusterInTeam($server, $team, $cluster);
}
/**
* Determine whether the user can move the server's Caddy ingress card on
* the canvas. Non-ingress servers must stay invisible on the canvas, and
* moving a card mutates persisted layout so it requires an admin/owner.
*/
public function updateCanvasPosition(User $user, Server $server, Team $team): Response
{
if (! ($server->team_id === $team->id && $server->isIngress())) {
return Response::denyAsNotFound();
}
return $this->allowIfAdmin($user, $team);
}
/**
* Run the team/cluster scoping check first (mismatch stays hidden as a 404)
* and only then the role check (403 for members on their own team's server).
*/
private function allowIfAdminAndScoped(User $user, Server $server, Team $team, Cluster $cluster): Response
{
$scope = $this->belongsToClusterInTeam($server, $team, $cluster);
if ($scope->denied()) {
return $scope;
}
return $this->allowIfAdmin($user, $team);
}
/**
* Members may read but not mutate; only admins/owners of the team pass.
*/
private function allowIfAdmin(User $user, Team $team): Response
{
return $user->isAdminOfTeam($team->id)
? Response::allow()
: Response::deny('You do not have permission to manage servers in this team.');
}
/**
* Servers outside the current team (or outside the addressed cluster)
* must stay invisible, so mismatches deny as not found.
*/
private function belongsToClusterInTeam(Server $server, Team $team, Cluster $cluster): Response
{
return $cluster->team_id === $team->id
&& $server->team_id === $team->id
&& $server->cluster_id === $cluster->id
? Response::allow()
: Response::denyAsNotFound();
}
}
+15
View File
@@ -3,7 +3,9 @@
namespace App\Providers;
use App\Models\PersonalAccessToken;
use App\Models\V5\Application;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Relations\Relation;
use Illuminate\Support\Facades\App;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Http;
@@ -27,6 +29,7 @@ class AppServiceProvider extends ServiceProvider
public function boot(): void
{
$this->configureCommands();
$this->configureMorphMap();
$this->configureModels();
$this->configurePasswords();
$this->configureSanctumModel();
@@ -41,6 +44,18 @@ class AppServiceProvider extends ServiceProvider
}
}
/**
* Map v5 models to stable morph aliases so polymorphic rows survive class
* renames. Deliberately NOT enforced: v4 polymorphic relations store FQCNs
* and must keep resolving them.
*/
private function configureMorphMap(): void
{
Relation::morphMap([
'v5.application' => Application::class,
]);
}
private function configureModels(): void
{
// Disabled because it's causing issues with the application
+14
View File
@@ -36,6 +36,10 @@ use App\Models\StandaloneRedis;
use App\Models\SwarmDocker;
use App\Models\Team;
use App\Models\TelegramNotificationSettings;
use App\Models\V5\Application as V5Application;
use App\Models\V5\Cluster as V5Cluster;
use App\Models\V5\ResourceConnection as V5ResourceConnection;
use App\Models\V5\Server as V5Server;
use App\Models\WebhookNotificationSettings;
use App\Policies\ApiTokenPolicy;
use App\Policies\ApplicationPolicy;
@@ -61,6 +65,10 @@ use App\Policies\SharedEnvironmentVariablePolicy;
use App\Policies\StandaloneDockerPolicy;
use App\Policies\SwarmDockerPolicy;
use App\Policies\TeamPolicy;
use App\Policies\V5\ApplicationPolicy as V5ApplicationPolicy;
use App\Policies\V5\ClusterPolicy as V5ClusterPolicy;
use App\Policies\V5\ResourceConnectionPolicy as V5ResourceConnectionPolicy;
use App\Policies\V5\ServerPolicy as V5ServerPolicy;
use Illuminate\Foundation\Support\Providers\AuthServiceProvider as ServiceProvider;
use Illuminate\Support\Facades\Gate;
use Laravel\Sanctum\PersonalAccessToken;
@@ -122,6 +130,12 @@ class AuthServiceProvider extends ServiceProvider
CloudProviderToken::class => CloudProviderTokenPolicy::class,
CloudInitScript::class => CloudInitScriptPolicy::class,
// V5 policies - scoped to the current team resolved from the request
V5Application::class => V5ApplicationPolicy::class,
V5Cluster::class => V5ClusterPolicy::class,
V5ResourceConnection::class => V5ResourceConnectionPolicy::class,
V5Server::class => V5ServerPolicy::class,
];
/**
+8
View File
@@ -60,6 +60,14 @@ class RouteServiceProvider extends ServiceProvider
return Limit::perMinute(5)->by($request->user()?->id ?: $request->ip());
});
// v5 authenticated web endpoints run synchronous SSH/Flux work per
// request (connectivity checks, bootstrap, diagnostics). Throttle per
// user so a single member cannot pin FPM workers by hammering them,
// while leaving ample headroom for the canvas's 3s cluster polling.
RateLimiter::for('v5', function (Request $request) {
return Limit::perMinute(120)->by($request->user()?->id ?: $request->ip());
});
RateLimiter::for('feedback', function (Request $request) {
return Limit::perMinute(3)->by($request->user()?->id ?: $request->ip());
});
+11 -2
View File
@@ -33,10 +33,19 @@ class ValidHostname implements ValidationRule
return;
}
// Reject ASCII control characters (including embedded newlines, which
// would otherwise slip through the trailing-newline-tolerant `$` anchor
// in the per-label regex below).
if (preg_match('/[\x00-\x1f\x7f]/', $hostname) === 1) {
$fail('The :attribute contains invalid characters. Only letters (a-z, A-Z), numbers (0-9), hyphens (-), and dots (.) are allowed.');
return;
}
// Check for dangerous shell metacharacters
$dangerousChars = [
';', '|', '&', '$', '`', '(', ')', '{', '}',
'<', '>', '\n', '\r', '\0', '"', "'", '\\',
'<', '>', "\n", "\r", "\0", '"', "'", '\\',
'!', '*', '?', '[', ']', '~', '^', ':', '#',
'@', '%', '=', '+', ',', ' ',
];
@@ -104,7 +113,7 @@ class ValidHostname implements ValidationRule
}
// Check if label contains only valid characters (letters, digits, hyphens)
if (! preg_match('/^[a-z0-9-]+$/', $label)) {
if (! preg_match('/^[a-z0-9-]+$/D', $label)) {
$fail('The :attribute contains invalid characters. Only letters (a-z, A-Z), numbers (0-9), hyphens (-), and dots (.) are allowed.');
return;
+31
View File
@@ -9,6 +9,12 @@ class ValidServerIp implements ValidationRule
{
/**
* Accepts a valid IPv4 address, IPv6 address, or RFC 1123 hostname.
*
* IP literals in private/reserved ranges (loopback, link-local, RFC 1918,
* etc.) are rejected by default to stop a member from pointing a server at
* the Coolify host's internal network and abusing the synchronous SSH check
* to probe it. Self-hosters on private LANs can allow them via
* config('coold.allow_private_server_ips').
*/
public function validate(string $attribute, mixed $value, Closure $fail): void
{
@@ -19,10 +25,14 @@ class ValidServerIp implements ValidationRule
$trimmed = trim($value);
if (filter_var($trimmed, FILTER_VALIDATE_IP, FILTER_FLAG_IPV4)) {
$this->failIfDisallowedRange($trimmed, $fail);
return;
}
if (filter_var($trimmed, FILTER_VALIDATE_IP, FILTER_FLAG_IPV6)) {
$this->failIfDisallowedRange($trimmed, $fail);
return;
}
@@ -37,4 +47,25 @@ class ValidServerIp implements ValidationRule
$fail('The :attribute must be a valid IPv4 address, IPv6 address, or hostname.');
}
}
/**
* Reject IPs in private/reserved ranges unless the operator has explicitly
* opted in. The IP is already known to be a valid literal here.
*/
private function failIfDisallowedRange(string $ip, Closure $fail): void
{
if (config('coold.allow_private_server_ips')) {
return;
}
$isPublic = filter_var(
$ip,
FILTER_VALIDATE_IP,
FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE
);
if ($isPublic === false) {
$fail('The :attribute must not be a private or reserved IP address.');
}
}
}
+190 -13
View File
@@ -2,20 +2,41 @@
namespace App\Services\Flux;
use App\Models\V5\RevokedAgentToken;
use App\Models\V5\Server as V5Server;
use Firebase\JWT\JWT;
use Illuminate\Support\Facades\File;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
use RuntimeException;
/**
* Mints the per-host ES256 JWT that authorizes a coold host agent against flux.
*
* Capability scoping: by default the token carries the EXPLICIT list of
* primitive capability strings coold advertises (config('flux.host_capabilities'),
* mirroring coold/coold/src/grpc/client.rs:204-231) rather than the
* `host-agent:default` wildcard profile. flux intersects the jwt `caps` with
* coold's advertised set (flux/src/main.rs:128-141), so the effective power is
* unchanged, but the token no longer depends on flux's
* `capability_profile_authorizes_all` wildcard bypass (main.rs:124-126).
*
* @see config/flux.php for the capability list, escape hatch, TTL and kid config.
*/
class AgentTokenIssuer
{
public const DEFAULT_PROFILE = 'host-agent:default';
private const TTL_FLOOR_SECONDS = 60;
/**
* @param array<int, string> $capabilities
* @param array<string, mixed> $extraClaims
* Mint a host JWT.
*
* @param array<int, string>|null $capabilities Explicit caps; null resolves the configured default set (or escape-hatch profile).
* @param int|null $ttl Lifetime in seconds; null resolves config('flux.host_token_ttl'). Clamped to a 60s floor.
* @param array<string, mixed> $extraClaims Extra claims merged in (a `jti` here is honored, otherwise one is generated).
*/
public function issue(string $hostId, array $capabilities = [self::DEFAULT_PROFILE], int $ttl = 86400, array $extraClaims = []): string
public function issue(string $hostId, ?array $capabilities = null, ?int $ttl = null, array $extraClaims = []): string
{
if ($hostId === '') {
throw new RuntimeException('Flux host id is required.');
@@ -27,30 +48,186 @@ class AgentTokenIssuer
throw new RuntimeException("Flux JWT private key not found at {$privateKeyPath}.");
}
$this->assertPrivateKeyPermissions($privateKeyPath);
$capabilities ??= $this->defaultCapabilities();
$ttl ??= (int) config('flux.host_token_ttl', 3600);
$jti = $extraClaims['jti'] ?? (string) Str::uuid();
unset($extraClaims['jti']);
$now = time();
$keyId = (string) config('flux.jwt_kid', 'flux-default');
return JWT::encode(array_merge($extraClaims, [
'sub' => $hostId,
'aud' => 'coold',
'caps' => $this->normalizeCapabilities($capabilities),
'jti' => $jti,
'iat' => $now,
'exp' => $now + max(60, $ttl),
]), File::get($privateKeyPath), 'ES256');
'exp' => $now + max(self::TTL_FLOOR_SECONDS, $ttl),
]), File::get($privateKeyPath), 'ES256', $keyId !== '' ? $keyId : null);
}
public function issueForServer(V5Server $server, int $ttl = 86400): string
public function issueForServer(V5Server $server, ?int $ttl = null): string
{
$hostId = $server->wireguard_management_ip ?: $server->node_address;
$hostId = $server->fluxHostId();
if (! is_string($hostId) || $hostId === '') {
throw new RuntimeException('Server is missing its Flux host id.');
if ($hostId === '') {
throw new RuntimeException('Server is missing a valid Flux host id.');
}
return $this->issue($hostId, [self::DEFAULT_PROFILE], $ttl, [
'team_id' => $server->team_id,
'cluster_id' => $server->cluster_id,
'server_id' => $server->id,
$jti = (string) Str::uuid();
$ttl ??= (int) config('flux.host_token_ttl', 3600);
// team_id/cluster_id/server_id are minted as STRINGS: flux deserializes
// the `team_id` claim as a string (coold/flux/src/auth.rs Claims), and
// rejects the whole token with a JSON type error if it arrives as a JSON
// integer. Keep the sibling ids string-typed for consistency.
$token = $this->issue($hostId, $this->defaultCapabilities(), $ttl, [
'jti' => $jti,
'team_id' => (string) $server->team_id,
'cluster_id' => (string) $server->cluster_id,
'server_id' => $hostId,
'wireguard_management_ip' => (string) $server->wireguard_management_ip,
]);
// Persist the freshly issued jti (so a later destroy/re-home knows which
// token to revoke) and its expiry (so the scheduled rotation loop knows
// when to re-mint). Use a targeted update keyed by id so this neither
// inserts an unsaved model nor flushes unrelated dirty attributes, and
// does not depend on the Server model's $fillable.
if ($server->exists) {
$expiresAt = now()->addSeconds(max(self::TTL_FLOOR_SECONDS, $ttl));
V5Server::query()->whereKey($server->getKey())->update([
'agent_token_jti' => $jti,
'agent_token_expires_at' => $expiresAt,
]);
$server->setAttribute('agent_token_jti', $jti);
$server->setAttribute('agent_token_expires_at', $expiresAt);
$server->syncOriginalAttribute('agent_token_jti');
$server->syncOriginalAttribute('agent_token_expires_at');
}
return $token;
}
/**
* Record the server's currently-issued host token jti as revoked AND push
* the revocation to flux so it rejects the jti at verify immediately.
*
* flux now consults a revocation denylist (flux/src/auth.rs `is_revoked`,
* fed by `POST /v1/tokens/revoke` on the flux UDS), and Laravel pushes to it
* here. The local `RevokedAgentToken` record remains the source of truth
* Laravel owns; the flux push is best-effort if flux is unreachable the
* revocation is logged and the local record still stands, with the short TTL
* and hourly rotation bounding the exposure until flux is reachable again.
*/
public function revoke(V5Server $server): void
{
$jti = $server->agent_token_jti;
if (! is_string($jti) || $jti === '') {
return;
}
$expiresAt = $server->agent_token_expires_at;
$expiresAtUnix = $expiresAt instanceof \DateTimeInterface ? $expiresAt->getTimestamp() : null;
RevokedAgentToken::query()->updateOrCreate(
['jti' => $jti],
[
'server_id' => $server->id,
'revoked_at' => now(),
'expires_at' => $expiresAt,
]
);
// Best-effort: a destroy/teardown must never fail because flux is down.
try {
app(FluxClient::class)->revokeToken($jti, $expiresAtUnix);
} catch (\Throwable $exception) {
Log::warning('Failed to push agent token revocation to Flux.', [
'server_id' => $server->id,
'jti' => $jti,
'error' => $exception->getMessage(),
]);
}
if ($server->exists) {
V5Server::query()->whereKey($server->getKey())->update(['agent_token_jti' => null]);
$server->setAttribute('agent_token_jti', null);
$server->syncOriginalAttribute('agent_token_jti');
}
}
/**
* Revoke the server's currently-issued host token. Alias of {@see revoke()}
* kept as the name the team-teardown job resolves via `method_exists`.
*/
public function revokeForServer(V5Server $server): void
{
$this->revoke($server);
}
public function isRevoked(string $jti): bool
{
if ($jti === '') {
return false;
}
return RevokedAgentToken::query()->where('jti', $jti)->exists();
}
/**
* The default capability set for production host tokens: the explicit
* advertised primitive list, unless the emergency escape hatch profile is
* configured (then that single profile is minted instead).
*
* @return array<int, string>
*/
private function defaultCapabilities(): array
{
$profile = config('flux.host_capability_profile');
if (is_string($profile) && trim($profile) !== '') {
return [trim($profile)];
}
$configured = config('flux.host_capabilities');
if (is_array($configured) && $configured !== []) {
return array_values($configured);
}
return [self::DEFAULT_PROFILE];
}
/**
* Warn (but do not hard-fail that could break existing installs) when the
* private key file is readable by group/other or is not owner-readable. The
* key should be generated 0600, e.g.:
* openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 \
* -out storage/app/flux/jwt.priv && chmod 600 storage/app/flux/jwt.priv
*/
private function assertPrivateKeyPermissions(string $path): void
{
$perms = @fileperms($path);
if ($perms === false) {
return;
}
$mode = $perms & 0777;
if (($mode & 0077) !== 0 || ($mode & 0400) === 0) {
Log::warning('Flux JWT private key has insecure permissions.', [
'path' => $path,
'mode' => sprintf('%04o', $mode),
'expected' => '0600',
]);
}
}
/**
+151 -22
View File
@@ -2,6 +2,7 @@
namespace App\Services\Flux;
use App\Exceptions\V5\UnsupportedCooldVerb;
use Illuminate\Support\Str;
use RuntimeException;
@@ -60,6 +61,28 @@ class FluxClient
return $this->output($payload, 'Container started.');
}
public function stopContainer(string $hostId, string $id, int $timeoutSeconds = 10): string
{
$payload = $this->dispatch($hostId, [
'type' => 'containers.stop',
'id' => $id,
'timeout_seconds' => max(0, $timeoutSeconds),
]);
return $this->output($payload, 'Container stopped.');
}
public function removeContainer(string $hostId, string $id, bool $force = false): string
{
$payload = $this->dispatch($hostId, [
'type' => 'containers.delete',
'id' => $id,
'force' => $force,
]);
return $this->output($payload, 'Container removed.');
}
/**
* @return array<string, mixed>
*/
@@ -151,6 +174,19 @@ class FluxClient
return $this->output($payload, 'No coold logs returned.');
}
public function containerLogs(string $hostId, string $containerId, int $tail = 200): string
{
$payload = $this->dispatch($hostId, [
'type' => 'containers.logs',
'id' => $containerId,
'tail' => max(1, min($tail, 1000)),
'stdout' => true,
'stderr' => true,
]);
return $this->output($payload, 'No container logs returned.');
}
public function corrosionTables(string $hostId, int $limit = 200): string
{
$payload = $this->dispatch($hostId, [
@@ -161,11 +197,107 @@ class FluxClient
return $this->output($payload, '{"limit":200,"tables":[]}');
}
/**
* Deliver a freshly minted host JWT to the node over the live coold RPC
* stream (flux gates the `host.jwt.set` capability; the token must carry
* it). Preferred over the SSH push because it reuses the already
* authenticated flux<->coold channel and works while the current token is
* still valid exactly the rotation window. Throws like the sibling
* dispatch methods (host not connected / UnsupportedCooldVerb / generic
* failure) so the caller can catch and fall back to SSH.
*/
public function pushHostToken(string $hostId, string $token): void
{
$this->dispatch($hostId, [
'type' => 'host.jwt.set',
'jwt' => $token,
]);
}
/**
* Revoke a host token by its `jti` on the flux revocation store so flux
* rejects it at verify immediately, instead of waiting for the token's TTL
* to lapse (flux/src/unix_bridge.rs `POST /v1/tokens/revoke`,
* flux/src/auth.rs `is_revoked`). The optional `expiresAt` (the token `exp`,
* unix seconds) lets flux prune the denylist entry once it can no longer
* matter.
*
* Best-effort like the sibling dispatch methods: throws a RuntimeException on
* connection failure / timeout / non-2xx so the caller can catch and treat
* an unreachable flux as non-fatal (the local revocation record still
* stands and the short TTL + rotation bound the exposure).
*/
public function revokeToken(string $jti, ?int $expiresAt = null): void
{
if (trim($jti) === '') {
return;
}
$requestBody = ['jti' => $jti];
if ($expiresAt !== null) {
$requestBody['expires_at'] = $expiresAt;
}
$body = json_encode($requestBody, JSON_THROW_ON_ERROR);
$response = $this->sendOverSocket('/v1/tokens/revoke', $body);
$statusCode = $this->statusCode($response);
if ($statusCode < 200 || $statusCode >= 300) {
$responseBody = $this->responseBody($response);
$payload = $responseBody === '' ? null : json_decode($responseBody, true);
throw new RuntimeException(
$this->errorMessage($payload, $responseBody) ?? "Flux token revocation returned HTTP {$statusCode}."
);
}
}
/**
* @param array<string, mixed> $command
* @return array<string, mixed>
*/
private function dispatch(string $hostId, array $command): array
{
$body = json_encode([
'host_id' => $hostId,
'request_id' => (string) Str::uuid(),
'command' => $command,
], JSON_THROW_ON_ERROR);
$response = $this->sendOverSocket('/v1/coold/dispatch', $body);
$statusCode = $this->statusCode($response);
$responseBody = $this->responseBody($response);
$payload = $responseBody === '' ? null : json_decode($responseBody, true);
if ($statusCode < 200 || $statusCode >= 300) {
throw $this->dispatchException(
$command,
$statusCode,
$this->errorMessage($payload, $responseBody) ?? "Flux dispatch returned HTTP {$statusCode}."
);
}
if (! is_array($payload)) {
throw new RuntimeException('Flux dispatch returned an invalid response.');
}
if (($payload['status'] ?? null) === 'error') {
$message = is_string($payload['message'] ?? null) ? $payload['message'] : 'Flux dispatch failed.';
throw $this->dispatchException($command, $statusCode, $message);
}
return $payload;
}
/**
* Send a single HTTP/1.1 request over the flux Unix-domain socket and return
* the raw response. Shared by every flux verb (coold dispatch, host token
* rotation, token revocation) only the request path and JSON body differ.
*/
private function sendOverSocket(string $path, string $body): string
{
$socketPath = config('flux.unix_socket_path');
@@ -177,11 +309,6 @@ class FluxClient
throw new RuntimeException('Flux socket was not found.');
}
$body = json_encode([
'host_id' => $hostId,
'request_id' => (string) Str::uuid(),
'command' => $command,
], JSON_THROW_ON_ERROR);
$connectionTimeout = (float) config('flux.connection_timeout_seconds', 1.0);
$dispatchTimeout = (float) config('flux.dispatch_timeout_seconds', 35.0);
$stream = @stream_socket_client("unix://{$socketPath}", $errorCode, $errorMessage, $connectionTimeout);
@@ -193,7 +320,7 @@ class FluxClient
stream_set_timeout($stream, (int) ceil($dispatchTimeout));
fwrite($stream, implode("\r\n", [
'POST /v1/coold/dispatch HTTP/1.1',
"POST {$path} HTTP/1.1",
'Host: flux',
'Accept: application/json',
'Content-Type: application/json',
@@ -206,25 +333,27 @@ class FluxClient
$response = stream_get_contents($stream) ?: '';
fclose($stream);
$statusCode = $this->statusCode($response);
$responseBody = $this->responseBody($response);
$payload = $responseBody === '' ? null : json_decode($responseBody, true);
return $response;
}
if ($statusCode < 200 || $statusCode >= 300) {
throw new RuntimeException($this->errorMessage($payload, $responseBody) ?? "Flux dispatch returned HTTP {$statusCode}.");
/**
* Flux answers a verb the node's coold did not advertise with HTTP 501 and
* the message "primitive <verb> is not supported by host" (coold repo:
* flux/src/routing.rs:50-53, flux/src/unix_bridge.rs:227-245). Anything
* else including coold-side command failures relayed with their own
* status code is a generic dispatch failure.
*
* @param array<string, mixed> $command
*/
private function dispatchException(array $command, int $statusCode, string $message): RuntimeException
{
$verb = is_string($command['type'] ?? null) ? $command['type'] : 'unknown';
if ($statusCode === 501 || preg_match('/primitive .+ is not supported by host/i', $message) === 1) {
return new UnsupportedCooldVerb($verb, $message);
}
if (! is_array($payload)) {
throw new RuntimeException('Flux dispatch returned an invalid response.');
}
if (($payload['status'] ?? null) === 'error') {
$message = is_string($payload['message'] ?? null) ? $payload['message'] : 'Flux dispatch failed.';
throw new RuntimeException($message);
}
return $payload;
return new RuntimeException($message);
}
private function statusCode(string $response): int
@@ -0,0 +1,87 @@
<?php
namespace App\Support\V5;
use App\Models\V5\Application as V5Application;
use App\Models\V5\Server as V5Server;
/**
* Single source of truth for the canvas resource payloads served by the
* dashboard Inertia props and broadcast by V5CanvasResourceUpdated the two
* must stay identical for websocket vs. initial-load parity.
*/
class CanvasResourceSerializer
{
public const CARD_WIDTH = 320;
public const CARD_HEIGHT = 144;
public const CARD_GAP = 32;
/**
* @return array<string, mixed>
*/
public function serializeApplication(V5Application $application): array
{
$application->loadMissing(['server', 'domains', 'project', 'environment']);
$server = $application->server;
$isServerReachable = ! $server instanceof V5Server || $this->isServerReachable($server);
return [
'id' => $application->uuid,
'name' => $application->name,
'image' => $application->image,
'containerName' => $application->container_name,
'status' => $application->status,
'statusMessage' => $application->status_message,
'effectiveStatus' => $isServerReachable ? $application->status : 'unknown',
'effectiveStatusMessage' => $isServerReachable
? $application->status_message
: $this->serverStatusMessage($server),
'runtimeContainerId' => $application->runtime_container_id,
'serverName' => $server?->name,
'serverStatus' => $server?->status,
'serverStatusMessage' => $server instanceof V5Server ? $this->serverStatusMessage($server) : null,
'isServerReachable' => $isServerReachable,
'serverIngressEnabled' => (bool) $server?->isIngress(),
'meshNamespace' => $application->mesh_namespace,
'ingressEnabled' => $application->ingress_enabled,
'internalPort' => $application->internal_port,
'domains' => $application->domains->pluck('domain')->values()->all(),
'meshFqdn' => $application->container_name.'.'.($application->mesh_namespace ?: 'default').'.coolify.internal',
'projectUuid' => $application->project?->uuid,
'environmentUuid' => $application->environment?->uuid,
'canvasX' => $application->canvas_x,
'canvasY' => $application->canvas_y,
];
}
/**
* @return array<string, mixed>
*/
public function serializeCaddyIngress(V5Server $server, int $index = 0): array
{
$isServerReachable = $this->isServerReachable($server);
return [
'id' => $server->uuid,
'name' => $server->name,
'host' => $server->host,
'type' => $server->ingressType(),
'status' => $isServerReachable ? $server->ingressStatus() : 'unreachable',
'statusMessage' => $isServerReachable ? null : $this->serverStatusMessage($server),
'canvasX' => $server->canvas_x ?? -(self::CARD_WIDTH + self::CARD_GAP),
'canvasY' => $server->canvas_y ?? $index * (self::CARD_HEIGHT + self::CARD_GAP),
];
}
private function isServerReachable(V5Server $server): bool
{
return $server->status !== 'unreachable';
}
private function serverStatusMessage(?V5Server $server): ?string
{
return $server?->last_status_output ?: null;
}
}
+90
View File
@@ -0,0 +1,90 @@
<?php
namespace App\Support\V5;
use App\Models\V5\Cluster as V5Cluster;
use App\Models\V5\Server as V5Server;
/**
* Single source of truth for the cluster payload served by the Clusters
* Inertia props and broadcast by V5ClusterUpdated the two must stay
* identical for websocket vs. initial-load parity.
*/
class ClusterSerializer
{
/**
* @return array<string, mixed>
*/
public function serialize(V5Cluster $cluster): array
{
return [
'id' => $cluster->uuid,
'name' => $cluster->name,
'description' => $cluster->description,
'wireguardInterface' => $cluster->wireguard_interface,
'wireguardManagementPool' => $cluster->wireguard_management_pool,
'wireguardListenPort' => $cluster->wireguard_listen_port,
'containerNetworkPool' => $cluster->container_network_pool,
'containerNetworkPrefix' => $cluster->container_network_prefix,
'namespaces' => $cluster->namespaces ?? V5Cluster::DEFAULT_NAMESPACES,
'defaultDenyContainers' => $cluster->default_deny_containers,
'cooldVersion' => $cluster->coold_version,
'corrosionVersion' => $cluster->corrosion_version,
'corrosionGossipPort' => $cluster->corrosion_gossip_port,
'corrosionApiPort' => $cluster->corrosion_api_port,
'builderEnabled' => $cluster->builder_enabled,
'builderCapacity' => $cluster->builder_capacity,
'builderCpuQuota' => $cluster->builder_cpu_quota,
'builderMemoryMax' => $cluster->builder_memory_max,
'builderTimeoutSecs' => $cluster->builder_timeout_secs,
'lastCliAction' => $cluster->last_cli_action,
'lastCliStatus' => $cluster->last_cli_status,
'lastCliSummary' => $cluster->last_cli_summary,
'lastCliRanAt' => $cluster->last_cli_ran_at?->toJSON(),
'serversCount' => $cluster->servers_count ?? $cluster->servers->count(),
'servers' => $cluster->servers->map(fn (V5Server $server) => [
'id' => $server->uuid,
'name' => $server->name,
'host' => $server->host,
'status' => $server->status,
'capabilities' => $server->capabilities ?? [],
'builderEnabled' => $server->builder_enabled,
'builderCapacity' => $server->builder_capacity,
'builderCpuQuota' => $server->builder_cpu_quota,
'ingressEnabled' => $server->isIngress(),
'ingressType' => $server->ingress_type,
'uuid' => $server->uuid,
'nodeAddress' => $server->node_address,
'wireguardListenPortOverride' => $server->wireguard_listen_port_override,
'wireguardEndpointOverride' => $server->wireguard_endpoint_override,
'wireguardManagementIp' => $server->wireguard_management_ip,
'wireguardPublicKey' => $server->wireguard_public_key,
'containerSubnets' => $server->container_subnets ?? [],
'privateKeyName' => $server->privateKey?->name,
'lastBootstrappedAt' => $server->last_bootstrapped_at?->toJSON(),
'lastBootstrapAction' => $server->last_bootstrap_action,
'lastBootstrapStatus' => $server->last_bootstrap_status,
'lastBootstrapOutput' => $server->last_bootstrap_output,
'lastBootstrapRanAt' => $server->last_bootstrap_ran_at?->toJSON(),
'lastStatusOutput' => $server->last_status_output,
'lastStatusCheckedAt' => $server->last_status_checked_at?->toJSON(),
])->all(),
];
}
/**
* Reload servers (with keys) and counts before serializing so the payload
* always reflects the latest database state.
*
* @return array<string, mixed>
*/
public function serializeFresh(V5Cluster $cluster): array
{
$cluster->load(['servers' => fn ($query) => $query
->with('privateKey')
->orderBy('name')]);
$cluster->loadCount('servers');
return $this->serialize($cluster);
}
}
+157
View File
@@ -0,0 +1,157 @@
<?php
namespace App\Support\V5;
use App\Exceptions\V5\UnsupportedCooldVerb;
use App\Models\V5\Application as V5Application;
use App\Models\V5\ResourceConnection;
use App\Models\V5\Server as V5Server;
use App\Services\Flux\FluxClient;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
/**
* Single source of truth for deriving node firewall rules from a resource
* connection's DB rules and converging them through Flux. Reusable from
* controllers, jobs, and events alike; deterministic rule ids keep repeated
* syncs and compensating rollbacks idempotent.
*/
class ConnectionFirewallSync
{
/**
* @return Collection<int, array{id: string, hostId: string, rule: array{id: string, namespace: string, src: string, dst: string, proto: string, port: int}}>
*/
public function rulesFor(ResourceConnection $connection): Collection
{
$applicationIds = $connection->rules
->flatMap(fn ($rule) => [$rule->source_resource_id, $rule->target_resource_id])
->unique()
->values();
$applications = V5Application::query()
->whereIn('id', $applicationIds)
->with('server')
->get()
->keyBy('id');
return $connection->rules
->flatMap(function ($rule) use ($applications, $connection): Collection {
$source = $applications->get($rule->source_resource_id);
$target = $applications->get($rule->target_resource_id);
if (! $source instanceof V5Application || ! $target instanceof V5Application) {
return collect();
}
$missingHost = collect([$source, $target])
->first(function (V5Application $application): bool {
$hostId = $application->server?->fluxHostId();
return ! is_string($hostId) || $hostId === '';
});
if ($missingHost instanceof V5Application) {
throw new \RuntimeException("Application {$missingHost->name} has no reachable server host id, so its firewall rules cannot be synced.");
}
$hostIds = collect([$source->server, $target->server])
->map(fn (V5Server $server) => $server->fluxHostId())
->unique()
->values();
$firewallRule = [
'id' => $this->ruleId($connection, $rule),
'namespace' => $target->mesh_namespace ?: 'default',
'src' => $source->container_name,
'dst' => $target->container_name,
'proto' => $rule->protocol ?: 'tcp',
'port' => (int) $rule->port,
];
return $hostIds->map(fn (string $hostId): array => [
'id' => $firewallRule['id'],
'hostId' => $hostId,
'rule' => $firewallRule,
]);
})
->values();
}
/**
* @param Collection<int, array{id: string, hostId: string, rule: array{id: string, namespace: string, src: string, dst: string, proto: string, port: int}}> $oldRules
* @param Collection<int, array{id: string, hostId: string, rule: array{id: string, namespace: string, src: string, dst: string, proto: string, port: int}}> $newRules
*/
public function sync(FluxClient $fluxClient, Collection $oldRules, Collection $newRules): void
{
$newRuleKeys = $newRules->map(fn (array $rule): string => $this->syncKey($rule))->all();
$oldRuleKeys = $oldRules->map(fn (array $rule): string => $this->syncKey($rule))->all();
$oldRules
->reject(fn (array $oldRule): bool => in_array($this->syncKey($oldRule), $newRuleKeys, true))
->each(fn (array $oldRule): ?string => $this->revokeRuleIfPresent($fluxClient, $oldRule['hostId'], $oldRule['id']));
$newRules
->reject(fn (array $newRule): bool => in_array($this->syncKey($newRule), $oldRuleKeys, true))
->each(function (array $newRule) use ($fluxClient): void {
try {
$fluxClient->applyFirewallRule($newRule['hostId'], $newRule['rule']);
} catch (UnsupportedCooldVerb $exception) {
Log::warning('V5 resource connection firewall rule skipped: coold verb unsupported', [
'host_id' => $newRule['hostId'],
'rule_id' => $newRule['id'],
'verb' => $exception->verb,
'message' => $exception->getMessage(),
]);
}
});
}
public function revokeRuleIfPresent(FluxClient $fluxClient, string $hostId, string $ruleId): ?string
{
try {
return $fluxClient->revokeFirewallRule($hostId, $ruleId);
} catch (UnsupportedCooldVerb $exception) {
Log::warning('V5 resource connection firewall revoke skipped: coold verb unsupported', [
'host_id' => $hostId,
'rule_id' => $ruleId,
'verb' => $exception->verb,
'message' => $exception->getMessage(),
]);
return null;
} catch (\RuntimeException $exception) {
if (str_contains(Str::lower($exception->getMessage()), 'not found')) {
return null;
}
throw $exception;
}
}
/**
* Deterministic node-side rule id derived only from the connection id and
* the rule's stable attributes — never from the rule row's primary key
* so rewritten or restored DB rows resolve to the same firewall rule ids
* and compensating re-syncs stay idempotent.
*/
public function ruleId(ResourceConnection $connection, mixed $rule): string
{
return implode(':', [
'v5-resource-connection',
$connection->id,
$rule->source_resource_id,
$rule->target_resource_id,
$rule->protocol ?: 'tcp',
(int) $rule->port,
]);
}
/**
* @param array{id: string, hostId: string, rule: array{id: string, namespace: string, src: string, dst: string, proto: string, port: int}} $rule
*/
private function syncKey(array $rule): string
{
return $rule['hostId'].'|'.$rule['id'];
}
}
@@ -0,0 +1,73 @@
<?php
namespace App\Support\V5;
use App\Models\V5\Application as V5Application;
use App\Models\V5\ResourceConnection;
use Illuminate\Support\Collection;
/**
* Single source of truth for the resource connection payloads served by the
* dashboard Inertia props and the connection endpoints the wire format is
* consumed by resources/js/v5/types.ts and must stay stable.
*/
class ResourceConnectionSerializer
{
/**
* @return array<string, mixed>
*/
public function serialize(ResourceConnection $connection): array
{
$applications = $this->applicationsById($connection);
$resourceOneUuid = $applications->get($connection->resource_one_id)?->uuid;
$resourceTwoUuid = $applications->get($connection->resource_two_id)?->uuid;
$applicationsById = $applications;
return [
'id' => $connection->uuid,
'applicationIds' => array_values(array_filter([
$resourceOneUuid,
$resourceTwoUuid,
])),
'fromApplicationId' => $resourceOneUuid,
'toApplicationId' => $resourceTwoUuid,
'portsByDirection' => $connection->rules
->groupBy(function ($rule) use ($applicationsById): string {
$sourceUuid = $applicationsById->get($rule->source_resource_id)?->uuid;
$targetUuid = $applicationsById->get($rule->target_resource_id)?->uuid;
return "{$sourceUuid}->{$targetUuid}";
})
->filter(fn (Collection $rules, string $direction): bool => ! str_starts_with($direction, '->') && ! str_ends_with($direction, '->'))
->map(fn (Collection $rules) => $rules
->sortBy('port')
->pluck('port')
->map(fn ($port) => (string) $port)
->values()
->all())
->all(),
];
}
/**
* @return Collection<string, V5Application>
*/
public function applicationsByUuid(ResourceConnection $connection): Collection
{
return $this->applicationsById($connection)->keyBy('uuid');
}
/**
* @return Collection<int, V5Application>
*/
public function applicationsById(ResourceConnection $connection): Collection
{
return V5Application::query()
->whereIn('id', [
(int) $connection->resource_one_id,
(int) $connection->resource_two_id,
])
->get()
->keyBy('id');
}
}
+68
View File
@@ -0,0 +1,68 @@
<?php
namespace App\Support\V5;
use App\Enums\V5\ApplicationStatus;
use App\Enums\V5\ContainerState;
use App\Enums\V5\IngressStatus;
use App\Enums\V5\ServerStatus;
use Carbon\CarbonInterface;
use Illuminate\Support\Facades\Log;
/**
* Shared status-observation watermarking and enum normalization used by every
* v5 status write path (the flux webhook, the reconcile job, and the manual
* refresh endpoint) so out-of-order updates are dropped and raw coold states
* are normalized identically everywhere.
*/
class StatusObservation
{
/**
* A write whose observation timestamp is older than the one already
* persisted is stale (delivered or computed out of order) and must not
* clobber the newer state.
*
* @param array<string, mixed> $logContext
*/
public static function isStale(?CarbonInterface $observedAt, ?CarbonInterface $currentObservedAt, string $context, array $logContext): bool
{
if ($observedAt === null || $currentObservedAt === null || ! $observedAt->lt($currentObservedAt)) {
return false;
}
Log::debug("Dropping stale flux {$context} update.", [
...$logContext,
'observed_at' => $observedAt->toIso8601String(),
'current_status_observed_at' => $currentObservedAt->toIso8601String(),
]);
return true;
}
/**
* Map a raw status string onto the given status enum. Unknown values are
* never written to the database: they fall back to the enum's Unknown case
* and are logged. Returns null only when no raw value is supplied.
*
* @param class-string<ApplicationStatus|ContainerState|IngressStatus|ServerStatus> $enumClass
*/
public static function normalize(?string $raw, string $enumClass): ?string
{
if ($raw === null || $raw === '') {
return null;
}
$status = $enumClass::tryFrom(strtolower($raw));
if ($status === null) {
Log::warning('Received unknown flux resource status; falling back to unknown.', [
'raw_status' => $raw,
'status_enum' => $enumClass,
]);
return $enumClass::Unknown->value;
}
return $status->value;
}
}
+17
View File
@@ -5,4 +5,21 @@ return [
'coold_version' => env('COOLIFY_COOLD_VERSION', 'nightly'),
'corrosion_version' => env('COOLIFY_CORROSION_VERSION', 'v1.0.0'),
'dev_ssh_user' => env('COOLIFY_CLI_SSH_USER', 'coolify'),
'flux_url' => env('COOLIFY_COOLD_FLUX_URL', env('COOLIFY_COOLD_VM_FLUX_URL')),
'flux_host_jwt_path' => env('COOLIFY_COOLD_HOST_JWT_PATH', '/etc/coolify/host-jwt'),
/*
* When false (the default), v5 server hosts/node addresses may not point at
* private or reserved IP ranges (loopback, link-local, RFC 1918, CGNAT is
* still allowed as it is the WireGuard mesh space). This blocks a team
* member from adding a server that targets the Coolify host's internal
* network and abusing the synchronous SSH connectivity check to probe it.
*
* Self-hosters running Coolify on a private LAN can opt back in by setting
* COOLIFY_ALLOW_PRIVATE_SERVER_IPS=true.
*/
'allow_private_server_ips' => filter_var(
env('COOLIFY_ALLOW_PRIVATE_SERVER_IPS', false),
FILTER_VALIDATE_BOOLEAN
),
];
+100
View File
@@ -7,5 +7,105 @@ return [
'health_timeout_seconds' => (float) env('COOLIFY_FLUX_HEALTH_TIMEOUT_SECONDS', 1.0),
'connection_timeout_seconds' => (float) env('COOLIFY_FLUX_CONNECTION_TIMEOUT_SECONDS', 1.0),
'dispatch_timeout_seconds' => (float) env('COOLIFY_FLUX_DISPATCH_TIMEOUT_SECONDS', 35.0),
'bootstrap_host_connection_timeout_seconds' => (int) env('COOLIFY_FLUX_BOOTSTRAP_HOST_CONNECTION_TIMEOUT_SECONDS', 30),
/*
|--------------------------------------------------------------------------
| Host agent (coold) token capabilities
|--------------------------------------------------------------------------
|
| The EXACT set of primitive capability strings coold advertises to flux on
| connect (coold/coold/src/grpc/client.rs, `primitive_capabilities`). Minting
| these explicit strings instead of the `host-agent:default` wildcard
| profile means the token no longer relies on flux's
| `capability_profile_authorizes_all` bypass. flux INTERSECTS the jwt `caps`
| with coold's advertised set, so as long as this list matches coold's
| advertised primitives the host retains exactly the same effective power.
|
| SAFETY: keep this list byte-for-byte in sync with coold's
| `primitive_capabilities()`. A string here that coold does not advertise is
| silently dropped by flux's intersection; a verb coold needs that is missing
| here means the host loses that ability.
|
| `host.jwt.set` authorizes RPC-delivered host-JWT rotation (the token can
| authorize its own replacement over the live stream; Laravel is the root of
| trust that holds the signing key).
*/
'host_capabilities' => [
'images.pull',
'images.list',
'images.delete',
'containers.create',
'containers.start',
'containers.stop',
'containers.restart',
'containers.delete',
'containers.inspect',
'containers.list',
'containers.logs',
'containers.exec',
'containers.healthcheck.run',
'ingress.apply',
'ingress.stop',
'firewall.allow',
'firewall.revoke',
'firewall.list',
'firewall.reconcile',
'coold.logs',
'corrosion.tables',
'host.jwt.set',
],
/*
| Emergency escape hatch: when set (e.g. to `host-agent:default`), minted
| host tokens carry ONLY this single capability profile instead of the
| explicit list above. This re-enables flux's wildcard bypass and is meant
| purely for rollback without a code change if the explicit list ever drifts
| from coold's advertised set and breaks the data plane. Leave NULL in
| production so tokens are explicitly scoped.
*/
'host_capability_profile' => env('COOLIFY_FLUX_HOST_CAPABILITY_PROFILE'),
/*
| Host JWT lifetime and rotation.
|
| `host_token_ttl` is the token `exp` window (default 1h) the maximum time
| a leaked/undetected token stays valid if BOTH rotation and revocation fail.
| Keep this at or below flux's `COOLIFY_FLUX_MAX_TOKEN_LIFETIME_SECS`
| default (3600), otherwise flux rejects coold streams at connect.
| `host_token_refresh_threshold` is the remaining-lifetime below which the
| rotation job re-mints and re-delivers a fresh token (default 30m).
| Keep the threshold below the TTL.
*/
'host_token_ttl' => (int) env('COOLIFY_FLUX_HOST_TOKEN_TTL', 3600),
'host_token_refresh_threshold' => (int) env('COOLIFY_FLUX_HOST_TOKEN_REFRESH_THRESHOLD', 1800),
/*
| JWT header `kid` minted into host tokens. flux selects the verification key
| by this id (single default key today; a per-cluster keys directory can map
| `kid = cluster-<id>` to `<kid>.pub` for per-tenant signing keys later).
*/
'jwt_kid' => env('COOLIFY_FLUX_JWT_KID', 'flux-default'),
/*
|--------------------------------------------------------------------------
| Inbound flux -> Laravel API token(s)
|--------------------------------------------------------------------------
|
| flux authenticates to Laravel's internal status-ingest endpoint with a
| bearer token. `laravel_api_tokens` accepts SEVERAL tokens at once so an
| operator can rotate with zero downtime:
| 1. add the new token alongside the old:
| COOLIFY_FLUX_LARAVEL_API_TOKENS=<old>,<new>
| then `php artisan config:clear`
| 2. cut every flux instance over to <new>
| 3. drop <old> from the list and `config:clear` again
| Generate tokens with `openssl rand -hex 32`. The single `laravel_api_token`
| remains as a fallback so existing single-token installs keep working.
*/
'laravel_api_token' => env('COOLIFY_FLUX_LARAVEL_API_TOKEN'),
'laravel_api_tokens' => array_values(array_filter(array_map(
'trim',
explode(',', (string) env('COOLIFY_FLUX_LARAVEL_API_TOKENS', ''))
))),
];
+28 -1
View File
@@ -192,6 +192,24 @@ return [
'sleep' => 3,
'timeout' => env('HORIZON_TIMEOUT', 36000),
],
// Dedicated low-priority pool for the v5 reconcile + host-token rotation
// jobs (queue `v5-reconcile`, set via onQueue()). Isolated from the
// user-facing high/default deploy pool so a starved rotation cannot let
// host tokens drift to expiry, and so the 5-minute fleet fan-out never
// blocks deploys.
'v5reconcile' => [
'connection' => 'redis',
'balance' => env('HORIZON_V5_RECONCILE_BALANCE', 'false'),
'queue' => 'v5-reconcile',
'maxTime' => env('HORIZON_V5_RECONCILE_MAX_TIME', 0),
'maxJobs' => 200,
'memory' => 128,
'tries' => 1,
'nice' => 10,
'sleep' => 3,
'timeout' => env('HORIZON_V5_RECONCILE_TIMEOUT', 300),
],
],
'environments' => [
@@ -203,7 +221,11 @@ return [
'balanceMaxShift' => env('HORIZON_BALANCE_MAX_SHIFT', 1),
'balanceCooldown' => env('HORIZON_BALANCE_COOLDOWN', 1),
],
'v5reconcile' => [
'autoScalingStrategy' => 'size',
'minProcesses' => env('HORIZON_V5_RECONCILE_MIN_PROCESSES', 1),
'maxProcesses' => env('HORIZON_V5_RECONCILE_MAX_PROCESSES', 2),
],
],
'local' => [
's6' => [
@@ -213,6 +235,11 @@ return [
'balanceMaxShift' => env('HORIZON_BALANCE_MAX_SHIFT', 1),
'balanceCooldown' => env('HORIZON_BALANCE_COOLDOWN', 1),
],
'v5reconcile' => [
'autoScalingStrategy' => 'size',
'minProcesses' => env('HORIZON_V5_RECONCILE_MIN_PROCESSES', 1),
'maxProcesses' => env('HORIZON_V5_RECONCILE_MAX_PROCESSES', 1),
],
],
],
];
@@ -0,0 +1,48 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
/**
* Run the migrations.
*/
public function up(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->index('wireguard_management_ip');
$table->index('node_address');
$table->index('host');
});
Schema::table('v5_applications', function (Blueprint $table) {
$table->index('runtime_container_id');
});
Schema::table('v5_container_statuses', function (Blueprint $table) {
$table->index('last_seen_at');
});
}
/**
* Reverse the migrations.
*/
public function down(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->dropIndex(['wireguard_management_ip']);
$table->dropIndex(['node_address']);
$table->dropIndex(['host']);
});
Schema::table('v5_applications', function (Blueprint $table) {
$table->dropIndex(['runtime_container_id']);
});
Schema::table('v5_container_statuses', function (Blueprint $table) {
$table->dropIndex(['last_seen_at']);
});
}
};
@@ -0,0 +1,39 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;
use Illuminate\Support\Str;
return new class extends Migration
{
/**
* Run the migrations.
*/
public function up(): void
{
DB::table('v5_servers')
->whereNull('uuid')
->pluck('id')
->each(function (int $id): void {
DB::table('v5_servers')
->where('id', $id)
->update(['uuid' => Str::lower(Str::random(24))]);
});
Schema::table('v5_servers', function (Blueprint $table) {
$table->string('uuid')->nullable(false)->change();
});
}
/**
* Reverse the migrations.
*/
public function down(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->string('uuid')->nullable()->change();
});
}
};
@@ -0,0 +1,44 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
/**
* Run the migrations.
*/
public function up(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->timestamp('status_observed_at')->nullable()->after('status');
});
Schema::table('v5_applications', function (Blueprint $table) {
$table->timestamp('status_observed_at')->nullable()->after('status_message');
});
Schema::table('v5_container_statuses', function (Blueprint $table) {
$table->timestamp('status_observed_at')->nullable()->after('status_message');
});
}
/**
* Reverse the migrations.
*/
public function down(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->dropColumn('status_observed_at');
});
Schema::table('v5_applications', function (Blueprint $table) {
$table->dropColumn('status_observed_at');
});
Schema::table('v5_container_statuses', function (Blueprint $table) {
$table->dropColumn('status_observed_at');
});
}
};
@@ -0,0 +1,28 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
/**
* Run the migrations.
*/
public function up(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->string('coold_version')->nullable()->after('wireguard_public_key');
});
}
/**
* Reverse the migrations.
*/
public function down(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->dropColumn('coold_version');
});
}
};
@@ -0,0 +1,74 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;
/**
* Rewrites v5 polymorphic rows that stored the Application FQCN (written
* before the 'v5.application' morph alias existed) so a future class rename
* cannot orphan them. New rows pick up the alias via getMorphClass().
*/
return new class extends Migration
{
private const FQCN = 'App\Models\V5\Application';
private const ALIAS = 'v5.application';
public function up(): void
{
$this->rewriteMorphTypes(self::FQCN, self::ALIAS);
$this->rewritePairKeys(self::FQCN, self::ALIAS);
}
public function down(): void
{
$this->rewriteMorphTypes(self::ALIAS, self::FQCN);
$this->rewritePairKeys(self::ALIAS, self::FQCN);
}
private function rewriteMorphTypes(string $from, string $to): void
{
if (Schema::hasTable('v5_resource_connections')) {
foreach (['resource_one_type', 'resource_two_type'] as $column) {
DB::table('v5_resource_connections')
->where($column, $from)
->update([$column => $to]);
}
}
if (Schema::hasTable('v5_resource_connection_rules')) {
foreach (['source_resource_type', 'target_resource_type'] as $column) {
DB::table('v5_resource_connection_rules')
->where($column, $from)
->update([$column => $to]);
}
}
}
private function rewritePairKeys(string $from, string $to): void
{
if (! Schema::hasTable('v5_resource_connections')) {
return;
}
// Backslash escaping in LIKE patterns differs between Postgres and
// SQLite, so match the FQCN in PHP instead of in SQL.
DB::table('v5_resource_connections')
->select(['id', 'resource_pair_key'])
->orderBy('id')
->chunkById(100, function ($connections) use ($from, $to): void {
foreach ($connections as $connection) {
if (! str_contains((string) $connection->resource_pair_key, $from)) {
continue;
}
DB::table('v5_resource_connections')
->where('id', $connection->id)
->update([
'resource_pair_key' => str_replace($from, $to, $connection->resource_pair_key),
]);
}
});
}
};
@@ -0,0 +1,74 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;
/**
* Replaces the unindexable v5_servers.capabilities JSON array of magic strings
* with indexed has_coold / is_ingress booleans. The Server model keeps
* exposing a computed `capabilities` array so serialized payloads stay
* identical.
*/
return new class extends Migration
{
public function up(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->boolean('has_coold')->default(false)->index();
$table->boolean('is_ingress')->default(false)->index();
});
DB::table('v5_servers')
->select(['id', 'capabilities'])
->orderBy('id')
->chunkById(100, function ($servers): void {
foreach ($servers as $server) {
$capabilities = json_decode($server->capabilities ?? '[]', true);
if (! is_array($capabilities) || $capabilities === []) {
continue;
}
DB::table('v5_servers')->where('id', $server->id)->update([
'has_coold' => in_array('coold', $capabilities, true),
'is_ingress' => in_array('ingress', $capabilities, true),
]);
}
});
Schema::table('v5_servers', function (Blueprint $table) {
$table->dropColumn('capabilities');
});
}
public function down(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->json('capabilities')->nullable();
});
DB::table('v5_servers')
->select(['id', 'has_coold', 'is_ingress'])
->orderBy('id')
->chunkById(100, function ($servers): void {
foreach ($servers as $server) {
$capabilities = array_values(array_filter([
$server->has_coold ? 'coold' : null,
$server->is_ingress ? 'ingress' : null,
]));
DB::table('v5_servers')->where('id', $server->id)->update([
'capabilities' => json_encode($capabilities),
]);
}
});
Schema::table('v5_servers', function (Blueprint $table) {
$table->dropIndex(['has_coold']);
$table->dropIndex(['is_ingress']);
$table->dropColumn(['has_coold', 'is_ingress']);
});
}
};
@@ -0,0 +1,22 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
public function up(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->string('agent_token_jti')->nullable()->after('coold_version');
});
}
public function down(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->dropColumn('agent_token_jti');
});
}
};
@@ -0,0 +1,25 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
public function up(): void
{
Schema::create('v5_revoked_agent_tokens', function (Blueprint $table) {
$table->id();
$table->string('jti')->unique();
$table->foreignId('server_id')->nullable();
$table->timestamp('revoked_at')->nullable();
$table->timestamp('expires_at')->nullable()->index();
$table->timestamps();
});
}
public function down(): void
{
Schema::dropIfExists('v5_revoked_agent_tokens');
}
};
@@ -0,0 +1,22 @@
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
public function up(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->timestamp('agent_token_expires_at')->nullable()->after('agent_token_jti');
});
}
public function down(): void
{
Schema::table('v5_servers', function (Blueprint $table) {
$table->dropColumn('agent_token_expires_at');
});
}
};
+1
View File
@@ -22,6 +22,7 @@ services:
COOLIFY_CONTAINER_ROLE: "${COOLIFY_CONTAINER_ROLE:-all}"
COOLIFY_COOLD_VERSION: "${COOLIFY_COOLD_VERSION:-nightly}"
COOLIFY_FLUX_VERSION: "${COOLIFY_FLUX_VERSION:-nightly}"
COOLIFY_FLUX_REQUIRE_HOST_BINDING: "${COOLIFY_FLUX_REQUIRE_HOST_BINDING:-0}"
COOLIFY_CLI_VERSION: "${COOLIFY_CLI_VERSION:-nightly}"
COOLIFY_CLI_SSH_USER: "${COOLIFY_CLI_SSH_USER:-}"
COOLIFY_CORROSION_VERSION: "${COOLIFY_CORROSION_VERSION:-v1.0.0}"
@@ -20,6 +20,7 @@ export COOLIFY_FLUX_UNIX_SOCKET_PATH="${COOLIFY_FLUX_UNIX_SOCKET_PATH:-/run/cool
export COOLIFY_FLUX_JWT_PRIVATE_KEY_PATH="${COOLIFY_FLUX_JWT_PRIVATE_KEY_PATH:-/var/www/html/storage/app/flux/jwt.priv}"
export COOLIFY_FLUX_JWT_PUBLIC_KEY_PATH="${COOLIFY_FLUX_JWT_PUBLIC_KEY_PATH:-/var/www/html/storage/app/flux/jwt.pub}"
export COOLIFY_FLUX_ALLOW_PUBLIC_BIND="${COOLIFY_FLUX_ALLOW_PUBLIC_BIND:-1}"
export COOLIFY_FLUX_REQUIRE_HOST_BINDING="${COOLIFY_FLUX_REQUIRE_HOST_BINDING:-0}"
export COOLIFY_FLUX_LARAVEL_API_URL="${COOLIFY_FLUX_LARAVEL_API_URL:-http://127.0.0.1:8080}"
if [ -z "${COOLIFY_FLUX_LARAVEL_API_TOKEN:-}" ] && [ -f .env ]; then
COOLIFY_FLUX_LARAVEL_API_TOKEN="$(grep -E '^COOLIFY_FLUX_LARAVEL_API_TOKEN=' .env 2>/dev/null | tail -n1 | cut -d= -f2- | sed "s/^['\"]//; s/['\"]$//")"
+26 -15
View File
@@ -81,7 +81,6 @@
"integrity": "sha512-RgHBCvtjbOK2gXSNBNIkNoEc9qoVEtau3hj8gEqKQuL3HZAibKarWFEI3Lfm6EYKkLalOh8eSrj9b+ch9H/VBA==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/code-frame": "^7.29.7",
"@babel/generator": "^7.29.7",
@@ -788,12 +787,36 @@
"@noble/ciphers": "^1.0.0"
}
},
"node_modules/@emnapi/core": {
"version": "1.11.2",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.11.2.tgz",
"integrity": "sha512-TC8MkTuZUtcTSiFeuC0ksCh9QIJ5+F21MvZ4Wn4ORfYaFJ/0dsiudv5tVkejgwZlwQ39jL9WWDe2lz8x0WglOA==",
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"@emnapi/wasi-threads": "1.2.2",
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/runtime": {
"version": "1.11.2",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.11.2.tgz",
"integrity": "sha512-kyOl3X0DuTiT1h2ft8r2fYO8JYtU9a9Xis/zBSiGArNaagCOWx90N1k2wxp18czFDH+OgcWGb5ZP/XMt3dcyPA==",
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
},
"node_modules/@emnapi/wasi-threads": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.2.2.tgz",
"integrity": "sha512-c95qOXkHdydNKhscBTebqEC1CVAZpyqOfVfBzQ1qgzyl3gfeldUjIggDbIZgDKsHLgnsM+igH7TJ/eAasaVuMA==",
"license": "MIT",
"optional": true,
"peer": true,
"dependencies": {
"tslib": "^2.4.0"
}
@@ -1017,7 +1040,6 @@
"integrity": "sha512-2I0gnIVPtfnMw9ee9h1dJG7tp81+8Ob3OJb3Mv37rx5L40/b0i7djjCVvGOVqc9AEIQyvyu1i6ypKdFw8R8gQw==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": "^14.21.3 || >=16"
},
@@ -1852,7 +1874,6 @@
"integrity": "sha512-MXfmqaVPEVgkBT/aY0aGCkRWWtByiYQXo3xdQ8r5RzuFrPiRn8Gar2tQdXSUQ2GKV3bkXckek89V8wQBY2Q/Aw==",
"devOptional": true,
"license": "MIT",
"peer": true,
"dependencies": {
"csstype": "^3.2.2"
}
@@ -1908,8 +1929,7 @@
"version": "5.5.0",
"resolved": "https://registry.npmjs.org/@xterm/xterm/-/xterm-5.5.0.tgz",
"integrity": "sha512-hqJHYaQb5OptNunnyAnkHyM8aCjZ1MEIDTQu1iIbbTD/xops91NB5yq1ZK/dC2JDbVWtF23zUtl9JE2NqwT87A==",
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/accepts": {
"version": "2.0.0",
@@ -2128,7 +2148,6 @@
}
],
"license": "MIT",
"peer": true,
"dependencies": {
"baseline-browser-mapping": "^2.10.12",
"caniuse-lite": "^1.0.30001782",
@@ -2949,7 +2968,6 @@
"integrity": "sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"accepts": "^2.0.0",
"body-parser": "^2.2.1",
@@ -3399,7 +3417,6 @@
"integrity": "sha512-2NFaIyNVgJmBs/ecmtGzlmluTFs5cHEWGTdu0t1HBwYzoGXOL5nUQBRMXsXWla5i4KkG//QMzVP88m1+I3fdAQ==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=16.9.0"
}
@@ -5056,7 +5073,6 @@
"resolved": "https://registry.npmjs.org/react/-/react-19.2.7.tgz",
"integrity": "sha512-HNe9WslTbXmFK8o8cmwgAeJFSBvt1bPdHCVKtaaV+WlAN36mpT4hcRpwbf3fY56ar2oIXzsBpOAiIRHAdY0OlQ==",
"license": "MIT",
"peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -5066,7 +5082,6 @@
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.2.7.tgz",
"integrity": "sha512-t0BRVXvbiE/o20Hfw669rLbMCDWtYZLvmJigy2f0MxsXF+71pxhR3xOkspmsO8h3ZlNzyibAmtCa3l4lYKk6gQ==",
"license": "MIT",
"peer": true,
"dependencies": {
"scheduler": "^0.27.0"
},
@@ -5732,8 +5747,7 @@
"version": "4.1.18",
"resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.1.18.tgz",
"integrity": "sha512-4+Z+0yiYyEtUVCScyfHCxOYP06L5Ne+JiHhY2IjR2KWMIWhJOYZKLSGZaP5HkZ8+bY0cxfzwDE5uOmzFXyIwxw==",
"license": "MIT",
"peer": true
"license": "MIT"
},
"node_modules/tapable": {
"version": "2.3.0",
@@ -5876,7 +5890,6 @@
"integrity": "sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==",
"dev": true,
"license": "Apache-2.0",
"peer": true,
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
@@ -5999,7 +6012,6 @@
"resolved": "https://registry.npmjs.org/vite/-/vite-8.0.16.tgz",
"integrity": "sha512-h9bXPmJichP5fLmVQo3PyaGSDE2n3aPuomeAlVRm0JLmt4rY6zmPKd59HYI4LNW8oTK7tlTsuC7l/m7awx9Jcw==",
"license": "MIT",
"peer": true,
"dependencies": {
"lightningcss": "^1.32.0",
"picomatch": "^4.0.4",
@@ -6451,7 +6463,6 @@
"integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==",
"dev": true,
"license": "MIT",
"peer": true,
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
+3
View File
@@ -23,6 +23,9 @@
<env name="SESSION_DRIVER" value="array" force="true"/>
<env name="TELESCOPE_ENABLED" value="false"/>
<env name="NIGHTWATCH_ENABLED" value="false"/>
<!-- The v5 bootstrap endpoint refuses to queue without a Flux URL; tests
that exercise the unconfigured path blank it via Config::set. -->
<env name="COOLIFY_COOLD_FLUX_URL" value="http://flux.testing:6443" force="true"/>
</php>
<source>
<include>
+394 -247
View File
@@ -4,6 +4,7 @@ import { useEffect, useMemo, useState } from 'react';
import type { FormEvent } from 'react';
import { AppNavbar } from '@/components/app-navbar';
import { CanvasNotice } from '@/components/canvas/canvas-notice';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
@@ -25,8 +26,9 @@ import {
DialogTitle,
} from '@/components/ui/dialog';
import { Select, SelectContent, SelectGroup, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select';
import { csrfToken } from '@/lib/csrf';
import { apiRequest } from '@/lib/api';
import { usePendingIds } from '@/lib/use-pending-ids';
import { useTeamChannel } from '@/lib/use-team-channel';
import type { V5Cluster, V5DashboardProps, V5Server } from '@/types';
type ClusterFormErrors = {
@@ -90,11 +92,13 @@ type DeleteServerResponse = {
type CooldLogsResponse = {
output: string;
fetchedAt: string;
source: 'flux' | 'ssh';
};
type CorrosionTablesResponse = {
output: string;
fetchedAt: string;
source: 'flux' | 'ssh';
};
type FirewallRule = {
@@ -127,16 +131,28 @@ type BootstrapServerResponse = {
message?: string;
};
type ServerSshCheck = {
status: string;
output: string;
checkedAt: string;
type ServerConnectionNotice = {
message: string;
description: string;
variant: 'danger' | 'success';
};
type V5ClusterUpdatedEvent = {
cluster: V5Cluster | null;
};
type ParsedBootstrapLogSummary = {
label: string;
value: string;
tone: 'success' | 'muted';
};
type ParsedBootstrapLogs = {
summary: ParsedBootstrapLogSummary[];
visibleOutput: string;
rawOutput: string;
};
function formatCorrosionCell(value: unknown): string {
if (value === null || value === undefined) {
return 'null';
@@ -163,6 +179,147 @@ function parseCorrosionTables(output: string): CorrosionTableDump | null {
}
}
function jsonValueToString(value: unknown): string {
if (value === null || value === undefined || value === '') {
return 'n/a';
}
if (typeof value === 'string' || typeof value === 'number' || typeof value === 'boolean') {
return String(value);
}
return JSON.stringify(value);
}
function initialJsonEnd(output: string): number | null {
const trimmedStart = output.search(/\S/);
if (trimmedStart === -1 || output[trimmedStart] !== '{') {
return null;
}
let depth = 0;
let inString = false;
let isEscaped = false;
for (let index = trimmedStart; index < output.length; index += 1) {
const character = output[index];
if (isEscaped) {
isEscaped = false;
continue;
}
if (character === '\\') {
isEscaped = inString;
continue;
}
if (character === '"') {
inString = !inString;
continue;
}
if (inString) {
continue;
}
if (character === '{') {
depth += 1;
}
if (character === '}') {
depth -= 1;
}
if (depth === 0) {
return index + 1;
}
}
return null;
}
function hideRawBootstrapPlan(output: string): string {
const visibleLines: string[] = [];
let isSkippingPlan = false;
output.split('\n').forEach((line) => {
if (line.trim() === 'Plan:') {
isSkippingPlan = true;
return;
}
if (isSkippingPlan) {
const trimmedLine = line.trim();
if (trimmedLine === '' || trimmedLine.startsWith('[') || line.startsWith(' ')) {
return;
}
isSkippingPlan = false;
}
visibleLines.push(line);
});
return visibleLines.join('\n').trim();
}
function parseBootstrapOutput(output: string | null): ParsedBootstrapLogs {
const rawOutput = output?.trim() || 'No install logs captured yet.';
const jsonEnd = initialJsonEnd(rawOutput);
if (jsonEnd === null) {
return {
summary: [],
visibleOutput: rawOutput,
rawOutput,
};
}
try {
const parsed = JSON.parse(rawOutput.slice(0, jsonEnd)) as {
results?: Array<{
action?: { action?: unknown; host?: unknown };
status?: unknown;
detail?: unknown;
}>;
verified?: Array<Record<string, unknown>>;
};
const summary = [
...(parsed.results ?? []).map((result) => ({
label: jsonValueToString(result.action?.action ?? result.action?.host ?? 'Action'),
value: `${jsonValueToString(result.status)}${result.detail ? ` · ${jsonValueToString(result.detail)}` : ''}`,
tone: result.status === 'ok' ? ('success' as const) : ('muted' as const),
})),
...(parsed.verified ?? []).map((node) => ({
label: `Verified ${jsonValueToString(node.host)}`,
value: [
`status ${jsonValueToString(node.status)}`,
`wg ${jsonValueToString(node.wireguard_ip)}`,
`peers ${jsonValueToString(node.peer_count)}`,
].join(' · '),
tone: node.status === 'ok' ? ('success' as const) : ('muted' as const),
})),
];
const remainingOutput = rawOutput.slice(jsonEnd).trim();
return {
summary,
visibleOutput: hideRawBootstrapPlan(remainingOutput),
rawOutput,
};
} catch {
return {
summary: [],
visibleOutput: rawOutput,
rawOutput,
};
}
}
function statusLabel(status: string): string {
return status
.split(/[-_\s]+/)
@@ -187,24 +344,6 @@ function statusBadgeClass(status: string): string {
return 'border-border bg-muted/40 text-muted-foreground';
}
type EchoChannel = {
listen: (event: string, callback: (payload: unknown) => void) => EchoChannel;
subscribed?: (callback: () => void) => EchoChannel;
error?: (callback: (error: unknown) => void) => EchoChannel;
};
type EchoClient = {
private: (channel: string) => EchoChannel;
leave?: (channel: string) => void;
leaveChannel?: (channel: string) => void;
};
declare global {
interface Window {
Echo?: EchoClient;
}
}
const clusterDefaults = {
wireguardInterface: 'wg0',
wireguardManagementPool: '100.64.0.0/16',
@@ -242,6 +381,18 @@ function formatDate(value: string | null): string {
}).format(new Date(value));
}
function diagnosticsSourceLabel(source: 'flux' | 'ssh' | null): string {
if (source === 'ssh') {
return 'SSH';
}
if (source === 'flux') {
return 'Flux';
}
return 'Unknown';
}
export default function Clusters({
flux,
currentTeam = null,
@@ -297,8 +448,9 @@ export default function Clusters({
const [isServerSubmitting, setIsServerSubmitting] = useState(false);
const [isServerUpdateSubmitting, setIsServerUpdateSubmitting] = useState(false);
const checkingServers = usePendingIds<string>();
const [sshChecks, setSshChecks] = useState<Record<string, ServerSshCheck>>({});
const [visibleBootstrapLogs, setVisibleBootstrapLogs] = useState<Record<string, boolean>>({});
const [serverConnectionNotice, setServerConnectionNotice] = useState<ServerConnectionNotice | null>(null);
const [isBootstrapLogsDialogOpen, setIsBootstrapLogsDialogOpen] = useState(false);
const [bootstrapLogsServerId, setBootstrapLogsServerId] = useState<string | null>(null);
const bootstrappingServers = usePendingIds<string>();
const [bootstrapServerError, setBootstrapServerError] = useState<string | null>(null);
const deletingServers = usePendingIds<string>();
@@ -314,12 +466,14 @@ export default function Clusters({
const [cooldLogsServer, setCooldLogsServer] = useState<V5Server | null>(null);
const [cooldLogsOutput, setCooldLogsOutput] = useState('');
const [cooldLogsFetchedAt, setCooldLogsFetchedAt] = useState<string | null>(null);
const [cooldLogsSource, setCooldLogsSource] = useState<'flux' | 'ssh' | null>(null);
const [cooldLogsError, setCooldLogsError] = useState<string | null>(null);
const [isLoadingCooldLogs, setIsLoadingCooldLogs] = useState(false);
const [isCorrosionTablesDialogOpen, setIsCorrosionTablesDialogOpen] = useState(false);
const [corrosionTablesServer, setCorrosionTablesServer] = useState<V5Server | null>(null);
const [corrosionTablesOutput, setCorrosionTablesOutput] = useState('');
const [corrosionTablesFetchedAt, setCorrosionTablesFetchedAt] = useState<string | null>(null);
const [corrosionTablesSource, setCorrosionTablesSource] = useState<'flux' | 'ssh' | null>(null);
const [corrosionTablesError, setCorrosionTablesError] = useState<string | null>(null);
const [isLoadingCorrosionTables, setIsLoadingCorrosionTables] = useState(false);
const [isFirewallRulesDialogOpen, setIsFirewallRulesDialogOpen] = useState(false);
@@ -340,60 +494,23 @@ export default function Clusters({
const initializedServers = selectedCluster?.servers.filter((server) => server.lastBootstrappedAt !== null) ?? [];
const hasBootstrapInProgress =
selectedCluster?.servers.some((server) => ['queued', 'running'].includes(server.lastBootstrapStatus ?? '')) ?? false;
const bootstrapLogsServer = useMemo(
() => clusterList.flatMap((cluster) => cluster.servers).find((server) => server.id === bootstrapLogsServerId) ?? null,
[bootstrapLogsServerId, clusterList],
);
const parsedBootstrapLogs = parseBootstrapOutput(bootstrapLogsServer?.lastBootstrapOutput ?? null);
useEffect(() => {
if (!currentTeam) {
useTeamChannel(currentTeam?.id ?? null, '.v5.cluster.updated', (payload) => {
const event = payload as V5ClusterUpdatedEvent;
if (!event.cluster) {
return;
}
let isCancelled = false;
let attempts = 0;
const channelName = `team.${currentTeam.id}`;
const interval = window.setInterval(() => {
attempts += 1;
if (!window.Echo) {
if (attempts === 1) {
console.debug('Waiting for window.Echo before subscribing to cluster updates');
}
if (attempts >= 20) {
window.clearInterval(interval);
}
return;
}
window.clearInterval(interval);
if (isCancelled) {
return;
}
const channel = window.Echo.private(channelName);
channel.subscribed?.(() => console.debug(`Subscribed to private-${channelName} for cluster updates`));
channel.error?.((error) => console.error(`Subscription error on private-${channelName}`, error));
channel.listen('.v5.cluster.updated', (payload) => {
const event = payload as V5ClusterUpdatedEvent;
if (!event.cluster) {
return;
}
setClusterList((currentClusters) =>
currentClusters.map((cluster) => (cluster.id === event.cluster?.id ? event.cluster : cluster)),
);
});
}, 500);
return () => {
isCancelled = true;
window.clearInterval(interval);
window.Echo?.leave?.(channelName) ?? window.Echo?.leaveChannel?.(`private-${channelName}`);
};
}, [currentTeam]);
setClusterList((currentClusters) =>
currentClusters.map((cluster) => (cluster.id === event.cluster?.id ? event.cluster : cluster)),
);
});
useEffect(() => {
if (!selectedCluster || !hasBootstrapInProgress) {
@@ -407,15 +524,9 @@ export default function Clusters({
return;
}
const response = await fetch(`/v5/clusters/${selectedCluster.id}`, {
method: 'GET',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
},
});
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}`, { method: 'GET' }).catch(() => null);
if (!response.ok || isCancelled) {
if (!response?.ok || isCancelled) {
return;
}
@@ -441,15 +552,9 @@ export default function Clusters({
setIsSubmitting(true);
setErrors({});
const response = await fetch('/v5/clusters', {
const response = await apiRequest('/v5/clusters', {
method: 'POST',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
body: JSON.stringify({
body: {
name,
description: description.trim() === '' ? null : description,
wireguard_interface: wireguardInterface,
@@ -471,10 +576,10 @@ export default function Clusters({
builder_cpu_quota: builderCpuQuota,
builder_memory_max: builderMemoryMax,
builder_timeout_secs: Number(builderTimeoutSecs),
}),
});
},
}).catch(() => null);
if (response.status === 422) {
if (response?.status === 422) {
const payload = (await response.json()) as {
errors?: ClusterFormErrors;
};
@@ -484,7 +589,7 @@ export default function Clusters({
return;
}
if (!response.ok) {
if (!response?.ok) {
setErrors({
name: ['Unable to create this cluster. Please try again.'],
});
@@ -517,15 +622,9 @@ export default function Clusters({
setIsServerSubmitting(true);
setServerErrors({});
const response = await fetch(`/v5/clusters/${selectedCluster.id}/servers`, {
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}/servers`, {
method: 'POST',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
body: JSON.stringify({
body: {
name: serverName,
host: serverHost,
ssh_user: serverSshUser,
@@ -540,10 +639,10 @@ export default function Clusters({
wireguard_listen_port_override:
wireguardListenPortOverride.trim() === '' ? null : Number(wireguardListenPortOverride),
wireguard_endpoint_override: wireguardEndpointOverride.trim() === '' ? null : wireguardEndpointOverride,
}),
});
},
}).catch(() => null);
if (response.status === 422) {
if (response?.status === 422) {
const payload = (await response.json()) as {
errors?: ServerFormErrors;
};
@@ -553,7 +652,7 @@ export default function Clusters({
return;
}
if (!response.ok) {
if (!response?.ok) {
setServerErrors({
name: ['Unable to add this server. Please try again.'],
});
@@ -582,24 +681,18 @@ export default function Clusters({
setIsServerUpdateSubmitting(true);
setEditServerErrors({});
const response = await fetch(`/v5/clusters/${selectedCluster.id}/servers/${editingServer.id}`, {
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}/servers/${editingServer.id}`, {
method: 'PATCH',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
body: JSON.stringify({
body: {
builder_enabled: editServerBuilderEnabled,
ingress_enabled: editServerIngressEnabled,
ingress_type: editServerIngressEnabled ? editServerIngressType : null,
builder_capacity: Number(editServerBuilderCapacity),
builder_cpu_quota: editServerBuilderCpuQuota,
}),
});
},
}).catch(() => null);
if (response.status === 422) {
if (response?.status === 422) {
const payload = (await response.json()) as {
errors?: ServerFormErrors;
};
@@ -609,8 +702,8 @@ export default function Clusters({
return;
}
if (!response.ok) {
const payload = (await response.json().catch(() => null)) as { message?: string } | null;
if (!response?.ok) {
const payload = (await response?.json().catch(() => null)) as { message?: string } | null;
setEditServerErrors({
builder_capacity: [payload?.message ?? 'Unable to update this server. Please try again.'],
@@ -637,23 +730,20 @@ export default function Clusters({
checkingServers.start(server.id);
const response = await fetch(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/check`, {
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/check`, {
method: 'POST',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
});
}).catch(() => null);
const payload = (await response?.json().catch(() => null)) as (CheckServerResponse & { message?: string }) | null;
if (response.ok) {
const payload = (await response.json()) as CheckServerResponse;
setSshChecks((currentChecks) => ({
...currentChecks,
[server.id]: payload,
}));
}
setServerConnectionNotice({
message: response?.ok
? `Connection check for ${server.name}: ${payload?.status ?? 'unknown'}`
: `Connection check failed for ${server.name}`,
description: response?.ok
? (payload?.output ?? 'No output returned.')
: (payload?.message ?? 'Unable to check server connection.'),
variant: response?.ok ? 'success' : 'danger',
});
checkingServers.finish(server.id);
}
@@ -665,17 +755,12 @@ export default function Clusters({
bootstrappingServers.start(server.id);
setBootstrapServerError(null);
openBootstrapLogs(server);
const response = await fetch(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/bootstrap`, {
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/bootstrap`, {
method: 'POST',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
});
const payload = (await response.json()) as BootstrapServerResponse;
}).catch(() => null);
const payload = ((await response?.json().catch(() => null)) ?? {}) as BootstrapServerResponse;
if (payload.cluster) {
setClusterList((currentClusters) =>
@@ -683,13 +768,17 @@ export default function Clusters({
);
}
if (!response.ok) {
if (!response?.ok) {
setBootstrapServerError(payload.message ?? 'Unable to queue bootstrap for this server.');
}
bootstrappingServers.finish(server.id);
}
function openBootstrapLogs(server: V5Server): void {
setBootstrapLogsServerId(server.id);
setIsBootstrapLogsDialogOpen(true);
}
async function loadCooldLogs(server: V5Server): Promise<void> {
if (!selectedCluster) {
@@ -702,18 +791,15 @@ export default function Clusters({
setCooldLogsError(null);
setCooldLogsOutput('');
setCooldLogsFetchedAt(null);
setCooldLogsSource(null);
const response = await fetch(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/coold-logs?tail=200`, {
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/coold-logs?tail=200`, {
method: 'GET',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
},
});
}).catch(() => null);
const payload = (await response.json().catch(() => null)) as CooldLogsResponse & { message?: string } | null;
const payload = (await response?.json().catch(() => null)) as CooldLogsResponse & { message?: string } | null;
if (!response.ok) {
if (!response?.ok) {
setCooldLogsError(payload?.message ?? 'Unable to load coold logs.');
setIsLoadingCooldLogs(false);
@@ -722,6 +808,7 @@ export default function Clusters({
setCooldLogsOutput(payload?.output ?? '');
setCooldLogsFetchedAt(payload?.fetchedAt ?? null);
setCooldLogsSource(payload?.source ?? null);
setIsLoadingCooldLogs(false);
}
@@ -736,18 +823,15 @@ export default function Clusters({
setCorrosionTablesError(null);
setCorrosionTablesOutput('');
setCorrosionTablesFetchedAt(null);
setCorrosionTablesSource(null);
const response = await fetch(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/corrosion-tables?limit=200`, {
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/corrosion-tables?limit=200`, {
method: 'GET',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
},
});
}).catch(() => null);
const payload = (await response.json().catch(() => null)) as CorrosionTablesResponse & { message?: string } | null;
const payload = (await response?.json().catch(() => null)) as CorrosionTablesResponse & { message?: string } | null;
if (!response.ok) {
if (!response?.ok) {
setCorrosionTablesError(payload?.message ?? 'Unable to load Corrosion tables.');
setIsLoadingCorrosionTables(false);
@@ -756,6 +840,7 @@ export default function Clusters({
setCorrosionTablesOutput(payload?.output ?? '');
setCorrosionTablesFetchedAt(payload?.fetchedAt ?? null);
setCorrosionTablesSource(payload?.source ?? null);
setIsLoadingCorrosionTables(false);
}
@@ -772,17 +857,13 @@ export default function Clusters({
setFirewallRules([]);
setFirewallRulesFetchedAt(null);
const response = await fetch(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/firewall-rules`, {
const response = await apiRequest(`/v5/clusters/${selectedCluster.id}/servers/${server.id}/firewall-rules`, {
method: 'GET',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
},
});
}).catch(() => null);
const payload = (await response.json().catch(() => null)) as FirewallRulesResponse & { message?: string } | null;
const payload = (await response?.json().catch(() => null)) as FirewallRulesResponse & { message?: string } | null;
if (!response.ok) {
if (!response?.ok) {
setFirewallRulesError(payload?.message ?? 'Unable to load firewall rules.');
setIsLoadingFirewallRules(false);
@@ -820,25 +901,32 @@ export default function Clusters({
async function deleteServer(cluster: V5Cluster, server: V5Server): Promise<void> {
deletingServers.start(server.id);
setDeleteClusterError(null);
const response = await fetch(`/v5/clusters/${cluster.id}/servers/${server.id}`, {
const response = await apiRequest(`/v5/clusters/${cluster.id}/servers/${server.id}`, {
method: 'DELETE',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
});
}).catch(() => null);
if (response.ok) {
const payload = (await response.json()) as DeleteServerResponse;
if (!response?.ok) {
const payload = (await response?.json().catch(() => null)) as { message?: string } | null;
setClusterList((currentClusters) =>
currentClusters.map((cluster) => (cluster.id === payload.cluster.id ? payload.cluster : cluster)),
setDeleteClusterError(
payload?.message ??
(response?.status === 422
? 'Delete or move applications from this server before deleting it.'
: 'Unable to delete this server. Please try again.'),
);
deletingServers.finish(server.id);
return;
}
const payload = (await response.json()) as DeleteServerResponse;
setClusterList((currentClusters) =>
currentClusters.map((cluster) => (cluster.id === payload.cluster.id ? payload.cluster : cluster)),
);
deletingServers.finish(server.id);
setIsDeleteDialogOpen(false);
setClusterPendingDelete(null);
@@ -866,16 +954,11 @@ export default function Clusters({
setIsDeletingCluster(true);
setDeleteClusterError(null);
const response = await fetch(`/v5/clusters/${cluster.id}`, {
const response = await apiRequest(`/v5/clusters/${cluster.id}`, {
method: 'DELETE',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
});
}).catch(() => null);
if (response.status === 422) {
if (response?.status === 422) {
const payload = (await response.json()) as { message?: string };
setDeleteClusterError(payload.message ?? 'Only empty clusters can be deleted.');
setIsDeletingCluster(false);
@@ -883,14 +966,14 @@ export default function Clusters({
return;
}
if (!response.ok) {
if (!response?.ok) {
setDeleteClusterError('Unable to delete this cluster. Please try again.');
setIsDeletingCluster(false);
return;
}
const nextClusters = clusterList.filter((cluster) => cluster.id !== selectedCluster.id);
const nextClusters = clusterList.filter((remainingCluster) => remainingCluster.id !== cluster.id);
setClusterList(nextClusters);
setSelectedClusterId(nextClusters[0]?.id ?? '');
@@ -969,11 +1052,8 @@ export default function Clusters({
const isBootstrappingServer = bootstrappingServers.has(server.id) || isBootstrapInProgress;
const isDeletingServer = deletingServers.has(server.id);
const isServerInitialized = server.lastBootstrappedAt !== null;
const latestSshCheck = sshChecks[server.id] ?? null;
const hasBootstrapLogs = server.lastBootstrapOutput !== null && server.lastBootstrapOutput.trim() !== '';
const canShowBootstrapLogs = hasBootstrapLogs || isBootstrapInProgress;
const isBootstrapLogVisible =
isBootstrapInProgress || (canShowBootstrapLogs && (visibleBootstrapLogs[server.id] ?? false));
return (
<article key={server.id} className="rounded-lg border border-border bg-background p-4">
@@ -1035,20 +1115,8 @@ export default function Clusters({
{isCheckingServer ? 'Checking...' : 'Check connection'}
</DropdownMenuItem>
{canShowBootstrapLogs ? (
<DropdownMenuItem
disabled={isBootstrapInProgress}
onClick={() =>
setVisibleBootstrapLogs((currentLogs) => ({
...currentLogs,
[server.id]: !isBootstrapLogVisible,
}))
}
>
{isBootstrapInProgress
? 'Install logs shown'
: isBootstrapLogVisible
? 'Hide install logs'
: 'Show install logs'}
<DropdownMenuItem onClick={() => openBootstrapLogs(server)}>
View install logs
</DropdownMenuItem>
) : null}
<DropdownMenuItem onClick={() => void loadCooldLogs(server)}>
@@ -1113,37 +1181,6 @@ export default function Clusters({
</dd>
</div>
</dl>
{latestSshCheck ? (
<div className="mt-4 rounded-md border border-border bg-muted/30 p-3 text-xs">
<div className="flex flex-col gap-1 sm:flex-row sm:items-center sm:justify-between">
<span className="font-medium text-foreground">Latest SSH check: {latestSshCheck.status}</span>
<span className="text-muted-foreground">{formatDate(latestSshCheck.checkedAt)}</span>
</div>
<pre className="mt-2 max-h-40 overflow-auto whitespace-pre-wrap rounded bg-background p-2 text-muted-foreground">
{latestSshCheck.output}
</pre>
</div>
) : null}
{isBootstrapLogVisible ? (
<div className="mt-4 rounded-md border border-border bg-muted/30 p-3 text-xs">
<div className="flex flex-col gap-1 sm:flex-row sm:items-center sm:justify-between">
<span className="font-medium text-foreground">
Install logs
{server.lastBootstrapStatus ? `: ${server.lastBootstrapStatus}` : ''}
</span>
<span className="text-muted-foreground">{formatDate(server.lastBootstrapRanAt)}</span>
</div>
{server.lastBootstrapOutput ? (
<pre className="mt-2 max-h-48 overflow-auto whitespace-pre-wrap rounded bg-background p-2 text-muted-foreground">
{server.lastBootstrapOutput}
</pre>
) : (
<p className="mt-2 text-muted-foreground">No install logs captured yet.</p>
)}
</div>
) : null}
</article>
);
}
@@ -1161,6 +1198,15 @@ export default function Clusters({
selectedEnvironmentUuid={selectedEnvironmentUuid}
/>
{serverConnectionNotice ? (
<CanvasNotice
message={serverConnectionNotice.message}
description={serverConnectionNotice.description}
variant={serverConnectionNotice.variant}
onDismiss={() => setServerConnectionNotice(null)}
/>
) : null}
<main className="flex min-h-dvh overflow-visible px-4 pt-16 lg:h-full lg:min-h-0 lg:overflow-hidden lg:px-6">
<section className="flex w-full flex-col gap-4 py-4 lg:min-h-0 lg:py-6">
<div className="rounded-lg border border-border bg-card p-4">
@@ -1487,6 +1533,7 @@ export default function Clusters({
setIsDeleteDialogOpen(open);
if (!open) {
setDeleteClusterError(null);
setClusterPendingDelete(null);
setServerPendingDelete(null);
}
@@ -1501,6 +1548,14 @@ export default function Clusters({
: `Delete cluster ${clusterPendingDelete?.name ?? ''}? This cannot be undone.`}
</DialogDescription>
</DialogHeader>
{deleteClusterError ? (
<p
role="alert"
className="rounded-md border border-destructive/30 bg-destructive/10 p-3 text-sm text-destructive"
>
{deleteClusterError}
</p>
) : null}
<DialogFooter>
<Button
type="button"
@@ -2027,6 +2082,82 @@ export default function Clusters({
</DialogContent>
</Dialog>
<Dialog
open={isBootstrapLogsDialogOpen}
onOpenChange={(open) => {
setIsBootstrapLogsDialogOpen(open);
if (!open) {
setBootstrapLogsServerId(null);
}
}}
>
<DialogContent className="max-w-6xl">
<DialogHeader>
<DialogTitle>Install logs</DialogTitle>
<DialogDescription>
Bootstrap output for {bootstrapLogsServer?.name ?? 'this server'}.
</DialogDescription>
</DialogHeader>
<div className="mt-5 flex flex-col gap-3">
<div className="flex flex-col gap-1 text-xs text-muted-foreground sm:flex-row sm:items-center sm:justify-between">
<p>
Status: {bootstrapLogsServer?.lastBootstrapStatus ?? 'unknown'}
{['queued', 'running'].includes(bootstrapLogsServer?.lastBootstrapStatus ?? '')
? ' · Auto-refreshing while bootstrap runs'
: ''}
</p>
<p>{formatDate(bootstrapLogsServer?.lastBootstrapRanAt ?? null)}</p>
</div>
{parsedBootstrapLogs.summary.length > 0 ? (
<div className="rounded-lg border border-border bg-muted/20 p-4">
<p className="mb-3 text-sm font-medium text-foreground">Action results</p>
<div className="grid gap-2 sm:grid-cols-2">
{parsedBootstrapLogs.summary.map((item, index) => (
<div
key={`${item.label}-${index}`}
className="rounded-md border border-border bg-background p-3"
>
<div className="flex items-start justify-between gap-3">
<p className="text-sm font-medium text-foreground">{item.label}</p>
<span
className={`rounded-full px-2 py-0.5 text-xs ${
item.tone === 'success'
? 'bg-emerald-500/10 text-emerald-400'
: 'bg-muted text-muted-foreground'
}`}
>
{item.tone === 'success' ? 'OK' : 'Info'}
</span>
</div>
<p className="mt-1 text-xs text-muted-foreground">{item.value}</p>
</div>
))}
</div>
</div>
) : null}
{parsedBootstrapLogs.visibleOutput ? (
<pre className="max-h-[70dvh] max-w-full overflow-auto whitespace-pre-wrap rounded-lg border border-border bg-black p-4 font-mono text-xs leading-relaxed text-white">
{parsedBootstrapLogs.visibleOutput}
</pre>
) : null}
{parsedBootstrapLogs.summary.length > 0 ? (
<details className="rounded-lg border border-border bg-muted/20 p-4">
<summary className="cursor-pointer text-sm font-medium text-foreground">
Raw JSON output
</summary>
<pre className="mt-3 max-h-80 max-w-full overflow-auto whitespace-pre-wrap rounded-md bg-black p-3 font-mono text-xs leading-relaxed text-white">
{parsedBootstrapLogs.rawOutput}
</pre>
</details>
) : null}
</div>
</DialogContent>
</Dialog>
<Dialog
open={isCooldLogsDialogOpen}
@@ -2037,6 +2168,7 @@ export default function Clusters({
setCooldLogsServer(null);
setCooldLogsOutput('');
setCooldLogsFetchedAt(null);
setCooldLogsSource(null);
setCooldLogsError(null);
}
}}
@@ -2051,9 +2183,16 @@ export default function Clusters({
<div className="mt-5 flex flex-col gap-3">
<div className="flex items-center justify-between gap-3">
<p className="text-xs text-muted-foreground">
{cooldLogsFetchedAt ? `Fetched ${formatDate(cooldLogsFetchedAt)}` : 'Last 200 lines'}
</p>
<div className="flex flex-wrap items-center gap-2">
<p className="text-xs text-muted-foreground">
{cooldLogsFetchedAt ? `Fetched ${formatDate(cooldLogsFetchedAt)}` : 'Last 200 lines'}
</p>
{cooldLogsSource ? (
<span className="rounded-full border border-border bg-muted/40 px-2 py-0.5 text-xs text-muted-foreground">
Source: {diagnosticsSourceLabel(cooldLogsSource)}
</span>
) : null}
</div>
{cooldLogsServer ? (
<Button
type="button"
@@ -2089,6 +2228,7 @@ export default function Clusters({
setCorrosionTablesServer(null);
setCorrosionTablesOutput('');
setCorrosionTablesFetchedAt(null);
setCorrosionTablesSource(null);
setCorrosionTablesError(null);
}
}}
@@ -2103,11 +2243,18 @@ export default function Clusters({
<div className="mt-5 flex flex-col gap-3">
<div className="flex items-center justify-between gap-3">
<p className="text-xs text-muted-foreground">
{corrosionTablesFetchedAt
? `Fetched ${formatDate(corrosionTablesFetchedAt)}`
: 'First 200 rows per table'}
</p>
<div className="flex flex-wrap items-center gap-2">
<p className="text-xs text-muted-foreground">
{corrosionTablesFetchedAt
? `Fetched ${formatDate(corrosionTablesFetchedAt)}`
: 'First 200 rows per table'}
</p>
{corrosionTablesSource ? (
<span className="rounded-full border border-border bg-muted/40 px-2 py-0.5 text-xs text-muted-foreground">
Source: {diagnosticsSourceLabel(corrosionTablesSource)}
</span>
) : null}
</div>
{corrosionTablesServer ? (
<Button
type="button"
File diff suppressed because it is too large Load Diff
+14 -66
View File
@@ -1,10 +1,11 @@
import { Head } from '@inertiajs/react';
import { useEffect, useState } from 'react';
import { useCallback, useEffect, useState } from 'react';
import { AppNavbar } from '@/components/app-navbar';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { csrfToken } from '@/lib/csrf';
import { useTeamChannel } from '@/lib/use-team-channel';
import type { V5DashboardProps } from '@/types';
type RealtimeTestProps = V5DashboardProps & {
@@ -19,24 +20,6 @@ type RealtimeTestEvent = {
sentAt: string;
};
type EchoChannel = {
listen: (event: string, callback: (payload: unknown) => void) => EchoChannel;
subscribed?: (callback: () => void) => EchoChannel;
error?: (callback: (error: unknown) => void) => EchoChannel;
};
type EchoClient = {
private: (channel: string) => EchoChannel;
leave?: (channel: string) => void;
leaveChannel?: (channel: string) => void;
};
declare global {
interface Window {
Echo?: EchoClient;
}
}
function formatLogPayload(payload: unknown): string {
if (typeof payload === 'string') {
return payload;
@@ -50,65 +33,30 @@ export default function RealtimeTest({ currentTeam, flux, projects = [], selecte
const [isBroadcasting, setIsBroadcasting] = useState(false);
const [logs, setLogs] = useState<string[]>([]);
function addLog(label: string, payload?: unknown): void {
const addLog = useCallback((label: string, payload?: unknown): void => {
const timestamp = new Date().toLocaleTimeString();
setLogs((currentLogs) => [
`[${timestamp}] ${label}${payload === undefined ? '' : `\n${formatLogPayload(payload)}`}`,
...currentLogs,
]);
}
}, []);
useEffect(() => {
if (!currentTeam) {
addLog('No current team was provided to the page.');
return;
}
}, [currentTeam, addLog]);
let isCancelled = false;
let attempts = 0;
const channelName = `team.${currentTeam.id}`;
useTeamChannel(
currentTeam?.id ?? null,
'.v5.realtime.test',
(payload) => {
const event = payload as RealtimeTestEvent;
const interval = window.setInterval(() => {
attempts += 1;
if (!window.Echo) {
if (attempts === 1) {
addLog('Waiting for window.Echo...');
}
if (attempts >= 20) {
window.clearInterval(interval);
addLog('window.Echo was not available after 10 seconds.');
}
return;
}
window.clearInterval(interval);
if (isCancelled) {
return;
}
addLog(`Subscribing to private-${channelName}`);
const channel = window.Echo.private(channelName);
channel.subscribed?.(() => addLog(`Subscribed to private-${channelName}`));
channel.error?.((error: unknown) => addLog(`Subscription error on private-${channelName}`, error));
channel.listen('.v5.realtime.test', (payload) => {
const event = payload as RealtimeTestEvent;
addLog('Received .v5.realtime.test', event);
});
}, 500);
return () => {
isCancelled = true;
window.clearInterval(interval);
window.Echo?.leave?.(channelName) ?? window.Echo?.leaveChannel?.(`private-${channelName}`);
};
}, [currentTeam]);
addLog('Received .v5.realtime.test', event);
},
{ onDebug: addLog, onError: addLog },
);
async function broadcastTestEvent(): Promise<void> {
setIsBroadcasting(true);
+57 -17
View File
@@ -7,25 +7,44 @@ import { csrfToken } from '@/lib/csrf';
import { cn } from '@/lib/utils';
import type { SelectItemOption, V5DashboardProps, V5Project } from '@/types';
async function persistSelection(projectUuid: string, environmentUuid: string): Promise<void> {
await fetch('/v5/selection', {
method: 'POST',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
body: JSON.stringify({
project_uuid: projectUuid,
environment_uuid: environmentUuid,
}),
});
async function persistSelection(projectUuid: string, environmentUuid: string): Promise<boolean> {
try {
const response = await fetch('/v5/selection', {
method: 'POST',
credentials: 'same-origin',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-CSRF-TOKEN': csrfToken(),
},
body: JSON.stringify({
project_uuid: projectUuid,
environment_uuid: environmentUuid,
}),
});
if (!response.ok) {
console.error(`Could not persist the project/environment selection (HTTP ${response.status}).`);
}
return response.ok;
} catch (error) {
console.error('Could not persist the project/environment selection.', error);
return false;
}
}
function refreshCurrentPageSelection(): void {
router.reload({
only: ['applications', 'selectedProjectUuid', 'selectedEnvironmentUuid'],
only: [
'applications',
'caddyIngresses',
'resourceConnections',
'nginxServers',
'selectedProjectUuid',
'selectedEnvironmentUuid',
],
});
}
@@ -65,10 +84,21 @@ export function AppNavbar({
const nextProject = projects.find((project) => project.uuid === nextProjectUuid);
const nextEnvironmentUuid = nextProject?.environments?.[0]?.uuid ?? '';
const previousProjectUuid = projectUuid;
const previousEnvironmentUuid = environmentUuid;
setProjectUuid(nextProjectUuid);
setEnvironmentUuid(nextEnvironmentUuid);
void persistSelection(nextProjectUuid, nextEnvironmentUuid).then(refreshCurrentPageSelection);
void persistSelection(nextProjectUuid, nextEnvironmentUuid).then((persisted) => {
if (persisted) {
refreshCurrentPageSelection();
return;
}
setProjectUuid(previousProjectUuid);
setEnvironmentUuid(previousEnvironmentUuid);
});
}
function selectEnvironment(nextEnvironmentUuid: string | null): void {
@@ -76,8 +106,18 @@ export function AppNavbar({
return;
}
const previousEnvironmentUuid = environmentUuid;
setEnvironmentUuid(nextEnvironmentUuid);
void persistSelection(projectUuid, nextEnvironmentUuid).then(refreshCurrentPageSelection);
void persistSelection(projectUuid, nextEnvironmentUuid).then((persisted) => {
if (persisted) {
refreshCurrentPageSelection();
return;
}
setEnvironmentUuid(previousEnvironmentUuid);
});
}
const projectItems: SelectItemOption[] = projects.map((project) => ({
@@ -0,0 +1,129 @@
import { memo, type MouseEvent, type PointerEvent } from 'react';
import { ApplicationIngressButton } from '@/components/canvas/application-ingress-button';
import { statusBadgeClass } from '@/components/canvas/status-badge';
import { CONNECTOR_SIDES, type ConnectorSide } from '@/lib/canvas-geometry';
import { cn } from '@/lib/utils';
import type { V5Application } from '@/types';
type ApplicationCardProps = {
application: V5Application;
isSelected: boolean;
isDeleting: boolean;
isIngressSaving: boolean;
onDragStart: (event: PointerEvent<HTMLDivElement>, application: V5Application) => void;
onOpenInspector: (event: MouseEvent<HTMLElement>, application: V5Application) => void;
onDelete: (application: V5Application) => void;
onToggleIngress: (application: V5Application) => void;
onConnectorPointerDown: (event: PointerEvent<HTMLButtonElement>, applicationId: string, side: ConnectorSide) => void;
};
export const ApplicationCard = memo(function ApplicationCard({
application,
isSelected,
isDeleting,
isIngressSaving,
onDragStart,
onOpenInspector,
onDelete,
onToggleIngress,
onConnectorPointerDown,
}: ApplicationCardProps) {
return (
<div
data-application-card="application-card"
data-application-id={application.id}
className="group/application absolute h-40 w-80 select-none overflow-visible rounded-xl border border-border bg-card p-4 shadow-xl transition-shadow hover:shadow-2xl"
style={{
transform: `translate3d(${application.canvasX}px, ${application.canvasY}px, 0)`,
}}
onPointerDown={(event) => onDragStart(event, application)}
onDoubleClick={(event) => onOpenInspector(event, application)}
>
{CONNECTOR_SIDES.map((side) => (
<button
key={side}
type="button"
aria-label={`${application.name} ${side} connector`}
data-application-connector="application-connector"
data-application-id={application.id}
data-connector-side={side}
onPointerDown={(event) => onConnectorPointerDown(event, application.id, side)}
className={cn(
'application-connector group/connector absolute z-10 flex size-8 items-center justify-center rounded-full opacity-0 transition group-hover/application:opacity-100 md:size-3',
isSelected && 'opacity-100',
side === 'top' && 'left-1/2 top-0 -translate-x-1/2 -translate-y-1/2',
side === 'right' && 'right-0 top-1/2 -translate-y-1/2 translate-x-1/2',
side === 'bottom' && 'bottom-0 left-1/2 -translate-x-1/2 translate-y-1/2',
side === 'left' && 'left-0 top-1/2 -translate-x-1/2 -translate-y-1/2',
)}
>
<span className="size-3 rounded-full border-2 border-card bg-warning shadow ring-2 ring-background transition group-hover/connector:scale-125 group-hover/connector:bg-warning/90" />
</button>
))}
<div className="flex items-start justify-between gap-3">
<div className="min-w-0">
<div className="truncate text-sm font-semibold text-foreground">{application.name}</div>
<div className="mt-1 truncate text-xs text-muted-foreground">{application.image}</div>
</div>
<div className="flex shrink-0 items-center gap-2">
<span
className={cn(
'rounded-full px-2 py-1 text-[0.625rem] font-semibold uppercase tracking-wide',
statusBadgeClass(application.effectiveStatus),
)}
title={application.effectiveStatusMessage ?? undefined}
>
{application.effectiveStatus}
</span>
<button
type="button"
onPointerDown={(event) => event.stopPropagation()}
onClick={(event) => onOpenInspector(event, application)}
className="rounded-md border border-border px-2 py-1 text-[0.625rem] font-semibold uppercase tracking-wide text-foreground transition hover:bg-muted"
>
Configure
</button>
<button
type="button"
onPointerDown={(event) => event.stopPropagation()}
onClick={(event) => {
event.stopPropagation();
onDelete(application);
}}
disabled={isDeleting}
className="rounded-md border border-destructive/40 px-2 py-1 text-[0.625rem] font-semibold uppercase tracking-wide text-destructive transition hover:bg-destructive/10 disabled:cursor-not-allowed disabled:opacity-60"
>
{isDeleting ? 'Deleting…' : 'Delete'}
</button>
</div>
</div>
<dl className="mt-4 grid gap-2 text-xs">
<div className="grid grid-cols-[auto_minmax(0,1fr)] gap-3">
<dt className="shrink-0 text-muted-foreground">Server</dt>
<dd className="truncate text-right font-medium text-foreground">
{application.serverName ?? 'Unknown'}
{!application.isServerReachable && <span className="ml-2 text-destructive">(unreachable)</span>}
</dd>
</div>
<div className="grid grid-cols-[auto_minmax(0,1fr)] gap-3">
<dt className="shrink-0 text-muted-foreground">Container</dt>
<dd className="truncate text-right font-mono text-[0.6875rem] text-foreground">{application.containerName}</dd>
</div>
<div className="grid grid-cols-[auto_minmax(0,1fr)] items-center gap-3">
<dt className="shrink-0 text-muted-foreground">Ingress</dt>
<dd className="flex items-center justify-end gap-2 text-right">
<span className="truncate text-muted-foreground">
{application.ingressEnabled
? `${application.domains.length} domain${application.domains.length === 1 ? '' : 's'}${application.internalPort ?? 'no port'}`
: 'Private'}
</span>
<ApplicationIngressButton application={application} isSaving={isIngressSaving} onToggle={onToggleIngress} />
</dd>
</div>
</dl>
</div>
);
});
@@ -0,0 +1,39 @@
import { Tooltip, TooltipContent, TooltipTrigger } from '@/components/ui/tooltip';
import type { V5Application } from '@/types';
type ApplicationIngressButtonProps = {
application: V5Application;
isSaving: boolean;
onToggle: (application: V5Application) => void;
};
export function ApplicationIngressButton({ application, isSaving, onToggle }: ApplicationIngressButtonProps) {
const isDisabled = !application.ingressEnabled && !application.serverIngressEnabled;
const button = (
<button
type="button"
onPointerDown={(event) => event.stopPropagation()}
disabled={isDisabled || isSaving}
onClick={(event) => {
event.stopPropagation();
onToggle(application);
}}
className="rounded-sm border border-border px-2 py-1 text-[0.625rem] font-semibold uppercase tracking-wide text-foreground transition hover:bg-muted disabled:cursor-not-allowed disabled:opacity-50"
>
{isSaving ? 'Saving...' : application.ingressEnabled ? 'Disable' : 'Enable'}
</button>
);
if (!isDisabled) {
return button;
}
return (
<Tooltip>
<TooltipTrigger render={<span className="inline-flex" />}>{button}</TooltipTrigger>
<TooltipContent side="top">
<p>You need to enable ingress in server settings first.</p>
</TooltipContent>
</Tooltip>
);
}
@@ -0,0 +1,276 @@
import { ApplicationIngressButton } from '@/components/canvas/application-ingress-button';
import { Button } from '@/components/ui/button';
import { Field, FieldLabel } from '@/components/ui/field';
import { Input } from '@/components/ui/input';
import { Sheet, SheetContent, SheetDescription, SheetHeader, SheetTitle } from '@/components/ui/sheet';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { Textarea } from '@/components/ui/textarea';
import { apiRequest } from '@/lib/api';
import type { V5Application } from '@/types';
import { useCallback, useEffect, useState } from 'react';
type ApplicationInspectorSheetProps = {
application: V5Application | null;
isIngressSaving: boolean;
onToggleIngress: (application: V5Application) => void;
onClose: () => void;
};
type ApplicationLogsResponse = {
status: string;
statusMessage: string | null;
containerId: string | null;
logs: string | null;
logsError: string | null;
};
function ApplicationLogsTab({ application }: { application: V5Application }) {
const [isLoading, setIsLoading] = useState(false);
const [logs, setLogs] = useState<string | null>(null);
const [logsError, setLogsError] = useState<string | null>(null);
const [hasLoaded, setHasLoaded] = useState(false);
const fetchLogs = useCallback(async () => {
setIsLoading(true);
setLogsError(null);
try {
const response = await apiRequest(`/v5/applications/${application.id}/logs`, { method: 'GET' });
if (!response.ok) {
throw new Error(`Request failed with status ${response.status}`);
}
const data = (await response.json()) as ApplicationLogsResponse;
setLogs(data.logs);
setLogsError(data.logsError);
} catch {
setLogs(null);
setLogsError('Could not reach the server to fetch container logs. Try again.');
} finally {
setIsLoading(false);
setHasLoaded(true);
}
}, [application.id]);
// Lazily fetch when the Logs tab first mounts for this app, and reset/refetch
// whenever a different application is inspected.
useEffect(() => {
setLogs(null);
setLogsError(null);
setHasLoaded(false);
void fetchLogs();
}, [fetchLogs]);
const statusMessage = application.effectiveStatusMessage ?? application.statusMessage;
return (
<div className="flex flex-col gap-4">
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2">
<Field>
<FieldLabel>Deploy status</FieldLabel>
<Input value={application.effectiveStatus} readOnly />
</Field>
<Field>
<FieldLabel>Container</FieldLabel>
<Input value={application.runtimeContainerId ?? 'Not created yet'} readOnly />
</Field>
</div>
<Field>
<FieldLabel>Status message</FieldLabel>
<Textarea value={statusMessage ?? 'No status message yet.'} readOnly className="min-h-16" />
</Field>
<Field>
<div className="flex items-center justify-between">
<FieldLabel>Container logs</FieldLabel>
<Button variant="outline" size="sm" onClick={() => void fetchLogs()} disabled={isLoading}>
{isLoading ? 'Refreshing…' : 'Refresh'}
</Button>
</div>
<Textarea
value={
isLoading && !hasLoaded
? 'Loading container logs…'
: logsError
? logsError
: logs
? logs
: hasLoaded
? 'No container logs yet — the container has not been created. See the status message above for the deploy result.'
: ''
}
readOnly
className="min-h-80 font-mono text-xs"
/>
</Field>
</div>
);
}
export function ApplicationInspectorSheet({ application, isIngressSaving, onToggleIngress, onClose }: ApplicationInspectorSheetProps) {
return (
<Sheet
open={application !== null}
onOpenChange={(open) => {
if (!open) {
onClose();
}
}}
>
<SheetContent side="right" className="w-full overflow-hidden bg-background sm:rounded-l-xl sm:border data-[side=right]:sm:!inset-y-4 data-[side=right]:sm:!h-auto data-[side=right]:sm:!w-[45vw] data-[side=right]:sm:!max-w-[45vw]" showCloseButton blurOverlay={false}>
{application && (
<>
<SheetHeader className="p-6 pb-4">
<SheetTitle>App configuration</SheetTitle>
<SheetDescription>
Double-click an application card to open configuration. Review runtime, networking, and advanced settings for{' '}
{application.name}.
</SheetDescription>
</SheetHeader>
<div className="flex flex-1 flex-col gap-6 px-6 pb-6">
<Tabs defaultValue="overview" className="gap-4">
<TabsList className="w-full justify-start" variant="line">
<TabsTrigger value="overview">Overview</TabsTrigger>
<TabsTrigger value="networking">Networking</TabsTrigger>
<TabsTrigger value="logs">Logs</TabsTrigger>
<TabsTrigger value="advanced">Advanced</TabsTrigger>
</TabsList>
<TabsContent value="overview" className="flex flex-col gap-4">
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2">
<Field>
<FieldLabel>Name</FieldLabel>
<Input value={application.name} readOnly />
</Field>
<Field>
<FieldLabel>Status</FieldLabel>
<Input value={application.effectiveStatus} readOnly />
</Field>
{application.effectiveStatus !== application.status && (
<Field>
<FieldLabel>Last known container status</FieldLabel>
<Input value={application.status} readOnly />
</Field>
)}
<Field>
<FieldLabel>Image</FieldLabel>
<Input value={application.image} readOnly />
</Field>
<Field>
<FieldLabel>Server</FieldLabel>
<Input
value={
application.isServerReachable
? (application.serverName ?? 'Unknown')
: `${application.serverName ?? 'Unknown'} (unreachable)`
}
readOnly
/>
</Field>
<Field>
<FieldLabel>Container</FieldLabel>
<Input value={application.containerName} readOnly />
</Field>
<Field>
<FieldLabel>Runtime container ID</FieldLabel>
<Input value={application.runtimeContainerId ?? 'Not available'} readOnly />
</Field>
</div>
<Field>
<FieldLabel>Status message</FieldLabel>
<Textarea
value={application.effectiveStatusMessage ?? 'No status message yet.'}
readOnly
className="min-h-20"
/>
</Field>
</TabsContent>
<TabsContent value="networking" className="flex flex-col gap-4">
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2">
<Field>
<FieldLabel>Mesh namespace</FieldLabel>
<Input value={application.meshNamespace} readOnly />
</Field>
<Field>
<FieldLabel>Mesh FQDN</FieldLabel>
<Input value={application.meshFqdn} readOnly />
</Field>
<Field>
<FieldLabel>Internal port</FieldLabel>
<Input value={application.internalPort?.toString() ?? 'Not configured'} readOnly />
</Field>
<Field>
<FieldLabel>Public ingress</FieldLabel>
<Input value={application.ingressEnabled ? 'Enabled' : 'Private'} readOnly />
</Field>
</div>
<Field>
<FieldLabel>Domains</FieldLabel>
<Textarea
value={
application.domains.length > 0
? application.domains.join('\n')
: 'No public domains configured.'
}
readOnly
/>
</Field>
<div className="flex flex-wrap items-center gap-2">
<ApplicationIngressButton application={application} isSaving={isIngressSaving} onToggle={onToggleIngress} />
<span className="text-xs text-muted-foreground">
Use this action to publish or private-route the app through the server ingress.
</span>
</div>
</TabsContent>
<TabsContent value="logs">
<ApplicationLogsTab key={application.id} application={application} />
</TabsContent>
<TabsContent value="advanced" className="flex flex-col gap-4">
<div className="grid grid-cols-1 gap-4 sm:grid-cols-2">
<Field>
<FieldLabel>Application ID</FieldLabel>
<Input value={application.id} readOnly />
</Field>
<Field>
<FieldLabel>Canvas position</FieldLabel>
<Input value={`${application.canvasX}, ${application.canvasY}`} readOnly />
</Field>
</div>
<Field>
<FieldLabel>Raw app config</FieldLabel>
<Textarea
value={JSON.stringify(application, null, 2)}
readOnly
className="min-h-80 font-mono text-xs"
/>
</Field>
</TabsContent>
</Tabs>
</div>
</>
)}
</SheetContent>
</Sheet>
);
}
@@ -0,0 +1,49 @@
import { memo, type PointerEvent } from 'react';
import { statusBadgeClass } from '@/components/canvas/status-badge';
import { cn } from '@/lib/utils';
import type { V5CaddyIngress } from '@/types';
type CaddyIngressCardProps = {
ingress: V5CaddyIngress;
onDragStart: (event: PointerEvent<HTMLDivElement>, ingress: V5CaddyIngress) => void;
};
export const CaddyIngressCard = memo(function CaddyIngressCard({ ingress, onDragStart }: CaddyIngressCardProps) {
return (
<div
className="absolute w-80 select-none overflow-hidden rounded-xl border border-warning/40 bg-card p-4 shadow-xl transition-shadow hover:shadow-2xl"
style={{
transform: `translate3d(${ingress.canvasX}px, ${ingress.canvasY}px, 0)`,
}}
onPointerDown={(event) => onDragStart(event, ingress)}
>
<div className="flex items-start justify-between gap-3">
<div className="min-w-0">
<div className="truncate text-sm font-semibold text-foreground">Caddy ingress</div>
<div className="mt-1 truncate text-xs text-muted-foreground">{ingress.name}</div>
</div>
<span
className={cn(
'shrink-0 rounded-full px-2 py-1 text-[0.625rem] font-semibold uppercase tracking-wide',
statusBadgeClass(ingress.status),
)}
title={ingress.statusMessage ?? undefined}
>
{ingress.status}
</span>
</div>
<dl className="mt-4 grid gap-2 text-xs">
<div className="grid grid-cols-[auto_minmax(0,1fr)] gap-3">
<dt className="shrink-0 text-muted-foreground">Server</dt>
<dd className="truncate text-right font-medium text-foreground">{ingress.name}</dd>
</div>
<div className="grid grid-cols-[auto_minmax(0,1fr)] gap-3">
<dt className="shrink-0 text-muted-foreground">Host</dt>
<dd className="truncate text-right font-mono text-[0.6875rem] text-foreground">{ingress.host}</dd>
</div>
</dl>
</div>
);
});
@@ -0,0 +1,38 @@
type CanvasNoticeProps = {
message: string;
description?: string;
onDismiss: () => void;
variant?: 'danger' | 'success' | 'info';
};
const noticeClasses = {
danger: 'border-destructive/40 text-destructive',
success: 'border-emerald-500/40 text-emerald-400',
info: 'border-blue-500/40 text-blue-400',
};
export function CanvasNotice({ message, description, onDismiss, variant = 'danger' }: CanvasNoticeProps) {
return (
<div
className={`fixed right-4 top-20 z-50 flex max-w-sm items-start gap-3 rounded-lg border bg-card p-3 text-sm shadow-lg ${noticeClasses[variant]}`}
>
<div className="min-w-0">
<p className="font-medium">{message}</p>
{description ? (
<p className="mt-1 max-h-32 overflow-auto whitespace-pre-wrap break-words text-xs text-muted-foreground">
{description}
</p>
) : null}
</div>
<button
type="button"
aria-label="Dismiss notice"
onClick={onDismiss}
className="-m-1 rounded p-1 opacity-80 transition hover:bg-background/20 hover:opacity-100"
>
<span aria-hidden="true">×</span>
<span className="sr-only">Dismiss notice</span>
</button>
</div>
);
}
@@ -0,0 +1,139 @@
import { memo } from 'react';
import { MAX_CANVAS_ZOOM, MIN_CANVAS_ZOOM } from '@/lib/use-canvas-viewport';
import type { V5NginxServer } from '@/types';
export type CanvasStatusCounts = {
running: number;
failed: number;
unknown: number;
};
type CanvasToolbarProps = {
nginxServers: V5NginxServer[];
selectedNginxServerId: string;
onSelectNginxServer: (serverId: string) => void;
nginxImage: string;
onNginxImageChange: (image: string) => void;
isCreating: boolean;
onDeploy: () => void;
onCenter: () => void;
zoom: number;
onZoomIn: () => void;
onZoomOut: () => void;
isRefreshing: boolean;
onRefresh: () => void;
applicationsCount: number;
statusCounts: CanvasStatusCounts;
};
export const CanvasToolbar = memo(function CanvasToolbar({
nginxServers,
selectedNginxServerId,
onSelectNginxServer,
nginxImage,
onNginxImageChange,
isCreating,
onDeploy,
onCenter,
zoom,
onZoomIn,
onZoomOut,
isRefreshing,
onRefresh,
applicationsCount,
statusCounts,
}: CanvasToolbarProps) {
return (
<div className="absolute left-4 top-20 z-30 flex max-w-[calc(100%-2rem)] flex-wrap items-center gap-2 rounded-xl border border-border bg-background/95 p-2 shadow-lg backdrop-blur">
<select
aria-label="Select nginx server"
value={selectedNginxServerId}
onChange={(event) => onSelectNginxServer(event.target.value)}
disabled={isCreating || nginxServers.length === 0}
className="rounded-lg border border-border bg-background px-3 py-2 text-sm font-medium text-foreground transition disabled:cursor-not-allowed disabled:opacity-60"
>
{nginxServers.length === 0 ? (
<option value="">No servers available</option>
) : (
nginxServers.map((server) => (
<option key={server.id} value={server.id}>
{server.name} ({server.host})
</option>
))
)}
</select>
<input
type="text"
aria-label="Nginx image"
value={nginxImage}
onChange={(event) => onNginxImageChange(event.target.value)}
disabled={isCreating}
className="w-72 rounded-lg border border-border bg-background px-3 py-2 text-sm font-medium text-foreground transition disabled:cursor-not-allowed disabled:opacity-60"
/>
<button
type="button"
onClick={onDeploy}
disabled={isCreating || nginxServers.length === 0}
className="rounded-lg bg-warning px-3 py-2 text-sm font-semibold text-black transition hover:bg-warning/90 disabled:cursor-not-allowed disabled:opacity-60"
>
{isCreating ? 'Deploying…' : 'Deploy'}
</button>
<button
type="button"
onClick={onCenter}
className="rounded-lg border border-border px-3 py-2 text-sm font-medium text-foreground transition hover:bg-muted"
>
Center
</button>
<div className="flex items-center overflow-hidden rounded-lg border border-border">
<button
type="button"
aria-label="Zoom out"
onClick={onZoomOut}
className="px-3 py-2 text-sm font-semibold text-foreground transition hover:bg-muted disabled:cursor-not-allowed disabled:opacity-50"
disabled={zoom <= MIN_CANVAS_ZOOM}
>
</button>
<span className="min-w-14 border-x border-border px-2 py-2 text-center text-xs font-medium text-muted-foreground">
{Math.round(zoom * 100)}%
</span>
<button
type="button"
aria-label="Zoom in"
onClick={onZoomIn}
className="px-3 py-2 text-sm font-semibold text-foreground transition hover:bg-muted disabled:cursor-not-allowed disabled:opacity-50"
disabled={zoom >= MAX_CANVAS_ZOOM}
>
+
</button>
</div>
<button
type="button"
onClick={onRefresh}
disabled={isRefreshing}
className="rounded-lg border border-border px-3 py-2 text-sm font-medium text-foreground transition hover:bg-muted disabled:cursor-not-allowed disabled:opacity-60"
>
{isRefreshing ? 'Refreshing…' : 'Refresh state'}
</button>
<div className="hidden items-center gap-2 px-2 text-xs text-muted-foreground sm:flex">
<span>{applicationsCount} apps</span>
<span></span>
<span>{statusCounts.running} running</span>
{statusCounts.failed > 0 && (
<>
<span></span>
<span className="text-destructive">{statusCounts.failed} failed</span>
</>
)}
{statusCounts.unknown > 0 && (
<>
<span></span>
<span>{statusCounts.unknown} unknown</span>
</>
)}
</div>
</div>
);
});
@@ -0,0 +1,111 @@
import { memo, useMemo, type MouseEvent } from 'react';
import { connectorPoint, shortestConnectionPoints, type DraftConnection } from '@/lib/canvas-geometry';
import { cn } from '@/lib/utils';
import type { CanvasConnection } from '@/lib/use-canvas-connections';
import type { V5Application } from '@/types';
type ConnectionLineProps = {
connection: CanvasConnection;
fromApplication: V5Application;
toApplication: V5Application;
isSelected: boolean;
onSelect: (event: MouseEvent<SVGLineElement>, connectionId: string) => void;
};
const ConnectionLine = memo(function ConnectionLine({ connection, fromApplication, toApplication, isSelected, onSelect }: ConnectionLineProps) {
const points = useMemo(() => shortestConnectionPoints(fromApplication, toApplication), [fromApplication, toApplication]);
return (
<g>
<line
x1={points.from.x}
y1={points.from.y}
x2={points.to.x}
y2={points.to.y}
aria-label="Select connection"
className="pointer-events-auto cursor-pointer"
stroke="transparent"
strokeWidth={12}
strokeLinecap="round"
onPointerDown={(event) => event.stopPropagation()}
onClick={(event) => onSelect(event, connection.id)}
/>
<line
x1={points.from.x}
y1={points.from.y}
x2={points.to.x}
y2={points.to.y}
className={cn('pointer-events-none', isSelected ? 'stroke-destructive' : 'stroke-warning')}
strokeWidth={isSelected ? 4 : 2}
strokeDasharray="6 6"
strokeLinecap="round"
markerEnd={isSelected ? 'url(#dashboard-connection-arrow)' : undefined}
/>
</g>
);
});
type ConnectionLinesProps = {
connections: CanvasConnection[];
applications: V5Application[];
selectedConnectionId: string | null;
draftConnection: DraftConnection | null;
onSelectConnection: (event: MouseEvent<SVGLineElement>, connectionId: string) => void;
};
export function ConnectionLines({ connections, applications, selectedConnectionId, draftConnection, onSelectConnection }: ConnectionLinesProps) {
const applicationsById = useMemo(() => new Map(applications.map((application) => [application.id, application])), [applications]);
const draftFromApplication = draftConnection ? applicationsById.get(draftConnection.from.applicationId) : undefined;
const draftFrom = draftConnection && draftFromApplication ? connectorPoint(draftFromApplication, draftConnection.from.side) : null;
return (
<svg className="pointer-events-none absolute inset-0 overflow-visible">
<defs>
<marker
id="dashboard-connection-arrow"
viewBox="0 0 10 10"
refX="9"
refY="5"
markerWidth="16"
markerHeight="16"
orient="auto"
markerUnits="userSpaceOnUse"
>
<path d="M 0 0 L 10 5 L 0 10 z" fill="var(--destructive)" />
</marker>
</defs>
{connections.map((connection) => {
const fromApplication = applicationsById.get(connection.fromApplicationId);
const toApplication = applicationsById.get(connection.toApplicationId);
if (!fromApplication || !toApplication) {
return null;
}
return (
<ConnectionLine
key={connection.id}
connection={connection}
fromApplication={fromApplication}
toApplication={toApplication}
isSelected={selectedConnectionId === connection.id}
onSelect={onSelectConnection}
/>
);
})}
{draftConnection && draftFrom && (
<line
x1={draftFrom.x}
y1={draftFrom.y}
x2={draftConnection.toX}
y2={draftConnection.toY}
className="stroke-warning/70"
strokeWidth={2}
strokeDasharray="6 6"
strokeLinecap="round"
/>
)}
</svg>
);
}
@@ -0,0 +1,146 @@
import { useMemo } from 'react';
import { shortestConnectionPoints } from '@/lib/canvas-geometry';
import { activeConnectionPorts, type CanvasConnection } from '@/lib/use-canvas-connections';
import { cn } from '@/lib/utils';
import type { V5Application } from '@/types';
type ConnectionPortsEditorProps = {
connection: CanvasConnection;
applications: V5Application[];
portInput: string;
onPortInputChange: (connectionId: string, value: string) => void;
onUpdateDirection: (connectionId: string, fromApplicationId: string, toApplicationId: string) => void;
onAddPort: (connectionId: string) => void;
onRemovePort: (connectionId: string, port: string) => void;
onDelete: (connectionId: string) => void;
};
export function ConnectionPortsEditor({
connection,
applications,
portInput,
onPortInputChange,
onUpdateDirection,
onAddPort,
onRemovePort,
onDelete,
}: ConnectionPortsEditorProps) {
const fromApplication = applications.find((candidate) => candidate.id === connection.fromApplicationId);
const toApplication = applications.find((candidate) => candidate.id === connection.toApplicationId);
const points = useMemo(
() => (fromApplication && toApplication ? shortestConnectionPoints(fromApplication, toApplication) : null),
[fromApplication, toApplication],
);
if (!points) {
return null;
}
function applicationDirectionLabel(applicationId: string): string {
const application = applications.find((candidate) => candidate.id === applicationId);
if (!application) {
return 'Unknown app';
}
return `${application.name} (${application.id.slice(0, 8)})`;
}
const activePorts = activeConnectionPorts(connection);
const firstApplicationId = connection.applicationIds[0];
const secondApplicationId = connection.applicationIds[1];
const isForwardDirection =
connection.fromApplicationId === firstApplicationId && connection.toApplicationId === secondApplicationId;
return (
<div
className="absolute z-20 flex w-72 -translate-x-1/2 -translate-y-1/2 flex-col gap-3 rounded-md border border-border bg-card p-3 shadow-lg"
style={{
left: (points.from.x + points.to.x) / 2,
top: (points.from.y + points.to.y) / 2,
}}
onPointerDown={(event) => event.stopPropagation()}
onClick={(event) => event.stopPropagation()}
>
<div className="space-y-2">
<div className="text-[0.625rem] font-semibold uppercase tracking-wide text-muted-foreground">Firewall</div>
<div className="grid gap-1">
<button
type="button"
onClick={() => onUpdateDirection(connection.id, firstApplicationId, secondApplicationId)}
className={cn(
'rounded-sm border px-2 py-1 text-left text-xs transition',
isForwardDirection
? 'border-warning/40 bg-warning/10 text-foreground hover:bg-warning/20'
: 'border-border text-muted-foreground hover:bg-muted hover:text-foreground',
)}
>
{applicationDirectionLabel(firstApplicationId)} {applicationDirectionLabel(secondApplicationId)}
</button>
<button
type="button"
onClick={() => onUpdateDirection(connection.id, secondApplicationId, firstApplicationId)}
className={cn(
'rounded-sm border px-2 py-1 text-left text-xs transition',
!isForwardDirection
? 'border-warning/40 bg-warning/10 text-foreground hover:bg-warning/20'
: 'border-border text-muted-foreground hover:bg-muted hover:text-foreground',
)}
>
{applicationDirectionLabel(secondApplicationId)} {applicationDirectionLabel(firstApplicationId)}
</button>
</div>
</div>
<div className="space-y-2">
<div className="text-[0.625rem] font-semibold uppercase tracking-wide text-muted-foreground">Allowed ports</div>
<div className="flex flex-wrap gap-1">
{activePorts.length === 0 && <span className="text-xs text-muted-foreground">No ports yet.</span>}
{activePorts.map((port) => (
<button
key={port}
type="button"
onClick={() => onRemovePort(connection.id, port)}
className="rounded-full border border-border bg-muted px-2 py-1 text-[0.625rem] font-medium text-foreground transition hover:border-destructive/40 hover:text-destructive"
>
{port} ×
</button>
))}
</div>
<div className="flex gap-1">
<input
type="number"
min={1}
max={65535}
placeholder="Port"
value={portInput}
onChange={(event) => onPortInputChange(connection.id, event.target.value)}
onKeyDown={(event) => {
if (event.key === 'Enter') {
onAddPort(connection.id);
}
}}
className="min-w-0 flex-1 rounded-sm border border-border bg-background px-2 py-1 text-xs text-foreground outline-none transition focus:border-warning"
/>
<button
type="button"
onClick={() => onAddPort(connection.id)}
className="rounded-sm border border-border px-2 py-1 text-xs font-medium text-foreground transition hover:bg-muted"
>
Add
</button>
</div>
</div>
<button
type="button"
aria-label="Delete connection"
onClick={() => onDelete(connection.id)}
className="rounded-sm border border-destructive/40 px-2 py-1 text-[0.625rem] font-semibold uppercase tracking-wide text-destructive transition hover:bg-destructive/10"
>
Delete
</button>
</div>
);
}
@@ -0,0 +1,82 @@
import { Button } from '@/components/ui/button';
import { Dialog, DialogContent, DialogDescription, DialogHeader, DialogTitle } from '@/components/ui/dialog';
import { Field, FieldLabel } from '@/components/ui/field';
import { Input } from '@/components/ui/input';
import type { IngressModalState } from '@/lib/use-application-ingress';
type IngressDialogProps = {
modal: IngressModalState;
isSaving: boolean;
onDomainsChange: (domains: string) => void;
onInternalPortChange: (internalPort: string) => void;
onSubmit: () => void;
onClose: () => void;
};
export function IngressDialog({ modal, isSaving, onDomainsChange, onInternalPortChange, onSubmit, onClose }: IngressDialogProps) {
return (
<Dialog
open
onOpenChange={(open) => {
if (!open && !isSaving) {
onClose();
}
}}
>
<DialogContent className="max-w-lg" showCloseButton>
<DialogHeader>
<DialogTitle>Enable app ingress</DialogTitle>
<DialogDescription>
Route public domains to {modal.application.name} through the server ingress.
</DialogDescription>
</DialogHeader>
<form
className="mt-6 flex flex-col gap-4"
onSubmit={(event) => {
event.preventDefault();
onSubmit();
}}
>
<Field>
<FieldLabel>Domains</FieldLabel>
<Input
type="text"
value={modal.domains}
onChange={(event) => onDomainsChange(event.target.value)}
placeholder="example.com, www.example.com"
/>
<span className="text-xs text-muted-foreground">
Use hostnames only, separated by commas. No scheme, path, wildcard, or port.
</span>
</Field>
<Field>
<FieldLabel>Internal port</FieldLabel>
<Input
type="number"
min="1"
max="65535"
value={modal.internalPort}
onChange={(event) => onInternalPortChange(event.target.value)}
placeholder="3000"
/>
</Field>
{modal.error && (
<div className="rounded-lg border border-destructive/30 bg-destructive/10 px-3 py-2 text-sm text-destructive">
<p className="font-medium">Ingress update failed</p>
<p className="mt-1 text-destructive/90">{modal.error}</p>
</div>
)}
<div className="flex justify-end">
<Button type="submit" variant="coolify" disabled={isSaving}>
{isSaving ? 'Saving...' : 'Enable ingress'}
</Button>
</div>
</form>
</DialogContent>
</Dialog>
);
}
@@ -0,0 +1,19 @@
export function statusBadgeClass(status: string): string {
if (status === 'running') {
return 'bg-emerald-500/15 text-emerald-400';
}
if (['creating', 'starting'].includes(status)) {
return 'bg-warning/15 text-warning';
}
if (status === 'unknown') {
return 'bg-muted text-muted-foreground';
}
if (['failed', 'exited', 'unreachable'].includes(status)) {
return 'bg-destructive/15 text-destructive';
}
return 'bg-muted text-muted-foreground';
}
+7
View File
@@ -0,0 +1,7 @@
/**
* General-purpose JSON request helper for v5 pages: same-origin credentials,
* CSRF header, and a 30 second timeout on every request. The implementation
* currently lives in canvas-api.ts; import from this module in non-canvas
* code so the canvas-specific name can be retired later.
*/
export { canvasRequest as apiRequest } from '@/lib/canvas-api';
+24
View File
@@ -0,0 +1,24 @@
import { csrfToken } from '@/lib/csrf';
type CanvasRequestOptions = {
method: 'GET' | 'POST' | 'PATCH' | 'DELETE';
body?: unknown;
};
/**
* Shared JSON fetch for the canvas endpoints: same-origin credentials,
* CSRF header, and a 30 second timeout on every request.
*/
export function canvasRequest(url: string, { method, body }: CanvasRequestOptions): Promise<Response> {
return fetch(url, {
method,
credentials: 'same-origin',
headers: {
Accept: 'application/json',
...(body !== undefined ? { 'Content-Type': 'application/json' } : {}),
'X-CSRF-TOKEN': csrfToken(),
},
body: body !== undefined ? JSON.stringify(body) : undefined,
signal: AbortSignal.timeout(30_000),
});
}
+132
View File
@@ -0,0 +1,132 @@
import { resolveCanvasNodeLayout, resolveCanvasNodePosition, type CanvasNodeBounds } from '@/lib/canvas-collision';
import type { V5Application, V5CaddyIngress } from '@/types';
export const APPLICATION_CARD_WIDTH = 320;
export const APPLICATION_CARD_HEIGHT = 160;
export const CANVAS_CARD_GAP = 16;
export type ConnectorSide = 'top' | 'right' | 'bottom' | 'left';
export const CONNECTOR_SIDES: ConnectorSide[] = ['top', 'right', 'bottom', 'left'];
export type CanvasPoint = {
x: number;
y: number;
};
export type CanvasCardNode = {
canvasX: number;
canvasY: number;
};
export type ConnectionEndpoint = {
applicationId: string;
side: ConnectorSide;
};
export type DraftConnection = {
from: ConnectionEndpoint;
toX: number;
toY: number;
};
export function connectorPoint(node: CanvasCardNode, side: ConnectorSide): CanvasPoint {
switch (side) {
case 'top':
return { x: node.canvasX + APPLICATION_CARD_WIDTH / 2, y: node.canvasY };
case 'right':
return { x: node.canvasX + APPLICATION_CARD_WIDTH, y: node.canvasY + APPLICATION_CARD_HEIGHT / 2 };
case 'bottom':
return { x: node.canvasX + APPLICATION_CARD_WIDTH / 2, y: node.canvasY + APPLICATION_CARD_HEIGHT };
case 'left':
return { x: node.canvasX, y: node.canvasY + APPLICATION_CARD_HEIGHT / 2 };
}
}
export function shortestConnectionPoints(fromNode: CanvasCardNode, toNode: CanvasCardNode): { from: CanvasPoint; to: CanvasPoint } {
let shortest = {
from: connectorPoint(fromNode, 'top'),
to: connectorPoint(toNode, 'top'),
distance: Number.POSITIVE_INFINITY,
};
for (const fromSide of CONNECTOR_SIDES) {
const from = connectorPoint(fromNode, fromSide);
for (const toSide of CONNECTOR_SIDES) {
const to = connectorPoint(toNode, toSide);
const distance = Math.hypot(from.x - to.x, from.y - to.y);
if (distance < shortest.distance) {
shortest = { from, to, distance };
}
}
}
return { from: shortest.from, to: shortest.to };
}
function applicationBounds(application: V5Application): CanvasNodeBounds {
return {
id: `application-${application.id}`,
x: application.canvasX,
y: application.canvasY,
width: APPLICATION_CARD_WIDTH,
height: APPLICATION_CARD_HEIGHT,
};
}
function ingressBounds(ingress: V5CaddyIngress): CanvasNodeBounds {
return {
id: `ingress-${ingress.id}`,
x: ingress.canvasX,
y: ingress.canvasY,
width: APPLICATION_CARD_WIDTH,
height: APPLICATION_CARD_HEIGHT,
};
}
export function canvasCollisionNodes(applications: V5Application[], ingresses: V5CaddyIngress[]): CanvasNodeBounds[] {
return [...applications.map(applicationBounds), ...ingresses.map(ingressBounds)];
}
export function settleCanvasResources(
nextApplications: V5Application[],
nextIngresses: V5CaddyIngress[],
): { applications: V5Application[]; ingresses: V5CaddyIngress[] } {
const settledNodes = resolveCanvasNodeLayout(canvasCollisionNodes(nextApplications, nextIngresses), CANVAS_CARD_GAP);
const positionsById = new Map(settledNodes.map((node) => [node.id, node]));
return {
applications: nextApplications.map((application) => {
const position = positionsById.get(`application-${application.id}`);
return position ? { ...application, canvasX: position.x, canvasY: position.y } : application;
}),
ingresses: nextIngresses.map((ingress) => {
const position = positionsById.get(`ingress-${ingress.id}`);
return position ? { ...ingress, canvasX: position.x, canvasY: position.y } : ingress;
}),
};
}
export function resolveApplicationPosition(
application: V5Application,
applications: V5Application[],
ingresses: V5CaddyIngress[],
): V5Application {
const position = resolveCanvasNodePosition(applicationBounds(application), canvasCollisionNodes(applications, ingresses), CANVAS_CARD_GAP);
return { ...application, canvasX: position.x, canvasY: position.y };
}
export function resolveIngressPosition(
ingress: V5CaddyIngress,
applications: V5Application[],
ingresses: V5CaddyIngress[],
): V5CaddyIngress {
const position = resolveCanvasNodePosition(ingressBounds(ingress), canvasCollisionNodes(applications, ingresses), CANVAS_CARD_GAP);
return { ...ingress, canvasX: position.x, canvasY: position.y };
}
+57
View File
@@ -0,0 +1,57 @@
export type OptimisticRequestResult<TPayload> =
| { ok: true; payload: TPayload }
| { ok: false; errorMessage?: string };
export type OptimisticUpdateOptions<TPayload> = {
/** Apply the optimistic state change before the request is sent. */
apply?: () => void;
/** Restore the previous state after a failed request. */
rollback?: () => void;
/** Perform the persistence request; return ok=false (or throw) on failure. */
request: () => Promise<OptimisticRequestResult<TPayload>>;
/** Notice used when the failure carries no specific error message. */
fallbackErrorMessage: string;
notify: (message: string | null) => void;
/** Reconcile local state with the server payload after success. */
onSuccess?: (payload: TPayload) => void;
/** Runs after success or failure, e.g. to clear pending markers. */
onSettled?: () => void;
};
/**
* Unified optimistic-update flow: apply the local change, persist it, and on
* any failure roll the local state back and surface a notice.
*/
export async function runOptimisticUpdate<TPayload = void>({
apply,
rollback,
request,
fallbackErrorMessage,
notify,
onSuccess,
onSettled,
}: OptimisticUpdateOptions<TPayload>): Promise<boolean> {
apply?.();
try {
const result = await request();
if (!result.ok) {
rollback?.();
notify(result.errorMessage ?? fallbackErrorMessage);
return false;
}
onSuccess?.(result.payload);
return true;
} catch (error) {
rollback?.();
notify(error instanceof Error ? error.message : fallbackErrorMessage);
return false;
} finally {
onSettled?.();
}
}
@@ -0,0 +1,169 @@
import { useCallback, useRef, useState } from 'react';
import { canvasRequest } from '@/lib/canvas-api';
import { usePendingIds } from '@/lib/use-pending-ids';
import type { CanvasNotify } from '@/lib/use-canvas-connections';
import type { V5Application } from '@/types';
export type IngressModalState = {
application: V5Application;
domains: string;
internalPort: string;
error: string | null;
};
export function isValidDomain(domain: string): boolean {
if (domain.length < 1 || domain.length > 253 || domain.startsWith('.') || domain.endsWith('.')) {
return false;
}
return domain.split('.').every((label) => /^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$/.test(label));
}
/**
* Owns the application ingress modal and the enable/disable persistence flow,
* tracking per-application saving state through usePendingIds.
*/
export function useApplicationIngress(options: {
notify: CanvasNotify;
onApplicationUpdated: (application: V5Application) => void;
}) {
const { notify, onApplicationUpdated } = options;
const [ingressModal, setIngressModal] = useState<IngressModalState | null>(null);
const savingIngressApplications = usePendingIds<string>();
const ingressModalRef = useRef<IngressModalState | null>(null);
ingressModalRef.current = ingressModal;
const saveApplicationIngress = useCallback(
async (application: V5Application, enabled: boolean, domains: string[], internalPort: number | null): Promise<void> => {
notify(null);
savingIngressApplications.start(application.id);
const reportError = (message: string): void => {
if (ingressModalRef.current) {
setIngressModal((currentModal) => (currentModal ? { ...currentModal, error: message } : currentModal));
} else {
notify(message);
}
};
try {
const response = await canvasRequest(`/v5/applications/${application.id}/ingress`, {
method: 'PATCH',
body: {
ingress_enabled: enabled,
internal_port: internalPort,
domains,
},
});
if (!response.ok) {
const payload = (await response.json().catch(() => null)) as { message?: string } | null;
reportError(payload?.message ?? 'Could not update application ingress.');
return;
}
const payload = (await response.json()) as { application: V5Application };
onApplicationUpdated(payload.application);
setIngressModal(null);
} catch (error) {
reportError(error instanceof Error ? error.message : 'Could not update application ingress.');
} finally {
savingIngressApplications.finish(application.id);
}
},
[notify, onApplicationUpdated, savingIngressApplications.start, savingIngressApplications.finish],
);
const openApplicationIngressModal = useCallback(
(application: V5Application): void => {
notify(null);
if (!application.serverIngressEnabled) {
notify('Enable ingress on the server before enabling app ingress.');
return;
}
setIngressModal({
application,
domains: application.domains.join(', '),
internalPort: application.internalPort ? String(application.internalPort) : '',
error: null,
});
},
[notify],
);
const toggleApplicationIngress = useCallback(
(application: V5Application): void => {
if (application.ingressEnabled) {
void saveApplicationIngress(application, false, application.domains, application.internalPort);
} else {
openApplicationIngressModal(application);
}
},
[saveApplicationIngress, openApplicationIngressModal],
);
const submitApplicationIngress = useCallback(async (): Promise<void> => {
const currentModal = ingressModalRef.current;
if (!currentModal) {
return;
}
const domains = currentModal.domains
.split(',')
.map((domain) => domain.trim().toLowerCase())
.filter(Boolean);
const internalPort = Number(currentModal.internalPort);
const invalidDomain = domains.find((domain) => !isValidDomain(domain));
if (domains.length === 0) {
setIngressModal({ ...currentModal, error: 'Add at least one valid domain.' });
return;
}
if (invalidDomain) {
setIngressModal({ ...currentModal, error: `${invalidDomain} is not a valid domain.` });
return;
}
if (!Number.isInteger(internalPort) || internalPort < 1 || internalPort > 65535) {
setIngressModal({ ...currentModal, error: 'Choose a valid internal port between 1 and 65535.' });
return;
}
await saveApplicationIngress(currentModal.application, true, [...new Set(domains)], internalPort);
}, [saveApplicationIngress]);
const closeIngressModal = useCallback((): void => {
setIngressModal(null);
}, []);
const setIngressModalDomains = useCallback((domains: string): void => {
setIngressModal((currentModal) => (currentModal ? { ...currentModal, domains, error: null } : currentModal));
}, []);
const setIngressModalInternalPort = useCallback((internalPort: string): void => {
setIngressModal((currentModal) => (currentModal ? { ...currentModal, internalPort, error: null } : currentModal));
}, []);
return {
ingressModal,
closeIngressModal,
setIngressModalDomains,
setIngressModalInternalPort,
submitApplicationIngress,
toggleApplicationIngress,
savingIngressApplications,
};
}
+19
View File
@@ -0,0 +1,19 @@
import { useTeamChannel } from '@/lib/use-team-channel';
import type { V5Application, V5CaddyIngress } from '@/types';
export type V5CanvasResourceUpdatedEvent = {
application: V5Application | null;
applications?: V5Application[];
caddyIngress: V5CaddyIngress | null;
};
/**
* Subscribes the canvas to the private team channel and forwards
* `.v5.canvas.resource.updated` payloads to the latest onEvent callback
* without resubscribing when the callback identity changes.
*/
export function useCanvasResourceChannel(teamId: number | null, onEvent: (event: V5CanvasResourceUpdatedEvent) => void): void {
useTeamChannel(teamId, '.v5.canvas.resource.updated', (payload) => {
onEvent(payload as V5CanvasResourceUpdatedEvent);
});
}

Some files were not shown because too many files have changed in this diff Show More