feat: add container restart tracking and crash loop detection

Track container restart counts from Docker and detect crash loops to provide better visibility into application health issues.

- Add restart_count, last_restart_at, and last_restart_type columns to applications table
- Detect restart count increases from Docker inspect data and send notifications
- Show restart count badge in UI with warning icon on Logs navigation
- Distinguish between crash restarts and manual restarts
- Implement 30-second grace period to prevent false "exited" status during crash loops
- Reset restart count on manual stop, restart, and redeploy actions
- Add unit tests for restart count tracking logic

This helps users quickly identify when containers are in crash loops and need attention, even when the container status flickers between states during Docker's restart backoff period.
This commit is contained in:
Andras Bacsai
2025-11-10 13:04:31 +01:00
parent 775216e7a5
commit 68a9f2ca77
9 changed files with 231 additions and 21 deletions

View File

@@ -12,6 +12,13 @@
@else
<x-status.stopped :status="$resource->status" />
@endif
@if (isset($resource->restart_count) && $resource->restart_count > 0 && !str($resource->status)->startsWith('exited'))
<div class="flex items-center pl-2">
<span class="text-xs dark:text-warning" title="Container has restarted {{ $resource->restart_count }} time{{ $resource->restart_count > 1 ? 's' : '' }}. Last restart: {{ $resource->last_restart_at?->diffForHumans() }}">
({{ $resource->restart_count }}x restarts)
</span>
</div>
@endif
@if (!str($resource->status)->contains('exited') && $showRefreshButton)
<button wire:loading.remove.delay.shortest wire:target="manualCheckStatus" title="Refresh Status" wire:click='manualCheckStatus'
class="mx-1 dark:hover:fill-white fill-black dark:fill-warning">

View File

@@ -12,7 +12,14 @@
</a>
<a class="{{ request()->routeIs('project.application.logs') ? 'dark:text-white' : '' }}"
href="{{ route('project.application.logs', $parameters) }}">
Logs
<div class="flex items-center gap-1">
Logs
@if ($application->restart_count > 0 && !str($application->status)->startsWith('exited'))
<svg class="w-4 h-4 dark:text-warning" viewBox="0 0 24 24" fill="currentColor" xmlns="http://www.w3.org/2000/svg" title="Container has restarted {{ $application->restart_count }} time{{ $application->restart_count > 1 ? 's' : '' }}">
<path d="M12 2L1 21h22L12 2zm0 4l7.53 13H4.47L12 6zm-1 5v4h2v-4h-2zm0 5v2h2v-2h-2z"/>
</svg>
@endif
</div>
</a>
@if (!$application->destination->server->isSwarm())
@can('canAccessTerminal')