mirror of
https://github.com/tiennm99/coolify.git
synced 2026-04-17 21:20:29 +00:00
feat(jobs): improve scheduled tasks with retry logic and queue cleanup
- Add retry configuration to CoolifyTask (3 tries, 600s timeout) - Add retry configuration to ScheduledTaskJob (3 tries, configurable timeout) - Add retry configuration to DatabaseBackupJob (2 tries) - Implement exponential backoff for all jobs (30s, 60s, 120s intervals) - Add failed() handlers with comprehensive error logging to scheduled-errors channel - Add execution tracking: started_at, retry_count, duration (decimal), error_details - Add configurable timeout field to scheduled tasks (60-3600s, default 300s) - Update UI to include timeout configuration in task creation/editing forms - Increase ScheduledJobManager lock expiration from 60s to 90s for high-load environments - Implement safe queue cleanup with restart vs runtime modes - Restart mode: aggressive cleanup (marks all processing jobs as failed) - Runtime mode: conservative cleanup (only marks jobs >12h as failed, skips deployments) - Add cleanup:redis --restart flag for system startup - Integrate cleanup into Dev.php init() for development environment - Increase scheduled-errors log retention from 7 to 14 days - Create comprehensive test suite (unit and feature tests) - Add TESTING_GUIDE.md with manual testing instructions Fixes issues with jobs failing after single attempt and "attempted too many times" errors
This commit is contained in:
@@ -45,6 +45,16 @@ class Dev extends Command
|
||||
} else {
|
||||
echo "Instance already initialized.\n";
|
||||
}
|
||||
|
||||
// Clean up stuck jobs and stale locks on development startup
|
||||
try {
|
||||
echo "Cleaning up Redis (stuck jobs and stale locks)...\n";
|
||||
Artisan::call('cleanup:redis', ['--restart' => true, '--clear-locks' => true]);
|
||||
echo "Redis cleanup completed.\n";
|
||||
} catch (\Throwable $e) {
|
||||
echo "Error in cleanup:redis: {$e->getMessage()}\n";
|
||||
}
|
||||
|
||||
CheckHelperImageJob::dispatch();
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user