This commit adds comprehensive validation improvements and DRY principles for handling Coolify's custom Docker Compose extensions.
## Changes
### 1. Created Reusable stripCoolifyCustomFields() Function
- Added shared helper in bootstrap/helpers/docker.php
- Removes all Coolify custom fields (exclude_from_hc, content, isDirectory, is_directory)
- Handles both long syntax (arrays) and short syntax (strings) for volumes
- Well-documented with comprehensive docblock
- Follows DRY principle for consistent field stripping
### 2. Fixed Docker Compose Modal Validation
- Updated validateComposeFile() to use stripCoolifyCustomFields()
- Now removes ALL custom fields before Docker validation (previously only removed content)
- Fixes validation errors when using templates with custom fields (e.g., traccar.yaml)
- Users can now validate compose files with Coolify extensions in UI
### 3. Enhanced YAML Validation in CalculatesExcludedStatus
- Added proper exception handling with ParseException vs generic Exception
- Added structure validation (checks if parsed result and services are arrays)
- Comprehensive logging with context (error message, line number, snippet)
- Maintains safe fallback behavior (returns empty collection on error)
### 4. Added Integer Validation to ContainerStatusAggregator
- Validates maxRestartCount parameter in both aggregateFromStrings() and aggregateFromContainers()
- Corrects negative values to 0 with warning log
- Logs warnings for suspiciously high values (> 1000)
- Prevents logic errors in crash loop detection
### 5. Comprehensive Unit Tests
- tests/Unit/StripCoolifyCustomFieldsTest.php (NEW) - 9 tests, 43 assertions
- tests/Unit/ContainerStatusAggregatorTest.php - Added 6 tests for integer validation
- tests/Unit/ExcludeFromHealthCheckTest.php - Added 4 tests for YAML validation
- All tests passing with proper Log facade mocking
### 6. Documentation
- Added comprehensive Docker Compose extensions documentation to .ai/core/deployment-architecture.md
- Documents all custom fields: exclude_from_hc, content, isDirectory/is_directory
- Includes examples, use cases, implementation details, and test references
- Updated .ai/README.md with navigation links to new documentation
## Benefits
- Better UX: Users can validate compose files with custom fields
- Better Debugging: Comprehensive logging for errors
- Better Code Quality: DRY principle with reusable validation
- Better Reliability: Prevents logic errors from invalid parameters
- Better Maintainability: Easy to add new custom fields in future
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Refine regex pattern to prevent false positives with flags like -foo, -from, -feature
- Change from \S (any non-whitespace) to [.~/]|$ (path characters or end of word)
- Add comprehensive tests for false positive prevention (4 test cases)
- Add path normalization tests for baseDirectory edge cases (6 test cases)
- Add @example documentation to injectDockerComposeFlags function
Prevents incorrect detection of:
- -foo, -from, -feature, -fast as the -f flag
- Ensures -f flag is only detected when followed by path characters or end of word
All 45 tests passing with 135 assertions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces several improvements to the Traefik version tracking
feature and proxy configuration UI:
## Caching Improvements
1. **New centralized helper functions** (bootstrap/helpers/versions.php):
- `get_versions_data()`: Redis-cached access to versions.json (1 hour TTL)
- `get_traefik_versions()`: Extract Traefik versions from cached data
- `invalidate_versions_cache()`: Clear cache when file is updated
2. **Performance optimization**:
- Single Redis cache key: `coolify:versions:all`
- Eliminates 2-4 file reads per page load
- 95-97.5% reduction in disk I/O time
- Shared cache across all servers in distributed setup
3. **Updated all consumers to use cached helpers**:
- CheckTraefikVersionJob: Use get_traefik_versions()
- Server/Proxy: Two-level caching (Redis + in-memory per-request)
- CheckForUpdatesJob: Auto-invalidate cache after updating file
- bootstrap/helpers/shared.php: Use cached data for Coolify version
## UI/UX Improvements
1. **Navbar warning indicator**:
- Added yellow warning triangle icon next to "Proxy" menu item
- Appears when server has outdated Traefik version
- Uses existing traefik_outdated_info data for instant checks
- Provides at-a-glance visibility of version issues
2. **Proxy sidebar persistence**:
- Fixed sidebar disappearing when clicking "Switch Proxy"
- Configuration link now always visible (needed for proxy selection)
- Dynamic Configurations and Logs only show when proxy is configured
- Better navigation context during proxy switching workflow
## Code Quality
- Added comprehensive PHPDoc for Server::$traefik_outdated_info property
- Improved code organization with centralized helper approach
- All changes formatted with Laravel Pint
- Maintains backward compatibility
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add automated Traefik version checking job running weekly on Sundays
- Implement version detection from running containers and comparison with versions.json
- Add notifications across all channels (Email, Discord, Slack, Telegram, Pushover, Webhook) for outdated versions
- Create dismissible callout component with localStorage persistence
- Display cross-branch upgrade warnings (e.g., v3.5 -> v3.6) with changelog links
- Show patch update notifications within same branch
- Add warning icon that appears when callouts are dismissed
- Prevent duplicate notifications during proxy restart by adding restarting parameter
- Fix notification spam with transition-based logic for status changes
- Enable system email settings by default in development mode
- Track last saved/applied proxy settings to detect configuration drift
- Refine regex pattern to prevent false positives with flags like -foo, -from, -feature
- Change from \S (any non-whitespace) to [.~/]|$ (path characters or end of word)
- Add comprehensive tests for false positive prevention (4 test cases)
- Add path normalization tests for baseDirectory edge cases (6 test cases)
- Add @example documentation to injectDockerComposeFlags function
Prevents incorrect detection of:
- -foo, -from, -feature, -fast as the -f flag
- Ensures -f flag is only detected when followed by path characters or end of word
All 45 tests passing with 135 assertions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces several improvements to the Traefik version tracking
feature and proxy configuration UI:
## Caching Improvements
1. **New centralized helper functions** (bootstrap/helpers/versions.php):
- `get_versions_data()`: Redis-cached access to versions.json (1 hour TTL)
- `get_traefik_versions()`: Extract Traefik versions from cached data
- `invalidate_versions_cache()`: Clear cache when file is updated
2. **Performance optimization**:
- Single Redis cache key: `coolify:versions:all`
- Eliminates 2-4 file reads per page load
- 95-97.5% reduction in disk I/O time
- Shared cache across all servers in distributed setup
3. **Updated all consumers to use cached helpers**:
- CheckTraefikVersionJob: Use get_traefik_versions()
- Server/Proxy: Two-level caching (Redis + in-memory per-request)
- CheckForUpdatesJob: Auto-invalidate cache after updating file
- bootstrap/helpers/shared.php: Use cached data for Coolify version
## UI/UX Improvements
1. **Navbar warning indicator**:
- Added yellow warning triangle icon next to "Proxy" menu item
- Appears when server has outdated Traefik version
- Uses existing traefik_outdated_info data for instant checks
- Provides at-a-glance visibility of version issues
2. **Proxy sidebar persistence**:
- Fixed sidebar disappearing when clicking "Switch Proxy"
- Configuration link now always visible (needed for proxy selection)
- Dynamic Configurations and Logs only show when proxy is configured
- Better navigation context during proxy switching workflow
## Code Quality
- Added comprehensive PHPDoc for Server::$traefik_outdated_info property
- Improved code organization with centralized helper approach
- All changes formatted with Laravel Pint
- Maintains backward compatibility
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Major architectural improvements:
- Merged download and restore into single atomic operation
- Eliminated separate S3DownloadFinished event (redundant)
- Files now transfer directly: S3 → helper container → server → database container
- Removed download progress tracking in favor of unified restore progress
UI/UX improvements:
- Unified restore method selection with visual cards
- Consistent "File Information" display between local and S3 restore
- Single slide-over for all restore operations (removed separate S3 download monitor)
- Better visual feedback with loading states
Security enhancements:
- Added isSafeTmpPath() helper for path traversal protection
- URL decode validation to catch encoded attacks
- Canonical path resolution to prevent symlink attacks
- Comprehensive path validation in all cleanup events
Cleanup improvements:
- S3RestoreJobFinished now handles all cleanup (helper container + all temp files)
- RestoreJobFinished uses new isSafeTmpPath() validation
- CoolifyTask dispatches cleanup events even on job failure
- All cleanup uses non-throwing commands (2>/dev/null || true)
Other improvements:
- S3 storage policy authorization on Show component
- Storage Form properly syncs is_usable state after test
- Removed debug code and improved error handling
- Better command organization and documentation
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
- Add automated Traefik version checking job running weekly on Sundays
- Implement version detection from running containers and comparison with versions.json
- Add notifications across all channels (Email, Discord, Slack, Telegram, Pushover, Webhook) for outdated versions
- Create dismissible callout component with localStorage persistence
- Display cross-branch upgrade warnings (e.g., v3.5 -> v3.6) with changelog links
- Show patch update notifications within same branch
- Add warning icon that appears when callouts are dismissed
- Prevent duplicate notifications during proxy restart by adding restarting parameter
- Fix notification spam with transition-based logic for status changes
- Enable system email settings by default in development mode
- Track last saved/applied proxy settings to detect configuration drift
The formatBytes function was used in the view but never defined, causing
a runtime error. This function was needed to display S3 file sizes in
human-readable format (e.g., "1.5 MB" instead of "1572864").
Added formatBytes() helper to bootstrap/helpers/shared.php:
- Converts bytes to human-readable format (B, KB, MB, GB, TB, PB)
- Uses base 1024 for proper binary conversion
- Configurable precision (defaults to 2 decimal places)
- Handles zero bytes case
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces functionality for integrating S3 storage into the import process. It allows users to select S3 storage, check for file existence, and download files directly from S3. This enhancement improves the flexibility of the import feature by enabling users to work with files stored in S3, addressing a common use case for teams that utilize cloud storage solutions.
Upgrade default Traefik proxy configuration from v3.5 to v3.6, with Coolify version bump to beta.444.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
Fixes Docker installation failures on non-root servers by properly handling
complex shell commands with pipes and operators. Previously, the sudo parser
would insert sudo throughout command chains, breaking pipe structures like
'curl URL | sh || curl URL2 | sh'.
The fix detects complex piped commands (containing '| sh', '| bash', or
pipes combined with && or || operators) and wraps them in 'sudo bash -c'
instead of inserting sudo mid-command. This preserves the command structure
and prevents syntax errors.
Changes:
- Detect complex piped commands in parseCommandsByLineForSudo
- Wrap complex commands in 'sudo bash -c' with proper quote escaping
- Preserve original behavior for simple commands
- Add 27 comprehensive unit tests covering all scenarios
Fixes#7116
- Fix container filtering to properly distinguish base deployments (pullRequestId=0) from PR deployments
- Add deployment cancellation when PR closes via webhook to prevent race conditions
- Prevent CleanupHelperContainersJob from killing active deployment containers
- Enhance error messages with exit codes and actual errors instead of vague "Oops" messages
- Protect status transitions in finally blocks to ensure proper job failure handling
- Added `requiredPort` property to `ServiceApplicationView` to track the required port for services.
- Introduced modal confirmation for removing required ports, including methods to confirm or cancel the action.
- Enhanced `Service` model with `getRequiredPort` and `requiresPort` methods to retrieve port information from service templates.
- Implemented `extractPortFromUrl` method in `ServiceApplication` to extract port from FQDN URLs.
- Updated frontend views to display warnings when required ports are missing from domains.
- Created unit tests for service port validation and extraction logic, ensuring correct behavior for various scenarios.
- Added feature tests for Livewire component handling of domain submissions with required ports.
- Preserve empty string environment variables instead of converting to null
Empty strings and null have different semantics in Docker Compose:
* Empty string (VAR: ""): Variable is set to "" in container (e.g., HTTP_PROXY="" means "no proxy")
* Null (VAR: null): Variable is unset/removed from container environment
- Remove empty top-level sections (volumes, configs, secrets) from generated compose files
These sections now only appear when they contain actual content, following Docker Compose best practices
- Add safety check for missing volumes in validateComposeFile to prevent iteration errors
- Add comprehensive unit tests for both fixes
Fixes#7126
The custom_network_aliases attribute in the Application model was being cast to an array directly. This commit refactors the attribute to provide both a string representation (for compatibility with older configurations and hashing) and an array representation for internal use. This ensures that network aliases are correctly parsed and utilized, preventing potential issues during deployment and configuration updates.
The redis_password column was removed from standalone_redis table
in migration 2024_10_16_120026_move_redis_password_to_envs.php,
moving passwords to environment_variables table.
However, the API endpoint still accepted redis_password parameter
and tried to mass-assign it via fill(), causing SQL error:
"column redis_password of relation standalone_redis does not exist"
Fixed by extracting redis_password from $otherData before fill()
and using it when creating the REDIS_PASSWORD environment variable,
similar to how the migration handled the transition.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The previous fix (a956e11b3) incorrectly set docker_compose_raw to the
fully processed compose file, which included all Coolify additions like
labels, environment variables, networks, and modified container names.
This broke the separation between user input (docker_compose_raw) and
Coolify's processed output (docker_compose).
Changes:
- Store original compose at parser start before any processing
- Only remove content/isDirectory fields from original compose
- Save clean version to docker_compose_raw
- Save fully processed version to docker_compose
Now docker_compose_raw contains:
✓ Original user input with only content fields removed
✓ User's template variables ($SERVICE_FQDN_*, $SERVICE_URL_*)
✓ User's original labels and environment variables
And docker_compose contains:
✓ All Coolify additions (labels, networks, COOLIFY_* env vars)
✓ Modified container names with UUIDs
✓ Resolved template variables
Added comprehensive unit tests to verify the fix.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When users define volumes with content: in compose files, the content
was being removed from docker_compose but not docker_compose_raw.
This caused files to be overwritten on every deployment/save when
users edited the compose file.
Now both docker_compose and docker_compose_raw are updated with the
cleaned version (without content:), allowing users to freely edit
files in the persistent storage view without them being overwritten.
The content: field now acts as an initial value only - it creates
the file on first deployment, then is removed so users have full
control over file contents.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changes:
- Extended validateDockerComposeForInjection to recognize env vars with defaults
- Added pattern check for ${VAR:-default} format alongside simple ${VAR} check
- Maintains consistency with parseDockerVolumeString behavior for string format
Test coverage:
- Added test for safe environment variable defaults in array format
- Verifies ${DATA_PATH:-./data} is allowed in array-format volumes
- All 79 security tests pass (215 assertions)
This allows users to specify environment variables with safe default values
in array-format Docker Compose volumes, matching the behavior already
supported in string-format volumes.
Changes:
- Added tab character ("\t") to dangerous characters list as token separator
- Removed redundant regex-based preg_match block (lines 147-152)
- Characters $(, ${, and backticks were already covered in $dangerousChars array
- Simplified function to rely solely on $dangerousChars loop
Security improvement:
- Tab characters can act as token separators in shell contexts
- Now explicitly blocked with descriptive error message
Tests:
- Added test for tab character blocking
- All 78 security tests pass (213 assertions)
- No regression in existing functionality
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Problem:
- validateVolumeStringForInjection used explode(':') to parse volume strings
- This incorrectly splits Windows paths like "C:\host\path:/container" at the drive letter colon
- Could lead to false positives/negatives in injection detection
Solution:
- Replace custom parsing in validateVolumeStringForInjection with call to parseDockerVolumeString
- parseDockerVolumeString already handles Windows paths, environment variables, and performs validation
- Eliminates code duplication and uses single source of truth for volume string parsing
Tests:
- All 77 existing security tests pass (211 assertions)
- Added 6 new Windows path tests (8 assertions)
- Fixed pre-existing test bug: preg_match returns int 1, not boolean true
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When catching and re-throwing exceptions, preserve the original exception
chain by passing the caught exception as the third parameter to new Exception.
This retains the full stack trace for debugging while keeping descriptive
error messages.
Changes:
- validateDockerComposeForInjection(): 4 locations fixed
- validateVolumeStringForInjection(): 3 locations fixed
Before:
throw new \Exception('Invalid Docker volume definition: '.$e->getMessage());
After:
throw new \Exception('Invalid Docker volume definition: '.$e->getMessage(), 0, $e);
Benefits:
- Full stack trace preserved for debugging
- Original exception context retained
- Better error diagnostics in production logs
All 60 security tests pass (176 assertions).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit addresses a critical security issue where malicious Docker Compose
data was being saved to the database before validation occurred.
Problem:
- Service models were saved to database first
- Validation ran afterwards during parse()
- Malicious data persisted even when validation failed
- User saw error but damage was already done
Solution:
1. Created validateDockerComposeForInjection() to validate YAML before save
2. Added pre-save validation to all Service creation/update points:
- Livewire: DockerCompose.php, StackForm.php
- API: ServicesController.php (create, update, one-click)
3. Validates service names and volume paths (string + array formats)
4. Blocks shell metacharacters: backticks, $(), |, ;, &, >, <, newlines
Security fixes:
- Volume source paths (string format) - validated before save
- Volume source paths (array format) - validated before save
- Service names - validated before save
- Environment variable patterns - safe ${VAR} allowed, ${VAR:-$(cmd)} blocked
Testing:
- 60 security tests pass (176 assertions)
- PreSaveValidationTest.php: 15 tests for pre-save validation
- ValidateShellSafePathTest.php: 15 tests for core validation
- VolumeSecurityTest.php: 15 tests for volume parsing
- ServiceNameSecurityTest.php: 15 tests for service names
Related commits:
- Previous: Added validation during parse() phase
- This commit: Moves validation before database save
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
