Fix S3 carve cleanup never running and panic on empty carves (#43045)

[juan-fdz-hawa]

Related issue: Resolves #43045

Fixed a bug where the carve cleanup cron job called the MySQL implementation instead of the S3-aware implementation on S3-configured deployments, meaning expired carves were never marked as expired in S3. Also fixed a panic in S3 carve cleanup that occurred when there were no non-expired carves.

Checklist for submitter

If some of the following don't apply, delete the relevant line.

• Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
  See Changes files for more information.

Testing

• Added/updated automated tests
• QA'd all new/changed functionality manually

[qodo-free-for-open-source-projects]

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: aggregate-result

Failed stage: Check for failures [❌]

Failed test name: main-mysql8.0.44, integration-mdm-mysql8.0.44

Failure summary: The action failed because the workflow’s status aggregation step detected failed test jobs and exited with code 1. - The main-mysql8.0.44 job wrote fail to ./main-mysql8.0.44-status/status. - The integration-mdm-mysql8.0.44 job wrote fail to ./integration-mdm-mysql8.0.44-status/status. The script collects any status files containing fail, prints ❌ One or more test jobs failed: main-mysql8.0.44, integration-mdm-mysql8.0.44, and then terminates the action.

Relevant error logs: 1: ##[group]Runner Image Provisioner 2: Hosted Compute Agent ... 119: Artifact download completed successfully. 120: Extracting artifact entry: /home/runner/work/fleet/fleet/main-mysql8.0.44-status/status 121: Artifact download completed successfully. 122: Extracting artifact entry: /home/runner/work/fleet/fleet/scripts-status/status 123: Artifact download completed successfully. 124: Extracting artifact entry: /home/runner/work/fleet/fleet/fast-status/status 125: Artifact download completed successfully. 126: Extracting artifact entry: /home/runner/work/fleet/fleet/vuln-mysql8.0.44-status/status 127: Artifact download completed successfully. 128: Extracting artifact entry: /home/runner/work/fleet/fleet/service-mysql8.0.44-status/status 129: Artifact download completed successfully. 130: Extracting artifact entry: /home/runner/work/fleet/fleet/integration-core-mysql8.0.44-status/status 131: Artifact download completed successfully. 132: Total of 10 artifact(s) downloaded 133: Download artifact has finished successfully 134: ##[group]Run failed_tests="" 135: �[36;1mfailed_tests=""�[0m 136: �[36;1mstatus_count=0�[0m 137: �[36;1m# Find all status files (they are in directories like 'fleetctl-mysql8.0.44-status/status')�[0m 138: �[36;1mfor status_file in $(find ./ -type f -name 'status'); do�[0m 139: �[36;1m status_count=$((status_count + 1))�[0m 140: �[36;1m # Extract test name from parent directory (e.g., 'fleetctl-mysql8.0.44-status')�[0m 141: �[36;1m test_dir=$(basename $(dirname "$status_file"))�[0m 142: �[36;1m # Remove '-status' suffix to get the test name�[0m 143: �[36;1m test_name="${test_dir%-status}"�[0m 144: �[36;1m status_content=$(cat "$status_file")�[0m 145: �[36;1m echo "Processing: $status_file (Test: $test_name) with status content: $status_content"�[0m 146: �[36;1m if grep -q "fail" "$status_file"; then�[0m 147: �[36;1m echo " ❌ Test failed: $test_name"�[0m 148: �[36;1m failed_tests="${failed_tests}${test_name}, "�[0m 149: �[36;1m else�[0m 150: �[36;1m echo " ✅ Test passed: $test_name"�[0m 151: �[36;1m fi�[0m 152: �[36;1mdone�[0m 153: �[36;1mif [[ $status_count -eq 0 ]]; then�[0m 154: �[36;1m echo "❌ ERROR: No status files found! This indicates a workflow issue."�[0m 155: �[36;1m exit 1�[0m 156: �[36;1mfi�[0m 157: �[36;1mif [[ -n "$failed_tests" ]]; then�[0m 158: �[36;1m echo "❌ One or more test jobs failed: ${failed_tests%, }"�[0m 159: �[36;1m exit 1�[0m 160: �[36;1mfi�[0m 161: �[36;1mecho "✅ All test jobs succeeded."�[0m 162: shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 163: ##[endgroup] 164: Processing: ./integration-core-mysql8.0.44-status/status (Test: integration-core-mysql8.0.44) with status content: success 165: ✅ Test passed: integration-core-mysql8.0.44 166: Processing: ./vuln-mysql8.0.44-status/status (Test: vuln-mysql8.0.44) with status content: success 167: ✅ Test passed: vuln-mysql8.0.44 168: Processing: ./service-mysql8.0.44-status/status (Test: service-mysql8.0.44) with status content: success 169: ✅ Test passed: service-mysql8.0.44 170: Processing: ./main-mysql8.0.44-status/status (Test: main-mysql8.0.44) with status content: fail 171: ❌ Test failed: main-mysql8.0.44 172: Processing: ./fleetctl-mysql8.0.44-status/status (Test: fleetctl-mysql8.0.44) with status content: success 173: ✅ Test passed: fleetctl-mysql8.0.44 174: Processing: ./integration-enterprise-mysql8.0.44-status/status (Test: integration-enterprise-mysql8.0.44) with status content: success 175: ✅ Test passed: integration-enterprise-mysql8.0.44 176: Processing: ./fast-status/status (Test: fast) with status content: success 177: ✅ Test passed: fast 178: Processing: ./mysql-mysql8.0.44-status/status (Test: mysql-mysql8.0.44) with status content: success 179: ✅ Test passed: mysql-mysql8.0.44 180: Processing: ./integration-mdm-mysql8.0.44-status/status (Test: integration-mdm-mysql8.0.44) with status content: fail 181: ❌ Test failed: integration-mdm-mysql8.0.44 182: Processing: ./scripts-status/status (Test: scripts) with status content: success 183: ✅ Test passed: scripts 184: ❌ One or more test jobs failed: main-mysql8.0.44, integration-mdm-mysql8.0.44 185: ##[error]Process completed with exit code 1. 186: Post job cleanup.

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.81%. Comparing base (33b77fb) to head (5c26d4a).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
server/datastore/s3/carves.go	57.14%	2 Missing and 1 partial ⚠️
cmd/fleet/serve.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #46462      +/-   ##
==========================================
+ Coverage   66.78%   66.81%   +0.02%     
==========================================
  Files        2803     2803              
  Lines      223565   223569       +4     
  Branches    11345    11345              
==========================================
+ Hits       149309   149377      +68     
+ Misses      60694    60616      -78     
- Partials    13562    13576      +14     

Flag	Coverage Δ
backend	68.55% <66.66%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.