Noticed during the adversarial review of PR #1682. Generic CI-gating hole, not specific to that PR.
What
ci-success (.github/workflows/ci.yml) is the summary job every other job feeds into. It fails the build on failure and on cancelled:
if: always()
steps:
- run: |
if [[ "${{ contains(needs.*.result, 'failure') }}" == "true" ]]; then ... exit 1; fi
if [[ "${{ contains(needs.*.result, 'cancelled') }}" == "true" ]]; then ... exit 1; fi
echo "All CI checks passed!"
It does not test for skipped. A needed job that is skipped therefore reports as an overall pass.
Why it is latent rather than live
No job in needs currently carries an if: or a paths: filter, so nothing can be skipped today, and ci-success is accurate as things stand. The hole opens the first time any gating job gains a condition: from then on the gate silently stops covering it, and the failure mode is a green ci-success with a check that never ran. That is the same defect class as a guard asserting a property it does not inspect.
Fix direction
Add the third branch:
if [[ "${{ contains(needs.*.result, 'skipped') }}" == "true" ]]; then
echo "One or more required CI jobs were skipped"
exit 1
fi
If a job ever legitimately needs to be skippable, it should be excluded from needs deliberately, or use if: always() internally and report a neutral success, rather than relying on ci-success not looking.
Worth checking the other workflows for the same summary-job pattern while in there.
Noticed during the adversarial review of PR #1682. Generic CI-gating hole, not specific to that PR.
What
ci-success(.github/workflows/ci.yml) is the summary job every other job feeds into. It fails the build onfailureand oncancelled:It does not test for
skipped. A needed job that is skipped therefore reports as an overall pass.Why it is latent rather than live
No job in
needscurrently carries anif:or apaths:filter, so nothing can be skipped today, andci-successis accurate as things stand. The hole opens the first time any gating job gains a condition: from then on the gate silently stops covering it, and the failure mode is a greenci-successwith a check that never ran. That is the same defect class as a guard asserting a property it does not inspect.Fix direction
Add the third branch:
If a job ever legitimately needs to be skippable, it should be excluded from
needsdeliberately, or useif: always()internally and report a neutral success, rather than relying onci-successnot looking.Worth checking the other workflows for the same summary-job pattern while in there.