Let a red E2E leg block the run - #109
Merged
Merged
Conversation
continue-on-error hid a real failure on main for three merges: the run was reported success while the Ubuntu leg was red. A job whose failure reads as a pass is a false signal, not a weak one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #108.
continue-on-error: truestood ontest-e2eto buy the suite time to proveitself, on the reasoning that a flaky job everybody ignores is worth less than
no job. That reasoning has a worse failure mode than the one it avoids, and it
happened: on
mainate4ed5d9the Ubuntu leg failed on a real bug — a notekeeping its title and losing the body typed after it — and the run was reported
success.gh run listsaid success, the badge said passing, and the failuresat there across three merges until it was looked for by hand.
A job whose failure reads as a pass is not a weak signal, it is a false one.
The flag is gone. The price it was paying — a genuinely flaky run blocking a
merge until it is re-run — is the one worth paying.
The suite earned it: five consecutive green runs on #106 (ten E2E jobs across
both platforms, no failure and no cancellation), then green again on
mainafter the merge.
The decision and its reason are written down in
ci.ymlbeside the job and indocs/architecture.mdwhere the suite is described, as the ticket asked.Checked locally
npm test960 passing,npm run lintclean,npm run test:e2e15/15 in 3:52.🤖 Generated with Claude Code