diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a10236f..f019c06 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -223,13 +223,19 @@ jobs: # keeps them apart. It is the only runner able to disprove a path — it has already caught a # `preferences.json` looked for in the wrong place. # - # `continue-on-error` on purpose: a flaky E2E job everyone ignores is worth less than no job - # at all. To be removed once it has proven itself over a few dozen runs. + # ⚠️ Blocking, and it was not always. `continue-on-error: true` stood here to buy the suite + # time to prove itself, and the bill came due: on `main` at `e4ed5d9` the Ubuntu leg failed + # on a real bug — a note keeping its title and losing the body typed after it — and the run + # was reported `success` all the same. `gh run list` said success, the badge said passing, + # and the failure sat there across three merges until someone went looking for it by hand. + # + # A job whose failure reads as a pass is worse than no job: it is a claim that nothing is + # wrong. So the flag is gone. The cost is the one it was avoiding — a genuinely flaky run + # blocks a merge until it is re-run — and that is the right way round. test-e2e: name: Test E2E (${{ matrix.os }}) runs-on: ${{ matrix.os }} timeout-minutes: 45 - continue-on-error: true strategy: fail-fast: false matrix: diff --git a/docs/architecture.md b/docs/architecture.md index 2824766..24bf5a3 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -2227,9 +2227,18 @@ case the control is asserted on — it exists, it is labelled — and never clic #### In CI -The suite is a job of its own, on a matrix of `windows-latest` and `ubuntu-22.04`, kept -`continue-on-error` until it has proved itself — a flaky E2E job that everybody ignores is -worse than no job. The two platforms do not break the same way, and **Linux is the one that +The suite is a job of its own, on a matrix of `windows-latest` and `ubuntu-22.04`, and it +**blocks**. It did not always: `continue-on-error: true` stood on it while it earned its +keep, on the reasoning that a flaky E2E job everybody ignores is worse than no job. That +reasoning turned out to have a worse failure mode than the one it was avoiding. On `main` +at `e4ed5d9` the Ubuntu leg failed on a real bug — a note keeping its title and losing the +body typed after it — and the run was reported `success` regardless: `gh run list` said +success, the badge said passing, and the failure sat there across three merges until it was +looked for by hand. ⚠️ A job whose failure reads as a pass is not a weak signal, it is a +false one. The flag is gone, and the price it was paying — a flaky run blocking a merge +until it is re-run — is the one worth paying. + +The two platforms do not break the same way, and **Linux is the one that can tell paths apart**: the WebView is WebView2 on one and WebKitGTK on the other, and `dirs::data_dir()` and `dirs::config_dir()` are the same `%APPDATA%\` on Windows but `~/.local/share` against `~/.config` on Linux.