test(sync): stop watchAndSync tests flaking on slow CI runners - #94
Merged
Conversation
The watchAndSync block intermittently failed CI on unrelated dependency PRs (#81 Jun 30 Node 22, #91 and #93 Jul 27 Node 24), always as a 10s test timeout and never as an assertion, hitting a different test each time. Both #81 and #91 passed on plain re-run with no code change. Three changes, in order of importance: 1. jest.setTimeout(30_000) for the block. The suite runs in ~1.8s locally but was observed at 10.8-10.9s on GitHub runners with 232 suites competing for ~4 cores — i.e. sitting on the 10s default, which tipped an arbitrary test over each time. This is the headroom the block actually needed. 2. flushWatchSetup now waits for an observable readiness signal (mockWatcherOn having been called) instead of burning a fixed round count, then drains a settle budget. attachDebouncedWatchLoop calls watcher.on() in the same synchronous continuation as the process.on('SIGINT') registration, so listeners-attached implies the handler each test uses for shutdown is registered. A fixed count could under-wait; this cannot. On a genuine stall it throws with the pending state instead of timing out mutely. 3. Settle rounds 250 -> 25. Every test in the block passes with as few as 5 (measured), so 25 keeps a 5x margin. On method: the historical escalation was 20 -> 50 -> 250 rounds, each assuming the budget was too small. Suite duration is flat across 250/25/5 rounds, so widening it could never have worked. Four further hypotheses were tested and disproven — slow sweepStaleBackups I/O (its projectRoot '/test' does not exist, so readdir ENOENTs immediately), a hang in controller.shutdown() (microtask-only), expensive flush rounds (0.00ms per 250), and CPU contention (old code passed 12/12 under 16 spinners on 11 cores). Details in bead sync-94ua. Honest limitation: the flake could not be reproduced locally — ~58 runs across five contention profiles (CPU, I/O+CPU, full-suite, starved budget) produced one failure whose identity was lost. So this targets the signature the evidence supports rather than a reproduction, and the new diagnostic ensures the next occurrence identifies itself. Verified: 49 tests in the block pass; 20/20 under combined I/O and CPU contention; lint, type-check, and 5501 tests green on Node 24.18.0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stops the
watchAndSynctest block intermittently failing CI on unrelated PRs. Addresses beadsync-94ua.The problem
Four sightings, always a 10s test timeout, never an assertion, hitting a different test in the block each time:
config cache across watch ticks › includes the .deepl-sync.yaml config file itself…should log the number of watched patterns#81 and #91 both passed on plain re-run with no code change. Each occurrence costs a re-run and, worse, destroys signal — during a release you cannot distinguish a flake from a real regression.
Root cause
The block runs in ~1.8s locally but was observed at 10.8–10.9s on CI, with 232 suites competing for ~4 cores on a shared runner. It was sitting on jest's 10s default, so an arbitrary test tipped over each time. Runner CPU starvation, not a hang.
Changes Made
jest.setTimeout(30_000)for the block — the headroom it actually needs. These are the heaviest tests in the file, each driving a full watch lifecycle through fake timers.flushWatchSetupwaits on an observable readiness signal rather than a fixed round count:Sound because
attachDebouncedWatchLoopcallswatcher.on('change'/'add')in the same synchronous continuation asprocess.on('SIGINT', …)(sync-command.ts:394–420) — so listeners-attached implies the handler each test uses for shutdown is registered. A fixed count could under-wait; this cannot. Two-phase so all 21 call sites keep working: 14 are re-invocations after change events that need the drain, and readiness is already true for those.Settle rounds 250 → 25 — every test passes with as few as 5 (measured), so 25 keeps a 5× margin.
On method — why the previous fixes didn't stick
The in-file comment records an escalation of 20 → 50 → 250 rounds, each assuming the budget was too small. Suite duration is flat across 250/25/5 rounds (1.80s / 1.95s / 1.40s), so widening it could never have helped.
Four further hypotheses were tested and disproven:
sweepStaleBackupsI/O exhausts the budgetprojectRootis/test, which does not exist →readdirENOENTs immediatelycontroller.shutdown()close(), empty backup setTest Coverage
The block's 49 tests pass. The new diagnostic path is verified by sabotage — forcing readiness never to arrive yields:
instead of a mute timeout.
Verified on Node 24.18.0:
Honest limitation
The flake could not be reproduced locally. Roughly 58 runs across five contention profiles — CPU-only, I/O+CPU, full-suite, and a deliberately starved 5-round budget — produced exactly one failure, whose identity was lost. So this targets the signature the evidence supports rather than a reproduction, and I would not claim it is proven until CI has run it repeatedly.
That is precisely why change 2 matters beyond correctness: if the mechanism turns out to be a genuine stall rather than starvation, the next occurrence will say so explicitly instead of prompting a sixth guess.
Backward Compatibility
✅ No production code touched — test file and changelog only.
⚠️ A genuinely hung setup in this block now surfaces after 30s rather than 10s, but with a diagnostic rather than a bare timeout.
✅ All 21
flushWatchSetupcall sites unchanged and passing.❌ Breaking changes: none.
Size: Small ✓
One test file, one changelog entry. No source changes.