Skip to content

Take the history deadline tests off the clock - #96

Merged
portdeveloper merged 1 commit into
portdeveloper:mainfrom
BeeHiveTeam:fix/history-fixture-timing
Sep 17, 2026
Merged

portdeveloper merged 1 commit into
portdeveloper:mainfrom
BeeHiveTeam:fix/history-fixture-timing

Conversation

@BeeHiveTeam

@BeeHiveTeam BeeHiveTeam commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Closes #95

The two partial-history tests needed a real loopback response to beat the fixture deadline. On the Windows image that race went red: both requests exceeded 150 ms, the healthy one included, so the assertion received an AggregateError instead of a partial history.

What I measured first

A probe workflow on my fork, actions/setup-node@v4 node 22, current main, loopback round-trip:

environment first request the next four
windows-latest 23.5 ms 2.3, 3.3, 11.3, 15.4 ms
ubuntu-latest 21.2 ms 2.1, 3.1, 3.3, 3.4 ms
Windows 11 laptop 14.4 ms 2.7, 7.8, 14.2, 15.2 ms
Linux dev box 8.7 ms 1.4, 2.1, 2.0, 1.8 ms

Twenty suite runs on each runner image and twenty on the laptop produced no failure, so the rate is under 1 in 20 and I could not reproduce it. 23 ms is also nowhere near 150 ms, so cold start alone does not explain a 455 ms failure. That left no number I could validate — which is the argument for removing the dependency rather than tuning the constant.

What changed

Only a stalled endpoint goes over a real connection now. Cancellation is the one thing a fetch double cannot demonstrate, because a double that ignores signal is indistinguishable from one that honours it; an answered endpoint has nothing to prove that way, so it is a plain double and never touches the network. No assertion waits on a real response beating a clock, on any OS.

The new test asks for "without waiting" the way the event loop defines it rather than the way a stopwatch does: an answer that is already known settles on the microtask queue and beats a macrotask, while one that subscribes to a close already past loses to it. No wall clock, so there is nothing in it to be flaky about either.

wasCancelled subscribed to a close that may already have fired — measured at false after 1000 ms, 1008 ms and 1014 ms on Linux, on a Windows 11 laptop and on both runner images. It reads the settled state before waiting now, and a new test pins that against a socket closed before the check.

The deadline the stall tests pass is named and justified rather than guessed. Only the stalled request's connection setup has to fit inside it: the suite passes down to 10 ms locally, and 500 ms is twenty times the slowest first request seen on the runner images. settlesWithin's outer bound stays absolute, so a missing production deadline still fails promptly instead of hanging.

Production is untouched — no src/ change, and the 10 s default stands.

Both stall tests now assert the stalled request reached the fixture before indexing its socket. Without that, an unreached fixture failed as Cannot read properties of undefined (reading 'destroyed'), which points away from the cause on the one platform where these tests have actually been unstable.

Tests

345, up from 344. 30 consecutive local runs, no failure.

Mutation-checked, each guard reverted on its own with a no-op edit as a control:

reverted result
signal not passed to fetch 3 red
clearTimeout removed 1 red
both-reason message → results[0].reason 1 red
timeoutMs fallback removed 1 red
deadline ends at the headers 2 red
socket.destroyed check removed 1 red
control: no-op edit 345 pass

The timeoutMs fallback is worth a note: moving the healthy endpoint to a double silently dropped that mutation's coverage, because an instant response resolves before an abort could reach it. That one test therefore keeps a real connection — and it is not a race, since the passing path has the full 10 s default while the broken path aborts at once regardless of speed.

Verified after the change

This branch, twenty runs of the full suite in each environment:

environment runs with a failure
windows-latest 0 of 20
ubuntu-latest 0 of 20
Windows 11 laptop 0 of 20

That is not proof the original flake is gone — it never reproduced in sixty runs before the change either. It does show the change costs nothing in stability, and the mechanism that failed is no longer present.

On verification

I cannot reproduce the original failure, so I can't claim to have observed it fixed. What I can say is narrower and checkable: the assertions no longer depend on a real response arriving within a deadline, which is the mechanism that failed. The matrix on this PR is the gate, and one green pass is not proof I'd offer — happy to re-run it a few times before you merge.

The two partial-history tests needed a real loopback response to beat the fixture
deadline. On the Windows CI image that race went red: both requests exceeded 150 ms,
including the healthy one, so the assertion got an AggregateError instead of a
partial history.

Only a stalled endpoint needs a real connection, because cancellation is the one
thing a fetch double cannot demonstrate — a double that ignores the signal looks
exactly like one that honours it. An answered endpoint has nothing to prove that
way, so it is now a plain double and never touches the network. Nothing waits on a
real response beating a clock.

wasCancelled subscribed to a close event that may already have fired, which reported
a cancelled request as uncancelled after burning the full wait. It now reads the
settled state first, and a test pins that against a socket closed before the check.

The deadline the stall tests pass is named and justified rather than guessed: the
suite passes down to 10 ms here, and 500 ms is also twenty times the slowest first
request measured on the CI runner images.

@portdeveloper portdeveloper left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, thanks

@portdeveloper
portdeveloper merged commit ab89591 into portdeveloper:main Sep 17, 2026
3 checks passed
ColinkaMir added a commit to ColinkaMir/nad-agent that referenced this pull request Sep 17, 2026
Review on portdeveloper#97: the cancellation observer repeated the race portdeveloper#95 tracks. It only
subscribed to close, so a socket already destroyed when the check ran was
reported as uncancelled, and only after the full wait.

Same three fixes portdeveloper#96 made for history, applied here:

- wasCancelled reads the settled state before it waits, so an already-closed
  socket answers true on the microtask queue.
- A regression covers exactly that: destroy a socket, wait for its close, then
  assert the observer answers true and answers without waiting. Removing the
  settled-state check turns this test red.
- The stall deadline moves from 120ms to 500ms and the outer bound to an absolute
  4000ms. 120ms could expire while a loaded runner was still opening the
  connection, which leaves no socket to assert on; the independent outer timeout
  still turns a missing production deadline into a prompt failure.
- Both stall cases assert the request reached the fixture before indexing the
  socket, so an unreached fixture reads as itself instead of as a TypeError.
portdeveloper pushed a commit that referenced this pull request Sep 17, 2026
Review on #97: the cancellation observer repeated the race #95 tracks. It only
subscribed to close, so a socket already destroyed when the check ran was
reported as uncancelled, and only after the full wait.

Same three fixes #96 made for history, applied here:

- wasCancelled reads the settled state before it waits, so an already-closed
  socket answers true on the microtask queue.
- A regression covers exactly that: destroy a socket, wait for its close, then
  assert the observer answers true and answers without waiting. Removing the
  settled-state check turns this test red.
- The stall deadline moves from 120ms to 500ms and the outer bound to an absolute
  4000ms. 120ms could expire while a loaded runner was still opening the
  connection, which leaves no socket to assert on; the independent outer timeout
  still turns a missing production deadline into a prompt failure.
- Both stall cases assert the request reached the fixture before indexing the
  socket, so an unreached fixture reads as itself instead of as a TypeError.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make the history timeout fixture reliable on Windows

2 participants