Skip to content

test(labs): isolate a test by naming it *.isolate.test.ts - #25559

Merged
charlielye merged 1 commit into
nextfrom
cl/isolate-suffix
Sep 30, 2026
Merged

charlielye merged 1 commit into
nextfrom
cl/isolate-suffix

Conversation

@charlielye

Copy link
Copy Markdown
Contributor

A test that binds a port has to run in its own container. Which tests those were was a list of package prefixes in labs' test_cmds:

# These need isolation due to network stack usage (p2p, anvil, etc).
if [[ "$test" =~ ^(prover-node|p2p|ethereum|aztec|prover-client/src/test|...) ]]; then
  prefix+=":ISOLATE=1:NAME=$test"

A list only works if whoever adds a test in a new package knows to extend it, and an unisolated test that spawns anvil passes until the day it races another one.

Two tests were already in that position

cli/src/cmds/l1/attester_exit, added on 18 September, fails before reaching any assertion:

Anvil exited with code 0 before listening.
Output: Error: Address already in use (os error 98)

three retries, all the same. It failed the x-full-no-test-cache run on #25362, a bb and bb.js migration it has nothing to do with. no-test-cache is what surfaced it: every test executes in the same window rather than most being served from cache, which is exactly when two unisolated anvils meet.

The two it lost to were ethereum/src/test/blob_kzg_warmup, isolated because ethereum is listed, and epoch-cache/src/epoch_cache.integration, which is not listed and has been unisolated since April. Neither is at fault; cli and epoch-cache were simply never on the list.

The change

Name a test *.isolate.test.ts and it gets ISOLATE=1, wherever it lives. The two above are renamed accordingly.

The package list stays, so nothing loses the isolation it has today. Prefer the suffix for anything new; an entry can leave the list later by renaming its tests. The point is that the requirement is stated where someone writes the test, instead of somewhere else they have to remember.

Not a default-port change: fixed ports with full isolation is the established preference here, and ephemeral ports have brought their own flakiness before.

Verified

  • The condition isolates both renamed tests, leaves ordinary tests shared, and keeps every currently listed package isolated.
  • The glob still enumerates both renamed files, 649 tests in all, and .isolate.test.ts is not caught by the .bench.test.ts skip.
  • bootstrap.sh parses with extglob and globstar, which is how ci3 runs it. Plain bash -n reports a syntax error on the !(...) glob both before and after this change.

Landing

Rides as a labs patch, labs-patches/0009, so CI gets it without waiting on a pin bump. It should be upstreamed to aztec-node and the patch dropped once the pin passes it. Worth a look from whoever owns cli, since it is their test failing other people's PRs.

A test that binds a port has to run in its own container, and which tests those were was
a list of package prefixes in labs' test_cmds. A list only works if whoever adds a test
in a new package knows to extend it, and an unisolated test that spawns anvil passes
until the day it races another one.

Two were already in that position. cli/src/cmds/l1/attester_exit, added this month,
fails with "Address already in use (os error 98)" before reaching an assertion when it
meets another anvil in the same window. It failed the x-full run on #25362, a bb and
bb.js migration it has nothing to do with; the two anvils it lost to were
ethereum/blob_kzg_warmup, which is isolated because ethereum is listed, and
epoch-cache/epoch_cache.integration, which is not listed and has been unisolated since
April.

The suffix says it where someone writes the test rather than somewhere else that has to
be remembered. The package list stays, so nothing loses the isolation it has today.

Rides as a labs patch; upstream to aztec-node separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuUzj3qpkpJor6GMpWB4T4
@charlielye
charlielye merged commit 1062206 into next Sep 30, 2026
13 of 15 checks passed
@charlielye
charlielye deleted the cl/isolate-suffix branch September 30, 2026 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant