Fix macOS e2e unpack Broken pipe evicting queue entries

[Mikola Lysenko (mikolalysenko)]

Problem

#1179 (merged 00:13 UTC) made every e2e leg unpack its binaries with

zstd -q -d -c target/e2e-archive/e2e-bin.tar.zst | tar -xf - -C target/e2e-bin

under set -o pipefail. Ten minutes later the first merge_group run built on it went red. In run 37863817362, e2e-macos (macos-latest, e2e_vex_build, pdm:: --ignored, 2.29.2) failed in "Unpack the e2e binaries" with only:

zstd: /*stdout*\: Broken pipe
##[error]Process completed with exit code 1.

tar reported no error. The fail-fast watcher cancelled the run and the queue evicted #1152 (00:25:58). This step runs in every e2e, e2e-macos, e2e-full and cargo-vex leg (dozens per run), so it is a per-leg race on every queue entry.

Root cause

A tar that stops reading at the end-of-archive marker and exits without draining stdin leaves zstd writing the rest of the stream (block padding) into a closed pipe. zstd fails with EPIPE, and pipefail makes the step fail even though every file was already extracted. Whether it fires depends on timing: how much zstd still has to write when tar exits. Apple's libarchive-based tar does this. GNU tar does the same locally:

$ (cat small.tar; head -c 2000000 /dev/zero) | zstd -q -o trail.tar.zst
$ bash -e -o pipefail -c 'zstd -q -d -c trail.tar.zst | tar -xf - -C p1'; echo $?
141   # zstd killed by SIGPIPE
$ bash -e -o pipefail -c 'zstd -q -d -f trail.tar.zst -o p2.tar && tar -xf p2.tar -C p2'; echo $?
0

Fix

Both "Unpack the e2e binaries" sites (the &e2e-steps anchor, shared by e2e, e2e-macos and e2e-full, plus the &cargo-vex-steps anchor) now decompress to target/e2e-bin.tar, extract that file, and delete it. No pipe means no reader can close it early. The extra cost is one temporary write of the uncompressed bundle, which is deleted right away.

Packing (tar -cf - . | zstd ...) is unchanged: zstd reads its stdin to EOF, so that side has no early-close race.

Proof

• New test_consumer_survives_a_tar_that_stops_at_the_end_marker in scripts/tests/test_ci_e2e_archive.py runs each consumer's real commands (pulled from ci.yml) against an archive with bytes after the end marker. It fails on origin/main's ci.yml and passes with this change.
• python3 -B -m unittest discover -s scripts/tests: 281 tests OK.
• zizmor --offline on ci.yml finds the same 44 issues (24 suppressed) as on main. YAML parses.

Where tests run

Nothing is removed or moved. All legs run where they did before. Required checks (ci-ok, clippy) are unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QNubt2ZznwzckQai6WxCLa

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

bugbot run

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c4d96d7. Configure here.}