fix(nodeswap): run the staged binary's own node-swap, and cross-drive/cross-fs installs - #481
Merged
Merged
Conversation
…/cross-fs installs The 2026-09-24 c23b5c6 rollout's own first real use found two defects in the swap tooling itself: 1. Both launchers (setup/linux-node-swap-launch.sh, setup/windows-node-swap-launch.ps1) defaulted the exe that RUNS `node-swap` to the currently-INSTALLED binary, not the newly-staged one. A fix landing inside node-swap's own engine code could therefore never take effect on the deploy that staged it: two Linux nodes hit exactly this, running the OLD deps_other.go stub via the old installed binary even though the staged build already had the fix. Both launchers now default the runner to the STAGED binary, hash-verified by the launcher itself before it is ever executed, falling back to the installed binary (with a clear log line) only when the staged build provably lacks the node-swap subcommand. Support detection uses a narrow exit-code allowlist (0 or 1 only, never a broad "anything but 2") and the PowerShell probe is wrapped in try/catch, so a corrupt/wrong-arch/non-PE staged file is never misread as safe to run and never aborts the launcher outright — both gaps found by code review and pinned with new self-test cases (a real non-PE file for the PowerShell CreateProcess-failure path, signal-death exit codes for the bash path). 2. node-swap's install-new step used plain os.Rename, which fails outright when the staged file and the target live on different volumes/ filesystems (Windows: a different drive letter; Linux/macOS: EXDEV) — hit for real staging at C:\tmp\ against a D:\ target: the tool rolled back safely, but the swap never happened. installNewBinary now falls back to copying the staged binary into the target's own directory, re-verifying its sha256 before trusting it, and renaming from there (same-device, so the fast path applies); the temp file is cleaned up on any failure along that path. Both launchers' runner-selection logic (resolve_runner_exe / Resolve-RunnerExe) is now independently unit-testable with fakes: the bash launcher's body moved into main(), only run when executed directly, so setup/linux-node-swap-launch.tests.sh (new, wired into CI) can source it; the PowerShell launcher's existing -SelfTest convention gained matching assertions. Tests: Go unit tests with fakes for the cross-device fallback (happy path, copy failure, corruption caught before install, ordinary failures never taking the fallback, nil-hooks degrading safely) plus real isCrossDeviceRenameErr/copyFile coverage (synthetic EXDEV, synthetic and a real live cross-drive rename on Windows); bash and PowerShell self-tests for runner selection and support detection. Every new guard broken once (red) and restored (green) by hand during review. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two node-swap defects found by
c23b5c6f's own first real production rollout (2026-09-24), fixed independently:setup/linux-node-swap-launch.sh,setup/windows-node-swap-launch.ps1): both detached launchers defaulted the exe that runsnode-swapto the currently-installed binary, not the newly-staged one. A fix landing inside node-swap's own engine code could therefore never take effect on the very deploy that staged it — this is what actually happened to the Lenovo/binxarn nodes in the rollout (they ran the OLD, still-buggydeps_other.gologic via the old installed binary, even though the staged build already had it fixed; the rollout doc's own diagnosis of that failure — "gateFindProcessesByExeonruntime.GOOS" — was therefore wrong, sincedeps_other.goalready returnsnil, nilon non-Windows inc23b5c6f). Both launchers now default the runner to the staged binary, hash-verified by the launcher itself before it is ever executed, with--runner-exe/-RunnerExestill available as an explicit override, and a fallback to the installed binary (logged clearly) only when the staged build provably does not supportnode-swapat all.internal/nodeswap): the install-new step used plainos.Rename, which fails outright when the staged file and the target live on different volumes (Windows: a different drive letter; Linux:EXDEV) — hit for real on the Aorus node (staged atC:\tmp\against aD:\target): it rolled back safely, but the swap never happened.installNewBinarynow falls back to copying the staged binary into the target's own directory, re-verifying its sha256 before trusting it, and renaming from there (now same-device); the temp file is cleaned up on any failure.Both launchers' runner-selection logic is refactored to be independently unit-testable with fakes (the bash launcher's body moved into
main(), only run when executed directly, so a new self-test cansourceit; the PowerShell launcher's existing-SelfTestconvention gained matching assertions).Review findings addressed
Two independent reviews ran on this diff (
offload_review_diffon a free local seat, and a pinned Sonnetpr-review-toolkit:code-reviewer):rc != 2= "supported"), which would misread a crashed/corrupt/wrong-architecture staged binary as safe to run. Tightened to a narrow allowlist (exit 0 or 1 only) in both launchers, with new self-test cases covering signal-death-shaped exit codes (bash) and a real crash exit code (PowerShell).Test-NodeSwapSupport) had notry/catch: a genuinely non-PE staged file (corrupt/wrong-arch) makesCreateProcessfail before$LASTEXITCODEis ever set, which PowerShell surfaces as a terminatingApplicationFailedException— uncaught under the script's$ErrorActionPreference = 'Stop', this aborted the whole launcher instead of falling back to the installed binary. Fixed withtry/catch, and pinned with a new self-test using an actual text-file-renamed-.exe(the only way to exercise the realCreateProcess-level failure, which the.cmdexit-code tests can't reach).-eq/-neare case-insensitive by default, confirmed live.--backup-suffixcollision on the temp-file naming (pre-existing pattern, not introduced by this change).How tested
go build/vet/test ./...— all green except one pre-existing, unrelated, timing-flaky test ininternal/gpuactivity(confirmed pre-existing: passes 5/5 in isolation; package untouched by this diff).bash setup/linux-node-swap-launch.tests.shandpowershell -File setup/windows-node-swap-launch.tests.ps1— both green, now wired into this PR's CI (buildjob gained one new step for the bash self-test;installer-windowsalready ran the PowerShell one).docs/systems/node-swap.mdupdated to document both fixes and the new invariants.Risk
Low. Both fixes are additive fallback paths (same-device rename and a build that already supports node-swap both take the exact same fast path as before); the only behavior change on the happy path is which binary a launcher runs
node-swapfrom, which is now hash-verified before execution either way.🤖 Generated with Claude Code