Surfaced by the PR #77 hardware round: 5/141 four-volume claims (3.5%) failed
with mount: /volumes/vol-X: /dev/vdY is not a valid block device → HTTP 500.
Zero failures in 112 single-volume claims. Reproduces on main (ce55029), so it
predates #77 — but concurrent bring-up makes multi-volume claims worth using,
so the race will be hit more.
Cause
engine/volume.go waitForVolumeDevice polls /sys/block/<dev>/serial and
returns /dev/<dev> as soon as the serial matches. The kernel publishes the
sysfs entry before devtmpfs creates the device node, so under load the guest
mount can run before /dev/<dev> exists (util-linux reports "not a valid
block device", exit 32).
Fix direction
Extend the wait predicate: after the serial matches, also confirm /dev/<dev>
exists (a silkd stat — the fs.stat verb already exists) before returning, under
the same poll interval and probe budget. Deterministic, no retry-semantics
change on the mount itself. Failed claims already clean up correctly (verified
on hardware: every failure settled to claimed==0 with warm back at target), so
the fix is purely about not failing spuriously.
Acceptance
- A multi-volume soak (≥100 four-volume claims on a multi-vCPU tier) shows zero
"not a valid block device" failures; before the fix the same soak reproduces
the ~3-4% rate.
- Single-volume latency unchanged (the extra stat rides the existing poll loop
only until the node appears).
Surfaced by the PR #77 hardware round: 5/141 four-volume claims (3.5%) failed
with
mount: /volumes/vol-X: /dev/vdY is not a valid block device→ HTTP 500.Zero failures in 112 single-volume claims. Reproduces on main (ce55029), so it
predates #77 — but concurrent bring-up makes multi-volume claims worth using,
so the race will be hit more.
Cause
engine/volume.gowaitForVolumeDevicepolls/sys/block/<dev>/serialandreturns
/dev/<dev>as soon as the serial matches. The kernel publishes thesysfs entry before devtmpfs creates the device node, so under load the guest
mountcan run before/dev/<dev>exists (util-linux reports "not a validblock device", exit 32).
Fix direction
Extend the wait predicate: after the serial matches, also confirm
/dev/<dev>exists (a silkd stat — the fs.stat verb already exists) before returning, under
the same poll interval and probe budget. Deterministic, no retry-semantics
change on the mount itself. Failed claims already clean up correctly (verified
on hardware: every failure settled to claimed==0 with warm back at target), so
the fix is purely about not failing spuriously.
Acceptance
"not a valid block device" failures; before the fix the same soak reproduces
the ~3-4% rate.
only until the node appears).