feat(guest-image): derive the guest rootfs from the pinned agent OCI image - #1252
Merged
trunk-io[bot] merged 3 commits intoSep 17, 2026
Conversation
…image The guest userland was a second nix evaluation of the agent image's expressions under a different nixpkgs pin, so the microVM guest and the published container could drift silently. The rootfs now unpacks the published image itself: per-layer fixed-output fetches keyed by the locked descriptor digests, applied in manifest order with each layer's whiteouts resolved against the lower layers before its own content lands. A boot layer is overlaid on top and wins every conflict: guestd as /sbin/init, modprobe, /lib/modules from the root-pinned kernel, and a writable /etc/resolv.conf. Two build-time gates keep it honest. The eval re-checks the lock's shape, and the userland contract is verified on the assembled tree by resolving every path component inside it -- the way the kernel will after switch_root -- so a missing /bin/sh breaks the build instead of shipping a guest that cannot arm egress. Refs RIG-3787 Co-authored-by: Matt Wilkinson <matt@rigel.build>
|
Compass engineering docs preview: https://compass-managed-rig-3787-der.compass-eng-docs.pages.dev Deployed from |
…unpacking Review findings on the OCI-derived rootfs, each with a reproduced counterexample: - The lock's manifest digest constrained nothing: only the layer descriptors decided what was unpacked, so a lock pairing that digest with another image's valid layers built clean. The manifest is now fetched fixed-output against the digest (a manifest digest is the sha256 of its body) and its own ordered layer list must equal the lock's. - A lower layer's absolute symlink used as a parent path let tar write, and the whiteout `rm -rf` delete, outside the staged root. Such parents are dropped before extraction and a `..` member now breaks the build. - The userland contract passed on a non-executable or a directory; it now requires an executable regular file. - setuid/setgid files are asserted absent rather than assumed. The packed image is byte-identical: the published layers trip none of these. Refs RIG-3787
…arkers Correctness-lens findings on the OCI-derived rootfs, each reproduced first: - Extraction used --no-same-permissions, so every member took the builder's umask instead of its published mode: /bin packed 0755 where the image ships 0555. Extract with -p and re-open staging write access on directories only, then restore the directories' published modes before packing. Directory members are selected by tar's type flag, since these layers list directories with no trailing slash. - The whiteout scan matched the marker prefix anywhere in a path, so an ordinary directory whose name contained it became a delete order. Reproduced deleting a lower-layer file outright. The match is now anchored to a marker-prefixed final component. - An opaque marker naming a missing directory was swallowed by a redirect to /dev/null plus a true fallback; it now breaks the build, since it means the layer stack is not what this build thinks it is. - The image root's mode was a chmod side effect (0555 inherited from the boot layer's store path until a later pass happened to widen it); it is now set outright. - The store-path loop word-split an unquoted command substitution under nullglob, so a path containing a glob metacharacter was silently dropped, yielding an incomplete rootfs. It now reads line-wise. - The contract resolver counted path components, not symlink hops, so a deep symlink-free path could be reported as a loop. - The lock guard called builtins.match on a non-string tag, replacing the curated error with a raw nix type error. Not fixed, measured: mkfs.erofs drops the sticky bit on /tmp whatever the staged mode, so /tmp packs 0777 rather than 1777. The rootfs boots read-only, so the lower layer's mode is not what makes /tmp writable. Refs RIG-3787
rigel-mintaka
marked this pull request as ready for review
September 16, 2026 22:35
mattwilkinsonn
added this pull request to stack #1261
September 17, 2026 02:01
mattwilkinsonn
approved these changes
Sep 17, 2026
|
😎 Stack merged successfully - details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is part of a stack containing 3 PRs:
mainThe guest userland was a second nix evaluation of the agent image's
expressions under a different nixpkgs pin, so the microVM guest and the
published container could drift silently.
The rootfs now unpacks the published image itself: per-layer
fixed-output fetches keyed by the locked descriptor digests, applied in
manifest order with each layer's whiteouts resolved against the lower
layers before its own content lands. A boot layer is overlaid on top
and wins every conflict: guestd as /sbin/init, modprobe, /lib/modules
from the root-pinned kernel, and a writable /etc/resolv.conf.
Two build-time gates keep it honest. The eval re-checks the lock's
shape, and the userland contract is verified on the assembled tree by
resolving every path component inside it -- the way the kernel will
after switch_root -- so a missing /bin/sh breaks the build instead of
shipping a guest that cannot arm egress.
Refs RIG-3787
Co-authored-by: Matt Wilkinson matt@rigel.build