From 34940c9a2b32b611e8bc0e31da53c33a3bd8b22e Mon Sep 17 00:00:00 2001 From: Anastassios Nanos Date: Tue, 29 Sep 2026 22:34:19 +0300 Subject: [PATCH 1/4] feat(bundle): Take the amd64 guest kernel from hull-assets The amd64 kernel came from bunny's linux-kernel-cloud-hypervisor:latest, a moving tag on an image nothing signs, copied out with docker. arm64 already took its kernel from hull-assets, the artifact hull and brig boot from, so the two arches drew from different places. Both arches now take the kernel from hull-assets. The build resolves the version tag once, checks the signature on that digest with the bundled cosign against the identity brig checks (the build-assets.yml workflow in NOFireAI/hull-assets), and pulls that digest. pins.env names it as KERNEL_SOURCE, :@. KERNEL_IMAGE_AMD64 and the docker path are gone; ASSETS_SIGNER and ASSETS_ISSUER set the identity for a fork. On arm64 the kernel is the file hull boots on macOS. On amd64 under cloud-hypervisor, a sandbox's agent answers about 0.25 s later than with bunny's kernel (20 boots each on two hosts). Refs: brig-sh/brig#234 Signed-off-by: Anastassios Nanos --- DESIGN.md | 7 ++-- README.md | 7 ++-- docs/variants.md | 44 ++++++++++-------------- scripts/build-bundle.sh | 74 ++++++++++++++++------------------------- 4 files changed, 52 insertions(+), 80 deletions(-) diff --git a/DESIGN.md b/DESIGN.md index acff765..4d6a91f 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -44,10 +44,9 @@ now done and is what `install.sh` and `scripts/build-bundle.sh` produce. protocol must match the urunc shim, so hull-assets' prebuilt initrd (hull's agent) is not usable; the build assembles a brig initrd from urunit + `urunit-agent` (from the same urunc commit as the shim) + busybox + urunc's - `container-init`. The kernel is still fetched, not built: on amd64 from - the bunny Cloud-Hypervisor kernel image - (`harbor.nbfc.io/nubificus/bunny/linux-kernel-cloud-hypervisor`), on arm64 from - `ghcr.io/nofireai/hull-assets`. + `container-init`. The kernel is still fetched, not built: from + `ghcr.io/nofireai/hull-assets` on both arches, by digest, once its cosign + signature checks out. Because urunc and the initrd ship in one tarball built together, their agent commit matches by construction. - **The installer's own job** is the host wiring the tarball cannot carry: create diff --git a/README.md b/README.md index 4241f4e..55d1caf 100644 --- a/README.md +++ b/README.md @@ -274,8 +274,7 @@ the three that do not: | urunc, containerd-shim-urunc-v2 | built from `urunc-dev/urunc` at commit `74dd0cc` (branch `feat/unchanged_containers-exec-fixes`) | CGO-static, built in a Go container | | urunit | built from `NOFireAI/urunit` at commit `71bfdee` (branch `urunit_agent`) | C-static; goes into the initrd | | container-initrd | built from the above | assembled for brig, not fetched | -| guest kernel (amd64) | `harbor.nbfc.io/nubificus/bunny/linux-kernel-cloud-hypervisor` | fetched; extracted from the bunny image's `/.boot/kernel` | -| guest kernel (arm64) | `ghcr.io/nofireai/hull-assets` | fetched; the same kernel hull and brig use | +| guest kernel | `ghcr.io/nofireai/hull-assets` | fetched by digest once its cosign signature checks out; the kernel hull and brig boot | | monitors, virtiofsd | `urunc-dev/monitors-build` | fetched | | containerd, runc, nerdctl, CNI | upstream releases | fetched, upstream checksums | | cosign | `sigstore/cosign` | fetched, pinned by sha256 | @@ -326,8 +325,8 @@ carries hull's agent. So `build-bundle.sh` builds a brig initrd with urunc's own `packaging/container-initrd` tooling, from `urunit`, a `urunit-agent` built from the same urunc commit as the shim, a static `busybox`, and urunc's `container-init`. The urunc binary and the initrd's agent ship in one tarball, so -they always match. Only the kernel is fetched rather than built — it is generic: -on amd64 from the bunny Cloud-Hypervisor kernel image, on arm64 from hull-assets. +they always match. Only the kernel is fetched rather than built. It is generic, +and comes from hull-assets on both arches, checked against its signature. Two properties of the initrd are checked when the build packs it and again when the runtime boots it, because each fails in a way that does not name itself: diff --git a/docs/variants.md b/docs/variants.md index 802787f..d165697 100644 --- a/docs/variants.md +++ b/docs/variants.md @@ -65,33 +65,23 @@ unpacks what the build produced. The two files a generic boot names are the guest kernel and the initrd, and they come from different places. -**The kernel** is fetched per architecture, and the two arches draw from -different sources: +**The kernel** is fetched from hull-assets on both arches, the OCI artifact hull +and brig already boot from, one tag per platform, pulled with the bundled `oras`: -- **amd64** takes the kernel from the bunny-built Cloud-Hypervisor kernel image, - a plain OCI image that carries the kernel at `/.boot/kernel`: - - ``` - harbor.nbfc.io/nubificus/bunny/linux-kernel-cloud-hypervisor:latest - ``` - - The image is scratch-style (no shell), so `build-bundle.sh` copies the kernel - out of a throwaway container with `docker create` + `docker cp` rather than - `oras`. The default is overridable with `KERNEL_IMAGE_AMD64`; set it empty to - fall back to the hull-assets path below. - -- **arm64** takes the kernel from hull-assets, the OCI artifact hull and brig - already use, one tag per platform, pulled with the bundled `oras`: - - ``` - ghcr.io/nofireai/hull-assets:-linux- immutable - ghcr.io/nofireai/hull-assets:linux- moving - ``` +``` +ghcr.io/nofireai/hull-assets:-linux- immutable +ghcr.io/nofireai/hull-assets:linux- moving +``` - The artifact's layers are the files themselves, each named by its - `org.opencontainers.image.title`. +The build resolves the `` tag once. The bundled `cosign` checks the +signature on the digest it names, against the identity brig checks: the +`build-assets.yml` workflow in `NOFireAI/hull-assets`. The build then pulls that +digest. The artifact's layers are the files themselves, each named by its +`org.opencontainers.image.title`, so the kernel's sha256 is its layer digest in +the signed manifest. -Either way the kernel is generic and not brig-specific. +The kernel is generic and not brig-specific. On arm64 it is the same file hull +boots on macOS. **The initrd is not taken from hull-assets.** brig execs into a guest through an in-guest agent, `urunit-agent`, whose wire protocol (`pkg/agentproto`) is a @@ -176,9 +166,9 @@ refuses a stock binary, rather than printing a table of zeroes. assets optional? A separate build keeps a stock tarball small. 2. Where do the Option-C patches live once they are pushed? A branch is enough to build from; upstream is better. -3. Settled: the guest kernel is fetched — on amd64 from the bunny - Cloud-Hypervisor kernel image, on arm64 from `hull-assets` by the same tags - hull uses; the runtime is built from `urunc-dev/urunc` at a pinned commit +3. Settled: the guest kernel is fetched from `hull-assets`, by the same tags hull + uses, and checked against its signature; the runtime is built from + `urunc-dev/urunc` at a pinned commit (`74dd0cc`, on `feat/unchanged_containers-exec-fixes`); and the initrd is built for brig from `NOFireAI/urunit` at a pinned commit (`71bfdee`, on `urunit_agent`) plus urunc's own `packaging/container-initrd`, so its agent diff --git a/scripts/build-bundle.sh b/scripts/build-bundle.sh index 09504e9..02b5156 100755 --- a/scripts/build-bundle.sh +++ b/scripts/build-bundle.sh @@ -161,11 +161,11 @@ MONITORS="${MONITORS:-firecracker cloud-hypervisor solo5-hvt solo5-spt}" ASSETS_REGISTRY="${ASSETS_REGISTRY:-ghcr.io}" ASSETS_REPO="${ASSETS_REPO:-nofireai/hull-assets}" ASSETS_VERSION="${ASSETS_VERSION:-0.1.4}" -# amd64 kernel: a bunny-built Cloud-Hypervisor kernel image (a plain OCI image -# carrying the kernel at /.boot/kernel), extracted with docker like busybox is. -# Empty falls back to the hull-assets oras pull. arm64 has no such image, so it -# always takes the hull-assets path below. -KERNEL_IMAGE_AMD64="${KERNEL_IMAGE_AMD64:-harbor.nbfc.io/nubificus/bunny/linux-kernel-cloud-hypervisor:latest}" +# Who signs hull-assets: the identity brig checks before it boots a kernel from +# there. The build checks the same one before it packs the kernel. +ASSETS_SIGNER_DEFAULT='^https://github\.com/NOFireAI/hull-assets/\.github/workflows/build-assets\.yml@refs/heads/main$' +ASSETS_SIGNER="${ASSETS_SIGNER:-$ASSETS_SIGNER_DEFAULT}" +ASSETS_ISSUER="${ASSETS_ISSUER:-https://token.actions.githubusercontent.com}" VIRTIOFSD="${VIRTIOFSD:-true}" # Fixed install layout, baked into the config files the bundle carries. install.sh @@ -1142,51 +1142,35 @@ fi chmod 0755 "$STAGE"/bin/* # ---- guest boot assets (generic-boot / introspection) ------------------------ -# The kernel on amd64 comes from KERNEL_IMAGE_AMD64 (a bunny-built OCI image with -# the kernel at /.boot/kernel), extracted with docker; on arm64 (or if that is -# unset) it comes from hull-assets, pulled with the oras we just bundled. The -# initrd never does: hull-assets' initrd carries hull's agent, so it is thrown -# away and the brig-built one above takes its place. A fresh bundle.json records -# the urunc the agent was built against, so the installer can check coherence. +# The kernel comes from hull-assets on both arches, pulled with the oras we just +# bundled. The tag is resolved once, the cosign we just bundled checks the +# signature on the digest it names, and that digest is what is pulled, so the +# kernel is the one hull-assets signed. The initrd is not taken from there: +# hull-assets' initrd carries hull's agent, so it is thrown away and the +# brig-built one above takes its place. A fresh bundle.json records the urunc +# the agent was built against, so the installer can check coherence. ASSETS_URUNC_REF="" KERNEL_SOURCE="" KERNEL_FROM_ASSETS=false if [ "$VARIANT" != "stock" ]; then case "$ARCH" in amd64) kernel=bzImage ;; arm64) kernel=Image ;; esac - if [ "$ARCH" = "amd64" ] && [ -n "$KERNEL_IMAGE_AMD64" ]; then - info "extracting the guest kernel from $KERNEL_IMAGE_AMD64 (linux/amd64)" - # Pull explicitly with retries: the registry sometimes resets the - # connection mid-handshake, and one reset should not kill the build. - kpull="" - katt=1 - while [ "$katt" -le 5 ]; do - if docker pull --platform linux/amd64 "$KERNEL_IMAGE_AMD64" >/dev/null 2>&1; then - kpull=ok; break - fi - info " pull attempt $katt/5 failed, retrying in $((katt * 3))s" - sleep $((katt * 3)) - katt=$((katt + 1)) - done - [ "$kpull" = ok ] || fatal "could not pull $KERNEL_IMAGE_AMD64 after 5 attempts" - # A scratch-style image (no shell), so copy the kernel out of a throwaway - # container instead of exec'ing cat inside it. The dummy command is never - # run; docker create just needs one, and docker cp works on a created - # container. - kcid="$(docker create --platform linux/amd64 "$KERNEL_IMAGE_AMD64" /nonexistent)" \ - || fatal "could not create a container from $KERNEL_IMAGE_AMD64" - docker cp "$kcid:/.boot/kernel" "$STAGE/share/guest/$kernel"; kcp=$? - docker rm "$kcid" >/dev/null 2>&1 || true - [ "$kcp" -eq 0 ] || fatal "could not copy /.boot/kernel from $KERNEL_IMAGE_AMD64" - KERNEL_SOURCE="$KERNEL_IMAGE_AMD64" - else - a_tag="$ASSETS_VERSION-linux-$ARCH" - info "fetching the guest kernel $ASSETS_REPO:$a_tag with oras" - ( cd "$STAGE/share/guest" \ - && "$STAGE/bin/oras" pull "$ASSETS_REGISTRY/$ASSETS_REPO:$a_tag" ) \ - || fatal "could not pull $ASSETS_REPO:$a_tag" - KERNEL_SOURCE="$ASSETS_REGISTRY/$ASSETS_REPO:$a_tag" - KERNEL_FROM_ASSETS=true - fi + a_tag="$ASSETS_VERSION-linux-$ARCH" + a_ref="$ASSETS_REGISTRY/$ASSETS_REPO" + a_digest="$("$STAGE/bin/oras" resolve "$a_ref:$a_tag")" \ + || fatal "could not resolve $ASSETS_REPO:$a_tag" + printf '%s\n' "$a_digest" | grep -Eqx 'sha256:[0-9a-f]{64}' \ + || fatal "oras resolved $ASSETS_REPO:$a_tag to '$a_digest', not a sha256 digest" + a_cosign="$("$STAGE/bin/cosign" verify \ + --certificate-identity-regexp "$ASSETS_SIGNER" \ + --certificate-oidc-issuer "$ASSETS_ISSUER" \ + "$a_ref@$a_digest" 2>&1 > /dev/null)" \ + || fatal "the signature on $ASSETS_REPO@$a_digest did not verify against $ASSETS_SIGNER: + $(printf '%s\n' "$a_cosign" | tail -n 1)" + info "fetching the guest kernel $ASSETS_REPO:$a_tag ($a_digest, signature verified) with oras" + ( cd "$STAGE/share/guest" && "$STAGE/bin/oras" pull "$a_ref@$a_digest" ) \ + || fatal "could not pull $ASSETS_REPO@$a_digest" + KERNEL_SOURCE="$a_ref:$a_tag@$a_digest" + KERNEL_FROM_ASSETS=true [ -s "$STAGE/share/guest/$kernel" ] || fatal "no $kernel in the guest assets" # Replace hull's initrd with the brig-built one. From eb5889f452872e7177315d847b4157450595b1b1 Mon Sep 17 00:00:00 2001 From: Anastassios Nanos Date: Tue, 29 Sep 2026 02:02:40 +0300 Subject: [PATCH 2/4] feat(bundle): Publish the digests of the kernel and initrd On Linux the kernel and initrd that boot come from this bundle, and nothing recorded what they hash to, so brig had nothing to compare them with before a boot. The build now writes share/guest/SHA256SUMS with the sha256 of the kernel and of the initrd, and publishes the same bytes as .boot-assets.sha256. The release job checksums every asset, so the signed checksums.txt covers the record. The kernel's line is its layer digest in the signed hull-assets manifest. CI checks that the published record is the one in the tree, that it lists the kernel and the initrd, that it matches them, and that pins.env names the kernel by digest. --help prints the whole header, which now lists the record. Refs: brig-sh/brig#234 Signed-off-by: Anastassios Nanos --- .github/workflows/ci.yml | 19 +++++++++++++++++++ README.md | 7 ++++++- docs/variants.md | 4 +++- scripts/build-bundle.sh | 18 ++++++++++++++++-- 4 files changed, 44 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0d97f12..3121487 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -107,6 +107,25 @@ jobs: scripts/build-bundle.sh --arch "${{ matrix.arch }}" --version ci \ --variant generic-boot ${{ matrix.flag }} --out dist + # The published record must be the one in the tree, and it must match the + # kernel and initrd the tree carries. pins.env names the kernel by digest. + - name: Check the boot-asset digests + run: | + set -eu + r="$(ls dist/*.boot-assets.sha256)" + d="$(mktemp -d)" + tar -xzf "$(ls dist/*.tar.gz)" -C "$d" + g="$(echo "$d"/*/share/guest)" + cmp "$r" "$g/SHA256SUMS" + (cd "$g" && sha256sum -c SHA256SUMS) + awk '{print $2}' "$g/SHA256SUMS" | LC_ALL=C sort | tr '\n' ' ' > "$d/names" + case "$(cat "$d/names")" in + "bzImage container-initrd "|"Image container-initrd ") ;; + *) echo "SHA256SUMS lists $(cat "$d/names")rather than the kernel and the initrd"; exit 1 ;; + esac + grep -Eq '^KERNEL_SOURCE=.+@sha256:[0-9a-f]{64}$' dist/*.pins.env \ + || { echo "pins.env does not name the kernel by digest"; exit 1; } + # install.sh retargets a user install by rewriting the layout the bundle # was generated with, which it reads from here. - name: Check the bundle records its layout diff --git a/README.md b/README.md index 55d1caf..f4ee289 100644 --- a/README.md +++ b/README.md @@ -105,7 +105,7 @@ $ sha256sum -c checksums.txt --ignore-missing libexec/cni/ CNI plugins etc/ urunc.toml, containerd.toml, nerdctl.toml, brig-env.sh, cni/net.d/, systemd/, certs.d/ - share/guest/ Image or bzImage, container-initrd, bundle.json + share/guest/ Image or bzImage, container-initrd, bundle.json, SHA256SUMS share/completions/ bash, zsh, fish completions pins.env every bundled version, the one manifest .install-stamp what this install created, read by the uninstaller @@ -340,6 +340,11 @@ through `BRIG_BOOT_ASSETS`, so brig uses the packed assets rather than fetching anything on first run. The annotations that carry them are `com.urunc.unikernel.bootKernel` and `com.urunc.unikernel.bootInitrd`. +`share/guest/SHA256SUMS` holds the sha256 of the kernel and of the initrd. Each +release publishes the same bytes as `.boot-assets.sha256`, which the +release's signed `checksums.txt` covers, so brig can check the files it is about +to boot against a record that the release signed. + ## What it touches outside the base directories Six things, all recorded in `.install-stamp` and all undone by the uninstaller. diff --git a/docs/variants.md b/docs/variants.md index d165697..23a8f0a 100644 --- a/docs/variants.md +++ b/docs/variants.md @@ -81,7 +81,9 @@ digest. The artifact's layers are the files themselves, each named by its the signed manifest. The kernel is generic and not brig-specific. On arm64 it is the same file hull -boots on macOS. +boots on macOS. `pins.env` records it as `KERNEL_SOURCE`, by the digest the build +pulled. `share/guest/SHA256SUMS` holds the sha256 of the kernel and the initrd, +and each release publishes it as `.boot-assets.sha256`: see the README. **The initrd is not taken from hull-assets.** brig execs into a guest through an in-guest agent, `urunit-agent`, whose wire protocol (`pkg/agentproto`) is a diff --git a/scripts/build-bundle.sh b/scripts/build-bundle.sh index 02b5156..978a467 100755 --- a/scripts/build-bundle.sh +++ b/scripts/build-bundle.sh @@ -30,8 +30,11 @@ # Produces, under --out: # brig-standalone--linux-.tar.gz (the install tarball) # brig-standalone--linux-.pins.env (the version manifest) +# brig-standalone--linux-.boot-assets.sha256 +# (the kernel and initrd digests; not +# for --variant stock) # -# and with --rootless, the same two named brig-standalone--rootless-*. +# and with --rootless, the same files named brig-standalone--rootless-*. set -eu @@ -63,7 +66,7 @@ while [ $# -gt 0 ]; do --variant) VARIANT="$2"; shift 2 ;; --rootless) ROOTLESS=true; shift ;; --urunc-version) URUNC_VERSION_STOCK="$2"; shift 2 ;; - --help|-h) sed -n '2,30p' "$0"; exit 0 ;; + --help|-h) awk 'NR > 1 && /^#/ { sub(/^# ?/, ""); print; next } NR > 1 { exit }' "$0"; exit 0 ;; *) fatal "unknown argument '$1'" ;; esac done @@ -1182,6 +1185,11 @@ if [ "$VARIANT" != "stock" ]; then cat > "$STAGE/share/guest/bundle.json" < "$STAGE/share/guest/SHA256SUMS" info " kernel from $KERNEL_SOURCE, initrd built for brig (urunc $URUNC_REF)" fi @@ -1479,6 +1487,12 @@ tar --sort=name \ gzip -n -9 -c "$TMP_DIR/$NAME.tar" > "$OUT/$NAME.tar.gz" cp "$STAGE/pins.env" "$OUT/$NAME.pins.env" +# A stock build names its outputs as a generic-boot one does, so a record left +# in --out by an earlier build would pass for this one's. +rm -f "$OUT/$NAME.boot-assets.sha256" +if [ -f "$STAGE/share/guest/SHA256SUMS" ]; then + cp "$STAGE/share/guest/SHA256SUMS" "$OUT/$NAME.boot-assets.sha256" +fi info "done" info " $OUT/$NAME.tar.gz ($(du -h "$OUT/$NAME.tar.gz" | awk '{print $1}'))" From b93e44b0aebb84dab23a34a7e73fd59d05afc0b0 Mon Sep 17 00:00:00 2001 From: Anastassios Nanos Date: Tue, 29 Sep 2026 02:21:24 +0300 Subject: [PATCH 3/4] feat(install): Keep the release's signed checksums The release's checksums.txt lists .boot-assets.sha256, which is share/guest/SHA256SUMS. The installer checked the tarball against it, and its signature too when cosign was present, then dropped all of it, so nothing on the host tied the kernel and initrd to the release. A release install now keeps checksums.txt, checksums.txt.sig and checksums.txt.pem in share/guest, beside SHA256SUMS, when checksums.txt lists that SHA256SUMS. The signature is fetched even when cosign is missing at install time, because brig checks it with the bundle's own cosign before a boot. A local tarball keeps the three files when they were copied across beside it. A remote install under INSTALL_BRIG_SKIP_SIGCHECK, a bundle with no SHA256SUMS, and a local tarball with nothing beside it keep none. A release whose signature is missing, or whose checksums.txt does not list the record, keeps none and says so, because brig would warn before every run. Refs: brig-sh/brig#234 Signed-off-by: Anastassios Nanos --- README.md | 21 ++++- install.sh | 63 +++++++++++++- tests/install-record.sh | 178 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 255 insertions(+), 7 deletions(-) create mode 100644 tests/install-record.sh diff --git a/README.md b/README.md index f4ee289..847ef40 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,14 @@ verify it on a machine that has cosign, copy it across, then: # INSTALL_BRIG_BUNDLE=./brig-standalone-v0.1.0-linux-amd64.tar.gz sh install.sh ``` +Copy the release's `checksums.txt`, `checksums.txt.sig` and `checksums.txt.pem` +across too, into the same directory as the tarball. `install.sh` keeps them +beside the kernel and initrd, which is what lets brig check those two files +against the release before a boot. Without them, brig warns before every run +that it cannot check the kernel, and refuses under `BRIG_VERIFY=require`. That +check asks Sigstore online, so a host with no network at all still gets the +warning. + ### Verifying A downloaded tarball is checked against the release's `checksums.txt`, which the @@ -105,7 +113,8 @@ $ sha256sum -c checksums.txt --ignore-missing libexec/cni/ CNI plugins etc/ urunc.toml, containerd.toml, nerdctl.toml, brig-env.sh, cni/net.d/, systemd/, certs.d/ - share/guest/ Image or bzImage, container-initrd, bundle.json, SHA256SUMS + share/guest/ Image or bzImage, container-initrd, bundle.json, SHA256SUMS, + and the release's checksums.txt, .sig and .pem share/completions/ bash, zsh, fish completions pins.env every bundled version, the one manifest .install-stamp what this install created, read by the uninstaller @@ -223,7 +232,7 @@ Everything is driven by environment variables and a couple of flags. | `INSTALL_BRIG_POOL_SIZE` | `100G` | thin pool data size, sparse | | `INSTALL_BRIG_POOL_PREALLOC` | `false` | `true` to `fallocate` the backing files | | `INSTALL_BRIG_SKIP_START` | `false` | lay the tree down without starting it | -| `INSTALL_BRIG_SKIP_SIGCHECK` | `false` | install a remote tarball unverified | +| `INSTALL_BRIG_SKIP_SIGCHECK` | `false` | install a remote tarball unverified, keeping no signed record of the kernel and initrd (brig then warns before every run) | | `INSTALL_BRIG_FORCE` | `false` | take over an `/var/lib/brig/data` we did not create | | `INSTALL_BRIG_DEBUG` | `false` | `set -x` | @@ -342,8 +351,12 @@ anything on first run. The annotations that carry them are `share/guest/SHA256SUMS` holds the sha256 of the kernel and of the initrd. Each release publishes the same bytes as `.boot-assets.sha256`, which the -release's signed `checksums.txt` covers, so brig can check the files it is about -to boot against a record that the release signed. +release's signed `checksums.txt` covers. `install.sh` keeps that `checksums.txt`, +with `checksums.txt.sig` and `checksums.txt.pem`, in `share/guest`, so brig can +check the files it is about to boot against a record that the release signed. +It keeps them only when `checksums.txt` lists this `SHA256SUMS`. A local tarball +keeps them when they were copied across beside it. A remote install under +`INSTALL_BRIG_SKIP_SIGCHECK` keeps none. ## What it touches outside the base directories diff --git a/install.sh b/install.sh index d2a2e52..098a7e4 100755 --- a/install.sh +++ b/install.sh @@ -146,7 +146,9 @@ nothing. (default: ask; non-interactive falls back to yes) INSTALL_BRIG_VERBOSE true for a line per stage (default: false, quiet) INSTALL_BRIG_SKIP_START true to lay the tree down without starting it - INSTALL_BRIG_SKIP_SIGCHECK true to install a remote tarball unverified + INSTALL_BRIG_SKIP_SIGCHECK true to install a remote tarball unverified. It + keeps no signed record of the kernel and initrd, + so brig warns before every run INSTALL_BRIG_REQUIRE_SIGCHECK true to refuse unless the cosign signature over checksums.txt verifies INSTALL_BRIG_FORCE true to take over a /var/lib/brig/data we did not create @@ -554,6 +556,14 @@ fetch_tarball() { [ -f "$BUNDLE" ] || fatal "tarball '$BUNDLE' not found" say "installing from the local tarball $BUNDLE" cp "$BUNDLE" "$TARBALL" + # The release's checksums.txt, signature and certificate, when + # they were copied across beside the tarball, are the record + # keep_release_record keeps. The tarball is still taken as given. + for f in checksums.txt checksums.txt.sig checksums.txt.pem; do + if [ -f "$(dirname "$BUNDLE")/$f" ]; then + cp "$(dirname "$BUNDLE")/$f" "$TMP_DIR/$f" + fi + done ;; esac } @@ -572,9 +582,15 @@ verify_tarball() { || fatal "no checksums.txt next to the tarball, cannot verify it. Set INSTALL_BRIG_SKIP_SIGCHECK=true to install it unverified anyway." - if command -v cosign >/dev/null 2>&1 \ - && fetch_quiet "$base/checksums.txt.pem" "$TMP_DIR/checksums.txt.pem" \ + # The signature is fetched whether or not cosign is here to check it: + # keep_release_record puts it beside the boot assets, where brig checks it + # with the bundle's own cosign before a boot. + sig_fetched=false + if fetch_quiet "$base/checksums.txt.pem" "$TMP_DIR/checksums.txt.pem" \ && fetch_quiet "$base/checksums.txt.sig" "$TMP_DIR/checksums.txt.sig"; then + sig_fetched=true + fi + if command -v cosign >/dev/null 2>&1 && [ "$sig_fetched" = "true" ]; then # A signature that fails is not a signature that is absent. The first # says the bytes or the identity are wrong and is fatal; only the second # degrades to the hash. @@ -640,6 +656,7 @@ unpack_tarball() { done [ -f "$root/pins.env" ] && cp "$root/pins.env" "$PREFIX/pins.env" chmod 0755 "$PREFIX" + keep_release_record mkdir -p "$DATA_DIR/containerd" "$DATA_DIR/nerdctl" "$DATA_DIR/log" "$RUN_DIR" retarget_tree @@ -648,6 +665,46 @@ unpack_tarball() { # switches at runtime without rewriting anything. } +# Keep the release's checksums.txt, and its signature and certificate, beside +# the boot assets. checksums.txt lists .boot-assets.sha256, which is +# share/guest/SHA256SUMS, so brig can check the kernel and initrd against a +# record the release signed before every boot. +# +# Without the three files brig cannot check that record: it warns before every +# run, and refuses under BRIG_VERIFY=require. A local tarball with nothing +# beside it, a remote one under INSTALL_BRIG_SKIP_SIGCHECK, and a bundle built +# before the record (no SHA256SUMS) keep none, as asked. A release that gave +# checksums.txt and not its signature, or whose checksums.txt does not list +# this SHA256SUMS, keeps none and says so: brig would refuse that record before +# every boot. +keep_release_record() { + guest="$PREFIX/share/guest" + [ -f "$guest/SHA256SUMS" ] || return 0 + if [ ! -s "$TMP_DIR/checksums.txt" ]; then + say "no checksums.txt from a release, so brig cannot check the boot assets' record" + return 0 + fi + for f in checksums.txt.sig checksums.txt.pem; do + if [ ! -s "$TMP_DIR/$f" ]; then + warn "the release gave no $f, so no record of the kernel and initrd is kept. + brig warns before every run that it cannot check them, and refuses under BRIG_VERIFY=require." + return 0 + fi + done + sums="$(sha256sum "$guest/SHA256SUMS" | awk '{print $1}')" + if ! awk -v h="$sums" '$1 == h && $2 ~ /[.]boot-assets[.]sha256$/ { found = 1 } END { exit !found }' \ + "$TMP_DIR/checksums.txt"; then + warn "the release's checksums.txt does not list this bundle's SHA256SUMS, so no record of + the kernel and initrd is kept. brig warns before every run that it cannot check them." + return 0 + fi + for f in checksums.txt checksums.txt.sig checksums.txt.pem; do + cp "$TMP_DIR/$f" "$guest/$f" || fatal "could not keep $f in $guest" + chmod 0644 "$guest/$f" + done + say "kept the release's signed checksums.txt in $guest" +} + # Move the tree's idea of where it lives. The generated wrappers, configs and # units carry the build layout as literal paths; no binary does. The grep at the # end is what keeps that true: a tenth file added upstream fails the install diff --git a/tests/install-record.sh b/tests/install-record.sh new file mode 100644 index 0000000..a2e9c0e --- /dev/null +++ b/tests/install-record.sh @@ -0,0 +1,178 @@ +#!/bin/sh +# +# Install a minimal fake bundle from a release served over http, and from a +# local tarball, and check when the release's checksums.txt, .sig and .pem are +# kept beside the boot assets' SHA256SUMS: only when checksums.txt lists that +# SHA256SUMS under a *.boot-assets.sha256 name and the signature came with it, +# cosign or not. +# +# Needs a normal user with a subuid range, and python3 to serve the release, on +# Linux. + +set -eu + +here="$(cd "$(dirname "$0")/.." && pwd)" +user="$(id -un)" + +skip() { echo "SKIP: $*"; exit 0; } +fail() { echo "FAIL: $*" >&2; exit 1; } +ok() { echo "ok - $*"; } + +[ "$(uname -s)" = Linux ] || skip "install.sh installs on Linux only" +[ "$(id -u)" -ne 0 ] || skip "a user install refuses root; run this as a normal user" +grep -q "^$user:" /etc/subuid 2>/dev/null || skip "no subuid range for $user" +grep -q "^$user:" /etc/subgid 2>/dev/null || skip "no subgid range for $user" +command -v python3 >/dev/null 2>&1 || skip "no python3 to serve the release" + +T="$(mktemp -d)" +server="" +trap '[ -z "$server" ] || kill "$server" 2>/dev/null; rm -rf "$T"' EXIT +mkdir -p "$T/stub" "$T/nocosign" "$T/rel" "$T/bare" + +# make_release : a fake rootless bundle in $T/rel, +# with a checksums.txt that lists the tarball, and the record when is +# yes, and stand-ins for its signature and certificate. Any other but +# no is the name the record is listed under instead. +make_release() { + bdir="$T/src/$1" + rm -rf "$T/src" + mkdir -p "$bdir/bin" "$bdir/etc" "$bdir/share/guest" + printf '#!/bin/sh\necho setup-ran\n' > "$bdir/bin/brig-rootless-setup.sh" + printf '#!/bin/sh\necho brig 0.3.0\n' > "$bdir/bin/brig" + chmod 0755 "$bdir/bin"/* + : > "$bdir/etc/brig-env.sh" + echo kernel > "$bdir/share/guest/bzImage" + echo initrd > "$bdir/share/guest/container-initrd" + if [ "$2" = yes ]; then + ( cd "$bdir/share/guest" && sha256sum -- bzImage container-initrd ) \ + > "$bdir/share/guest/SHA256SUMS" + fi + cat > "$bdir/pins.env" <<'PINS' +BUNDLE_VERSION=v9.9.9 +BRIG_VERSION=v0.3.0 +ROOTLESS=true +PINS + tar -C "$T/src" -czf "$T/rel/$1.tar.gz" "$1" + ( cd "$T/rel" && sha256sum -- "$1.tar.gz" > checksums.txt ) + if [ "$2" = yes ] && [ "$3" != no ]; then + rec="$1.boot-assets.sha256" + [ "$3" = yes ] || rec="$3" + sum="$(sha256sum "$bdir/share/guest/SHA256SUMS" | awk '{print $1}')" + echo "$sum $rec" >> "$T/rel/checksums.txt" + fi + echo signature > "$T/rel/checksums.txt.sig" + echo certificate > "$T/rel/checksums.txt.pem" +} + +# The host checks pass, as in install-summary.sh. In $T/stub cosign answers +# yes, so the stand-in signature is taken whatever cosign this host has. +# $T/nocosign is the same without cosign. +cat > "$T/stub/getfacl" < "$T/stub/sysctl" +printf '#!/bin/sh\n[ "$*" = "-n true" ]\n' > "$T/stub/sudo" +printf '#!/bin/sh\nexit 1\n' > "$T/stub/modinfo" +for b in newuidmap setfacl systemctl cosign; do + printf '#!/bin/sh\nexit 0\n' > "$T/stub/$b" +done +chmod 0755 "$T/stub"/* +cp -p "$T/stub"/* "$T/nocosign/" +rm "$T/nocosign/cosign" + +port="$(python3 -c 'import socket; s = socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1])')" +python3 -m http.server --bind 127.0.0.1 --directory "$T/rel" "$port" > "$T/http.log" 2>&1 & +server=$! +i=0 +until curl -sf -o /dev/null "http://127.0.0.1:$port/"; do + i=$((i + 1)) + [ "$i" -lt 50 ] || fail "the release server did not come up" + sleep 0.1 +done +url="http://127.0.0.1:$port" + +# install_from []: a user install into a fresh HOME, from a +# URL or a path. +install_from() { + rm -rf "${T:?}/home" "${T:?}/run" + mkdir -p "$T/home" "$T/run" + if ! env -u XDG_CONFIG_HOME -u XDG_DATA_HOME \ + HOME="$T/home" XDG_RUNTIME_DIR="$T/run" PATH="${2:-$T/stub}:$PATH" \ + INSTALL_BRIG_BUNDLE="$1" \ + sh "$here/install.sh" > "$T/out.log" 2>&1; then + cat "$T/out.log" >&2 + fail "the install from $1 failed" + fi +} +guest="$T/home/.local/share/brig/data/share/guest" + +kept() { + for f in checksums.txt checksums.txt.sig checksums.txt.pem; do + cmp -s "$1/$f" "$guest/$f" || fail "$2: $f was not kept beside the boot assets" + done +} +none_kept() { + for f in checksums.txt checksums.txt.sig checksums.txt.pem; do + [ ! -e "$guest/$f" ] || fail "$1: $f was kept" + done +} + +name=brig-standalone-v9.9.9-rootless-linux-amd64 +make_release "$name" yes yes + +install_from "$url/$name.tar.gz" +kept "$T/rel" "release install" +[ -f "$guest/SHA256SUMS" ] || fail "the bundle's SHA256SUMS is missing" +ok "a release install keeps checksums.txt, its signature and its certificate" + +# Without cosign the signature is not checked here, and it is still fetched +# and kept, for brig to check with the bundle's own cosign. +if PATH="$T/nocosign:$PATH" command -v cosign >/dev/null 2>&1; then + echo "SKIP: this host has cosign on PATH, so the no-cosign install cannot be run" +else + before="$(grep -c 'GET /checksums.txt.sig ' "$T/http.log" || true)" + install_from "$url/$name.tar.gz" "$T/nocosign" + after="$(grep -c 'GET /checksums.txt.sig ' "$T/http.log" || true)" + [ "$after" -gt "$before" ] || fail "without cosign the signature was not fetched" + kept "$T/rel" "no-cosign install" + ok "a release install with no cosign still fetches and keeps the signature" +fi + +cp "$T/rel/$name.tar.gz" "$T/bare/" +install_from "$T/bare/$name.tar.gz" +none_kept "local tarball alone" +ok "a local tarball with nothing beside it keeps no release record" + +install_from "$T/rel/$name.tar.gz" +kept "$T/rel" "local tarball beside the release files" +ok "a local tarball keeps the release files copied across beside it" + +make_release "$name" yes no +install_from "$url/$name.tar.gz" +none_kept "unlisted record" +grep -q 'does not list this bundle' "$T/out.log" \ + || { cat "$T/out.log" >&2; fail "an unlisted record was dropped without a warning"; } +ok "a checksums.txt that does not list SHA256SUMS keeps none, and says so" + +# brig takes the record only under a *.boot-assets.sha256 name. +make_release "$name" yes "$name.pins.env" +install_from "$url/$name.tar.gz" +none_kept "record under another name" +grep -q 'does not list this bundle' "$T/out.log" \ + || { cat "$T/out.log" >&2; fail "a record under another name was dropped without a warning"; } +ok "a checksums.txt that lists SHA256SUMS under another name keeps none, and says so" + +make_release "$name" yes yes +rm "$T/rel/checksums.txt.sig" +install_from "$url/$name.tar.gz" +none_kept "no signature" +grep -q 'gave no checksums.txt.sig' "$T/out.log" \ + || { cat "$T/out.log" >&2; fail "a release with no signature was dropped without a warning"; } +ok "a release with no signature keeps none, and says so" + +old=brig-standalone-v9.9.8-rootless-linux-amd64 +make_release "$old" no no +install_from "$url/$old.tar.gz" +none_kept "no SHA256SUMS" +ok "a bundle with no SHA256SUMS keeps no release record" From 41aad3aa18020330df2051c44363df293e43bd1a Mon Sep 17 00:00:00 2001 From: Anastassios Nanos Date: Wed, 30 Sep 2026 09:15:50 +0300 Subject: [PATCH 4/4] feat(bundle): Take the minimal kernels from hull-assets 0.1.6 hull-assets 0.1.6 carries the minimal kernel profile, which drops the drivers for hardware no VMM presents and builds in the memory cgroup controller, cgroup BPF and virtio-rng: - amd64: bzImage 11.3 MB (was 16.6 MB) - arm64: Image 23.5 MB (was 52.4 MB) The bundle builds its own initrd, so the kernel is the only part it takes from 0.1.6. Signed-off-by: Anastassios Nanos --- scripts/build-bundle.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/build-bundle.sh b/scripts/build-bundle.sh index 978a467..b1cff53 100755 --- a/scripts/build-bundle.sh +++ b/scripts/build-bundle.sh @@ -163,7 +163,7 @@ COSIGN_SHA_LINUX_ARM64="c5d324e091826b0d7a78eb16fef316450b4eb9aaec045611c08ba06f MONITORS="${MONITORS:-firecracker cloud-hypervisor solo5-hvt solo5-spt}" ASSETS_REGISTRY="${ASSETS_REGISTRY:-ghcr.io}" ASSETS_REPO="${ASSETS_REPO:-nofireai/hull-assets}" -ASSETS_VERSION="${ASSETS_VERSION:-0.1.4}" +ASSETS_VERSION="${ASSETS_VERSION:-0.1.6}" # Who signs hull-assets: the identity brig checks before it boots a kernel from # there. The build checks the same one before it packs the kernel. ASSETS_SIGNER_DEFAULT='^https://github\.com/NOFireAI/hull-assets/\.github/workflows/build-assets\.yml@refs/heads/main$'