Skip to content

fix(ci): generate the SBOM with Syft instead of a BuildKit attestation - #3

Merged
grinidx merged 1 commit into
mainfrom
fix/sbom-out-of-band
Aug 16, 2026
Merged

fix(ci): generate the SBOM with Syft instead of a BuildKit attestation#3
grinidx merged 1 commit into
mainfrom
fix/sbom-out-of-band

Conversation

@grinidx

@grinidx grinidx commented Aug 16, 2026

Copy link
Copy Markdown
Member

What broke

The weekly release run on 16 Aug failed on both architectures at Push by digest (run 31924690964):

failed to solve: /tmp/buildkit-mount1937992494/sbom.spdx.json exceeds 41943040 bytes

No commit here since 9 Aug, and that morning's release was green.

Root cause

docker/setup-buildx-action boots the floating moby/buildkit:buildx-stable-1 tag:

run BuildKit result
9 Aug v0.31.2 green
16 Aug v0.32.2 red

Same syft scanner in both (buildkit-syft-scanner:stable-1, v1.11.0), so the SBOM itself is unchanged.

v0.32 replaced the plain json.NewDecoder(f) in exporter/attestation/unbundle.go with a new decodeStatement that wraps the reader in io.LimitedReader{N: maxAttestationBytes + 1}. maxAttestationBytes is 40 MiB, exactly the 41,943,040 in the error. The constant has existed since 2022; v0.32 is the first version to enforce it on that path (moby/buildkit@9cdf6429).

Our SBOM has been over that line for months. Off the last good release, v2026.08.09:

arch SBOM bytes
linux/amd64 52,276,285
linux/arm64 51,944,931

Breakdown: files 25.9 MB (50,656 entries), relationships 15.8 MB, packages 10.6 MB (6,838). Inherent to a container this broad, and there is no supported way to trim it - buildkit-syft-scanner reads only BUILDKIT_SCAN_DESTINATION, SOURCE and SOURCE_EXTRAS.

The change

Stop routing the SBOM through a mechanism with a ceiling we do not control.

  • build.yml: drop sbom: true from Push by digest; add a Syft scan of the image already built, loaded and tested, uploaded as a per-arch artifact. provenance: mode=max is a few KB and is untouched.
  • release.yml: attach the two SPDX documents instead of extracting one from the image. The GHCR login there existed only for the extraction, so it goes too.

The scan runs on every build, publishing or not, so this PR's own CI proves it works; only the upload is gated on publish.

What consumers see

  • Release assets become sbom-amd64.spdx.json.gz and sbom-arm64.spdx.json.gz, each a plain SPDX 2.3 document, replacing the single sbom.json.gz that wrapped both architectures in a bespoke buildx envelope. The old notes called that CycloneDX; it was SPDX then too.
  • The image no longer carries an SBOM attestation, so imagetools inspect --format '{{ json .SBOM }}' no longer returns one. README, SECURITY and ARCHITECTURE now point at the release assets.

Rejected

  • Pin moby/buildkit:v0.31.2. One line, green today, but gives up a hardening release and only defers this.
  • Shrink the SBOM. Would need a forked generator image to reach syft's config.

The weekly release failed on both architectures at "Push by digest":

  failed to solve: /tmp/buildkit-mount.../sbom.spdx.json
  exceeds 41943040 bytes

Nothing in this repo changed. docker/setup-buildx-action boots the floating
moby/buildkit:buildx-stable-1 tag, which rolled from v0.31.2 (9 Aug, green)
to v0.32.2 (16 Aug, red). v0.32 replaced the plain json.NewDecoder in
exporter/attestation/unbundle.go with a decodeStatement that wraps the reader
in an io.LimitedReader capped at maxAttestationBytes - 40 MiB, exactly the
41943040 in the error. The constant has existed since 2022; v0.32 is the first
version to enforce it on that path.

This image's SPDX document has been over that line for months. Off the last
good release, v2026.08.09: amd64 52,276,285 bytes, arm64 51,944,931. Of that,
files 25.9 MB across 50,656 entries, relationships 15.8 MB, packages 10.6 MB
across 6,838. The size is inherent to a container this broad, and there is no
supported knob to trim it - buildkit-syft-scanner reads only
BUILDKIT_SCAN_DESTINATION, SOURCE and SOURCE_EXTRAS.

So stop routing the SBOM through a mechanism with a ceiling we do not control.
Syft scans the image that was already built, loaded and tested, and each
release carries one SPDX document per architecture. Provenance is a few KB and
stays exactly where it was, still signed by actions/attest.

The scan runs on every build, publishing or not, so a pull request proves it
still works; only the artifact upload is gated on publish.

Release assets change shape: sbom-amd64.spdx.json.gz and sbom-arm64.spdx.json.gz
replace the single sbom.json.gz, which wrapped both architectures in a bespoke
buildx envelope. Each is now a plain SPDX 2.3 document. The old release notes
called it CycloneDX; it was SPDX then too.

The image no longer carries an SBOM attestation in the registry, so
`imagetools inspect --format '{{ json .SBOM }}'` no longer returns one. README,
SECURITY and ARCHITECTURE updated to point at the release assets instead.
@grinidx
grinidx merged commit 73e4e84 into main Aug 16, 2026
4 checks passed
@grinidx
grinidx deleted the fix/sbom-out-of-band branch August 16, 2026 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant