Skip to content

Add bbox agents import IMAGE (BYO agents Phase 5)#216

Merged
JAORMX merged 4 commits into
mainfrom
byo-agents-phase-5-oci-image-import
Jul 8, 2026
Merged

Add bbox agents import IMAGE (BYO agents Phase 5)#216
JAORMX merged 4 commits into
mainfrom
byo-agents-phase-5-oci-image-import

Conversation

@JAORMX

@JAORMX JAORMX commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements #204 — BYO agents Phase 5: self-describing OCI images. bbox agents import IMAGE extracts an embedded agent manifest from an OCI image and registers a runnable agent with no local YAML authoring.

bbox agents import ghcr.io/acme/aider-bbox:latest
bbox aider

An image ships its agent definition either at a path named by the org.stacklok.broodbox.agent config label, or at the well-known fallback path /usr/share/broodbox/agent.yaml. The imported ref is pinned to its resolved digest so the registered agent is reproducible.

What changed

  • internal/infra/ocimage/ (new) — Fetcher interface + RemoteFetcher using go-containerregistry. Pulls the image config, reads the label (or falls back to the well-known path), walks layers topmost-first, and extracts the manifest file. Pins the ref to repo@sha256:.... Default keychain auth (docker login/podman login creds work out of the box) + a 60s context timeout so a slow/malicious registry cannot hang the CLI. 401/403/404 errors are enriched with actionable hints. Lives in internal/infra/ (I/O layer) per the DDD rules; imports no domain code.
  • cmd/bbox/agents_import.go — overloads import to accept a local manifest file or an OCI image ref, auto-detected. isImageRef requires a / or @sha256: so a typo'd local path (aidr.yaml) does not trigger a network pull against docker.io. --name overrides the manifest's name. Warns only on a real repository mismatch (same-repo tag→digest pin is silent). Reuses the exact validate → collision-gate → UpsertAgent → receipt tail the file path uses, so behavior, receipts, and safety match.
  • Docsdocs/run-image.md and docs/USER_GUIDE.md document the self-describing image convention (label + well-known path, topmost-layer-wins, digest pinning) and the minimum image contract.

Design decisions (resolving the issue's open questions)

  • Local-cache vs live-pull: import IMAGE live-pulls just the manifest (config + the one layer holding the manifest file). The full image is pulled at VM run time by the go-microvm runner. import only needs the manifest bytes + digest.
  • Verify manifest's declared image: the imported ref is authoritative (pinned to digest). A warning fires only when the embedded manifest declares a different repository; a same-repo tag→digest pin is the normal case and is silent.
  • Doctor pullability: deferred. runDoctor is pure/offline (env + imageValidator injected); adding network I/O would break that invariant. A future --pull flag or separate command is the right home.

Tests

Hermetic — no network in CI:

  • extractFromImage unit tests via in-memory v1.Image fixtures (built with crane.Layer + mutate.AppendLayers + mutate.ConfigFile): label present, label absent→well-known path, neither→error, non-topmost layer, topmost shadows base, malformed bytes, empty/whitespace label, label to missing file.
  • pinnedDigestRef normalization (tag stripped, repo preserved).
  • enrichRemoteError table test (401/403/404/non-transport).
  • isImageRef table test (file vs image vs typo'd path vs docker hub short ref).
  • E2e CLI tests via an injected fake Fetcher: image import → digest-pinned image in config + receipt OK + doctor passes + round-trips through loader; manifest declares different repo → warning + override; same-repo tag → no warning; --name override; --name to a builtin → rejected; collision with existing custom agent (without/with --force); no image field → succeeds silently; fetcher error → CLI surfaces the ref; malformed manifest → parse error.
  • All existing file-import/export tests remain green.

Verification

task fmt    # clean
task lint   # 0 issues
task test   # all packages pass (-race)

Out of scope

  • agents doctor image-pullability check (deferred — would break doctor's pure/offline invariant).
  • Signed manifests / cosign verification.
  • Multi-arch selection beyond remote.Image's host-platform default.
  • Reading the manifest from an OCI image index (single-platform images only for Phase 5).
  • run-image reading an embedded manifest (run-image stays flag-driven).

Closes #204.

Self-describing OCI images: bbox agents import IMAGE extracts an
embedded agent manifest declared by the org.stacklok.broodbox.agent
config label (or the well-known /usr/share/broodbox/agent.yaml path)
and registers a runnable agent with no local YAML authoring.

- internal/infra/ocimage: Fetcher interface + RemoteFetcher using
  go-containerregistry. Walks layers topmost-first, extracts the
  manifest file, pins the ref to the resolved digest. Default
  keychain auth (docker login) + 60s context timeout. 401/403/404
  error enrichment with actionable hints.
- cmd/bbox/agents_import.go: overload import to accept a file OR an
  image ref. isImageRef heuristic requires / or @sha256: so a typo'd
  local path does not trigger a network pull. --name overrides the
  manifest's name. Warns only on real repo mismatch (same-repo
  tag->digest pin is silent). Reuses the validate->UpsertAgent->
  receipt tail shared with the file path.
- Tests: hermetic extractFromImage tests via in-memory v1.Image
  fixtures (no network); e2e CLI tests via an injected fake Fetcher;
  enrichRemoteError unit tests; edge cases (fetcher error, no image
  field, --name to builtin, collision, malformed manifest).
- Docs: run-image.md and USER_GUIDE.md document the self-describing
  image convention and minimum image contract.

Doctor image-pullability is deferred (would break doctor's pure/
offline invariant); signed manifests and multi-arch selection beyond
the host platform are out of scope.
JAORMX and others added 3 commits July 3, 2026 21:44
Panel-review second-pass fixes:

- Reject foreign-layer URLs (CWE-918): readLayerPath now checks
  partial.Descriptor for an OCI foreign-blob urls field and refuses
  the layer before any I/O. Foreign URLs are fetched on the host
  outside the sandbox egress policy and are vulnerable to DNS
  rebinding to cloud metadata. The agent manifest has no legitimate
  reason to be a foreign layer.
- Normalize tar entry names: cleanTarPath strips a leading / and ./
  from both the target path and tar entry names before comparison,
  so entries emitted by docker/podman build (./usr/share/...) match
  an absolute target (/usr/share/...). Previously these were
  silently skipped, producing a false 'manifest not found'.
- RemoteFetcher doc: clarify the zero value pulls anonymously; use
  NewRemoteFetcher for host credentials.
- Tests: foreign-layer rejection (mock layer panics on Uncompressed,
  proving no I/O), leading-./ and relative tar entry matches,
  keychain-wiring assertion, cancelled-context timeout test,
  isImageRef ../ and absolute-path rows.
Two fixes to `bbox agents import <source>` from PR review:

- isImageRef: a slash-containing source is treated as an OCI image
  only when the segment before the first "/" looks like a registry
  host (contains "." or ":", or is "localhost"). Previously any "/"
  made it an image, so a mistyped local path like
  "manifests/aider.yaml" fell through to a docker.io network pull —
  a needless pull and a typosquatting surface. Such paths now resolve
  to the file loader and its precise "no such file" error. An
  "@sha256:" digest still marks an unambiguous image ref.

- loadImportManifest no longer re-wraps the fetcher error: the
  Fetcher contract (and RemoteFetcher) already prefix errors with
  "importing agent from image %q", so the CLI wrap produced a doubled
  message. The test fake now mirrors that contract.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resource-exhaustion hardening for `bbox agents import IMAGE`, which
pulls from a possibly-untrusted registry on the host:

- Reject an image whose declared config-blob size exceeds 4 MiB (OCI
  config is small JSON) before fetching it, and refuse an image with
  more than 512 layers. Layer byte size is intentionally NOT capped —
  legitimate agent images are 1-2 GB and must still import.

- readLayerPath rejects an embedded manifest whose tar header size
  exceeds configfile.MaxSize (1 MiB) instead of silently truncating it
  via LimitReader and handing garbage to the decoder; the LimitReader
  stays as a backstop against a lying header.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jhrozek

jhrozek commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Review follow-up: 4 findings fixed (2 commits)

Addressed the highest-value findings from a multi-agent review of this PR. Two small, self-contained commits split by layer; full task test green.

d06cccf — Harden agents import source classification (cmd/bbox/agents_import.go)

  • isImageRef typo'd-path / typosquat gap (HIGH). The old rule treated any source containing / as an OCI image. A mistyped local path in a subdirectory — e.g. manifests/aider.yaml — therefore fell through to a docker.io network pull instead of a clean "no such file" error. That's both a needless pull and a typosquatting surface (an attacker could register a plausible mistyped repo). Now a slash-containing source is an image only when the segment before the first / looks like a registry host (contains . or :, or is localhost); otherwise it resolves to the file loader. An @sha256: digest still marks an unambiguous image ref.
    • Behavior note: namespaced Docker Hub shorthand like bitnami/nginx:latest is now treated as a file — import it fully-qualified as docker.io/bitnami/nginx:latest. Consistent with the existing ubuntu:24.04 full-qualification policy; help text updated.
  • Double error-wrap (MEDIUM). loadImportManifest re-wrapped the fetcher error with the same importing agent from image %q prefix that RemoteFetcher already adds, producing a doubled message. Dropped the CLI wrap (the Fetcher contract owns that prefix); the test fake now mirrors the contract.

020ac4f — Bound OCI config and manifest sizes on import (internal/infra/ocimage/ocimage.go)

  • Unbounded config blob / layer count (HIGH, DoS). A malicious/compromised registry could declare a multi-GB config blob or thousands of layers and make the host buffer them. Now: reject a declared config size > 4 MiB (OCI config is small JSON) before fetching it, and refuse > 512 layers. Layer byte size is intentionally not capped — legitimate agent images are 1–2 GB (verified against a real wolfi-based image) and must still import.
  • Silent manifest truncation (MEDIUM). An oversized embedded agent.yaml was silently truncated by the LimitReader and handed to the strict decoder as garbage. Now readLayerPath rejects a manifest whose tar header size exceeds configfile.MaxSize (1 MiB) with a clear error; the LimitReader stays as a backstop against a lying header.

Not addressed (deliberately deferred)

  • A single malicious giant layer is still bounded only by the 60s timeout. Capping per-layer bytes would reject legitimate large images, so it needs a smarter approach (streamed byte budget) — deferred.
  • LOW/NIT polish: host-platform digest pinning / --platform, swallowed ConfigFile() error, --dry-run manifest preview, and a manifest-decode dedup.

Each fix ships with unit tests (subdir-path classification, oversized-config, too-many-layers, oversized-manifest, and the fetcher-error contract).

@JAORMX JAORMX merged commit ffacbc4 into main Jul 8, 2026
8 checks passed
@JAORMX JAORMX deleted the byo-agents-phase-5-oci-image-import branch July 8, 2026 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BYO agents Phase 5: self-describing OCI images (bbox agents import IMAGE)

2 participants