Skip to content

fix(storage): re-pin native file and storage helper to oven/bun:1.4.4-slim - #600

Closed
roodboi wants to merge 2 commits into
nextfrom
claude/hack-1758-helper-bun-144
Closed

roodboi wants to merge 2 commits into
nextfrom
claude/hack-1758-helper-bun-144

Conversation

@roodboi

@roodboi roodboi commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

What changes

The native file and storage helper image moves from oven/bun:1.4.2-slim to oven/bun:1.4.4-slim. This was the last place next still shipped Bun 1.4.2.

  • Storage helper catalog (src/lib/native-compose-storage-witness-docker-artifact.ts): new index, platform manifest, config, Bun and libc pins for linux/arm64 and linux/amd64. The helper bundle bytes (helperHash) are unchanged; bun scripts/generate-native-storage-witness-helper.ts --check passes.
  • Artifact type (src/lib/native-compose-storage-witness-xattr-carrier.ts): NATIVE_STORAGE_HELPER_BUN_VERSION = "1.4.4". Dependency selection now also refuses any catalog entry whose Bun version is not the current one.
  • VM file helper (src/lib/native-compose-vm-file-protocol.ts): new preparation uses oven/bun:1.4.4-slim.
  • CI no longer pulls oven/bun:1.4.2-slim; the existing oven/bun:1.4.4-slim pull now covers the helper too.
  • Docs (native-compose-storage-xattr-carrier.md, native-config-files.md, native-compose-benchmark.md, tests/e2e/README.md) name the new image and explain what happens to earlier state.

State written by an earlier CLI (needs review)

The old pin was a single literal in the validator. Changing only that literal would make saved receipts, intents, references and carrier journals that hold the 1.4.2 artifact fail to parse. Saved stop and recovery for those instances would then refuse. To avoid stranding them:

  • Saved storage records with the earlier bunVersion: "1.4.2" still parse, and capture keeps their version rather than relabeling it.
  • That artifact is no longer cataloged, so storage enrolled under it refuses verification as artifact drift. It is never relabeled, re-seeded or adopted by the new helper. A succession contract for such storage would be a separate design; this PR does not add one.
  • A saved VM file manifest may name either oven/bun:1.4.2-slim or oven/bun:1.4.4-slim. The generation keeps its own pinned helper image ID for restore and retirement. Only new preparation uses the current tag.

This is acceptable for the candidate channel: each candidate version starts with fresh native homes, so storage prepared under the 1.4.2 helper refuses afterward and is prepared again under the current helper. docs/reference/native-compose-storage-xattr-carrier.md now says so.

No existing guard was relaxed. Every other image, manifest, config, platform, helper-hash and policy check is unchanged.

How the pins were derived (no local Docker)

Everything came from the registry HTTP API with an anonymous pull token, in a scratch directory:

token=$(curl -fsS "https://auth.docker.io/token?service=registry.docker.io&scope=repository:oven/bun:pull" | jq -r .token)
reg=https://registry-1.docker.io/v2/oven/bun
# 1. Index by tag, checking sha256(body) == Docker-Content-Digest.
curl -fsSL -H "Authorization: Bearer $token" \
  -H 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json' \
  -D index.headers -o index.json "$reg/manifests/1.4.4-slim"
# 2. For linux/arm64 and linux/amd64 (exactly one entry each): manifest by digest,
#    then config blob by digest, then every layer blob by digest. Each file's
#    sha256 must equal the digest it was fetched by. Config os/architecture checked.
curl -fsSL -H "Authorization: Bearer $token" \
  -H 'Accept: application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' \
  -o manifest.json "$reg/manifests/<platform-digest>"
curl -fsSL -H "Authorization: Bearer $token" -o <blob> "$reg/blobs/<digest>"
shasum -a 256 <blob>
# 3. Extract the layers in manifest order, resolve symlinks inside the rootfs
#    (/lib is a merged-usr symlink), then hash the two regular files.
tar -xf <layer> -C rootfs
shasum -a 256 rootfs/usr/local/bin/bun rootfs/usr/lib/<aarch64|x86_64>-linux-gnu/libc.so.6

Check of the method: the same script run against the old index sha256:cb3bbbb0… reproduced every existing 1.4.2 pin exactly: both manifests, both configs, both Bun hashes and both libc hashes.

Result for oven/bun:1.4.4-slim:

linux/arm64 linux/amd64
index sha256:1846adf22496bd711036aae8be147e57bdebbaea2e5264e3e0b5af1db187d9a4 same
manifest sha256:b0d169a695f779087d0fe8c08b5b17f0d3a927dceac2a087c87a1351340c6565 sha256:13b43f8bb2c63f5d89c56205c4279b85620363296a00ada33ff7ad568f3688e7
config sha256:f545ab0408208347c62fad5822a4621a4b7fc3a7da43a1dea66dc17335535b7e sha256:bca1fb36a1a809a0aec806c0a239364f4873ead989e7f069e0a47f17e0a83cfa
layers 7, all digest-verified 7, all digest-verified
/usr/local/bin/bun 354c2b049c716aca07b2e5cddb887928ac58950675d1258bb6312083917793b6 10702245766b225e70bbf6235fd6b47214fd94b4760d00ffaed8ab0df57ac235
libc.so.6 70fa4c4cf57eabeb96097cce47582d02379c4cd1de3db3e98ade4dd7089a1b13 9792e3cbb541c8f44c7acf5f14f4022ea62998ecc787d326bed4d8b6547dfd92

Other facts from the configs: the image label org.opencontainers.image.revision is 38afee6c…, matching the Bun 1.4.4 build CI uses. The base moved from Debian 13.6 to 13.7, and glibc from 2.41-12+deb13u3 to 2.41-12+deb13u4. That is the same glibc 2.41 ABI and the same fixed libc paths. User, Volumes, Entrypoint and Env are unchanged from 1.4.2.

Validation

  • bun run --cwd packages/cli typecheck: pass.
  • biome check on the changed TypeScript files: pass.
  • bun scripts/generate-native-storage-witness-helper.ts --check: pass (helper bytes unchanged).
  • bun run docs:prepare && bun run docs:check: pass.
  • Focused and directly affected tests, run through an isolated bounded runner (empty environment, private HOME, Docker and launchctl stubbed): 25 storage-witness and VM file test files (every test file that imports native-compose-storage-* or native-compose-vm-file-*): 613 pass, 15 skip, 1 fail. The failure pinned a hash of recorded daemon output that contains the old image ID. That test now keeps those recorded bytes as a literal with the hash unchanged, rather than rewriting the hash. Rerunning the two changed storage files after the fix: 111 pass, 0 fail. No outer Docker, launchctl or hack stub was called. The branch was then rebased onto next 6410d1c, whose two new commits touch none of these files, and typecheck passed again.
  • New regression tests:
    • Exact registry-derived ARM and AMD pins.
    • An earlier 1.4.2 artifact parses unchanged, is never referenced as an image, and is never selected, even when it reuses the current image identity.
    • Malformed Bun versions refuse.
    • A saved carrier journal carrying the earlier Bun version refuses before transport.
    • The list of saved VM file helper tags.
  • gitleaks protect --staged and gitleaks detect over this branch's commits: clean.
  • Not run locally: the full bun suite. The machine is heavily loaded, so this PR relies on hosted CI for it. No Rust changed. No TLA+ model covers helper image pins, so no model changed.

Remaining gaps

  • No native runtime qualification of the new helper. No daemon, VM or container ran it. Still unproven:
    • seed and verify through the Linux FFI adapter on both architectures;
    • VM file writer and observer behavior;
    • --pull never resolution of the new repository digest in classic and containerd image stores.
  • There is no test that drives a real saved 1.4.2 VM file manifest through restore and retirement. Only the accepted tag list is tested.
  • Storage enrolled under the 1.4.2 artifact refuses on the next storage-bearing command. Recovering it needs a separately designed succession contract, or the earlier CLI.
  • The application benchmark protocol still uses oven/bun:1.4.2-slim as its pinned small-fixture workload image, to keep cohorts comparable. Storage-bearing benchmark trials now also need oven/bun:1.4.4-slim cached.
  • Whether CVE-2026-111497 was ever reachable through the helper has not been established.

Refs HACK-1758

🤖 Generated with Claude Code


Run Proof
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

…-slim

The native file and storage helper image was the last place next still
shipped Bun 1.4.2. Catalog the registry-derived oven/bun:1.4.4-slim index,
platform manifests, configs and Bun/libc byte pins for linux/arm64 and
linux/amd64, and use the 1.4.4-slim tag for new VM file preparation.
Selection refuses any catalog entry whose Bun version is not current.

Saved storage records and VM file manifests written under the earlier
1.4.2 helper still parse, so saved stop and recovery stay available. That
artifact is no longer cataloged: storage enrolled under it refuses
verification as artifact drift and is never relabeled or re-seeded.

Refs HACK-1758

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@linear-code

linear-code Bot commented Nov 5, 2026

Copy link
Copy Markdown

HACK-1758

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T04:41:28.553661Z 7bcd47e Manual request
🔒 Security Review ✅ Completed 2026-11-06T04:45:00.489117Z 7bcd47e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 4d61ef2994

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 4d61ef2994

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…the candidate channel

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 7bcd47ea5b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 7bcd47ea5b

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

roodboi added a commit that referenced this pull request Nov 7, 2026
#633)

## What this is

Merge train batch 4 into `next`. Each component PR's reviewed head is
merged unchanged with `git merge --no-ff`, in this order: #603 is
stacked on #602, and #604 on #599. Integration conflicts are resolved in
the batch's merge commits, described below. Hosted CI on this branch is
the full-suite gate for the combined result.

| PR | Title | Linear | Head included | Review at that head |
| --- | --- | --- | --- | --- |
| #602 | feat(native): share many project roots in one development pool
| - | `33293eaa` | Codex code + security: no findings; 0 open threads |
| #603 | feat(native): add a new project root to the stopped shared pool
on up | - | `a6a301cc` | Codex code + security: no findings; 0 open
threads |
| #589 | feat(native): add hack down --remove-data for stopped native
projects | HACK-1120 | `8216321a` | Codex code + security: no findings;
0 open threads; restore/removal race closed at the runtime cleanup
boundary (`--retained-only`) |
| #599 | feat(native): review interpolated port defaults and IPv6
loopback ports | HACK-1907 | `fc41d228` | Codex code + security: no
findings; 0 open threads |
| #604 | feat(native): publish Compose ports on host loopback through
owned bridges | HACK-1907 | `17ad74b4` | Codex code + security: no
findings; 0 open threads |
| #600 | fix(storage): re-pin native file and storage helper to
oven/bun:1.4.4-slim | HACK-1758 | `7bcd47ea` | Codex code + security: no
findings; 0 open threads |

## Integration changes

- **#602** (`provider/network_update.rs`,
`docs/reference/v5-model-coverage.md`): a network update first refuses
while a share-update journal is pending (#602). Only then does it
restore a clean stop's short-home alias (#613, batch 3), so a refusal
makes no write. Both model-coverage rows are kept.
- **#589** (`src/commands/project.ts`): native `down` keeps #589's early
`--remove-data` branch and its config read after that branch, with batch
2's lazily computed lifecycle environment.
- **#604**: an all-features build failed because the
`native-config-plan` resources in `provider/graph/native/mod.rs` didn't
set #604's new `loopback_ports` field. The fix (`17ad74b4`) went to
#604's branch first, so the head included here is its reviewed head.

#603, #599 and #600 merged without conflicts.

## Validation on this branch

- CLI typecheck: pass, in a clean worktree. Biome on changed files: no
errors.
- Focused bun tests, bounded runner, the 11 changed test files plus the
startup-log and DNS preview tests: 482 pass, 0 fail.
- `RUSTUP_TOOLCHAIN=1.97.1 bun run check:local`: pass. All-features
clippy `-D warnings` with Zig 0.15.2: pass.
- Cargo:
- `--lib` filtered to `share_update`, `lifecycle`, `network_update`,
`state`, `graph` and `config_audit`: 596 pass.
- `--features native-config-plan` filtered to `provider::graph::native`,
`share_update` and `project_share`: 202 pass.
  - `project_contract`: 63 pass. `local_contract`: 14 pass.
- `bun run test:models` with the pinned TLA+ 1.7.4 jar: exit 0.
`tests/tla-runtime-models.test.ts`: 131 pass.
- `docs:prepare` + `docs:check`, `privacy:check`, gitleaks over
`origin/next..HEAD`: pass.

The final merge of `next` (`f2d53ac7`) changes no file: `next` holds
batch 3 with the same tree as the batch 3 head this branch already
contained.

Not run locally: the full bun suite, the Linux cargo jobs, docker-e2e
and native host qualification. Hosted CI covers the first three; host
qualification stays with each component's Linear issue.

After this merges, each component PR is closed with a link to the merge
commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- codesmith:footer -->
---
<a
href="https://backend.blacksmith.sh/track/run-proof?expires=1796592995&installation_model_id=17053&pr_number=633&ref=codesmith_pr_footer&repository=hack-dance%2Fhack&return_to=https%3A%2F%2Fgithub.com%2Fhack-dance%2Fhack%2Fpull%2F633&variant=run_proof_on_this_pr&signature=0bfa867ef8fed79ca75d8e8a3a69eb445c6b0bfbf57e889af50e0201d406f292"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/proof-run-proof-on-this-pr-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://pr-comments-assets.blacksmith.sh/codesmith/proof-run-proof-on-this-pr-light.svg"><img
alt="Run Proof on this PR"
src="https://pr-comments-assets.blacksmith.sh/codesmith/proof-run-proof-on-this-pr-dark-3x.png"
width="168" height="28"></picture></a>
<sup><a
href="https://app.blacksmith.sh/hack-dance/codesmith/hack/pr/633?autoLogin=true&ref=codesmith_pr_footer">Open
in [code]smith</a> · Need help on this PR? Tag
<code>@codesmith-bot</code> with what you need.</sup>

<!-- codesmith:autofix:disabled -->
<!-- /codesmith:footer -->

---------

Co-authored-by: Dimitri Kennedy <344842+roodboi@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 7, 2026

Copy link
Copy Markdown
Contributor Author

Merged into next through the train batch PR #633 (squash commit b844c76). This PR's reviewed head 7bcd47ea5b24f7942eb46ab4c0c05fbd51d72703 was included unchanged. #633 describes the integration resolutions. Closing as merged by batch; the branch is kept.

🤖 Addressed by Claude Code

@roodboi roodboi closed this Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant