Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -328,10 +328,9 @@ jobs:
docker pull alpine:3.20
docker pull alpine:3.22
docker pull node:24.22.1-bookworm-slim
# Also the cataloged native file and storage helper image
# (src/lib/native-compose-storage-witness-docker-artifact.ts).
docker pull oven/bun:1.4.4-slim
# The qualified native file and storage helper artifact stays on this image
# until its catalog pins are requalified (src/lib/native-compose-vm-file-protocol.ts).
docker pull oven/bun:1.4.2-slim
docker pull postgres:17.6-alpine
docker pull lucaslorentz/caddy-docker-proxy:2.10.0-alpine
- uses: dtolnay/rust-toolchain@1.97.1
Expand Down
3 changes: 2 additions & 1 deletion docs/reference/native-compose-benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ Actual trials require a separately qualified compiled CLI and companion compiler
their exact SHA-256 hashes and qualified source revision. Reserve the Docker test
slot before starting: the flag acknowledges coordination; it cannot detect every
other engine user. There must be no competing stateful tests. The harness requires
an existing local Unix Docker endpoint, cached `oven/bun:1.4.2-slim`, and the exact
an existing local Unix Docker endpoint, the cached `oven/bun:1.4.2-slim` workload image
and `oven/bun:1.4.4-slim` storage helper image, and the exact
already-running proxy with curl. By default that proxy must expose standard HTTP
port 80. The explicit fixture mode below accepts a separately qualified proxy with
no published ports. The harness does not pull images, start/repair global services,
Expand Down
24 changes: 19 additions & 5 deletions docs/reference/native-compose-storage-xattr-carrier.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ before hooks, intent or volume effects. PostgreSQL data roots, filesystem crash
durability and fresh-install acquisition of the pinned image remain unqualified
gates; synthetic controls do not qualify that runtime behavior.

Storage startup requires the exact cached Linux arm64 or amd64 Bun 1.4.2 helper
Storage startup requires the exact cached Linux arm64 or amd64 Bun 1.4.4 helper
image selected by the daemon architecture before any authored hook, enrollment
intent or volume effect. The command never pulls a dependency or falls back to
emulation. Uncached first-install dependency acquisition remains unsupported; the
Expand Down Expand Up @@ -49,7 +49,7 @@ is part of this source.
fixed bundled artifact. Qualification must bind this entry with its strict
codec and kernel adapter; an exact declared local Linux image, Bun executable,
libc and program digest must be checked before any carrier effect. The
dependency is an explicitly selected local `oven/bun:1.4.2-slim` image. A tag
dependency is an explicitly selected local `oven/bun:1.4.4-slim` image. A tag
alone is not an artifact pin. Bun FFI remains experimental; source inspection and
mocked ports do not qualify its behavior.

Expand Down Expand Up @@ -276,19 +276,33 @@ on the admitted shell platforms; acceptance is at most 4096 bytes per stream.
Unknown child disposition retains descriptors and the pending journal rather than
starting competing cleanup. No child redirects through a reopened named path.

The declared dependencies use the fixed Linux arm64 and amd64 Bun 1.4.2 image index
`sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61`.
The declared dependencies use the fixed Linux arm64 and amd64 Bun 1.4.4 image index
`sha256:1846adf22496bd711036aae8be147e57bdebbaea2e5264e3e0b5af1db187d9a4`.
The packaged helper pins its bytes and the qualified Bun/libc bytes separately.
Dependency/platform checks precede volume effects; every helper uses `--pull never`.
Each architecture has separate manifest/config identities and Bun/libc byte pins.
Inspection and create use the immutable `oven/bun@sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61`
Inspection and create use the immutable `oven/bun@sha256:1846adf22496bd711036aae8be147e57bdebbaea2e5264e3e0b5af1db187d9a4`
repository reference. Classic Engine stores resolve this reference to the selected
config ID; containerd stores may expose the index or manifest ID. The response
must retain one of the three qualified IDs and the exact selected platform.
The carrier policy requires that same configured reference and `--pull never`;
it never retries another image, tag or platform.
There is no image download, emulation fallback or platform substitution. Uncached
first installation remains outside this qualification.
The pins were derived from registry bytes, without a local engine: the OCI index,
each platform manifest and config, and every layer blob were checked against their
digests, then `/usr/local/bin/bun` and the architecture's `libc.so.6` were hashed
from the extracted layers. Matching pins do not qualify the helper's runtime
behavior on a daemon.

Saved receipts, intents, references and carrier journals written under the earlier
Bun 1.4.2 artifact still parse, so saved stop, inspection and explicit recovery stay
available. That artifact is no longer cataloged: storage enrolled under it refuses
verification as artifact drift. It is never relabeled, re-seeded or adopted by the
new helper; succession for such storage needs its own qualified contract.
This refusal is expected on the candidate channel: each candidate version starts
with fresh native homes, so storage prepared under the 1.4.2 helper is not carried
forward. Prepare that storage again under the current helper.
Maintain the packaged source with
`bun scripts/generate-native-storage-witness-helper.ts --check` or `--write`;
the generator refuses a changed artifact pin rather than approving new bytes.
Expand Down
10 changes: 6 additions & 4 deletions docs/reference/native-config-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ without launching a later child or inferring reaping from engine absence.
The VM client preserves the selected `docker` command name for multiplexer clients
while checking that its physical executable and selection remain unchanged.
The VM path requires an explicit eligible local Unix Docker selection and already
cached workload images plus `oven/bun:1.4.2-slim`. It neither pulls nor builds an
cached workload images plus `oven/bun:1.4.4-slim`. It neither pulls nor builds an
image. Images declaring volumes and workload pull policy `always` refuse. Hack
pins each physical workload image and its original `Config.User`; it does not
rewrite the app command or user. An omitted grant UID/GID defaults to `0:0` only
Expand Down Expand Up @@ -222,16 +222,18 @@ lifecycle/recovery qualification remain separate gates for this VM transport.
## Requirements

The file material VM path and native Compose storage witness verification both
run a fixed helper from the cataloged `oven/bun:1.4.2-slim` image. That exact
run a fixed helper from the cataloged `oven/bun:1.4.4-slim` image. That exact
image must already be in the selected daemon's local image store: the owner
never pulls, builds, emulates or substitutes another tag or platform, and
storage admission inspects and creates with `--pull never`. Before the first
file-bearing or storage-bearing native command on a daemon, pull it once:

```sh
docker pull oven/bun:1.4.2-slim
docker pull oven/bun:1.4.4-slim
```

The daemon's own architecture selects the cataloged linux/arm64 or linux/amd64
artifact; other daemons refuse. A tag is not a pin: storage admission checks the
artifact; other daemons refuse. A VM file generation prepared with the earlier
`oven/bun:1.4.2-slim` helper keeps its own pinned helper image for restore and
retirement; new preparation uses only the current tag. A tag is not a pin: storage admission checks the
exact cataloged image identity and refuses a tag that resolves to other bytes.
34 changes: 20 additions & 14 deletions src/lib/native-compose-storage-witness-docker-artifact.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,28 @@ import { isRecord } from "./guards.ts";
import { NATIVE_STORAGE_WITNESS_HELPER } from "./native-compose-storage-witness-helper-bundle.ts";
import {
captureNativeComposeStorageXattrArtifact,
NATIVE_STORAGE_HELPER_BUN_VERSION,
type NativeComposeStorageXattrArtifact,
nativeComposeStorageXattrArtifactValid,
} from "./native-compose-storage-witness-xattr-carrier.ts";
import { refuseNativeComposeStorageXattr as refuse } from "./native-compose-storage-witness-xattr-codec.ts";

/** Explicit cached dependency; the owner never pulls or builds this image. Other
* platforms require their own artifact/ABI/volume qualification before admission. */
* platforms require their own artifact/ABI/volume qualification before admission.
* Pins for `oven/bun:1.4.4-slim` were derived from registry bytes: the OCI index,
* each platform manifest and config, and every layer blob verified against its
* digest, then `/usr/local/bin/bun` and the architecture's `libc.so.6` hashed from
* the extracted layers. A tag alone is not a pin. */
export const NATIVE_STORAGE_DOCKER_ARTIFACT: NativeComposeStorageXattrArtifact =
Object.freeze({
version: 1,
imageId:
"sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61",
"sha256:1846adf22496bd711036aae8be147e57bdebbaea2e5264e3e0b5af1db187d9a4",
platform: "linux/arm64",
bunVersion: "1.4.2",
bunHash: "616f267a34278ff5ac282df37ffdfba1d7141f4f6926bca99af2cd6ef3ad32b1",
bunVersion: NATIVE_STORAGE_HELPER_BUN_VERSION,
bunHash: "354c2b049c716aca07b2e5cddb887928ac58950675d1258bb6312083917793b6",
libcHash:
"9a325944115b36e74077786b29531963d96cd55cd6c82f11750a352cdb58fab0",
"70fa4c4cf57eabeb96097cce47582d02379c4cd1de3db3e98ade4dd7089a1b13",
helperHash:
"9d9c322ee35f43e047a41fd4901926789673a072c1e0bbf9030a106e7050ba5f",
kernelAbi: 1,
Expand Down Expand Up @@ -53,29 +58,29 @@ export const NATIVE_STORAGE_DOCKER_DEPENDENCIES: Dependencies = Object.freeze({
artifact: NATIVE_STORAGE_DOCKER_ARTIFACT,
imageIds: Object.freeze([
NATIVE_STORAGE_DOCKER_ARTIFACT.imageId,
"sha256:5c51cee225076d3c7db2150683141476298062489de4660f2d1729e522641f91",
"sha256:1cb8f81099813a0ec61f99b69348f6594e188cf1689c20a0a53a5ddd23b37708",
"sha256:b0d169a695f779087d0fe8c08b5b17f0d3a927dceac2a087c87a1351340c6565",
"sha256:f545ab0408208347c62fad5822a4621a4b7fc3a7da43a1dea66dc17335535b7e",
]),
}),
"linux/amd64": Object.freeze({
artifact: Object.freeze({
version: 1,
imageId:
"sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61",
"sha256:1846adf22496bd711036aae8be147e57bdebbaea2e5264e3e0b5af1db187d9a4",
platform: "linux/amd64",
bunVersion: "1.4.2",
bunVersion: NATIVE_STORAGE_HELPER_BUN_VERSION,
bunHash:
"a83d263767d839e4d2649ca8e35d07159c7afc99afdc96d731ced29e056dda0c",
"10702245766b225e70bbf6235fd6b47214fd94b4760d00ffaed8ab0df57ac235",
libcHash:
"fa430b8f298f817a266046af84a77533185ad6fc4406c7d3787b5a0a0c207826",
"9792e3cbb541c8f44c7acf5f14f4022ea62998ecc787d326bed4d8b6547dfd92",
helperHash:
"9d9c322ee35f43e047a41fd4901926789673a072c1e0bbf9030a106e7050ba5f",
kernelAbi: 1,
}),
imageIds: Object.freeze([
"sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61",
"sha256:debbe76858f2e398d2937c1eceeb82c571ac1fcd78aadf00e9634578ac2b5ef7",
"sha256:808ae5d87c721a0c738c37d17ae4837367baa0c9c4353f5ea1762802b0fb6392",
"sha256:1846adf22496bd711036aae8be147e57bdebbaea2e5264e3e0b5af1db187d9a4",
"sha256:13b43f8bb2c63f5d89c56205c4279b85620363296a00ada33ff7ad568f3688e7",
"sha256:bca1fb36a1a809a0aec806c0a239364f4873ead989e7f069e0a47f17e0a83cfa",
]),
}),
});
Expand Down Expand Up @@ -109,6 +114,7 @@ export function selectNativeComposeStorageDockerDependency(opts: {
Array.isArray(dependency.imageIds)
) ||
dependency.artifact.platform !== platform ||
dependency.artifact.bunVersion !== NATIVE_STORAGE_HELPER_BUN_VERSION ||
dependency.artifact.helperHash !==
NATIVE_STORAGE_DOCKER_ARTIFACT.helperHash ||
dependency.imageIds.length !== 3 ||
Expand Down
22 changes: 19 additions & 3 deletions src/lib/native-compose-storage-witness-xattr-carrier.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,28 @@ const TOKEN = /^[a-f0-9]{32}$/;
const IMAGE = /^sha256:[a-f0-9]{64}$/;
const ENGINE = /^[a-zA-Z0-9][a-zA-Z0-9:-]{0,127}$/;

/** Bun version of the currently cataloged helper image. */
export const NATIVE_STORAGE_HELPER_BUN_VERSION = "1.4.4";
/** Every helper Bun version a released CLI may have written into a saved receipt,
* intent, reference or carrier journal. Earlier versions stay parseable so saved
* stop, inspection and recovery keep working; only the current version is
* cataloged, so storage enrolled under an earlier artifact refuses verification
* as artifact drift instead of being relabeled or silently re-enrolled. */
export type NativeComposeStorageHelperBunVersion =
| "1.4.2"
| typeof NATIVE_STORAGE_HELPER_BUN_VERSION;
function helperBunVersion(
value: unknown
): value is NativeComposeStorageHelperBunVersion {
return value === "1.4.2" || value === NATIVE_STORAGE_HELPER_BUN_VERSION;
}

/** Required private pins. A well-shaped pin is not an artifact/runtime qualification. */
export type NativeComposeStorageXattrArtifact = {
readonly version: 1;
readonly imageId: string;
readonly platform: "linux/arm64" | "linux/amd64";
readonly bunVersion: "1.4.2";
readonly bunVersion: NativeComposeStorageHelperBunVersion;
readonly bunHash: string;
readonly libcHash: string;
readonly helperHash: string;
Expand All @@ -42,7 +58,7 @@ export function nativeComposeStorageXattrArtifactValid(
) &&
value.version === 1 &&
value.kernelAbi === 1 &&
value.bunVersion === "1.4.2" &&
helperBunVersion(value.bunVersion) &&
(value.platform === "linux/arm64" || value.platform === "linux/amd64") &&
typeof value.imageId === "string" &&
IMAGE.test(value.imageId) &&
Expand All @@ -64,7 +80,7 @@ export function captureNativeComposeStorageXattrArtifact(
version: 1,
imageId: value.imageId,
platform: value.platform,
bunVersion: "1.4.2",
bunVersion: value.bunVersion,
bunHash: value.bunHash,
libcHash: value.libcHash,
helperHash: value.helperHash,
Expand Down
3 changes: 2 additions & 1 deletion src/lib/native-compose-vm-file-owner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ import { createNativeComposeVmFileClient } from "./native-compose-vm-file-client
import {
NATIVE_COMPOSE_VM_FILE_IMAGE,
NATIVE_COMPOSE_VM_FILE_LIMIT,
NATIVE_COMPOSE_VM_FILE_SAVED_IMAGES,
NATIVE_COMPOSE_VM_FILES_EXTENSION,
parseVmFileFacts,
parseVmFileJournal,
Expand Down Expand Up @@ -830,7 +831,7 @@ async function state(document: Document) {
const helper = parseImage(raw.helper);
requireValue(
helper.workload === "" &&
helper.reference === NATIVE_COMPOSE_VM_FILE_IMAGE &&
NATIVE_COMPOSE_VM_FILE_SAVED_IMAGES.includes(helper.reference) &&
helper.id === raw.observer.image
);
const images: Image[] = [];
Expand Down
8 changes: 7 additions & 1 deletion src/lib/native-compose-vm-file-protocol.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@ import { refuseNativeComposeFile } from "./native-compose-file-bytes.ts";
import { keys } from "./native-compose-private-state.ts";

export const NATIVE_COMPOSE_VM_FILES_EXTENSION = "x-hack-native-vm-files";
export const NATIVE_COMPOSE_VM_FILE_IMAGE = "oven/bun:1.4.2-slim";
/** Helper image tag for new VM file preparation. The owner never pulls it. */
export const NATIVE_COMPOSE_VM_FILE_IMAGE = "oven/bun:1.4.4-slim";
/** Helper tags a released CLI may have recorded in a saved VM file manifest. A
* saved generation keeps its own pinned helper image ID for restore, readiness
* and retirement; only new preparation uses the current tag. */
export const NATIVE_COMPOSE_VM_FILE_SAVED_IMAGES: readonly string[] =
Object.freeze(["oven/bun:1.4.2-slim", NATIVE_COMPOSE_VM_FILE_IMAGE]);
export const NATIVE_COMPOSE_VM_FILE_LIMIT = 2 * 1024 * 1024;
/** Docker omits optional empty image fields. Default absence only, retaining
* every present value for the image decoder's type and policy checks. */
Expand Down
7 changes: 3 additions & 4 deletions tests/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ define those two spellings.

The explicitly selected `native-config-vm-files` scenario exercises ordinary new
containers with protected VM-native material; it does not adopt existing binds.
It uses only the cached helper image `oven/bun:1.4.2-slim`, two linked worktrees, no published ports
It uses only the cached helper image `oven/bun:1.4.4-slim`, two linked worktrees, no published ports
and no image build or pull. Application-side checks independently require exact
0400/default-root and 0600/explicit-numeric owner identity, actual owner reads,
distinct-UID `EACCES`, privileged write-open `EROFS`, exact single-file mounts and
Expand Down Expand Up @@ -398,9 +398,8 @@ HACK_E2E_CLI_BIN=./dist/hack HACK_E2E_DOCKER=1 HACK_E2E_REQUIRE_DOCKER=1 bun tes
## Native file qualification

`native-config-files` requires the current compiled CLI, its matching compiler and
cached `oven/bun:1.4.4-slim` (the workload image) plus `oven/bun:1.4.2-slim` (the qualified
native file and storage helper image, which stays on Bun 1.4.2 until its catalog
pins are requalified). It pins the existing image, uses no builds or published
cached `oven/bun:1.4.4-slim`, which is both the workload image and the cataloged
native file and storage helper image. It pins the existing image, uses no builds or published
ports, and does not pull or activate DNS/trust. The primary and two linked worktrees
deliver distinct binary configs, encrypted managed secrets and empty files through
literal dollar-containing targets and a private runtime home outside the checkout.
Expand Down
2 changes: 1 addition & 1 deletion tests/helpers/native-compose-storage-docker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ const writeRows=async(path,rows)=>await Bun.write(path,JSON.stringify(rows));
const argument=key=>storageArgs[storageArgs.indexOf(key)+1];
const emit=value=>console.log(JSON.stringify(value));
const engineId="fixture-engine";
const imageId="sha256:cb3bbbb08e13a4a2ff400f24c7a2a1d5efa83f6ef8544d52d95a519631e2fc61";
const imageId="sha256:1846adf22496bd711036aae8be147e57bdebbaea2e5264e3e0b5af1db187d9a4";
const imageReference="oven/bun@"+imageId;
const labels=()=>Object.fromEntries(storageArgs.flatMap((value,index)=>value==="--label"?[storageArgs[index+1].split(/=(.*)/s).slice(0,2)]:[]));
const volumeMount=volume=>({Type:"volume",Name:volume.name,Source:"/var/lib/docker/volumes/"+volume.name+"/_data",Destination:"/storage/"+volume.storage,Driver:"local",RW:true,Propagation:""});
Expand Down
11 changes: 8 additions & 3 deletions tests/helpers/native-compose-vm-docker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { createHash } from "node:crypto";
import { appendFile, readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import {
NATIVE_COMPOSE_VM_FILE_IMAGE,
VM_FILE_IMAGE_FORMAT,
VM_FILE_OBSERVER_PROGRAM,
VM_FILE_VERIFY_PROGRAM,
Expand Down Expand Up @@ -267,16 +268,20 @@ if (args[0] === "synthetic-program") {
}
if (args[0] === "image") {
exact(["image", "inspect", "--format", VM_FILE_IMAGE_FORMAT, args[4]]);
if (!["oven/bun:1.4.2-slim", "synthetic/reader:1"].includes(args[4] ?? "")) {
if (
![NATIVE_COMPOSE_VM_FILE_IMAGE, "synthetic/reader:1"].includes(
args[4] ?? ""
)
) {
refuse();
}
console.log(
JSON.stringify({
id: args[4] === "oven/bun:1.4.2-slim" ? helper : app,
id: args[4] === NATIVE_COMPOSE_VM_FILE_IMAGE ? helper : app,
user: (await has("named-user")) ? "named" : "",
volumes: null,
labels:
args[4] === "oven/bun:1.4.2-slim"
args[4] === NATIVE_COMPOSE_VM_FILE_IMAGE
? (await has("image-label-collision"))
? { "io.hack.native-file.role": "foreign" }
: { "org.opencontainers.image.title": "Bun synthetic" }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ async function fixture(recordCreated = true) {
kernelAbi: 1,
platform: "linux/arm64",
imageId: `sha256:${"e".repeat(64)}`,
bunVersion: "1.4.2",
bunVersion: "1.4.4",
bunHash: "e".repeat(64),
libcHash: "f".repeat(64),
helperHash: "e".repeat(64),
Expand Down
2 changes: 1 addition & 1 deletion tests/native-compose-storage-command-reader.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -514,7 +514,7 @@ test(
version: 1,
imageId: `sha256:${"a".repeat(64)}`,
platform: "linux/arm64",
bunVersion: "1.4.2",
bunVersion: "1.4.4",
bunHash: "a".repeat(64),
libcHash: "b".repeat(64),
helperHash: "c".repeat(64),
Expand Down
2 changes: 1 addition & 1 deletion tests/native-compose-storage-witness-command.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ test("source CLI reports and explicitly stops saved resources for unknown enroll
version: 1,
imageId: `sha256:${"a".repeat(64)}`,
platform: "linux/arm64",
bunVersion: "1.4.2",
bunVersion: "1.4.4",
bunHash: "b".repeat(64),
libcHash: "c".repeat(64),
helperHash: "e".repeat(64),
Expand Down
Loading
Loading