Generate Apple Container provisioning, images, and MCP transport (stack layer 2) - #56131
Conversation
Layer 1 defined the apple-container contract and failed closed on everything
statically knowable. This layer generates the steps that prepare a self-hosted
bare-metal Apple Silicon runner, and wires the one transport that had no route
into the NIC-less guest.
Runtime provisioning
--------------------
Four steps now run before AWF, in dependency order, so an unusable host costs
seconds rather than a multi-gigabyte pull:
1. Host preflight rejects an ineligible runner: self-hosted provenance, macOS 26+,
native arm64 (checked via RUNNER_ARCH *and* uname -m, so a runner registered as
ARM64 but executing under Rosetta is caught), kern.hv_support=1, a usable
launchd user domain, bash 4+, and a working Docker daemon with Compose.
2. CLI setup verifies a preinstalled `container` CLI inside AWF's validated range
(>=0.4.0 <1.0.0). Only when runtime-install is not false does it install the
pinned apple/container release, verified by SHA-256 AND by its
"Developer ID Installer: Apple Inc. - Containerization" signature before
`installer` runs, over non-interactive sudo. No `latest` path, no unsigned
fallback, and the post-install version is re-checked against the window.
3. Service start runs `container system start --enable-kernel-install`. The flag is
not optional: the default prompts on stdin, and a headless runner would block
until the step is cancelled. The known headless launchd failure is detected and
reported with the actual remediation rather than an opaque bootstrap error.
4. Image pull populates Apple Container's own store, which `docker pull` cannot
reach. Only the agent and appleInit images, always --platform linux/arm64, and
only when digest-pinned -- the script re-rejects a floating reference rather
than trusting its caller. Docker pre-pulls for Squid, the API proxy and the CLI
proxy are untouched.
The pinned release is deliberately 0.12.3 rather than the newest one. Every
current apple/container release is 1.x, which is outside AWF's contract range: a
major bump may relocate the real vminitd inside the init image and boot a guest
with no capability relay at all.
State is run-scoped under ${RUNNER_TEMP} by default, so nothing an earlier job
left on the persistent runner can influence this one. CONTAINER_APP_ROOT is
exported so the service, the image pull, AWF and teardown all address the same
store -- a split root would surface as AWF failing to verify an image that had
just been pulled. Teardown runs with if: always(), stopping the guest, then the
services, then removing run-scoped state, and never rewrites the agent's outcome.
MCP gateway transport
---------------------
The gateway had no route into the guest. gh-aw runs awmg-mcpg as its own Docker
container, every other runtime reaches it through network.topologyAttach, and AWF
rejects topologyAttach for a NIC-less guest. gh-aw now publishes the gateway on
macOS loopback port 9100 -- bound to 127.0.0.1 only, never 0.0.0.0 -- and declares
it to AWF as appleContainer.mcpGatewayUpstreamPort (gh-aw-firewall#7768). AWF
health-probes the port, then publishes mcp-gateway.sock into the guest, whose
relay serves 127.0.0.1:8080.
The generated MCP client config addresses that guest URL. API-key authentication,
allowed mount roots, safe-output path permissions and containerized GitHub MCP
startup are unchanged. Because the guest port is compiled into both halves of
AWF's transport contract, a non-default sandbox.mcp.port is now rejected. No
topologyAttach, host networking, Compose service, enclave flag or guest Docker
socket is introduced.
Security review fix: the published host port and the gateway's own port had always
been equal, so the Stop MCP Gateway step passed MCP_GATEWAY_PORT and POSTed to
localhost:8080 with the gateway API key in an Authorization header. Under
apple-container nothing gh-aw owns listens on host 8080, so that request would
have handed the credential to whatever local process held the port on a long-lived
runner -- silently, because the curl failure is tolerated. The published port is
now threaded through as a step output, the stop script targets it and refuses to
send the key to a non-numeric port, and check_mcp_servers.sh re-anchors every
probe on the caller-supplied origin instead of the port the gateway reports.
Teardown additionally refuses to rm -rf any application root containing "..",
which closes the one path where an un-normalised operator-supplied root could
escape the prefix check.
Portability
-----------
Generated setup scripts that now have to run on macOS were made BSD- and
bash-3.2-safe without changing Linux behaviour: download_docker_images.sh resolves
a portable pull deadline instead of requiring GNU timeout,
compute_artifact_prefix.sh falls back to shasum, resolve_docker_socket_gid.sh also
tries BSD `stat -Lf` so socket group discovery works automatically on a Mac, and
the MCP gateway scripts resolve a millisecond clock instead of assuming
`date +%s%3N`. install_awf_binary.sh selects and verifies the Darwin arm64 AWF
build, skips the Linux-only chroot cleanup on macOS rather than invoking sudo for
nothing, and no longer uses the bash-4.4 ${var@Q} expansion. Because the wider
script corpus still uses bash 4 features, the preflight gates on bash 4+ rather
than letting an unrelated script fail with a syntax error mid-run.
Across all 294 compiled workflows the Linux delta is two lines, both
semantically identical to what they replace: MCP_GATEWAY_HOST_PORT equals
MCP_GATEWAY_PORT everywhere except apple-container.
Docs and tests
--------------
The agent-runtimes reference gains a full Apple Container section: runner labels
and prerequisites, why GitHub-hosted macOS can never work, the capability
transport table, provisioning, the APPLE_VMINIT_IMAGE publication dependency,
unsupported features, diagnostics and cleanup. Agent-runtime instructions are
updated so tools do not emit an invalid configuration.
Tests cover setup-step generation and ordering, the version/checksum/signature
gates, app-root consistency, the separate image stores in both directions, digest
pinning, the MCP endpoint and transport config, shell quoting, and the negative
compatibility cases. A wasm-golden fixture pins the generated lock file, and
end-to-end compiler tests assert the host bind, the AWF config field, the guest
URL, readiness ordering and topology absence.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Hey @lpcox 👋 — excellent work on bringing the Apple Container runtime to production-ready status! This PR is well-structured, thoroughly documented, and clearly aligned with the project's roadmap. ✅ What looks great:
This is ready for maintainer review.
|
There was a problem hiding this comment.
Warning
Firewall blocked 2 domains
The following domains were blocked by the firewall during workflow execution:
clients2.google.commtalk.google.com
To allow these domains, add them to the network.allowed list in your workflow frontmatter:
network:
allowed:
- defaults
- "clients2.google.com"
- "mtalk.google.com"See Network Configuration for more information.
💥 [THE END] — Illustrated by Smoke Claude · claude · sonnet46 · 66.8 AIC · ⌖ 17.3 AIC · ⊞ 7.1K
Comment /smoke-claude to run again
| @@ -0,0 +1,24 @@ | |||
| --- | |||
| "gh-aw": minor | |||
| --- | |||
There was a problem hiding this comment.
🤖 Smoke test review comment #1 (Run 33029847256): The changeset file looks well-structured. Consider adding a link to the relevant issue for traceability.
| @@ -25,13 +25,16 @@ Use these instructions when creating or updating workflows that mention Docker, | |||
| - Docker sbx requires KVM and normally does not work on ARC DinD because the sbx daemon must run on the runner host. | |||
| - Cloud Hypervisor requires `RUNNER_ENVIRONMENT=github-hosted`, Ubuntu Linux x86_64, and `/dev/kvm`; it is not supported on self-hosted or ARC DinD runners. | |||
| - Apple Container is the inverse: it requires a self-hosted bare-metal Apple Silicon runner and is never valid on a GitHub-hosted `macos-*` label, because those runners are virtual machines without nested virtualization. | |||
There was a problem hiding this comment.
🤖 Smoke test review comment #2 (Run 33029847256): Good documentation of runtime constraints. It would be helpful to add examples for the apple-container runtime configuration.
There was a problem hiding this comment.
Pull request overview
Adds runnable Apple Container provisioning and zero-NIC MCP transport to the compiler’s existing runtime contract.
Changes:
- Generates host validation, CLI/service setup, image preparation, and teardown.
- Routes MCP through a loopback-bound gateway and AWF capability socket.
- Adds macOS portability support, documentation, and regression fixtures.
Show a summary per file
| File | Description |
|---|---|
.changeset/apple-container-runtime-provisioning.md |
Records the runtime feature release. |
.github/aw/agent-runtime-instructions.md |
Adds Apple Container authoring guidance. |
.github/workflows/*.lock.yml (294 files) |
Regenerates host-port gateway wiring. |
Makefile |
Runs the new CLI setup test. |
actions/setup/sh/apple_container_host_preflight.sh |
Validates eligible macOS hosts. |
actions/setup/sh/apple_container_pull_images.sh |
Populates Apple’s image store. |
actions/setup/sh/apple_container_setup_cli.sh |
Verifies or installs the CLI. |
actions/setup/sh/apple_container_setup_cli_test.sh |
Tests CLI version handling. |
actions/setup/sh/apple_container_start_services.sh |
Starts services and selects state roots. |
actions/setup/sh/apple_container_teardown.sh |
Stops services and removes state. |
actions/setup/sh/check_mcp_servers.sh |
Anchors probes to the published gateway. |
actions/setup/sh/compute_artifact_prefix.sh |
Adds macOS hashing fallback. |
actions/setup/sh/download_docker_images.sh |
Adds portable pull deadlines. |
actions/setup/sh/install_awf_binary.sh |
Supports Darwin AWF installation. |
actions/setup/sh/resolve_docker_socket_gid.sh |
Supports BSD stat. |
actions/setup/sh/start_mcp_gateway.sh |
Publishes and reports the host port. |
actions/setup/sh/stop_mcp_gateway.sh |
Safely targets gateway shutdown. |
actions/setup/sh/verify_mcp_gateway_health.sh |
Adds portable timing. |
docs/src/content/docs/reference/agent-runtimes.md |
Documents the preview runtime. |
pkg/constants/version_constants.go |
Defines runtime versions and ports. |
pkg/workflow/apple_container.go |
Extends compatibility validation. |
pkg/workflow/apple_container_install.go |
Generates provisioning steps. |
pkg/workflow/apple_container_install_test.go |
Tests provisioning generation. |
pkg/workflow/apple_container_mcp_test.go |
Tests MCP transport contracts. |
pkg/workflow/apple_container_test.go |
Extends runtime validation tests. |
pkg/workflow/awf_config.go |
Models upstream Apple configuration. |
pkg/workflow/awf_config_build.go |
Emits Apple transport configuration. |
pkg/workflow/codex_engine.go |
Adds provisioning to Codex. |
pkg/workflow/compiler_yaml_ai_execution.go |
Emits post-agent teardown. |
pkg/workflow/mcp_setup_gateway.go |
Separates guest and host ports. |
pkg/workflow/nodejs.go |
Adds provisioning to Node engines. |
pkg/workflow/sandbox_runtime_profile.go |
Enables runtime installation. |
pkg/workflow/sandbox_runtime_profile_test.go |
Updates profile expectations. |
pkg/workflow/schemas/awf-config.schema.json |
Adds the upstream MCP port field. |
pkg/workflow/testdata/TestWasmGolden_AllEngines/claude.golden |
Updates generated Claude output. |
pkg/workflow/testdata/TestWasmGolden_AllEngines/codex.golden |
Updates generated Codex output. |
pkg/workflow/testdata/TestWasmGolden_AllEngines/copilot.golden |
Updates generated Copilot output. |
pkg/workflow/testdata/TestWasmGolden_AllEngines/gemini.golden |
Updates generated Gemini output. |
pkg/workflow/testdata/TestWasmGolden_AllEngines/pi.golden |
Updates generated Pi output. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/apple-container.golden |
Pins complete runtime output. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/basic-copilot.golden |
Updates gateway output wiring. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/playwright-cli-mode.golden |
Updates gateway output wiring. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/smoke-copilot.golden |
Updates gateway output wiring. |
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/with-imports.golden |
Updates gateway output wiring. |
pkg/workflow/testdata/wasm_golden/fixtures/apple-container.md |
Adds the runtime compile fixture. |
Review details
- Files reviewed: 300/338 changed files
- Comments generated: 4
- Review effort level: Balanced
| image := defaultAWFImageForRole(role, imageTag) | ||
| if manifest != nil { | ||
| if override, ok := manifest[role]; ok && override != "" { | ||
| image = override | ||
| } | ||
| } |
| APP_ROOT="${GH_AW_APPLE_CONTAINER_APP_ROOT:-${CONTAINER_APP_ROOT:-}}" | ||
| PRESERVE="${GH_AW_APPLE_CONTAINER_PRESERVE:-false}" |
| bash_major="${BASH_VERSINFO[0]:-0}" | ||
| if (( bash_major < 4 )); then | ||
| fail "bash ${BASH_VERSION:-unknown} is too old. gh-aw's generated setup scripts require bash 4 or newer, and macOS only ships bash 3.2. Install a newer bash on the runner (for example 'brew install bash') and make sure it precedes /bin/bash on PATH." | ||
| fi |
| # Timeout produces exit code 124 | ||
| if [ $exit_code -eq 124 ]; then | ||
| echo "docker pull timed out for $image after 5 minutes" | ||
| echo "docker pull timed out for $image after ${PULL_TIMEOUT_SECONDS}s" | ||
| return 1 |
Stack layer 2 of 2. Targets
lpcox-apple-container-runtime-contract, notmain.lpcox-apple-container-runtime-contract): the runtime enum, profile, schema, version gate, and fail-closed validation.Depends on gh-aw-firewall#7764 (the selectable Apple Container backend) and gh-aw-firewall#7768 (the external loopback MCP gateway upstream contract). Review
28c9251..HEADto see only this layer.Runtime provisioning
Four steps now run before AWF, in dependency order, so an unusable host costs seconds rather than a multi-gigabyte pull:
kern.hv_support=1, a usable launchd user domain, bash 4+, and a working Docker daemon with Compose. arm64 is checked viaRUNNER_ARCHanduname -m, so a runner registered as ARM64 but executing under Rosetta is caught rather than trusted.containerCLI inside AWF's validated range (>=0.4.0 <1.0.0). Only whenruntime-installis notfalsedoes it install the pinnedapple/containerrelease, verified by SHA-256 and by itsDeveloper ID Installer: Apple Inc. - Containerizationsignature beforeinstallerever runs, over non-interactivesudo. Nolatestpath, no unsigned fallback, and the post-install version is re-checked against the window.container system start --enable-kernel-install. The flag is not optional: the default prompts on stdin, and a headless runner would block until the step is cancelled. The known headless launchd failure is detected and reported with the actual remediation instead of an opaque bootstrap error.docker pullcannot reach. Only the agent andappleInitimages, always--platform linux/arm64, and only when digest-pinned — the script re-rejects a floating reference rather than trusting its caller. Docker pre-pulls for Squid, the API proxy and the CLI proxy are untouched.Note
The pin is 0.12.3, not the newest release. Every current
apple/containerrelease is 1.x, which is outside AWF's contract range: a major bump may relocate the realvminitdinside the init image and boot a guest with no capability relay at all. The digest, signing identity, and install paths were verified against the real signed package.State is run-scoped under
${RUNNER_TEMP}by default so nothing an earlier job left on the persistent runner can influence this one.CONTAINER_APP_ROOTis exported so the service, the image pull, AWF, and teardown all address the same store — a split root would surface as AWF failing to verify an image that had just been pulled. Teardown runs withif: always(), stopping the guest, then the services, then removing run-scoped state, and never rewrites the agent's outcome.MCP gateway transport
The gateway had no route into the guest: gh-aw runs
awmg-mcpgas its own Docker container, every other runtime reaches it throughnetwork.topologyAttach, and AWF rejectstopologyAttachfor a NIC-less guest.gh-aw now publishes the gateway on macOS loopback port 9100 — bound to
127.0.0.1only, never0.0.0.0— and declares it to AWF asappleContainer.mcpGatewayUpstreamPort. AWF health-probes the port, then publishesmcp-gateway.sockinto the guest, whose relay serves127.0.0.1:8080. The generated MCP client config addresses that guest URL.API-key authentication, allowed mount roots, safe-output path permissions, and containerized GitHub MCP startup are unchanged. Because the guest port is compiled into both halves of AWF's transport contract, a non-default
sandbox.mcp.portis now rejected. NotopologyAttach, host networking, Compose service, enclave flag, or guest Docker socket is introduced.MCP is wired end to end, subject to the live-hardware gap below.
Security review
A read-only security specialist reviewed the staged diff. It confirmed the no-NIC guarantee, the absence of any Docker socket path into the guest, loopback-only binding, correct supply-chain ordering, and safe shell interpolation. It raised one HIGH finding, fixed in this PR:
The published host port and the gateway's own port had always been equal, so the Stop MCP Gateway step passed
MCP_GATEWAY_PORTand POSTed tolocalhost:8080with the gateway API key in anAuthorizationheader. Under apple-container nothing gh-aw owns listens on host 8080, so that request would have handed the credential to whatever local process held the port on a long-lived self-hosted runner — silently, because thecurl -ffailure is tolerated. The gateway container holds the host Docker socket and GitHub tokens, so the API key is the only thing protecting it.Fixed by threading the published port through as a step output, targeting it from the stop script (which now refuses to send the key to a non-numeric port), and re-anchoring every
check_mcp_servers.shprobe on the caller-supplied origin instead of the port the gateway reports. Teardown additionally refuses torm -rfany application root containing.., closing the one path where an un-normalised operator-supplied root could escape the prefix check.Portability
Generated setup scripts that now have to run on macOS were made BSD- and bash-3.2-safe without changing Linux behaviour: a portable pull deadline instead of GNU
timeout, ashasumfallback forsha256sum, BSDstat -Lffor Docker socket group discovery (so it works automatically on a Mac rather than requiring manual configuration), and a resolved millisecond clock instead ofdate +%s%3N.install_awf_binary.shselects and verifies the Darwin arm64 AWF build, skips the Linux-only chroot cleanup on macOS rather than invokingsudofor nothing, and no longer uses the bash-4.4${var@Q}expansion. Because the wider script corpus still uses bash 4 features, the preflight gates on bash 4+ rather than letting an unrelated script fail with a syntax error mid-run.Across all 294 compiled workflows the Linux delta is exactly two lines, both semantically identical to what they replace, because
MCP_GATEWAY_HOST_PORTequalsMCP_GATEWAY_PORTeverywhere except apple-container:Exact prerequisites
To run a workflow with
sandbox.agent.runtime: apple-container:runs-on: [self-hosted, macOS, ARM64](extra pool labels allowed). Every GitHub-hostedmacos-*label is rejected at compile time.sysctl kern.hv_support=1../svc.sh install), running as an unprivileged user.container system startregisters a per-user service; a LaunchDaemon or bare SSH session cannot start it.PATH(brew install bash).sudoonly if you want the workflow to install the pinned Apple Container package; otherwise preinstallapple/container0.12.3 and setruntime-install: false.sandbox.agent.version(orfirewall.version) pinned to an AWF release containing Move JavaScript tests and production files to actions/setup/js #7764 and docs: fix campaign command description to match CLI behavior #7768. The gate stays at v0.28.9 as instructed; bumpAWFAppleContainerMinVersionif docs: fix campaign command description to match CLI behavior #7768 ships later, and never lower it.gh-aw-firewallside, anAPPLE_VMINIT_IMAGErepository variable must be set before any release carries anapple-initimage.Validation
go build ./...clean;make fmtapplied.make golint— 0 issues.make golint-custom— no findings in the changed files (the pre-existing function-length finding from layer 1 was resolved by splitting the validator).go test ./pkg/workflow ./pkg/constants— all pass exceptTestGeneratePackageLock_UsesNormalizedWorkflowDir, which is a pre-existing macOS-only failure (/varvs/private/varsymlink resolution) reproduced on the unmodified base commit.apple_container_setup_cli_test.sh(22 assertions on the CLI version window, including that 1.x is rejected and that0.12.3sorts above0.4.0numerically) wired intomake test-scripts;resolve_docker_socket_gid_test.shstill green.shellcheck -S warningon every new and modified script introduces no new warnings.make recompile— 294/294 succeed, with the two-line delta above.make update-golden/make update-wasm-goldenregenerated; a newapple-containerwasm-golden fixture pins the generated lock file for this runtime.Live validation gap
Important
No part of this has run on real hardware. Apple Container requires Virtualization.framework, and every GitHub-hosted macOS runner reports
kern.hv_support=0— the preflight fails there by design, with no fallback. No bare-metal Apple Silicon runner is registered in this repository, so no live smoke workflow is added here;gh-aw-firewallowns the live suite (workflow_dispatchonly, protected environment, maintainer-supplied SHA,persist-credentials: false).Specifically unvalidated on hardware: the launchd LaunchAgent registration path and its failure detection,
--enable-kernel-installbehaviour on a fresh app root, virtiofs uid/gid mapping, Apple Container image-store pull throughput, and the end-to-end MCP relay hop. Everything is unit- and contract-tested, and the CLI surface (system start,system status,system kernel set,image pull/inspect/list,list,delete,stop) was verified against the realcontainer0.12.3 binary rather than assumed.Treat the runtime as preview-only until it has run green on a real runner.
✨ PR Review Safe Output Test - Run 33029847256
Warning
Firewall blocked 2 domains
The following domains were blocked by the firewall during workflow execution:
clients2.google.commtalk.google.comTo allow these domains, add them to the
network.allowedlist in your workflow frontmatter:See Network Configuration for more information.