Provision self-hosted Apple Silicon runner to exercise HVF runtime tests

[jserv]

Background

The current CI (.github/workflows/main.yml) runs build-macos, tidy-macos, and scan-macos on GitHub-hosted macos-15 runners. Those runners are themselves virtualized and do not expose Hypervisor.framework: hv_vm_create returns HV_UNSUPPORTED, so the build-macos job stops at make elfuse + entitlement check, and the following are never exercised in CI:

• make test-hello (assembly smoke test through the shim)
• make check (aarch64 unit tests + busybox + syscall coverage)
• make test-multi-vcpu (multi-vCPU HVF validation)
• bash tests/test-matrix.sh all (full corpus across elfuse-aarch64 / qemu-aarch64 / elfuse-x86_64 modes, including the seven Rosetta sub-suites)

Regressions in the HVF bring-up path, syscall surface, fork/clone IPC, Rosetta integration, and FUSE transport therefore only surface in local developer runs. This issue tracks provisioning a self-hosted Apple Silicon runner so the matrix runs on every accepted PR.

Hardware available: M4-based Mac mini (detect_x86_64_host_class already handles M4 for the Rosetta
fast-path baseline).

Goals

1. Run all HVF-dependent jobs on real hardware, including tests/test-matrix.sh all.
2. Keep the hosted Linux lint and macOS build-macos / tidy-macos / scan-macos jobs unchanged.
3. Keep the repository public, but tightly limit which workflow runs reach the self-hosted M4. The runner is opt-in per run, not opt-out.

Security model

The repository stays public. Public-repo + self-hosted-runner is a documented RCE vector: anyone who can land code that executes on the runner gets a shell on the M4. The mitigation stack below treats the YAML guard, the GitHub setting, and the maintainer label as independent layers; none of them is sufficient on its own.

Dedicated macOS account

The M4 must have a dedicated local account whose only purpose is running the GitHub Actions agent. This is the primary isolation boundary between runner workloads and any personal data on the machine.

Required properties of the ghrunner account:

• Standard (non-admin) account. sysadminctl -addUser ghrunner -fullName "GH Actions Runner" (interactive password prompt). Do not run dseditgroup -o edit -a ghrunner -t user admin.
• No iCloud sign-in. Not linked to any Apple ID. No iMessage, no iCloud Drive, no Keychain sync.
• Separate login Keychain. Default behavior; just confirm by inspecting ~/Library/Keychains/login.keychain-db is local-only
  after first login.
• No auto-login. The runner runs as a launchd LaunchDaemon (set up via sudo ./svc.sh install ghrunner), so it starts at boot without a GUI session.
• FileVault enabled on the boot volume. Required for at-rest protection of the runner workspace.
• No Full Disk Access, Screen Recording, Accessibility, Input Monitoring, or Developer Tools privacy grants for ghrunner. The agent does not need them; if a future step does, grant only the minimum.
• Spotlight indexing disabled on ~/actions-runner/_work/: sudo mdutil -i off ~/actions-runner/_work after first configuration. The fixture trees are large, churn often, and indexing them just wastes IO.
• Limit ownership scope: the ghrunner home directory must be mode 700 and owned by ghrunner:staff. Personal data on the primary user's account is protected by macOS's per-user permissions, but treat it as an additional layer, not the only one.

Limiting which runs reach the M4

The repository is public; PRs from anyone can be opened. The runtime-macos job must NOT run on those by default. The three controls below run in series:

1. Repository setting: Settings > Actions > General > "Require approval for all outside collaborators" (or stricter:
   "Require approval for first-time contributors who are new to GitHub"). Fork-PR workflow runs cannot start until a maintainer clicks "Approve and run". Even at that point, only the workflows on the PR head are eligible to run.
2. Workflow YAML guard. The runtime-macos job is fenced by an if: clause that requires both (a) the PR head ref to live in sysprog21/elfuse, AND (b) the PR to carry the ci-hvf label. The label is applied by a maintainer only after reading the PR diff. Hosted-runner jobs (lint, build-macos, tidy, scan) are not gated and continue to run on every PR including from forks.
3. Maintainer discipline. Never push fork PR commits to a branch in sysprog21/elfuse without a code read. Never apply ci-hvf without reading the diff. Never approve fork PR workflow runs that modify .github/workflows/.

Forbidden events: do not extend the job to pull_request_target, workflow_run, or a workflow_dispatch that checks out arbitrary refs. pull_request_target paired with actions/checkout of the PR head is the canonical way to break every guard above.

needs: build-macos exists for signal/ordering only. It is not a security boundary; malicious code can pass make elfuse and execute arbitrary commands in make check or the matrix.

Host preparation (M4 Mac mini)

After the dedicated account exists, log in as ghrunner and do the following:

1. macOS: latest stable (>= 15.x). Apple Silicon only.
2. Command-line tools:
   • xcode-select --install
   • Homebrew (arm64 prefix /opt/homebrew).
3. Install Rosetta for Linux (required for elfuse-x86_64 mode):

   softwareupdate --install-rosetta --agree-to-license
   test -x /Library/Apple/usr/libexec/oah/RosettaLinux/rosetta
   

   tests/test-matrix.sh checks this exact path before running the
   Rosetta sub-suites; without it the suites skip silently.
4. Runtime/test dependencies via Homebrew:
   • binutils (GNU objcopy; mk/shim.mk requires the GNU variant; available at /opt/homebrew/opt/binutils/bin/objcopy)
   • make (GNU make, installed as gmake by Homebrew; either invoke as gmake in the workflow or prepend
     /opt/homebrew/opt/make/libexec/gnubin to the runner PATH)
   • llvm (clang-tidy + scan-build, for parity with hosted jobs)
   • qemu (for the qemu-aarch64 ground-truth mode)
   • coreutils (provides gtimeout, expected by
     tests/lib/test-runner.sh on macOS)
   • python (3.x; for scripts/check-syscall-coverage.py and
     scripts/gen-syscall-dispatch.py)
   • wget, curl, jq
5. Cross-toolchains under /opt/toolchain (default expected by
   mk/toolchain.mk:32, LINUX_TOOLCHAIN ?= /opt/toolchain/aarch64-linux-gnu):
   • /opt/toolchain/aarch64-none-elf/ (bare-metal, for the
     assembly smoke test)
   • /opt/toolchain/aarch64-linux-gnu/ with
     bin/aarch64-linux-gnu-gcc and
     aarch64-unknown-linux-gnu/sysroot/
/opt/toolchain/ is owned root:wheel mode 755; toolchains are read-only from ghrunner's perspective. Pin the exact toolchain source, version, and SHA-256 of the tarball in this issue once chosen so the runner image is reproducible.
6. HVF entitlement (no host configuration needed -- the binary just needs the embedded entitlement that make elfuse already adds).
   Verify after first build:

   codesign -d --entitlements - build/elfuse 2>&1 \
     | grep com.apple.security.hypervisor
   

7. Filesystem: APFS, >= 30 GiB free for
   externals/test-fixtures/ (Alpine packages, linux-virt kernel,
   Rosetta fixtures fetched by tests/fetch-fixtures.sh). Note that fetch-fixtures.sh downloads pinned-by-name but does not verify SHA-256 against the index; treat the on-disk fixture set as the source of truth and cache it across runs.
8. Power: sudo pmset -c sleep 0 disksleep 0 displaysleep 10.

Runner registration

Pick the latest runner from
https://github.com/actions/runner/releases and record the version
(X.Y.Z) and SHA-256 of the asset in this issue. Asset filename omits the leading v.

As ghrunner:

mkdir -p ~/actions-runner && cd ~/actions-runner
curl -L -o actions-runner-osx-arm64.tar.gz \
  https://github.com/actions/runner/releases/download/vX.Y.Z/actions-runner-osx-arm64-X.Y.Z.tar.gz
shasum -a 256 actions-runner-osx-arm64.tar.gz    # compare to release page
tar xzf actions-runner-osx-arm64.tar.gz

./config.sh --url https://github.com/sysprog21/elfuse \
            --token <registration-token-from-Settings/Actions/Runners> \
            --name macmini-m4-hvf \
            --labels self-hosted,macOS,arm64,m4,hvf \
            --work _work

Install as a LaunchDaemon (this is the form documented by GitHub for macOS):

sudo ./svc.sh install ghrunner
sudo ./svc.sh start
sudo ./svc.sh status

Verify in repo Settings > Actions > Runners that the runner appears as Idle with the expected labels.

Workflow changes (.github/workflows/main.yml)

Append the new job. The if: clause is the second layer of the defense-in-depth stack: same-repo head ref AND ci-hvf label. The concurrency group prevents two runs from contending for the single M4 if a maintainer pushes follow-up commits.

  runtime-macos:
    name: Runtime (macOS Apple Silicon, HVF)
    needs: build-macos
    if: >
      github.event_name == 'push' ||
      (github.event_name == 'pull_request' &&
       github.event.pull_request.head.repo.full_name == github.repository &&
       contains(github.event.pull_request.labels.*.name, 'ci-hvf'))
    runs-on: [self-hosted, macOS, arm64, hvf]
    timeout-minutes: 60
    concurrency:
      group: runtime-macos-${{ github.ref }}
      cancel-in-progress: ${{ github.event_name == 'pull_request' }}
    env:
      LINUX_TOOLCHAIN: /opt/toolchain/aarch64-linux-gnu
      GNU_OBJCOPY: /opt/homebrew/opt/binutils/bin/objcopy
    steps:
      - name: Checkout
        uses: actions/checkout@v6
      - name: Build elfuse
        run: gmake elfuse
      - name: test-hello
        run: gmake test-hello
      - name: test-multi-vcpu
        run: gmake test-multi-vcpu
      - name: make check
        run: gmake check
      - name: Test matrix
        run: bash tests/test-matrix.sh all

Allowed events here are exactly push (which only fires for refs already in this repo, i.e. post-merge to main or maintainer-pushed branches) and same-repo pull_request carrying the ci-hvf label. Do not add pull_request_target, workflow_run, or arbitrary-ref workflow_dispatch.

Operational concerns

• Disk hygiene: _work/ accumulates checkouts and fixtures. Add a daily LaunchDaemon job that prunes runs older than 7 days. Keep externals/test-fixtures/ outside _work/ so it survives prunes.
• Secrets: this workflow needs none beyond the auto-injected GITHUB_TOKEN. Do not add repository secrets to the runner host, and never store anything sensitive in the ghrunner home.
• Monitoring: enable runner-offline notifications in repo Settings.
• Updates: the runner agent auto-updates by default; keep that on. macOS minor updates require a manual reboot; the LaunchDaemon brings the runner back up after boot without needing a user session.

References

• GitHub: self-hosted runner security warning for public repos:
  https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security
• GitHub: approving fork-PR workflow runs:
  https://docs.github.com/en/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks
• GitHub: macOS service installation:
  https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/configuring-the-self-hosted-runner-application-as-a-service?platform=mac
• Apple: Hypervisor.framework entitlement
  (com.apple.security.hypervisor):
  https://developer.apple.com/documentation/hypervisor
• Apple: Rosetta for Linux:
  https://developer.apple.com/documentation/virtualization/running-intel-binaries-in-linux-vms-with-rosetta

Checklist

• Repo setting "Require approval for all outside collaborators"
  (or stricter) enabled.
• Repo label ci-hvf created.
• ghrunner macOS account created: standard, no iCloud,
  FileVault on, no Full Disk Access, mdutil off on _work/.
• Rosetta for Linux installed, RosettaLinux translator path
  verified.
• Cross-toolchain version + SHA-256 pinned in this issue.
• Runner agent version + SHA-256 pinned in this issue.
• Runner registered with labels
  self-hosted,macOS,arm64,m4,hvf.
• LaunchDaemon installed (sudo ./svc.sh install ghrunner)
  and starts at boot.
• runtime-macos job added behind the same-repo + ci-hvf
  label gate.
• First green run on a maintainer PR labeled ci-hvf.
• Disk-pruning LaunchDaemon in place.