Skip to content

SPTM Hypervisor - #8

Merged
ADevWithAnIdea merged 30 commits into
mainfrom
PR_sptm_hv
Nov 3, 2026
Merged

ADevWithAnIdea merged 30 commits into
mainfrom
PR_sptm_hv

Conversation

@ADevWithAnIdea

Copy link
Copy Markdown
Member

This PR adds support for the SPTM enabled hypervisor for the base M4 and MacBook Neo on firmware 26.6.2.

The majority of the code, including all of the SPTM emulator, was written by Niklas and carries his signed off by. In areas where I made modifications, his co-authorship is preserved.

It does NOT work on the M4 Max. I have a private branch where it does work, but I do not want to push this due to copyright reasons. See the attached document on the changes required to enable M4 Max (and presumably M4 Pro).

Tested on MacBook Neo.

Changes made to Niklas's patchset:

  • refactored some of the page table handling
  • dropped the smp patch
  • dropped the AMX patch now that the minimal commpage bits were identified
  • extended the power guard to m4 max
  • some trustcache and amcc fixes required for 26.6.2

M4 Max Bringup Requirements:

Support for my M4 max required four changes to the SPTM emulator, which I will
not contribute for copyright reasons:

## 1. Scale SPTM boot storage with guest memory

The SPTM emulator reserves memory for boot-time objects, metadata describing
each managed memory page, and the page tables needed to map the guest.
Currently, there is a 64 MiB boot arena and bookkeeping for 1024 page tables,
however, larger memory configurations require more metadata and more page
tables. My machine, with 48GB ram, requires roughly 162 MiB of reserved storage
and 1572 page tables.

The fix is to calculate the size of the reservation based on the actual amount
of memory.

## 2. Accommodate additional DART instances

The current emulator has room for 32 DART records. The M4 Max configuration exceeds
that capacity: recorded boots rejected additional DARTs, including identifiers
35 and 36.

Provide enough records for the platform's configured DARTs. The working bringup
environment raises the capacity to 64.

## 3. Preserve unused DART streams locked by firmware

Each DART can have multiple stream identifiers, allowing different device DMA
streams to have separate translation settings. During boot, firmware can lock
these settings so the hypervisor cannot rewrite them.

On the M4 Max, some unused stream slots remain locked with translation enabled
but without a valid root page table. The emulator's policy for those slots
requests a different translation setting. Its existing consistency check treats
that difference as a failure and rejects the DART power-up operation. A recorded
failure occurred on DART 23, stream 15.

Preserve these unused, firmware-locked slots when the requested policy does not
require translation and the hardware has no valid root.

## 4. Separate NVMe hardware tag capacity from queue depth

The NVMe controller uses hardware tags to track outstanding storage commands.
Its memory-management hardware supports 64 tags. The associated count register
uses a zero-based encoding, so the value representing that capacity is 63.

Configure the hardware tag count for its 64-tag capacity independently of the
queue depth. I know there was an issue here with 64 on the base M4, but for the
M4 Max using only 63 breaks.

niklassheth and others added 18 commits October 10, 2026 19:58
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Program the guarded guest event-stream state through CNTKCTL_EL12 and rescale
XNU's 24 MHz event index to the architectural counter frequency.

Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Rearm the physical EL2 timer after guest deadline changes, time-offset updates,
and host tick programming so the emulated Apple deadline is serviced on time.

Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Stage live PTE changes before publication and split existing block mappings with
break-before-make. Prepare child tables before installing their parent
descriptors, and keep asynchronous handlers out of identity-map breaks.

Co-Authored-By: Niklas Sheth <hi@niklassheth.com>
Signed-off-by: Cody Ho <codyho@stanford.edu>
Add mmu_retype_mapping() to invalidate an existing mapping and publish the break
before installing its replacement attributes and permissions. Use the live
page-table publication machinery for both phases.

Co-Authored-By: Niklas Sheth <hi@niklassheth.com>
Signed-off-by: Cody Ho <codyho@stanford.edu>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Convert all ordinary aliases of selected RAM pages between Normal-WB and
Normal-NC with cache maintenance and break-before-make. Provide an optional
dedicated NC alias for live IOMMU tables, and validate mapping state before
transitions.

Co-Authored-By: Niklas Sheth <hi@niklassheth.com>
Signed-off-by: Cody Ho <codyho@stanford.edu>
Clean framebuffer data before changing cache attributes, then invalidate all
four ordinary aliases before installing Normal-NC mappings.

Co-Authored-By: Niklas Sheth <hi@niklassheth.com>
Signed-off-by: Cody Ho <codyho@stanford.edu>
Remove the iBoot handoff lookup and special uncached mappings from MMU
initialization, including the early /vram conversion. Default RAM mappings no
longer carve out the handoff region; framebuffer mapping remains in the
framebuffer and display setup paths.

Co-Authored-By: Niklas Sheth <hi@niklassheth.com>
Signed-off-by: Cody Ho <codyho@stanford.edu>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Use the ADT-derived CPU/CPM power-register guard on T8132, T8140, T6040 and
T6041. EL2 retains control of the live power registers while the guest reads
back its writes from a width-aware shadow. Reads of registers without a shadow
continue to use the underlying hardware.

Replace the separate T8140 guard with this implementation and keep it in
hv_power_register_guard.c. Install its hooks after the guest device mappings are
established.

Co-Authored-By: Niklas Sheth <hi@niklassheth.com>
Signed-off-by: Cody Ho <codyho@stanford.edu>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Comment thread proxyclient/tools/run_guest.py Resolved
Comment thread src/hv_sptm_xnu.c Outdated
case 49: /* REG_SNAPSHOT */
ctx->regs[0] = SPTM_STATUS_SUCCESS;
return true;
case 12: /* CONFIGURE_ROOT */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading through my code again with more knowledge (and better models), I see a lot of room for improvement. There's a lot of duplicate code like this, I’ll send a PR to clean up the emulator

self.log("Guest is shutting down CPU")
self.p.hv_exit_cpu()
del self.started_cpus[self.ctx.cpu_id]
elif enc in self.sysreg_ro_overrides:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't hit either of these logs, are you hitting this path? If so, this might be neo specific

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very possibly, this is definitely required to boot on my neo-- this device appears particularly cursed, i'll do some A/B testing of the neo vs the m4 max

niklassheth and others added 7 commits November 2, 2026 23:18
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Track writable XNU commpage frames through SPTM retyping and publish the
fallback policy when the first shared region is nested. This replaces Python
patches that depend on fixed physical addresses.

Keep the existing timebase, continuous-clock and TPRO fallbacks. On the
recognized 64-bit commpage, clear only CPU capability bits 28 and 29 in both the
64-bit word and its 32-bit mirror. Either bit selects the unsupported AMX path
in the tested macOS vDSP dispatcher; all other capabilities are preserved. Check
the commpage signature before accessing these fields because other writable
commpage frames have a different layout.

This avoids locating or modifying XNU AMX globals. The commpage-only policy was
verified on Neo with About This Mac, large vDSP operations, and single- and
double-precision matrix multiplication.

Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Configure SMCR_EL2 for SME guests so primary and secondary CPUs expose the
maximum streaming vector length, including ZT0 access. Keep this independent of
the SPTM commpage fallback policy.

Assisted-By: OpenAI Codex (GPT-5.6 Sol)
Signed-off-by: Niklas Sheth <hi@niklassheth.com>
Pass amfi_allow_only_tc_override=2 in the default M4-class guest boot arguments
required by macOS 26.6.2.

Assisted-By: LLM
Signed-off-by: Cody Ho <codyho@stanford.edu>
The AMCC RORGN registers retain the host firmware physical range when XNU is
relocated in guest memory, causing validation of its kernel image to fail.
Derive the guest range from its Mach-O segments and override trapped reads.

Assisted-By: LLM
Signed-off-by: Cody Ho <codyho@stanford.edu>
Pass reads of S3_1_C15_C13_3 through to hardware locally. A boot profile
recorded 6,329 Python callbacks and 10.15 seconds of handler time in 22
seconds; local handling avoids those callbacks and their protocol
requests.

Handle per-CPU CLPC HVCs without the global hypervisor lock, retaining
HVC return-PC semantics and locking for shared SPTM services.

Assisted-By: LLM
Signed-off-by: Jan Marek <janmarek98@proton.me>
Size the boot reservation and page-table bookkeeping from managed RAM
and mapped ranges. Include per-page metadata and table storage so larger
memory configurations do not exhaust fixed boot limits.

Check allocator bounds without overflowing address arithmetic.

Assisted-By: LLM
Signed-off-by: Jan Marek <janmarek98@proton.me>
Raise the DART record capacity from 32 to 64 to accommodate the
instances configured on M4 Pro and Max platforms.

Assisted-By: LLM
Signed-off-by: Jan Marek <janmarek98@proton.me>
Program the NVMMU count for its 64 hardware tags using the zero-based
value 63, independently of the submission queue depth.

Assisted-By: LLM
Signed-off-by: Jan Marek <janmarek98@proton.me>
Set VMPIDR_EL2 from MPIDR_EL1 on primary and secondary guest entry.
Stale firmware values can select another CPU record during XNU startup
and leave a secondary CPU spinning before it comes online.

Assisted-By: LLM
Signed-off-by: Jan Marek <janmarek98@proton.me>
Allocate and initialize TXM channels for every supported CPU slot while
retaining the ADT CPU count for the guest CPU map. T6041 XNU consumes
sixteen channels even when the ADT contains fewer CPU IDs.

Assisted-By: LLM
Signed-off-by: Jan Marek <janmarek98@proton.me>
@ADevWithAnIdea
ADevWithAnIdea marked this pull request as ready for review November 3, 2026 07:58
@ADevWithAnIdea
ADevWithAnIdea merged commit a6c2ec4 into main Nov 3, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants