Repository navigation
Conversation
|
Actual full-size transport QA passed against Runner/supervisor exit0,47.96s total, sampled peak process-group RSS17.98MiB, minimum free disk491.61GiB. Bounded by1.5GiB RSS/60GiB free disk/20min deadline, nice10/GOMAXPROCS1/GOMEMLIMIT256MiB; no VM ran during upload. Sampled RSS does not include OS cache or account for physical APFS allocation. This demonstrates a viable streamed transport for the concrete publisher, not a completed PR5 publisher/build API. Cache namespace avoided image conversion; registry auth middleware, remote credentials/redirect/retry/resume and ambiguous commit reconciliation were not exercised. Default go-containerregistry |
f109add to
ed21906
Compare
|
Simplification update at ed21906: standalone production MachineBuildRunner replaced by internal MachineBuildBackend executing machine-specific phases through ordinary CreateBuild/runBuild. Normal manager owns queue, persisted request, deadline, private source staging/hash verification, status/log completion, provenance and image-readiness gate. Machine resources inherit base; unsupported Linux builder/cache/secret options fail admission; recovered source is verified before start. Source helper is shared without adding a Linux source-size cap. Normal-manager synthetic tests prove ready/failed status, source consumption and no publication after provision failure. Build race suite passed 3 runs excluding exactly five existing Linux config-volume tests requiring unavailable mkfs.ext4; focused image/machine tests, CGO0 machine/source tests and vet pass. CGO0 VZ OCI round-trip remains unsupported. No concrete VZ/GuestService driver, streamed publisher, recipe, secret/log stream or public HTTP machine mode: still a foundation. All PRs remain draft. This is source-level/synthetic validation, not a new live boot or production build proof. Default Codex independent review was attempted but is still blocked by authentication (HTTP401). Published atomically after checking reviewer heads with explicit per-ref force-with-lease; prior heads preserved locally. |
ed21906 to
951e8f6
Compare
|
Published Implemented concrete GuestService/VZ machine driver, bounded tar.gz/versioned JSON recipe admission, instance-locked strict stop/export, normal-job cancellation/quarantine, and scoped-auth streaming publisher with independent persisted blob verification and manifest-last digest verification. Darwin shutdown now schedules power-off after acceptance so vsock teardown cannot race the reply. Native isolated QA passed: ordinary queued build → root provisioning → source sanitation → acknowledged no-force shutdown + owned VMM exit → matching export → authenticated28.85GB streaming publication → shared image-readiness gate → fresh empty destination/cache HTTP pull → cold boot → root exec marker + source-absence check. Run483.74s, sampled test-process peak36.875MiB, minimum free313.17GiB. First attempt correctly refused export/publication when shutdown acknowledgement was lost. Final cold-boot test cleanup used ordinary short-timeout forced fallback; that stop was not used for export. Evidence limits: activation remains internal Go CreateBuild, not HTTP POST /builds/SDK. Base was a manually installed isolated development template, not an automatically provisioned or credential-sanitized production image. Identity exclusivity preserved; no rekey/fork/parallel claim. Common safe recipe logs, clean-base policy, storage quotas, native repeat/cancel/recovery and Linux runtime regressions remain gates. Repeated focused race regressions and supported full build suite pass; exactly five existing mkfs.ext4-dependent tests excluded on Darwin. Selected CGO0 synthetic tests and vet pass. Default independent autoreview remains401 authentication-blocked, not a completed clean review. All QA VMMs stopped/storage closed; original API and identity slot preserved. See updated docs/macos-builds.md for contracts and remaining gates. |
|
Published Default-off **Real HTTP/provider QA passed:**401 missing/invalid auth,403 read-only scope,202 POST /builds, ordinary queue → real guest provisioning/sanitation → acknowledged no-force export → verified scoped streaming publication → shared readiness → Ready. POST /instances cold-booted the returned image_ref; normal authenticated exec WebSocket verified UID0, the new marker and source absence. New output cold boot used converted cache; previous native run separately proved fresh pull. Private development template remains unsuitable for external distribution without credential sanitation. All QA storage closed/API stopped/slot released; original instance preserved. Repeated focused API/config/provider/scopes race tests, full config/provider/scopes race tests and vet pass. Default independent autoreview remains401-blocked; no clean review claim. Still gates: external SDK releases, safe recipe logs, clean-base policy, storage quotas, native cancellation/recovery/repeat and Linux runtime validation. Updated docs/macos-builds.md includes opt-in configuration and curl example. |
-->
✱ stlc build✅ go code · compare
✅ python code · compare
✅ typescript code · compare
Diagnostics: ❗ 0 new / 1 total error, 💡 0 new / 5 total note
Build metadata
This comment is auto-generated by stlc and is kept up to date as you push. |
0983495 to
d5871ad
Compare
49be29f to
b6a7a23
Compare
b6a7a23 to
d5fc996
Compare
d5fc996 to
a47ca78
Compare
a47ca78 to
c4a074d
Compare
chruffins
changed the title
c4a074d to
53825ac
Compare
53825ac to
37bc2d3
Compare
92f19b3 to
4812b8a
Compare
5122239 to
259b0cf
Compare
809ca6a to
f361ec8
Compare
f361ec8 to
71aa094
Compare
Shared GuestService on host-CID vsock 2222, root system agent, agent readiness probe, agent-gated exec, copy and stat, and a macOS stop floor. Darwin exec runs its own bounded lifecycle in exec_lifecycle.go. Linux exec, boot markers, stop and the guest client are unchanged.
The guest display was fixed at 1280x800 at 80 ppi. Use 3024x1964 at 160 ppi, which is 1512x982 points at 2x. Changes the display only on the macOS guest path; the Linux path is untouched.
The group watcher stopped when cmd.Wait returned. On a TTY, Wait returns as soon as the shell exits, so a timeout during the output drain never killed a background child that held the terminal open. The watcher now runs until the exec lifecycle finishes. The exit status was bounded only after cancellation. A client that stopped reading could hold the handler with no cancellation at all. The status send now has its own deadline, counted from the send. A timed-out exec whose status cannot be sent still reports the cancellation error. Tests: a TTY timeout during the drain kills the background child (failed before this change), and a stalled client cannot block the exit status.
Readiness probes, the readiness state and shutdown checked SkipGuestAgent alone. They now use GuestAgentEnabled, which also requires the image to declare the agent. For Linux guests the two are equal. macOS instances created before the boot.raw name keep their disk at overlay.raw. Start and VM-process lookup use overlay.raw when boot.raw is absent. New instances still use boot.raw. The guest-operation helper's comment no longer claims to be the only rule.
The direct exec path and the macOS lifecycle each built the command, started the PTY, ran the stdin and resize pump, and computed the exit status. Those now live in exec_common.go. The policy stays in each path: the direct path buffers stdout and waits without a bound, and the lifecycle streams output and bounds its drain. A deadline now reports 124 on the direct path too, which matches its existing comment. Before, a timed-out direct exec reported the signal exit code (-1).
Probes run concurrently with other callers on the same guest, so a failed probe evicted the pooled connection and cancelled their RPCs. KeepPooledConn only stopped that eviction. ProbeExec dials a connection for each attempt and closes it. The pooled connection is never touched by a probe. The readiness probe uses ProbeExec, and KeepPooledConn is removed.
Instances created before this change kept their disk at overlay.raw. macOS is experimental and breaking changes there are acceptable, so there is no fallback. New instances and existing ones both use boot.raw.
A kill after the leader was reaped could reach a reused process group id. Go's cmd.Wait reaps the leader, and the lifecycle needed that to read the pipes. The leader now waits for its exit without being reaped: waitid with WNOWAIT on Linux, and kqueue NOTE_EXIT on Darwin. Darwin checked on a macOS guest: the exit event fires before the reap, the group is still signalable, and a watch on an already-exited child still fires. The lifecycle reads its own pipes, drains within the bound, and reaps last. processGroup allows a kill only before reap. cmd.Cancel is now a no-op, so the group watcher is the only killer. Tests: the reuse guard now uses processGroup. A pipe-drain timeout test covers the drain on every platform. The TTY drain test is skipped on macOS, which revokes the terminal when the shell exits.
The shared exit-code helper reported 124 whenever the exec deadline had passed when the status was computed. On the direct (Linux) path that happens after the buffered output is sent, so a command that exited in time could report 124 if delivering its output outlasted the deadline. A timed-out command also changed from its signal status (-1) to 124. The direct path now reports the process status, as before the macOS work. The macOS lifecycle keeps reporting 124 on a deadline.
The macOS agent ships inside its image, so it can be older or newer than the host. A new GetAgentInfo RPC reports the agent's protocol version, build version and optional capabilities. The host calls it when a macOS agent first answers and records the reply on the instance. An agent that predates the RPC answers Unimplemented and is recorded as protocol 0. An agent newer than the host's protocol is refused, so the guest stays Initializing with that reason. Linux guests run the agent embedded in hypeman's initrd, so the host does not call GetAgentInfo for them.
An image that declares the guest agent but ships a missing, broken or incompatible one left the guest Initializing indefinitely. On macOS hosts a controller now stops a guest that has been Initializing for 10 minutes, after one last probe under the instance lock. The reason, with the last probe failure, is recorded in the exit message. Disks and logs are kept, start clears the message, and the restart policy applies as for any other stop. Linux guests are not checked.
…e, and stopped clones Stopped snapshots after a graceful stop, same-instance restore through a staged disk swap, experimental same-instance standby, and clones from a stopped source with a fresh machine identifier, MAC and SSH host keys.
Forward HTTP and WebSocket requests from /instances/{id}/ports/{port}/* to a TCP
port on a macOS guest's loopback interface, through the guest agent on vsock 2223.
Hypeman does not start or supervise services in the guest; the image does.
Opt-in macos_builds: pinned installed base, versioned recipe, strict export under the instance lock, streaming publication, abort without export, prior-builder reap. Linux builds are unchanged.
This branch has not been deployed
1 participant
Summary
PR 6 of 6. Experimental machine-image builds for macOS. Base: #514. Not needed for v0; this can land later.
macos_builds: true(requiresmacos_only) letsPOST /buildsacceptmachine_base_image(a SHA256-pinned installed base) and a versioned recipe (hypeman-macos-build.json, version 1).builds/{id}, and the build waits for the existing image-readiness gate.macos_onlymode, and Linux jobs stay untouched on disk.Linux code touched
lib/builds/manager.go: machine jobs dispatch tomacos_build_manager.go; the staged machine source is removed once the image is ready.lib/instances/stop.go: a strict stop variant used only for export; a normal stop takes the same path as before.cmd/api:macos_buildsconfig, the upload limiter, and multipart parsing for machine builds. Linux build requests are parsed as before.Limitations
The recipe format is internal (version 1). Recipe output isn't streamed. No clean-template sanitation policy, toolchain provenance or storage quota yet.
registry.urlmust be hypeman's own registry.Testing
Stack
Each PR is one commit on the one below it. macOS logic lives in its own files (
macos*.go,*_darwin.go,exec_lifecycle.go,port_forward.go); shared Linux files get small hooks such asif stored.MacOS != nil.