Repository navigation
fix(native): keep Compose-only projects on the Compose backend with closed refusals - #365
Merged
Merged
Conversation
…losed refusals Native image-project `up` and `restart` now refuse an external Compose network, a reserved ownership label (com.docker.compose.*, io.hack-local.*, io.hack.native-config.*), a build outside the import preview's basic form, or a non-arm64 service platform before branch, env, lifecycle, storage or runtime work. Each refusal is one value-free E_NATIVE_PROJECT_UNSUPPORTED line that names the feature and the HACK_RUNTIME_BACKEND=compose selection. `hack doctor` reports the same features in a `compose backend features` check, and the native candidate guide lists which Compose features keep a project on the Compose backend. Refs HACK-1291 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…se-feature-refusals-b21 # Conflicts: # src/commands/doctor.ts
This comment has been minimized.
This comment has been minimized.
…ation The HACK-1291 Compose-backend check read only the original Compose file, so an external network that a valid HACK_NATIVE_ADAPTATION selection isolates (the documented `isolatedNetworks` path) was refused before native start applied the adaptation. Native up, restart, restart --recover and native doctor now apply the selection's isolatedNetworks first. The selection is read only when the file has an external network; an invalid selection refuses with the same closed E_CONFIG_INVALID line native start gives, before any runtime request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
… CLI case Under heavy host load the stub runtime's reply can time out or arrive unreadable. Each of those lines is only reachable after the Compose-backend check passed, so the adaptation case accepts all three. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
Found 1 test failure on Blacksmith runners: Failure
|
roodboi
marked this pull request as ready for review
October 22, 2026 00:46
roodboi
added a commit
that referenced
this pull request
Oct 22, 2026
…l untracked (#369) ## Root cause Two gaps in the candidate guides (HACK-1315): 1. The candidate install guide told users not to run stable `hack` and `hack-next` against the same home at once, but it did not cover the stable `hackd` daemon. Stable `hack global up` (v4.2.0) runs `hack daemon start` whenever `daemon.autoStart` is on, which is the default (`src/commands/global.ts`, `src/control-plane/sdk/config.ts`). Other stable commands that talk to `hackd` also auto-start it (`src/daemon/client.ts`). So a user who stopped `hackd` and then ran `hack global up` to bring up routing had a stable daemon running on the same `~/.hack` during the Compose candidate run. 2. Nothing in the candidate guides said not to commit `.hack/.internal/`. A project that tracks it (Omega does) gets a dirty working tree every time a newer generator rewrites files such as `compose.override.yml`. ## Fix (docs only, no product behavior change) - `docs/guides/candidate-install.md`: new subsection "Stop stable hackd before a Compose run" with the order `hack global up`, then `hack daemon stop`, then check that `pgrep -fl "hack daemon start"` prints nothing, and no stable `hack` commands while the candidate run uses the same `~/.hack`. New subsection "Keep `.hack/.internal/` out of Git" pointing at the existing `hack doctor` generated-files check and `hack doctor --fix`. The shared-state bullet list links to both. - `docs/guides/native-candidate.md` (copied into the bundle as `README.md`): the `hack-v5` Compose paragraph now gives the same order and the `.hack/.internal/` rule. - `docs/guides/v5-dogfooding.md` step 2: links to the new hackd subsection and repeats the order. - `docs/architecture.md`: `.hack/.internal` is machine-local; do not hand-edit **or commit** it. ## Doctor warning follow-up No new code was needed. A doctor warning for tracked `.hack/.internal/` files already exists: `src/lib/doctor-generated-files.ts` ("generated files" check, pathspec `.hack/.internal`), and it also ships in stable 4.2.1. The new guide text points users to it. `src/commands/doctor.ts` was not touched (in-flight PR #365). ## Tests and evidence - Bun 1.4.2 (`mise x bun@1.4.2`). - Local focused: `tests/cli-reference-docs.test.ts`, `tests/prerelease-plan.test.ts` (checks `native-candidate.md` anchors), `tests/doctor-generated-files.test.ts`: 66 pass, 0 fail. - `bun run --cwd packages/cli typecheck` exit 0; `bun run --cwd packages/cli check` exit 0 (warnings only, none new). - Relative links and anchors in the 4 changed files: checked with an ad-hoc script, 0 broken. The script was proven to flag a bad anchor with a probe file. - gitleaks 8.30.1 over `origin/next..HEAD`: no leaks. - Full suite on the M3 at exact head `ea9b4ac8` (Bun 1.4.2): 7554 pass, 126 skip, 1 fail, across 540 files. The failure was `tests/native-authored-project-recovery.test.ts` "new no-host death uses real read-only quiescence facts and retires only its exact bound record" (`Native process live-session observation refused (changed)`). Serial reruns with `--timeout 60000` failed twice on the M3 and once locally, and each also failed "new no-host recovery retains original bindings on last-await". **The same 2 tests fail at bare `next` `ec223cdb` on the M3**, so the failure already exists on `next` and is unrelated to this change. This test does not read any doc changed here. It is reported as a separate finding. - No CLI surface or consumer agent guidance change, so `docs:cli-reference` and `generate:agent- plugins` do not apply. No unit tests were added, because this is a docs-only change. ## Gaps - I checked the commands against the source, not on a host with stable 4.2.0 installed: `global up` auto-start, `daemon stop`, and the `daemon start --foreground` argv that the `pgrep` pattern matches. - A launchd-managed `hackd` normally exits 0 on SIGTERM, so launchd (`KeepAlive.SuccessfulExit=false`) should not restart it. The guide still tells users to re- check with `pgrep` before starting the candidate. Refs HACK-1315 🤖 Generated with [Claude Code](https://claude.com/claude-code)
roodboi
added a commit
that referenced
this pull request
Oct 22, 2026
…hase (#375) ## Summary Fix unit G30: native e2e and storage budgets (HACK-1324, HACK-1320, HACK-1321). ### HACK-1324: storage preparation gets its own named budget Root cause, confirmed on three hosted first attempts (#365 twice, #370): - Every storage carrier took its absolute deadline from the start budget (`Date.now() + resolveComposeStartupTimeoutMs()` in `src/lib/native-compose-command-storage.ts`). `native-config-down-hooks` runs with `HACK_COMPOSE_STARTUP_TIMEOUT_MS=15000`, so its first storage carrier had 15 s for the dependency check, target observation and helper create/policy/start. On loaded Blacksmith runners preparation alone takes longer, and the scenario failed at 18.3-18.5 s with `storage-target`, `storage-helper-*`, `probe-timeout`/`probe-cancel`/`unclassified`. - In `native-config-build`, about a dozen full storage proofs run in the build and pre-spawn guards (`fresh()` in `buildNativeComposeImages`, then `runNativeComposeOwnedFileChild`). They consumed the 90 s start deadline, so the spawn admission refused (`compose-child` / `private-uncertain`, run 39676161586) or the outcome was `E_STARTUP_INCOMPLETE` (run 39663510147). Changes: - `HACK_NATIVE_STORAGE_TIMEOUT_MS` (`resolveNativeStorageTimeoutMs`, default 90 s, range 1,000-600,000 ms) bounds each storage carrier. It is never derived from the start budget. Default behavior on an unset host is unchanged (the old start default was also 90 s). - `createNativeComposeStartDeadline`: guard storage proofs during a native `up`/`restart` are excluded from the start budget; their monotonic elapsed time extends the start deadline. Overlaps count once, rejected proofs still count, the deadline never moves earlier, and each excluded proof stays bounded by its own carrier budget. Build and spawn admission read the live deadline. - Whole-phase cap on excluded storage time (7daca0d): the total excluded from one `up`/`restart` start deadline may not exceed 16 x `HACK_NATIVE_STORAGE_TIMEOUT_MS` or 1 hour, whichever is smaller (24 min at the 90 s default; 1 h from S = 225 s up). The deadline moves at most that far; a guard proof that would start past the cap is refused before it runs; one that ends past it refuses the phase with a closed, value-free `NativeStorageTotalBudgetError` (effect reason `storage-budget`, stage `guard-storage`). The phase ends within the start budget + the cap + at most one in-flight guard proof. - Closed effect reason `storage-budget`: a refusal that finds its carrier's own deadline passed (not cancelled) names the budget; the human summary names `HACK_NATIVE_STORAGE_TIMEOUT_MS`. - Scenario budgets: build hack commands 120 s -> 300 s, down-hooks 90 s -> 240 s. Both scenarios now log per-command duration plus closed storage phase timings from the passive phase trace, so hosted runs record measured preparation durations. No check, ownership fence, retention or recovery rule changed; per-probe caps (10-15 s) are unchanged. ### HACK-1320: networks whole-run budget - Named `NATIVE_NETWORK_FIXTURE_SCENARIO_TIMEOUT` = 90 min. 44 measured hosted passes took 2113-3494 s (median ~2650 s); this PR's own run hit 3494 s, 97% of the old 60-minute budget. 90 min keeps 1.55x the slowest pass; with cleanup and two command bounds the hard bound stays inside the 150-minute shard job. - Whole-run and cleanup budgets now gate only the start of a command; a started command keeps its full 180 s. Run 39431629301 failed because the remaining whole-run budget clipped and SIGKILLed an in-flight `down` carrier command, leaving a created helper that saved `down --recover` correctly refuses to repair. ### HACK-1321: M6 and monotonic group absence - The unrecorded-command group-absence deadline is now monotonic (`performance.now`), named `UNRECORDED_GROUP_BUDGET_MS` (3 s). - New fake-clock test pins the `probe-budget` refusal on the no-record path. - Carrier doc lists the fixed 10-15 s probe caps. ### Release note - Storage carriers no longer follow `HACK_COMPOSE_STARTUP_TIMEOUT_MS` in either direction (a short start budget no longer shortens them, a long one no longer lengthens them). Hosts that set `HACK_COMPOSE_STARTUP_TIMEOUT_MS` should also review `HACK_NATIVE_STORAGE_TIMEOUT_MS`, which now sets both the per-carrier bound and the whole-phase storage cap. ## Evidence ### Rework at 7daca0d (verifier gaps) - Wiring test: `tests/native-compose-start-storage-exclusion.test.ts` runs the real source CLI `up` with a 3 s start budget. The Docker stand-in reads the CLI's own phase trace (stderr written to a file, so it is on disk before the carrier spawns Docker) and delays the carrier dependency check (`image inspect`) by 4 s only while a `guard.storage` span is open. No call counting. About 43 s on a loaded Mac (the earlier prototype took about 80 s). Mutation: with `start.excludeStorage(...)` removed around the guard's `storage.verify`, the test fails (`up` exits 1, the workload is never admitted); restored, it passes. - Cap tests in `tests/compose-startup-budget.test.ts` (fake clocks). Each of four mutations fails them: no post-proof check, no entry check, no `min` on the deadline extension, and the cap error not classified as `storage-budget`. - Docs: `docs/cli.md`, `docs/reference/native-compose-commands.md` and `docs/reference/native-compose-storage-xattr-carrier.md` describe the per-proof bound, the phase cap and how the two variables interact. - Gates: CLI and root typecheck, root `bun run check` (privacy-check ok), biome on the changed files (only an existing complexity warning in `native-compose-command.ts`), gitleaks 8.30.1 over `origin/next..HEAD` (no leaks). - M3 full suite at 7daca0d: 7721 pass, 126 skip, 1 fail (`repository catalog > the checked-in .hack/tasks.json recipes are all reviewed`). It passes locally at the same head and in hosted `test`, and this PR does not touch `.hack/` or the task catalog. I could not rerun it on M3 next because SSH to M3 failed (local agent signing error). - Hosted CI at 7daca0d: all checks pass, including all four docker-e2e shards. ### Measured storage durations (hosted shard 4, this branch, 9 attempts) From the new per-command phase-trace lines: | Measure | Range across 9 attempts | | --- | --- | | Longest single storage proof | 3.6-11.7 s | | Cold enrollment (`storage.enroll`, first `up`) | 11.7-35.0 s | | `native-config-build` first `up` | 50-176 s, of which 16 proofs took up to 145 s | Old behavior on the slowest attempt: the 145 s of guard proofs alone exceed the 90 s start budget, and the 35 s cold enrollment exceeds the 15 s budget the down-hooks scenario gave every carrier. New budgets: 90 s per carrier (2.6x the slowest observed carrier work, 7.7x the slowest proof); build hack commands 300 s (1.7x the slowest `up`); down-hooks hack commands 240 s. ### Hosted CI on this branch (11 attempts across 0af0fb6, 94ce188, cee65eb, 4e2b056) | Scenario | Passes | Durations | | --- | --- | --- | | native-config-build | 11/11 | 116.7-336.9 s | | native-config-down-hooks | 11/11 | 84.9-181.4 s | | native-config-networks | 11/11 | 1766-3494 s | The merged head 4e2b056 (next a25160d merged in) passed both of its full runs (run 39721132414 attempts 1 and 2). Before this change, shard 4 failed on first attempts of #365, #370, #374 and on next itself (39663080763, 39670689406). Other attempt failures, all unrelated to these files: `tests/native-authored-project- start.test.ts` frontend cascade (also seen on 6+ other branches and next), `tests/private-env- file.test.ts` symlinked `.env` ENOENT, and one runtime-core Rust test (`rebind::tests::healthy_fixed_selection_is_noop_and_complete_shared_slot_refreshes_together`). ### Gate vs PR shard-4 comparison Same runner pool and labels (`blacksmith-4vcpu-ubuntu-2404`, runner 2.337.0), identical shard composition and order, no leftover volumes before `native-config-build` (end-of-job cleanup volumes are the failed scenarios' own retained fixtures). The failing jobs (including next push run 39663080763) ran unrelated earlier scenarios 1.3-1.9x slower, so the difference is runner speed, not composition or state. The logs print no `docker info` storage-driver lines, so that dimension could not be compared. ### Mutation proofs (each killed by a focused test) M6 (`groupAbsent` deadline -> plain `refuse()`), M6b (wall-clock deadline), carrier deadline from the start budget, no start-deadline exclusion, nested exclusion double-count, build deadline read once, carrier reason ignores budget, budget reported over cancellation, xattr lifetime unmarked, effect reason ignores mark, summary not naming the variable, networks command clipped to the whole-run remainder. ### Local and M3 gates - `bun run typecheck` and `bun run check`: 0 errors at 4e2b056. - gitleaks 8.30.1 over `origin/next..HEAD`: no leaks. - M3 full suite at 0af0fb6: 7587 pass, 126 skip, 0 fail. - M3 full suite at 4e2b056 (host load 25-39): 7681 pass, 126 skip, 12 fail. 10 of the 12 pass when their files rerun serially; the other 2 (`tests/native-authored-project-recovery.test.ts` no-host death/recovery) fail the same way on plain next a25160d. None touch the changed code. - M3 is not a qualifying host for the native e2e scenarios; hosted CI is the e2e proof. ### Gaps - The whole-phase cap is proven at unit level only. A command-level cap test cannot isolate it, because at S = 1 s the per-carrier budget fires first. - An in-flight guard proof is not aborted when the cap is reached. It finishes within its own per-carrier bound, then the phase refuses. - The networks scenario still SIGKILLs a command that exceeds its own 180 s bound; that genuine hang still leaves storage evidence for explicit recovery. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
roodboi
added a commit
that referenced
this pull request
Oct 22, 2026
…ough it, and support adopted projects (#392) ## Summary Two `hack doctor` fixes. - **HACK-1316:** doctor no longer follows a symlinked `.hack` or `.hack/.internal`. Before this fix, `doctor --fix` deleted a file outside the project. - **HACK-1311:** doctor now reports an adopted (`hack config adopt`) project correctly, and `doctor --fix` completes in one. Before this fix, doctor refused the project as native input, the `--fix` plan threw, and `--fix` with tracked generated files failed after it had already changed the Git index. ## Rework after review (commit `ff7f11e1`) The verifier found one blocking defect in the first version (`ffbc4ffd`): - **Defect.** In an active adopted project whose generated files are tracked by Git (for example a committed `.hack.secret.key`), `hack doctor --fix --no-interactive` exited 1 with `E_NATIVE_PROJECT_UNSUPPORTED`. It did this after `git rm --cached` had already run, so `.hack/.gitignore` was not written and the later fix steps (hackd, Docker networks, Caddy and the rest) never ran. - **Root cause.** `maybeUntrackGeneratedFiles` selected the adopted project, ran `git rm --cached`, and then called `ensureHackDirGitignore`. That writer calls `assertLegacyProjectDirectory`, which throws `ProjectInputSelectionError("native")` for an active adoption (`src/lib/project-input-selection.ts`). - **Fix.** For an adopted project, the step now changes only the Git index. It still adds a committed `.hack.secret.key` to the root `.gitignore`, which is not an adoption input. It leaves the adoption's `.hack/.gitignore` unchanged and says so in a closed note: `Removed N path(s) from the git index. Left .hack/.gitignore unchanged: an adopted Compose project's .hack/ belongs to 'hack config adopt'.` The plan line names the same change. Legacy projects behave exactly as before. - **Docs.** `docs/reference/native-compose-adoption.md` now matches the code: `--fix` leaves `.hack/.gitignore` unchanged and adds a committed key to the root `.gitignore`. - **Test gaps closed.** - The `findDoctorLegacyProjectContext` guards on the later `--fix` steps had no regression test. The new test runs the whole `--fix` flow with Docker reachable and `--migrate-env- config`, so the lifecycle, incomplete-runtime and env-migration steps run against the adopted project. - The writer's refusal before `mkdir` (HACK-1316) had no test. A new unit test covers it. ## Update: merge with `next` at `5c9cc82e` (commits `0d6aa587`, `51244fc9`) - `0d6aa587` merges `origin/next` at `5c9cc82e` (#375, #407, #412) with no conflicts. #407 changes the same `.hack/.gitignore` code in doctor: the repair helper now returns `{ changed, readOnly }` and the untrack step can leave a read-only file alone. After the merge, the adopted branch of the untrack step still skips the `.hack/.gitignore` writer, and only the legacy branch uses #407's read-only handling. - `51244fc9` is a one-line docs fix. `docs/reference/native-compose-adoption.md` listed every check that doctor skips for an adopted project except the `.hack/.gitignore` check. The code already skips it. - **The verifier's repro, rerun on the merged head.** It is the verifier's own scratch test: `retainedSourceBindFixture`, a real `git init`, `git add -f .hack.secret.key`, adopt (prepare and publish), then `runCli doctor --fix --no-interactive`, with shell `run` mocked. The output has no `E_NATIVE_PROJECT_UNSUPPORTED`. `git ls-files` is empty, `.hack/.gitignore` is not created, the closed note above is printed, the adoption is still `active`, and the later steps run (the global `daemon start` call and the closing `Re-run: hack doctor` note). - **Why that run still exits 1.** Doctor's exit code comes from the checks that run before `--fix`. This is existing behavior and is the same for legacy projects. The verifier's harness makes every non-Git `exec` fail, so `docker daemon` is an error, and the committed key is an error in the `generated files` and `env key` checks. The PR's `doctor-fix-adopted-project` test makes Docker reachable. Its `.hack/.env.state.json` case exits 0. Its key case exits 1 only because of the committed-key error. - **Mutations rerun on the merged head**, each killing its test for the intended reason: | Mutation | Result | | --- | --- | | Adopted untrack step writes `.hack/.gitignore` again (`ffbc4ffd` behavior) | `doctor-fix- adopted-project` 0/2, with `E_NATIVE_PROJECT_UNSUPPORTED` in the output | | `findDoctorLegacyProjectContext` no longer skips an adopted project (survived the first review) | `doctor-fix-adopted-project` 0/2, with `E_NATIVE_PROJECT_UNSUPPORTED` | | Writer's pre-`mkdir` refusal removed (survived the first review) | `compose-env-at-rest` 15/16; "the writer refuses a symlinked .hack before creating .internal behind it" fails | ## Root cause ### HACK-1316: symlinked `.hack/.internal` `src/lib/compose-env-at-rest.ts` checked that `.hack/.internal` was a real directory, but only inside `readReplaceablePrivateFile`, and it never checked `.hack`. When that read refused, `inspectEntry` called `lstat(path)` to choose between `absent` and `unverifiable`. That `lstat` followed the `.internal` link, found the outside `compose.env.override.yml`, and classified it as `unverifiable`. With `includeUnverifiable`, the `--fix` removal then ran `lstat`, `rename` and `unlink` through the same link, which deleted the outside file. The stale-aside sweep ran `readdir` on the linked directory before any symlink check, so it could also unlink `.remove`-shaped files in the outside directory. ### HACK-1311: adopted projects An active or pending adoption makes `inspectProjectInputsAtRoot` classify the root as `native`. Doctor's first step, `assertLegacyProjectDiscovery`, therefore threw `E_NATIVE_PROJECT_UNSUPPORTED`. Every project check and `buildDoctorRemediationPlanLines` call `findProjectContext`, which throws the same refusal. Every `--fix` step that writes legacy project files does too. ## Fix ### HACK-1316 (`src/lib/compose-env-at-rest.ts`) - One no-follow ownership check now covers both `.hack` and `.hack/.internal`. The writer and every read, removal and sweep use it. - A new `refused` state covers these cases: - a symlinked or non-directory `.hack` or `.hack/.internal`; - a symlinked or non-regular override. Doctor reports it with a closed, value-free message. A `refused` override is never eligible for removal. - Removal returns before the sweep when the directories are not real. It checks the directory identities again before the move and before each aside unlink. The aside listing runs only on the held directory. - The writer refuses a linked `.hack` or `.internal` before `mkdir`, so it creates nothing through the link. ### HACK-1311 (`src/commands/doctor.ts`, `src/lib/doctor-adopted-project.ts`) - Doctor selects an active or pending adoption first, with `selectLegacyComposeAdoptedRoot`, the selector that adopted `ps`, `logs` and the other adopted commands use. The legacy-discovery refusal runs only when no adoption is found. - **Checks for an adopted project:** - The `project` check names the adopted root. A pending adoption is a warning that asks for explicit recovery. - The env key, tracked generated files and Compose env at rest checks run against the adopted root. - Each check that reads adoption-owned inputs is listed as not checked. - Agent integrations use the adopted root, and sessions mux uses global settings. - **`--fix` for an adopted project:** - The plan says it leaves the adoption's inputs, retained containers and state alone, and it skips env migration. - Lifecycle, incomplete-runtime and env-migration repairs treat the project as having no legacy project. - Untracking generated files changes only the Git index (see the rework above). - The legacy project-check block moved unchanged into `runProjectChecks`, to stay under the complexity limit. The #365 compose backend features check and the #357 env-at-rest adopted fallback are unchanged. There are no CLI surface changes and no agent-guidance changes. ## Tests | Test | Proves | Mutation that kills it | | --- | --- | --- | | `doctor-fix-adopted-project` (new), 2 cases: a committed `.hack.secret.key` and a committed `.hack/.env.state.json` | Real `runCli doctor --fix --no-interactive --migrate-env-config` in an adopted fixture with a real `git init`. Docker is reachable through the mock, and Git runs only on the fixture with no global or system config. Each case checks:<ul><li>no `E_NATIVE_PROJECT` and no canary or key text;</li><li>the exact plan line and step note;</li><li>the closing notes, which prove every later step ran;</li><li>`git ls-files` is empty and the file is unchanged on disk;</li><li>the root `.gitignore` gets the key entry only in the key case, and `.hack/.gitignore` is not created;</li><li>the receipt is unchanged, the adoption is still active, and no stub engine command or project `run` call happens.</li></ul>Exit code: 0 for the env state file. For the key it is 1, because the `env key` check reports the committed key as an error before `--fix` runs. That is existing doctor behavior. | <ul><li>`ffbc4ffd` behavior (always write `.hack/.gitignore`): 0/2 pass, exit 1.</li><li>`findDoctorLegacyProjectContext` no longer skipping adopted projects: 0/2 pass, exit 1.</li><li>Plan line reverted to the legacy wording: 0/2 pass.</li></ul> | | `compose-env-at-rest`: "the writer refuses a symlinked .hack before creating .internal behind it" (new) | A link to an outside directory without `.internal` gets nothing created behind it, and the refusal is closed. | Writer's pre-`mkdir` refusal removed: fails (this mutation survived before). | | `doctor-fix-noninteractive`: "doctor --fix leaves a file behind a symlinked .hack/.internal untouched (HACK-1316)" | Real `runCli doctor --fix`: the outside file survives, and the plan and notes do not name it. | `origin/next`'s helper: fails with ENOENT, because the outside file was deleted. | | `compose-env-at-rest`: symlinked `.hack` and `.hack/.internal` (2 cases) | `refused` state, closed message, no repair planned, no asides removed with a stale-aged clock, writer refusal, outside contents byte-identical. | Old helper: both fail. Directory list without `.hack`: the `.hack` case fails. `inspectEntry` always `unverifiable`: the symlinked-override case fails. | | `doctor-adopted-project`: `doctor --json` and the `--fix` plan in an adopted project | Closed project message, the checks that run, the not-applicable list, the exact plan, the receipt unchanged, no container start or stop. | Adopted branch of `runProjectChecks` disabled: fails. Preflight forced back to legacy discovery: fails. | All tests use temporary directories only. They do not touch the real HOME, `~/.hack`, the Docker socket, the keychain or launchd. ## Evidence - **Toolchain.** Bun 1.4.2 (`mise x bun@1.4.2`), `bun install --frozen-lockfile`. `bun.lock` is unchanged. - **Local gates on the current head `51244fc9` (M5):** - `bun run --cwd packages/cli typecheck`: 0 errors. - Root `bun run check`: exit 0, including `privacy-check: ok` (34 existing warnings in other files). - Biome on the 7 changed source and test files: clean. - gitleaks 8.30.1: `gitleaks dir` over a `git log -p origin/next..HEAD` export: no leaks. - **Focused tests at `51244fc9`, each file run on its own with `--timeout 120000` (the M5 was under heavy load), all passing:** - `compose-env-at-rest` 16, `doctor-fix-noninteractive` 10, `doctor-adopted-project` 2, `doctor-fix-adopted-project` 2 - `native-compose-backend-features` 13, `native-compose-backend-features-cli` 15, `compose- env-at-rest-cli` 9 - `doctor-command` 28, `doctor-json` 3, `doctor-generated-files` 11, `doctor-env-key` 31, `doctor-utils` 8, `doctor-worktree` 6 - From #407 and #412, which touch the same code: `hack-gitignore` 14, `project-env-config` 58, `project-env-hydrate` 18, `compose-env-shared-references` 6 - The five doctor `--fix` and at-rest files also pass together in one process (41/41), which checks that the scoped module mocks do not interfere. - Under the same load, `doctor-fix-adopted-project` with Bun's default 5 s timeout failed in a fixture hook. It passes with the longer timeout. This is load, not a code change. - **Full suite on the M3: not run.** The runner for `91dae070` waited about 6.5 hours and never got the shared CI lock. I stopped it and queued a runner for `51244fc9`, which waited about 50 minutes. During that time the lock directory had existed since 12:37 with no suite process running on the M3, several other units were also waiting, and free space was about 57.6 GiB, under the 58 GiB floor. I did not touch the lock, since I could not prove who owned it. I stopped my runner. It created no worktree or ref. - **Hosted CI:** all 16 checks pass on the current head `51244fc9` (run 39839971608): the full- suite `test` job (39 min), all four `docker-e2e` shards, candidate core, config compiler, runtime state models, and secret scan. Earlier heads `0a94d881`, `a2470049` and `91dae070` also passed. ## Remaining gaps - **No M3 full suite.** It did not run because the shared lock was blocked and free space was under the floor (see Evidence). The hosted full-suite `test` job ran at `51244fc9` on ubuntu, not on macOS. - **Symlinked `.hack` now refused by the writer.** A Compose command that renders the env override now refuses a symlinked `.hack`, not only a symlinked `.internal`. This matches the existing `.hack/.env` rule. A project that relied on writing through a linked `.hack` will see a closed refusal. - **Small race window remains.** Directory identities are checked again before each move and unlink, not held as open descriptors. A same-user swap in that window remains, as already documented for the move-aside window. - **Other generated files can be committed again.** For an adopted project, `--fix` does not add the untracked generated files under `.hack/` to any ignore file. Only the key gets a root `.gitignore` entry. The note asks the user to keep them out of later commits. - **`--fix` with a committed key exits 1.** The exit code reflects the checks that ran before the fix, for legacy and adopted projects alike. The fix itself completes and removes the key from the Git index, which is what the tracked-key checks read. - **Pending adoption not tested.** The pending-adoption warning has no test: the fixture has no cheap pending state. - **Native backend skips adopted project checks.** With `HACK_RUNTIME_BACKEND=native`, native doctor still skips project checks for an adopted project. This is unchanged. Refs HACK-1316 Refs HACK-1311 🤖 Generated with [Claude Code](https://claude.com/claude-code)
roodboi
added a commit
that referenced
this pull request
Oct 24, 2026
…ls, null platform and external volumes (#404) ## Root cause The native Compose-backend classifier from #365 (`src/lib/native-compose-backend-features.ts`, used by native `hack up`/`hack restart` and `hack doctor`) had four gaps: 1. **The reserved label set was written by hand and was incomplete.** It listed `com.docker.compose.`, `io.hack-local.` and `io.hack.native-config.`. It missed `io.hack.storage-witness.*` (storage witness carriers) and `io.hack.native-file.*` (VM file ownership). A service labeled `io.hack.storage-witness.owner: x` got the general native configuration refusal, not the closed Compose-backend refusal. No single source listed Hack's ownership label namespaces: the keys were string literals in about 16 TypeScript files and in runtime-core. 2. **`platform: null` was classified as a foreign platform.** Compose and native review both treat a null platform as unset (runtime-core's `optional_text` filters null). 3. **External volumes were not classified.** Runtime-core refuses them as `external_volume`, but the classifier did not report them. 4. **`checkComposeBackendFeatures` was placed between the HACK-1127 JSDoc and `checkProjectEnvKey`**, so that JSDoc was attached to the wrong function. ## Fix - New `src/lib/hack-ownership-labels.ts` exports `HACK_OWNERSHIP_LABEL_NAMESPACES` (`io.hack- local.`, `io.hack.native-config.`, `io.hack.native-file.`, `io.hack.storage-witness.`). This list is now the single source of truth. `RESERVED_OWNERSHIP_LABEL_PREFIXES` is `com.docker.compose.` plus that list, and the refusal and doctor wording are built from it. - A parity test (`tests/hack-ownership-labels.test.ts`) scans `src/**/*.ts`, `packages/*/src/**/*.{ts,rs}` and `packages/*/tests/**/*.rs`. The scan is read-only. It checks two things: every `io.hack*` label key falls in a declared namespace, and every declared namespace is used somewhere. The two files that only restate the list are left out of the scan. - An absent or `null` `platform` is unset. Any other non-native value, including `""`, numbers and booleans, is still `foreign_platform`. - New closed feature `external_volume` (ordered after `external_network`). It covers `volumes.<v>.external: true` and the legacy `external: {name: ...}` form. It appears in the refusal, the doctor text and JSON (`composeBackend.features`), and the docs table. - Moved `checkComposeBackendFeatures` above the HACK-1127 JSDoc, so `checkProjectEnvKey` has its JSDoc again. - Documented that only `.hack/docker-compose.yml` and the explicit `platform` field are classified. Overrides, `extends`, `include` and `DOCKER_DEFAULT_PLATFORM` are not followed, which matches native start, because `prepareNativeProjectInput` reads only that one file. Extending the classifier to them is not cheap: native start does not merge them either, and the native runtime's own review reports `extends` and `include` as unsupported fields. The CLI interface did not change, so `docs/reference/cli.md` is not regenerated. Consumer agent guidance and `package.json` scripts did not change. ## Tests (each new behavior was checked by a mutation) | Mutation | Tests that failed | | --- | --- | | Remove `io.hack.storage-witness.` from `HACK_OWNERSHIP_LABEL_NAMESPACES` | parity scan, reserved-namespace unit test, CLI storage-witness refusal, CLI reserved-label and combined-feature refusals (5) | | Add a stale `io.hack.unused.` namespace | "every declared ownership namespace is written or verified somewhere" | | Treat `platform: null` as foreign again | unit platform test, CLI null-platform test (2) | | Disable `external_volume` detection | unit external-volume and order tests, CLI external-volume refusal, CLI combined refusal (4) | | Accept only `external: true` (drop the legacy `{name}` form) | unit external-volume test | New CLI tests use the real source CLI with a stub runtime. They show the following: - `io.hack.storage-witness.owner` refuses native `up` with the closed `E_NATIVE_PROJECT_UNSUPPORTED` line. The runtime received no requests and no files were written. - An external volume is refused the same way. - A null platform passes the check and reaches `runtime up`, so native review admits it. - `extends` and `include` are not followed into another file. - Native doctor JSON reports `external_volume` and leaves out the volume name. ## Evidence - Bun 1.4.2 (`mise x bun@1.4.2`), `bun install --frozen-lockfile`. `bun.lock` is unchanged. - Local (this Mac): `bun run --cwd packages/cli typecheck` passed. Root `bun run check` passed, including `privacy-check: ok` (only existing warnings). Biome on the changed files is clean. - Focused tests, local: `tests/hack-ownership-labels.test.ts`, `tests/native-compose-backend- features.test.ts` and `tests/native-compose-backend-features-cli.test.ts` all pass. So do `tests/doctor-command.test.ts`, `tests/doctor-json.test.ts`, `tests/doctor-env-key.test.ts` and `tests/cli-reference-docs.test.ts`. - gitleaks 8.30.1 over `origin/next..HEAD` (a `git log -p` export scanned with `gitleaks dir`): no leaks. - Merged the current `next` (d0c8184) into the branch as d4e4500 (fast-forward push, no force). The merge was clean. At d4e4500: CLI typecheck passed, root `bun run check` passed (`privacy-check: ok`), Biome on the changed files is clean, and the 98 focused tests above pass. gitleaks 8.30.1 over `origin/next..HEAD` found no leaks. - Mutations re-run at d4e4500: dropping `io.hack.storage-witness.` fails 2 unit tests; treating `null` as foreign fails 1; disabling external-volume detection fails 2; the unmutated baseline passes 17/17. - Full suite on the M3 at d4e4500: pending. It is queued behind another unit's long loaded- repetition run on the shared M3 lock. This description will be updated with the result. ## Gaps - The TypeScript and Rust writers still spell their label keys inline. They do not import `HACK_OWNERSHIP_LABEL_NAMESPACES`. The parity scan is what enforces the list: a new `io.hack*` label namespace anywhere in those trees fails the test until it is declared. Swift and other packages are not scanned, and none of them write `io.hack*` labels today. - The doc statement that the runtime review refuses `extends` and `include` comes from runtime- core's `keys()` allow-list (`unsupported_field`). The stub-runtime CLI test cannot exercise the real runtime review. - No native host or VM run was done. The CLI tests stop at the stub runtime. Refs HACK-1337 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
Summary
User decision (2026-10-19): in v5, unchanged real Compose projects stay on the supported Compose backend. Native support for these features is post-v5 work (HACK-1270). This PR makes native
hack upandhack restartrefuse each blocking Compose feature exactly, before any mutation. It also documents the features and adds ahack doctorcheck that reports them.The explicit
HACK_NATIVE_ADAPTATIONpath stays supported. A network that a valid adaptation lists inisolatedNetworksis not treated as an external network: the check classifies the Compose file after that adaptation, as native start does.Root cause
Native image-project
up(handleNativeUp→startNativeProject) had no early classification of Compose-only features:platformwere not checked on the TypeScript side. They failed only afterruntime check-project-share, run-storage preparation, lifecyclebeforehooks andruntime up(the runtime-core review rejectsexternal_network, and registry admission selects arm64 only).NATIVE_CONFIGURATION_REFUSAL. That came after the source preflight and storage preparation, and it did not say the project belongs on the Compose backend.Rework after verification (adaptation regression)
The first version of this PR (9ce0f3b) read only the original
.hack/docker-compose.yml. It ignored the documentedHACK_NATIVE_ADAPTATIONisolatedNetworksselection, which native start applies to the normalized copy (replaceNetworkssetsx-hack-isolated). So"isolatedNetworks": ["hack-dev"]with an externalhack-devnetwork was refused with the Compose-backend line before the adaptation was read. That blockedup,restartandrestart --recover, and native doctor warned about it. Onnextthe same project passes native review and reachesruntime check-project-shareandruntime up.Fix (76003b5):
findComposeBackendFeatures(compose, {isolatedNetworks})does not count a network named in the set as external. Other features are unaffected.readComposeBackendFeatures(composeFile, {adaptationFile})reads the selection only when the unadapted file has an external network. A project without one keeps its existing refusal order.readAdaptedIsolatedNetworksreads the file with native start's ownreadNativeSelection(absolute path, bounded, unchanged regular file). If the file cannot be read, the selection is not version 1, orisolatedNetworksis not a bounded list of unique strings, it refuses with the same one-lineE_CONFIG_INVALIDmessage native start gives, before any runtime request. Native start still validates the whole selection against the normalized Compose file, including that each listed name is an external network.handleNativeUp(coversup,restartandrestart --recover) passesprocess.env.HACK_NATIVE_ADAPTATION, the same value it givesstartNativeProject. Native doctor passes the same value. Compose doctor does not, because the adaptation is native-only.docs/guides/native-candidate.mdnow describe the adaptation. The adaptation section links back to the Compose-backend section, anddocs/reference/native-config-import.mdqualifies its summary. The two sections no longer contradict each other, andisolatedNetworksis not retired.Fix (unchanged from the first version)
src/lib/native-compose-backend-features.tsclassifies the project's Compose file into a closed, ordered set:external_network,reserved_label,unsupported_build,foreign_platform.external: trueorexternal: {name}, unless it isx-hack-isolated: trueor isolated by the adaptation.com.docker.compose.,io.hack-local.orio.hack.native-config., on service, network and volume labels. Routing (caddy*), cache (hack.dependencies.*), one-shot and other ordinary labels keep their native behavior.mapLegacyComposeBuild. The result matches whathack config import --dry-runreports asinvalid_or_unsupported_buildorimage_build_exclusive.linux/arm64orlinux/arm64/v8.handleNativeUpcallsrefuseNativeComposeBackendFeaturesafter option refusal and project selection, before branch, config, env-fingerprint, lifecycle, shared-source, storage and runtime work.--detachstill gets its one-lineE_USAGEfirst (fix(cli): pass child flags through and print closed native refusals #359).downis not affected. Authored and adopted native-config projects go throughtryNativeComposeCommandfirst and never reach this check.HackCliErrorwith the stable codeE_NATIVE_PROJECT_UNSUPPORTEDand one value-free line that names every feature found and theHACK_RUNTIME_BACKEND=composeselection.detailis{backend: "compose", features: [...]}.hack doctorhas acompose backend featurescheck, in both native and Compose doctor. On native, a feature found is awarn. On Compose it is informational (ok). With--json, the check carriescomposeBackend.features(closed names only).The CLI surface (options and help) is unchanged, so
docs/reference/cli.mdwas not regenerated.Tests
tests/native-compose-backend-features.test.ts(unit, 13 tests). The original cases are kept. New in the rework:adaptNativeProjecton the same selection and classifies its normalized output the same way. A network that is not listed is still external, and isolation never hides another feature.isolatedNetworksleaves the external network.E_CONFIG_INVALIDand the exact message. For each one, the realadaptNativeProjectthrows the same message. Without an external network the selection is not read. Missing and relative paths refuse.tests/native-compose-backend-features-cli.test.ts(real CLI throughprivateCliEnv, stub runtime, 15 tests). The original cases are kept. New in the rework:hack-dev+{"version":1,"isolatedNetworks":["hack-dev"]}+HACK_NATIVE_SHARED_SOURCE=1.hack uppasses the check and reachesruntime check-project-shareand thenruntime up, as onnext.hack restartwith the same selection reaches restart preflight's ownNative restart has no existing runline.E_CONFIG_INVALIDline, with 0 runtime calls and an unchanged tree.doctor --jsonwith the selection reportsokandfeatures: [].Mutation proof for the rework. Each mutation was applied, both files run, and the change reverted with
git checkout:handleNativeUpdoes not pass the adaptationisolatedNetworksshape not validatedversionnot validatedThe first version's mutation results (removing the call, each rule, doctor wiring) still apply to the unchanged cases.
Evidence
Bun 1.4.2 (
mise x bun@1.4.2), macOS arm64 host. Load average was between 30 and 95 during these runs.bun run --cwd packages/cli typecheck: 0 errors.bun run --cwd packages/cli check: 0 errors and 34 existing warnings.biome checkon the changed files is clean.Focused and required: the two PR test files,
tests/cli-spawn-env-isolation.test.ts,tests/cli-reference-docs.test.ts,tests/integration-ownership.test.tsandtests/native-refusal-cli.test.ts: 63 pass, 0 fail.tests/doctor-*.test.ts,tests/native-project-adaptation.test.tsandtests/native-config-import*.test.ts: 539 pass, 3 fail. The 3 failures are the capture process-group tests intests/native-config-import-build.test.ts(about 505 ms each). The verifier found the same failures withorigin/nextsource, so they come from this host, not this PR.After
bun run buildandbun run build:config-compiler, all sixscripts/check-native-*-plan-cli.tsexit 0. No native refusal code or message changed. TheE_CONFIG_INVALIDadaptation message is native start's existing one.gitleaks 8.30.1: no leaks. The scan ran as
gitleaks diron agit log -p origin/next..HEADexport, because the worktree sandbox blocksgitleaks git.Full
bun run test: exit 1 after 1 h 27 min on the loaded host. 114 files had failures in the parallel run. A serial rerun of each with--timeout 60000left 20 files failing. 19 are authored/adopted native-compose suites, which never reachhandleNativeUp's check, and their failures are fixture timeouts and unsettled children. The 20th,native-env-metadata, passed 32/32 on a later rerun with this PR's code. As a baseline,src/commands/project.tsandsrc/commands/doctor.tswere swapped to the merge baseec223cdb.native-compose-down-command(1/13),native-compose-storage-fence-cost(0/4),native-compose-retained-storage-command(0/5) andnative-authored-process-incarnation(9/1) failed the same way, so these failures are inherited under this load.The adaptation CLI case also accepts a timed-out or unreadable stub reply (94e12a9). Under load, both appeared in the parallel run, and each is reachable only after the check passed. The original
control: ordinary labels…case is sensitive to the same load in the parallel run and passes serially.Hosted CI on 76003b5: every job passed except docker-e2e shards 3 and 4, which hit the same inherited failures as on 9ce0f3b:
pattern-p01-dependency-bootstrapdocker inspect … No such objectrace.native-config-buildandnative-config-down-hooks(also failing on thenextbase run 39663080763).The run on 94e12a9 was in progress when this was written. Shards 1–3 and all other completed jobs passed.
Remaining gaps
platformfield is classified. An image with noplatformthat publishes only amd64 is still refused later, at native image admission, with the existing image refusal..hack/docker-compose.ymlis classified. Compose override files and env-overlay interpolation are not evaluated. An interpolatedplatformsuch as${PLATFORM}is treated as non-native.versionandisolatedNetworksparts of the adaptation. Native start still validates the rest of the selection, and that each listed name is an external network, in its existing order. A listed network declared in the legacyexternal: {name}form passes this check. Native start's adaptation then decides it as before, becausereplaceNetworksaccepts onlyexternal: truein the normalized file.Refs HACK-1291
🤖 Generated with Claude Code