Skip to content

fix(native): deliver runtime metadata env to native containers - #317

Merged
roodboi merged 19 commits into
nextfrom
wf/native-runtime-env-b10
Oct 12, 2026
Merged

roodboi merged 19 commits into
nextfrom
wf/native-runtime-env-b10

Conversation

@roodboi

@roodboi roodboi commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Native containers did not receive HACK_BRANCH, HACK_DEV_HOST, HACK_DEV_URL, HACK_ALIAS_URL, the current-service URL fields or HACK_RUNTIME_METADATA, although docs/cli.md promised them with no backend qualifier (found by a HACK-1163 acceptance run on the M3, next ccf526c). The native up path (handleNativeUp → startNativeProject) only delivered managed project env; buildRuntimeHostMetadataOverride was used only by the Compose path.

Design: public env, not managed delivery

The suggested fix was to merge the metadata into the managed (private) environment. That is not safe in the current runtime:

  • Managed delivery wraps a service with the private launcher. The launcher requires a pinned absolute entrypoint and a numeric user (launcher::validate, image_process::apply_private). Images like busybox, node, postgres and redis would fail to start, including the accept-native-frontend.py fixture.
  • Every declared bare key must be supplied on native one-off run-service (execution_environment_missing). Fresh exec also replaces values rather than inheriting them.

So native up adds the metadata as public environment in the normalized Compose that native review checks:

  • src/lib/runtime-host-metadata.ts: a new shared buildRuntimeHostMetadataEnvironments holds the metadata logic, so both backends emit the same keys and document shape. It takes an optional HTTPS port: URLs include :PORT unless the port is 443; host variables never include a port. The Compose override bytes are unchanged (checked against the origin/next implementation for 16 input combinations, and pinned by a byte-exact test).
  • src/backends/native-runtime-metadata.ts: applyNativeRuntimeMetadata runs after branch rewriting, adaptation and AWS, so the reviewed bytes are final.
    • Inputs: the route labels (normalized the way review does: lowercase, trailing dot removed), the effective branch (the review identity's branch when deferred), the base dev/alias hosts, and <project>--<branch> as the Compose project name.
    • Keys a service already declares are skipped. That includes the bare keys that Hack-managed project env adds, so explicit project values win.
    • List-form env keeps its shape, and $ is escaped.
    • verifyNativeRuntimeMetadataRoutes refuses before serving unless every active service's reviewed routing.hostnames match the advertised hosts.
  • startNativeProject takes a new optional runtimeMetadata selection, which handleNativeUp supplies. The port is taken from opts.https. preflightNativeRestart reviews the same input.
  • Retained graphs: the run mapping records runtimeMetadata: true. A restore re-adds metadata only for graphs that recorded it. Legacy retained graphs are restored exactly as admitted, so upgrading does not break their strict restore identity. A recorded graph restored without a selection fails early, before any effect.
  • packages/runtime-core/src/project/hostname_change.rs: the hostname-change fingerprint now ignores the 10 reserved metadata keys in the normalized Compose document only (map and list env), because that is where Hack generates them. Their values follow the hostnames, so without this change, metadata would have blocked hostname-only retaining restarts (domain migration, as exercised by accept-native-frontend.py). The original .hack/docker-compose.yml is not masked: a HACK_* value the user declares there is an ordinary literal (Hack never overrides a declared key), so changing it still refuses. Any other env literal still counts. A change to metadata alone still refuses, because the graph proof requires changed routes.
  • hack exec inherits the container's static env. Native hack run uses the admitted service environment.

Consumer guidance

src/agents/instruction-source.ts said containers receive HACK_RUNTIME_METADATA, HACK_DEV_URL and the other fields without qualification. It now says Compose-backed and native image projects receive them, and authored .hack/hack.project.json projects do not yet. The plugin bundle (bun run generate:agent-plugins) and the checked-in surfaces (.codex/skills/hack-cli, .cursor/rules/hack.mdc, examples/basic/AGENTS.md and CLAUDE.md, rendered through their installers with an explicit project root) are regenerated. The content revision is now 7c86dd4dda99, and the new hack-cli-skill and cursor-rules render fingerprints are appended to KNOWN_RENDER_FINGERPRINTS.

Authored native path (tryNativeAuthoredCommand)

It has the same gap, and so does the authored Compose renderer: no HACK_* metadata anywhere. It is not covered here. Authored env is compiled by the Rust config compiler into an environment_plan, and the authored frontend has no routing. Adding metadata needs a new compiler binding kind. Docs now state that authored .hack/hack.project.json configuration does not deliver these variables yet.

Validation

  • bun test focused: tests/native-runtime-metadata.test.ts (new), tests/runtime-host-metadata.test.ts, tests/native-project-start.test.ts, restart, run, branch and input, the native-compose-adoption suites (projection, generation, branch, source-bind-generation), and tests/integration-ownership.test.ts. Result: 288 pass, 1 skip, 0 fail. New cases:
    • Branch native input carries values with port 8443, using a real project fixture, branch rewrite and project env.
    • A project-env HACK_DEV_URL overrides the metadata and stays private.
    • Base instance (no branch) has portless URLs for 443 or no port.
    • Native output has the same keys and values as the Compose builder.
    • A reviewed route mismatch refuses before serve.
    • Start and restart preflight review byte-identical metadata; legacy runs get none.
    • The mapping flag is persisted.
    • The Compose override bytes are unchanged.
  • bun run --cwd packages/cli typecheck: 0 errors. bun run --cwd packages/cli check: 0 errors (34 existing warnings in other files). Changed tests linted.
  • Rust 1.97.1: bun run check:local (fmt + clippy -D warnings) and bun run test:local both pass (1134 tests). Tests: runtime_metadata_follows_hostnames_but_other_environment_still_counts, and user_declared_metadata_keys_still_count_in_the_original (a user-declared HACK_DEV_URL change in map or list env still changes the fingerprint; generated metadata differences in the normalized document do not).
  • Polish tests, each proven by a mutation that makes it fail:
    • Rust: masking the original document too makes user_declared_metadata_keys_still_count_in_the_original fail.
    • tests/native-project-start.test.ts "restart preflight refuses runtime host metadata that differs from reviewed routes": removing the verifyNativeRuntimeMetadataRoutes call in preflightNativeRestart makes it fail.
    • tests/native-up-runtime-metadata-selection.test.ts (new, CLI level through upCommand.handler with startNativeProject and authored dispatch mocked, temp HACK_HOME, no Docker): checks that handleNativeUp passes { devHost, aliasHost, projectName } for a branch with an OAuth alias and for a base instance without one. Removing the runtimeMetadata selection makes both cases fail.
  • Guidance: integration-ownership, agent-plugin-generation, agent-instruction-source, skill-references, hack-init-skill, onboarding-* and shared-agent-skill tests pass. Root bun run typecheck and bun run check pass.
  • Combined focused run after the polish (start, restart, run, branch, input, metadata, adoption projection/generation/branch/source-bind-generation, project-up-startup-state, integration-ownership): 348 pass, 1 skip, 0 fail. One earlier run of the same set timed out in native-compose-adoption-generation while a cargo build ran at the same time. That file passes alone, and the whole set passes on a rerun without the build.
  • Unit tests inject runners; none use real Docker or OrbStack.

Gaps

  • Not run on a live native runtime. No M3 or acceptance-script run has checked that containers see the env, that one-off/exec inherit it, or that domain-migration retaining restarts still pass with metadata present. Re-run scripts/accept-native-frontend.py and the HACK-1163 check.
  • Changing HACK_NATIVE_HTTPS_PORT alone for a retained graph now refuses the retaining start. So does selecting or removing the native HTTPS frontend when its port is not 443, because the advertised URLs gain or lose the port. Both cases are documented in docs/cli.md and docs/guides/native-candidate.md; the workaround is to start with the original frontend selection and port.
  • Retained graphs from earlier releases get metadata only after their next start without retained data.
  • An unrouted graph with an HTTPS selection still advertises the selected port in HACK_DEV_URL, even though no frontend runs for it.
  • Authored native and authored-Compose paths remain uncovered (see above).

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

roodboi and others added 18 commits October 12, 2026 13:52
…mands

A warm, image-only, unrouted `up` or `restart` with no hooks now runs
four directory-xattr storage proofs instead of six. Each proof is three
fresh helper containers.

- The warm enrollment tail (no new volume) reuses the effect-entry proof
  of the same generation under the same lease. It uses the existing
  adjacentProof record, which needs a byte-identical material binding,
  so any receipt write in between forces a full proof.
- Finalization skips the second proof pass when there is no route,
  after hook, or new or saved file material. In that case the
  completion step only reads.

Proof reuse stays within one command and one lease. Nothing is
persisted, so every command still opens with a full proof.

Each performed verification proof now emits a passive `storage.proof`
phase-trace span. The hosted native-config-compose scenario uses it to
assert four proofs on each of two consecutive warm ups.

Refs: HACK-1123

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sed file

The HACK-1211 experiment produced two iterations per Bun version: one
focused pass and one full `bun test` that started on a 600 s estimate and
overran the 30-minute budget, because the full suite takes 30-35 minutes
on hosted macOS.

- Never start an iteration whose estimate exceeds the remaining budget.
  The estimate is a conservative prior (focused 120 s, full 2400 s) until a
  run completes, then the longest completed run times 1.2. A command that
  does not fit is recorded once in summary.json `skippedForBudget` and
  leaves the rotation. Deadline stops remain a safety net (`overrun`).
- Add `--mode=focused|full|alternate` (default focused: only
  tests/native-https-owner.test.ts) and `--role=baseline|candidate`.
- The workflow gains a validated `mode` dispatch input; the label-triggered
  matched pair uses focused with baseline/candidate roles. Full mode needs a
  budget of at least 40 minutes. The job timeout stays budget + 15.
- summary.json gains per-command counts (iterations, passes, crashes, other
  failures, overruns, skipped-for-budget) and a one-line verdict such as
  "baseline 0/97 focused crashes".
- The shared classifier also treats exit 134 (SIGABRT) and 138 (SIGBUS) as
  crash exits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native `up` emitted a `--ready` goal for every compose service, including
services in profiles that were not selected. The runtime does not schedule
inactive services and refuses readiness for them ("Readiness includes
inactive or unknown services."), so any project with an unused profile
failed with graph_readiness.

Build readiness goals only for services the reviewed plan marks active,
and ignore dependency edges declared by inactive services so they cannot
raise an active service's goal. The runtime refusal stays as a guard; a
contract test now proves it still rejects inactive and unknown services.

Bridge pool capacity keeps counting inactive routed services: it is sized
before the profile-aware review and must remain an upper bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On macOS, print one closed hint line on stderr when a command opens or
prints a browser URL or dev host ending in .local (hack open, hack branch
open, hack status/projects, and Doctor's browser check). Browsers other
than Safari need the macOS Local Network permission once for .local names
(Apple TN3179), even when Hack's resolver answers with loopback.

The hint is shown at most once per command, never on other platforms,
never for non-.local hosts, and never inside --json payloads. Hack does
not read or change the permission.

Document the name-based rule, the Safari exemption and the alternatives
(OAuth alias, or a custom suffix under hack.gy) in the macOS browser
guide, and link it from the init and native domain docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under the native backend on macOS, `hack doctor` reports a
`native hack.local route` check (JSON `hackLocalRoute`) and
`hack doctor --fix` proposes and, after confirmation plus the native
administrator prompt, rewrites only the Hack-managed
`address=/.hack.local/<old>` dnsmasq line to 127.0.0.1 in place. It
creates /etc/resolver/hack.local with `nameserver 127.0.0.1` only when
missing. Preconditions are closed refusal reasons: verified relay owner
listening on 127.0.0.1, a single exact managed line, a Hack-written
target, a dead old target (bounded :443 probe), a Hack-owned resolver,
no pending receipt, and unchanged inputs across authorization.

A private receipt (old/new line, whole-file hashes, timestamps) is
published before effects; failures restore the earlier line. Rollback
via `hack global dns deactivate --domain hack.local` restores the exact
earlier bytes, verifies the hash and recovers interrupted repairs.

`hack global dns preview|activate|deactivate` without --domain no
longer targets a built-in default; it stops with a usage message naming
`hack doctor --fix` and the rollback command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native foreground up and restart now add the same runtime host metadata
the Compose backend generates (HACK_BRANCH, HACK_COMPOSE_PROJECT,
HACK_DEV_HOST/URL, HACK_ALIAS_HOST/URL, HACK_SERVICE_* and
HACK_RUNTIME_METADATA) to the normalized Compose input native review
checks. Values are public container environment derived from the route
labels review turns into its route plan, the effective branch and the
native HTTPS port (included in URLs when it is not 443). Startup refuses
before serving if the advertised hostnames differ from the reviewed
routes. Keys a service already declares, including Hack-managed project
values, are left untouched, so explicit project values win.

Both backends share one builder; the Compose override bytes are
unchanged. The run mapping records runtimeMetadata so restores of graphs
admitted with metadata reproduce it, while legacy retained graphs are
restored exactly as admitted. Restart preflight reviews the same input.

The runtime-core hostname-change fingerprint ignores the reserved
metadata keys, whose values follow the hostnames, so hostname-only
retaining restarts (domain migration) keep working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Rollback of a receipt whose dnsmasq.conf changed (or names another path)
  no longer refuses forever: after a plain confirmation it marks the
  receipt released and writes nothing else, so a pending receipt can
  always be cleared.
- Keep a leading BOM when reading DNS files, so rewrites keep every byte
  and receipt hashes equal the raw file hashes.
- Hold an exclusive lock for repair and rollback; refuse with busy.
- Repair refuses with restart-unavailable before any effect; rollback
  restores the file even when dnsmasq cannot be restarted and says so.
  Automatic cleanup restarts only when something changed and reports
  "nothing was changed" when the failure came first.
- Accept only the managed single-line form for a receipt's old line, and
  control-free absolute paths; diagnostics name lines by number and
  replace control characters.
- Cover the rollback non-interactive, recheck, restore, path, lock, file
  owner/link and private-directory guards with focused tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k.local repair

The hack.local repair now refuses with closed reasons instead of acting on
DNS data it never inspected or throwing raw read errors:

- uninspected-config-source: dnsmasq.conf or a dnsmasq.d include loads
  conf-file, conf-script, addn-hosts, hostsdir, servers-file, or a conf-dir
  other than <brew>/etc/dnsmasq.d,*.conf. The scan is shared with the scoped
  DNS plan (findUninspectedDnsmasqSource), which now also refuses
  servers-file and conf-script.
- unreadable-config: a DNS file is a symlink (O_NOFOLLOW), invalid UTF-8,
  linked or oversized. Inspection and rollback refuse without echoing the
  read error or file content.

Adds tests for the conf-file repro, every source directive, unreadable main
and include files, rollback of an unreadable file, and the three
defense-in-depth guards: the post-write hash recheck in restore,
replaceManagedLine with zero or two matches, and the null resolver identity
guard before removal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only `hack open --json` exercised the hint, so dropping it from the
human open path, branch open, projects/status or either doctor path left
tests green.

Run `hack open`, `hack branch open`, `hack projects` and `hack status`
from source with process.platform preloaded as darwin, stub `open` and
`docker` on a hermetic PATH, and an isolated HOME/state, asserting one
hint for a .local dev host and none for a non-.local host.

Doctor probes Docker, resolvers, the keychain and /usr/bin/curl, so its
compose and native tails now share `reportDoctorResults` with an
injectable hint, and each path's targets come from exported builders
that are tested directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a source CLI regression for a first up with new file material and no
saved file material: finalization keeps its completion step, the second
finalize pass, and the post-completion file-mount readiness check. The
test fails when the retirement condition keeps only the saved-files arm.

Native-compose CLI harnesses now give the child a private HOME under the
fixture root instead of the contributor's HOME.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mask the reserved HACK_* runtime metadata keys only in the normalized
Compose document, where Hack generates them. Values declared in
.hack/docker-compose.yml stay literals, so changing them still refuses a
retaining hostname-change restart.

Add regression tests for the restart preflight route check and for
handleNativeUp passing the runtime metadata selection to native startup.
Scope the consumer guidance to Compose-backed and native image projects,
regenerate the checked-in surfaces, and document that selecting or
removing a non-443 HTTPS frontend also refuses a retained restart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicted files: none. The plain merge applied cleanly; no generated
files needed hand resolution, and the generation and reference tests
pass on the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicted files: none. The plain merge resolved automatically. This
PR's change set (native readiness goals, the bridge capacity comment and
the project_contract.rs guard test) does not overlap witness reuse or
merge train 2. Regenerated plugins and docs/reference/cli.md were
byte-equal to the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-hint-b11

Conflicted files: none. The merge applied cleanly; src/commands/doctor.ts
and src/commands/project.ts auto-merged with both sides' changes kept.

Regenerated surfaces (generate:agent-plugins, docs:cli-reference) produced
no further changes. src/agents/integration-revision.ts,
src/agents/integration-ownership.ts and the checked-in renders match the
predecessor branch, and their tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pair-b11

Conflicted files: none. Git auto-merged docs/guides/init-project.md,
docs/guides/native-candidate.md and src/commands/doctor.ts, keeping both
sides' content.

Regenerated plugins (generate:agent-plugins) and docs/reference/cli.md
(docs:cli-reference); both produced no further changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-env-b10

Conflicted files and resolutions:
- src/agents/integration-revision.ts: set to f4120f477794, the value
  tests/agent-instruction-source.test.ts computes from the merged
  instruction sections.
- src/agents/integration-ownership.ts: KNOWN_RENDER_FINGERPRINTS is the
  append-only union of both sides, plus the merged renders requested by
  tests/integration-ownership.test.ts (hack-cli-skill 0e99791aa0edfd1895e31748,
  cursor-rules 0b82281933af514de8a1b8b8).
- plugins/hack/rules/hack.mdc, plugins/hack/skills/hack-cli/SKILL.md:
  regenerated with bun run generate:agent-plugins.
- .codex/skills/hack-cli/SKILL.md, .cursor/rules/hack.mdc,
  examples/basic/AGENTS.md, examples/basic/CLAUDE.md: re-rendered through
  installCodexSkill, installCursorRules and upsertAgentDocs with an explicit
  project root; byte-equal to their renderers (hack-init skills unchanged).
- docs/reference/cli.md: regenerated with bun run docs:cli-reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi changed the base branch from next to wf/hack-local-native-repair-b11 October 12, 2026 21:03
@roodboi
roodboi marked this pull request as ready for review October 12, 2026 21:10
Base automatically changed from wf/hack-local-native-repair-b11 to next October 12, 2026 21:54
Re-parents onto next after the previous stacked PR squashed in; next's tree equals 44eb2a1, which this branch already contains, so the branch tree is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi merged commit 63b614f into next Oct 12, 2026
30 of 31 checks passed
@roodboi
roodboi deleted the wf/native-runtime-env-b10 branch October 12, 2026 21:55
@blacksmith-sh

blacksmith-sh Bot commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
native-config-process-policy (30.1s) — Native --profile must succeed/
native-config-process-policy (30.1s) — Native --profile must succeed
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant