Skip to content

fix: hermetic test fixtures, load-sized budgets and a coded env lock refusal - #504

Draft
roodboi wants to merge 5 commits into
nextfrom
wf/g108-test-hermeticity-and-timeouts
Draft

roodboi wants to merge 5 commits into
nextfrom
wf/g108-test-hermeticity-and-timeouts

Conversation

@roodboi

@roodboi roodboi commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-ups from HACK-1459 (items 1-6 plus its comment) and the HACK-1460 flake. Test fixtures no longer read the caller's Git or Docker environment, load-sensitive cases get named budgets sized for their work, a timed-out case can no longer break the next one, and one env refusal gets a stable code.

Root causes

  1. Hermeticity (HACK-1459 item 1 and comment). Fixture git calls in about 50 test files inherited the runner's environment.
    • An exported GIT_DIR/GIT_WORK_TREE sent the fixture's init/commit/worktree add into that repository. tests/native-env-metadata.test.ts cleared those variables in beforeEach, but its fixture runs in beforeAll, before that.
    • Under a private HOME, git still read the contributor's $XDG_CONFIG_HOME/git/config, whose core.hooksPath guard refused fixture commits. That is the actual cause of the local-only failure of native-compose-generation "real linked worktree ...". The fixture already set its identity with -c user.*; it fails on the hook, not the identity.
  2. down --recover storage-witness cases fail locally. DOCKER_HOST did not cause this. The cases fail the same way with and without it, and with a private or a real HOME. invoke() defaults HACK_COMPOSE_STARTUP_TIMEOUT_MS to 1 s, and down --recover bounds its owned stop by that budget. On a loaded host (load average 60-150 here) the stop's synthetic Docker processes take longer than 1 s, so the CLI reports native_compose_effect_refusal {effect-execution, private-uncertain} instead of E_COMPOSE_FAILED. With a 60 s budget the same run gave E_COMPOSE_FAILED at load average 108. Two more wall-clock bounds expired under load: the in-process carrier deadline (Date.now() + 30_000, which gave "witness missing ... uncertain") and invoke's 15 s kill (exit 137).
  3. native-env-metadata beforeAll (items 2-3). The template build (five git processes) had Bun's 5 s hook budget. Bun starts afterAll as soon as beforeAll times out, while the build keeps creating native-env-metadata-template-* directories. Probe: a 200 ms hook budget with a 300 ms body showed afterAll running while the body was still in progress.
  4. HACK-1460 file-owner. The up cancellation case had an explicit 5 s budget. Other cases had 15 or 30 s. When a case times out, Bun runs afterEach while the body keeps running, and that body reads the module-level store/root, which the next beforeEach replaces. A late body can therefore act on the next case's store. That is how one timeout becomes "state unsafe" in the next case.
  5. Items 4-6.
    • Item 4: since fix(storage): bound storage helper probes by the carrier's remaining budget #437, a hung helper start is bounded only by the carrier's remaining budget, so it reports storage-budget. That is the right reason, because the remedy is the budget variable that the summary names. The code, the probe-budget test and the reference already agreed; the doc and the test now state it explicitly.
    • Item 5: hack env hydrate with a .hack/.internal file is refused by the earlier legacy adoption-state check (ENOTDIR counts as an input access failure, never as absence), so it reports E_CONFIG_INVALID. docs/env.md said E_PATH_CONFLICT.
    • Item 6: EnvLayerLockRefusedError was an uncoded ClosedCliError, so the CLI printed it without an E_* prefix and --json reported E_UNEXPECTED.

Fix

  • New tests/helpers/fixture-git.ts: fixtureGitEnv(), fixtureGitShellEnv() (for src/lib/shell.ts), runFixtureGit[Sync]().
    • It builds the environment from PATH/TMPDIR plus the caller's values.
    • It removes every GIT_* variable, DOCKER_HOST/DOCKER_CONTEXT and the runner's HOME/XDG_CONFIG_HOME.
    • It sets GIT_CONFIG_GLOBAL=/dev/null, GIT_CONFIG_NOSYSTEM=1, a fixed author and committer identity, and GIT_TERMINAL_PROMPT=0.
    • gitEnv lets a test set a Git variable on purpose (used for GIT_INDEX_FILE and GIT_CEILING_DIRECTORIES).
  • Every fixture git spawn in tests/ goes through it (about 50 files). The e2e runCommand gets an exactEnv option. The checkout write guard drops a caller's repository variables but keeps the contributor's config.
  • tests/fixture-hermeticity.test.ts:
    • Exports GIT_DIR, the work tree, the index, the object directory, the common dir, a hostile GIT_CONFIG_GLOBAL and a hostile identity, then runs every fixture route. The scratch repository stays byte-for-byte unchanged, and no hostile hook runs.
    • A fixture HOME's own .gitconfig cannot add hooks, signing or identity.
    • An ambient DOCKER_HOST/DOCKER_CONTEXT never reaches fixture git or the private CLI env.
    • A source scan refuses any new fixture git spawn that does not use the helper.
  • native-compose-storage-witness-command: named RECOVERY_DEADLINE_MS (30 s), CLI_COMMAND_TIMEOUT_MS (45 s) and a 3 min case budget. The carrier deadline uses the product default. The source-unavailable recovery case also runs with an ambient DOCKER_HOST at a missing socket plus a DOCKER_CONTEXT.
  • native-env-metadata: named TEMPLATE_BUDGET_MS (60 s) on both hooks. afterAll waits for the build to settle, and a build that outlives that wait removes its own directory.
  • native-compose-file-owner: one named FILE_OWNER_CASE_BUDGET_MS (60 s) via setDefaultTimeout replaces the per-case 5/15/30 s budgets. afterEach waits for the running body before teardown.
  • EnvLayerLockRefusedError is now a coded E_PATH_CONFLICT HackCliError. docs/env.md is aligned on hydrate's .internal file and the lock refusal. The storage-carrier reference names storage-budget as the reason for a hung start.

Assertions in the flaky cases are unchanged. Only budgets, deadlines and teardown order changed.

Evidence

In progress: local 30-run soaks under load and hosted CI at this head; this section is updated when they finish.

Gaps

Listed when the evidence above is complete.

Refs HACK-1459
Refs HACK-1460

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

roodboi and others added 2 commits October 24, 2026 18:37
… docs

A dangling or non-directory `.hack/.internal` beside an env layer made
`hack env set`/`unset` refuse with one line that had no stable code (and
`--json` reported E_UNEXPECTED). EnvLayerLockRefusedError is now a coded
E_PATH_CONFLICT refusal, the code `hack env hydrate` already gives the same
entry.

docs/env.md said hydrate reports E_PATH_CONFLICT for a `.hack/.internal`
that is a file; it reports E_CONFIG_INVALID ("Cannot inspect Hack project
inputs ...") from its earlier adoption-state check, which refuses ENOTDIR
rather than treating it as absent. The doc now says so, and a CLI test pins
it. The storage carrier reference and the probe-budget test now state the
decided reason for a hung helper start since #437: storage-budget, never
probe-timeout.

Refs HACK-1459

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixture `git` inherited the runner's environment: a caller-exported
GIT_DIR made tests/native-env-metadata.test.ts commit, branch and register
a worktree in another repository, and a contributor's global
core.hooksPath (read through XDG_CONFIG_HOME even under a private HOME)
refused fixture commits. tests/helpers/fixture-git.ts now builds every
fixture git environment from PATH/TMPDIR plus the caller's values, removes
every GIT_* variable, DOCKER_HOST/DOCKER_CONTEXT and the runner's
HOME/XDG_CONFIG_HOME, and sets GIT_CONFIG_GLOBAL=/dev/null,
GIT_CONFIG_NOSYSTEM=1 and a fixed identity. All fixture git spawns in
tests/ go through it; the e2e harness gains an exact-env option for them
and the checkout write guard drops caller repository variables.
tests/fixture-hermeticity.test.ts proves an exported GIT_DIR, work tree,
index and hostile global hooks leave a scratch repository byte-for-byte
unchanged, and refuses any new unrouted fixture git spawn.

Budgets sized for the work under load, assertions unchanged:
- native-env-metadata: the template beforeAll gets a named 60 s budget
  (24/36 runs exceeded 5 s under I/O load) and afterAll waits for a
  timed-out build, so template directories no longer leak.
- native-compose-file-owner (HACK-1460): one named 60 s per-case budget
  replaces the 5 s/15 s/30 s mix, and afterEach waits for a timed-out
  body before teardown so it cannot change the next case's store.
- native-compose-storage-witness-command: `down --recover` ran under
  invoke's 1 s HACK_COMPOSE_STARTUP_TIMEOUT_MS and reported
  private-uncertain once load stretched the stop; recovery now gets a
  30 s deadline, CLI processes a 45 s kill bound, the carrier the product
  default, and the case runs again with an ambient DOCKER_HOST at a
  missing socket.

Refs HACK-1459
Refs HACK-1460

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

HACK-1459

HACK-1460

roodboi and others added 3 commits October 24, 2026 19:21
A 30-run soak caught the GIT_DIR hermeticity case failing 2/30: `commit`
starts a detached `git maintenance run --auto` that takes
objects/maintenance.lock after the command returns, so the fixture
repository changed under the comparison. Fixture git now sets
maintenance.auto=false and gc.auto=0 through GIT_CONFIG_COUNT, which also
keeps an unowned background git from outliving any fixture; a test pins
both settings.

native-env-metadata cases and the generation linked-worktree case each run
the product's Git worktree verification and took more than 5 s at load
average above 100; they get named 30 s budgets. Assertions unchanged.

Refs HACK-1459

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 30-run soak at load average 120 still failed the recovery cases: the
30 s recovery deadline and the 45 s kill bound both expired mid-stop
(exit 137). Recovery now runs with DEFAULT_COMPOSE_STARTUP_TIMEOUT_MS, as
on a host that sets no budget, CLI processes get a kill bound a minute above
it, and cases an 8 minute budget. Assertions unchanged; in CI the file
still takes about 5 s.

Refs HACK-1459

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second 30-run soak passed 29/30 at load average 200-300; the one
failure was the unknown-enrolled case, whose `ps` and `down` still ran
under invoke's 1 s HACK_COMPOSE_STARTUP_TIMEOUT_MS. Every source CLI
process in the file now runs with the product default budget, the same
one recovery already uses. Assertions unchanged.

Refs HACK-1459

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant