[kixelated]

Problem

The macOS host never produced a trustworthy result, and collections collided:

• collect ran macOS's host side outside Nix over a non-login ssh session, so it got macOS's python3 (3.9, no tomllib) and wrote nothing.
• On macOS, run sampled top for 5 seconds before the build and the run, and measured nothing during it. kixtop came back ok twice while another session's gated cargo builds ran alongside (load 2 to 9; 177 of 357 timings differ by over 10% between the runs).
• Two collections against kixtop ran at once from different sessions. The second's rm -rf src deleted the first's source and build-cost scratch mid-build.

Approach

• Every host's run is held with quest gate --host (moq-dev/quest#526). That keeps every other gated command off the host until the benchmarks and build costs finish, and queues a second collection behind the first. The gate is the collector's own locked quest (nix shell github:moq-dev/quest/<rev> from its flake.lock), so a commit that pins an older quest still runs, in its own dev shell.
• macOS runs in the commit's Nix dev shell like Linux. kixtop has Nix, and rind's dev shell builds there, so the Homebrew and rustup special case is gone. macOS now links Nix's OpenSSL, like the Linux hosts.
• Each collection extracts into its own mktemp -d copy and removes it afterwards, so a second collection can't replace the first's source. The build directory stays shared, guarded by cargo's lock and the gate.
• collect refuses to run under quest gate, since the local host's nested gate would run at once without holding the host.
• macOS's busy measure reads CPU ticks from host_statistics(HOST_CPU_LOAD_INFO), the counterpart of /proc/stat, so one measure serves both platforms, and other_busy_cpus_during_run / noisy now cover macOS. Deltas wrap at 32 bits, as macOS's counters do after weeks of uptime.
• The flake pins quest at moq-dev/quest#526's merge, which also updates the quest CLI and skills rind's agents use.

Impact

• No public API change; admin/bench-report, the README, and the flake's quest pin only.
• macOS benchmarks link Nix's OpenSSL (3.5.9) instead of Homebrew's.

Validation

• just ci passes.
• End to end on kixtop (Apple M4) at c5f9d0b: the host hold recorded the gate's PID and UTC start, and the run came back ok with 357 results. The during-run measure read 0.55 other busy CPUs (not noisy), and build cost was ok, a first for kixtop. The hold released when the gate exited.
• The fetch's code=$? (zsh's status is read-only) was checked under kixtop's zsh over ssh. The collector's gate was checked on kixtop, and on nixos with f95b91d's tree, whose own quest lacks --host: its dev shell came up, and a nested gate ran at once.
• The busy measure reads 0.53 busy CPUs on idle kixtop and 0.09 on nixos2.

Alternatives

• Gating only run with the old shared src/: the copy step runs before any gate and would still wipe a running collection.
• Taking the gate from the measured commit's dev shell: commits pinning an older quest could no longer be collected.
• Keeping Homebrew and rustup on macOS: this needs quest installed separately on the laptop, and keeps two toolchains.

Review

Codex (gpt-6.1-sol, high), 3 rounds; round 3 on head 566a050 reported no findings.

• Round 1: status=$? fails under zsh, where status is read-only. Fixed: code=$?.
• Rounds 1 and 2: the gate came from the measured commit's dev shell, so commits pinning an older quest could not be collected on Linux. Fixed: the gate is the collector's locked quest, run via nix shell.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

[kixelated]

Heads-up from #397: at about 17:45 UTC I ran admin/bench-report collect --host kixtop, whose copy step runs rm -rf ~/.cache/rind-bench-report/src before any busy check. Your kixtop run at f95b91da (started 17:13 UTC) was in build-cost's openssl-vendored build then, so its build_cost failure (make build_libs in .tmp/build-cost/openssl-vendored) is most likely my doing, not the host's. Its benchmark timings had finished at 17:41 and are unaffected. Nothing of mine runs on kixtop now, and #397 leaves kixtop to this PR.

collect has no guard against two sessions measuring one host at once; I'll suggest a per-host lock as a follow-up in #397.

(Written by Claude Opus 5.5)

[pullfrog]

Your OpenCode Go account is out of credit. Top it up, then re-trigger Pullfrog.

OpenCode Go's response

provider error: Insufficient account funds

Top up OpenCode Go →

  ^{｜ Rerun failed job ➔ ｜ View workflow run ｜ via Pullfrog ｜ Using glm-5.3}

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 54 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: d85f56ba-5bb6-4b4c-b710-11720eb6c611

Commits

Reviewing files that changed from the base of the PR and between ba5ea58 and 566a050.

Files ignored due to path filters: 1

• flake.lock is excluded by !**/*.lock

Files selected for processing: 3

• README.md
• admin/bench-report
• flake.nix

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[kixelated]

Automated review by review (OpenAI)

Reviewed head: 566a050699c6bad634347d771845e798334accaf.

No actionable correctness findings in the reviewed changes. I checked the macOS CPU-tick sampling and wraparound arithmetic, the during-run subtraction of benchmark CPU time, unique source/result directories and cleanup, the collector's independently pinned Quest host gate, and the Nix/toolchain changes against the surrounding code and pinned Quest implementation.

Direction: no code revision requested by this review. Complete the required CI before merging.

Limits: static, GitHub-only review. I did not run tests, Nix builds, SSH commands, or benchmarks, and have not independently reproduced the host measurements reported in the PR. The existing bench-report tests cover rendering, not the new collection and CPU-accounting paths. CI was still in progress when checked.