Skip to content

fix(nix): dedupe the dev shell's PATH so mbx stays under E2BIG - #7594

Merged
kixelated merged 4 commits into
mainfrom
quest/m0/ci/mbx-arg-max
Nov 2, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m0/ci/mbx-arg-max

Conversation

@kixelated

@kixelated kixelated commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

just ci failed locally in RecipeTest.test_unexecuted_bench_is_not_compiled (sh/rs/select.test.py) and elsewhere: mbx could not launch Cargo, Argument list too long (os error 7) (E2BIG).

mbx 1.25.0 through 1.29.1 copies PATH into MBX_SESSION_PATH and MBX_SESSION_RESTORE, serialized as byte arrays at ~3.6× PATH's size (record_overlay). Linux rejects any single env string over 131,072 bytes (MAX_ARG_STRLEN), so a PATH over ~36 KB breaks every Cargo launch through mbx.

The repo-side cause is why PATH got that big. Each nested dev shell (direnv or an agent harness, then nix develop, then another nix develop inside a gate) prepends another copy of the shell's ~160 store paths, about 10.4 KB per level. Four levels reach 42 KB. Only PATH and XDG_DATA_DIRS accumulate across nesting; the other large Nix variables are reset each time.

Approach

The dev shell's shellHook (flake.nix) now keeps only the first occurrence of each PATH directory, after it puts the mbx Cargo shim in front. Command lookup takes the first matching directory, so dropping later duplicates can't change what any command resolves to. The fix lives in the shell that CI (nix develop --command …), direnv, and local gates all enter, so it covers CI and local runs alike. No test, assertion, or mbx path changes.

Measurements

PATH by nesting depth (depth 1 is this session's own dev shell; nix develop from a clean environment gives 10,464 bytes):

Depth PATH before PATH after
2 21,302 (335 entries, 175 unique) 10,769 (175)
3 31,671 (492 entries) 10,769 (175)
4 42,040 (649 entries) 10,769 (175)

At depth 4, strace -f -e trace=execve on cargo build of a dependency-free crate through the mbx 1.25.0 shim (largest entry, including name, =, and NUL):

Entry Before After
PATH 42,046 10,775
MBX_SESSION_PATH 148,919 (E2BIG) 38,246
MBX_SESSION_RESTORE 153,291 (E2BIG) 42,082

At depth 4, the original test fails before the fix with the reported mbx[error]: failed to run …/cargo: Argument list too long (os error 7). After the fix it passes.

Decisions

  • 2026-11-02 (maintainer): no upstream post to jdx/mr-boxington, which is outside the authorized orgs; find a repo-side workaround instead. This replaces this PR's earlier view that trimming PATH would hide the upstream defect. The quest file is deleted, so this decision is recorded here and in the commit message.

Choices made without asking, for review:

  • Deleted quest/m0/ci/mbx-arg-max.md. Its goal (the test passes in just ci) is met. Its rewritten goal (pin an upstream-fixed mbx) depended on an upstream report the maintainer declined. The remaining exposure needs a PATH over ~26 KB from outside the dev shell. The deduped shell PATH (10.8 KB) leaves mbx about 3.4× headroom.
  • Deduped only PATH, not XDG_DATA_DIRS. mbx doesn't copy XDG_DATA_DIRS. It grows ~7 KB per level and would need ~18 levels to reach the per-string limit.
  • A trailing empty PATH entry (the current directory) is dropped. awk with RS=: emits no final empty record. An implicit current-directory entry is a hazard anyway.

Checks

  • quest gate nix develop --command just ci: exit 1, on two failures this change doesn't cause. Cargo launched through mbx with no E2BIG (obs_compile: mbx[cache]: 429 hits, 21 misses). Passed: obs_compile, js check, quest, obs, shell, large_files, toml, nix, justfile, markdown, gh check, gh test.
    • check flake: the github-runner VM image build fails with unshare: write failed /proc/self/uid_map: Operation not permitted. This session's host restricts rootless user namespaces (kernel.apparmor_restrict_unprivileged_userns = 1), and CI_STRICT=1 turns the userns probe's skip into a failure (sh/dispatch.sh:331). The github-runner-vm and github-runner-vm-image check derivations are byte-identical to origin/main's, so main fails the same way here. flake.nix PRs still land through the host gate, e.g. moq-dev/moq#7217.
    • test js: @moq/net a peer close drains the overtaken group over moq-lite-02 (js/net/src/lite/connection.test.ts:394) failed under host load ~34. Rerun alone, bun test --only-failures src/lite/connection.test.ts passed 57/57. PATH dedupe can't affect it.
    • The diff selects no Rust module, so RecipeTest doesn't run in this gate. It ran at depth 4 instead (next item).
  • Depth-4 focused test (python3 sh/rs/select.test.py RecipeTest.test_unexecuted_bench_is_not_compiled): fails before, passes after.
  • quest check --base origin/main: 629 documents ok.
  • nix fmt -- --ci flake.nix: no changes.

Review

  • Reviewer: Codex, gpt-6.1-sol, medium effort (quest review --base origin/main, round 1, session 01a19aaa-2b48-70b0-b2ca-6827e6e947f5).
  • Reviewed head: 21f8dbb70d16a5723d090227a2a487af13502a26.
  • Findings: none.
  • Head cb7b0e5cfa adds only a merge of origin/main with no conflicts, which the guide exempts from re-review.

Follow-ups

  • If the maintainer later authorizes it, report the mbx metadata expansion upstream: file-backed session restoration would remove the ~36 KB PATH ceiling entirely.

Unblocks the E2BIG failures in drafts moq-dev/moq#7593, moq-dev/moq#7596, moq-dev/moq#7599, moq-dev/moq#7600, moq-dev/moq#7602, and moq-dev/moq#7606 once they merge this base.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 2 commits November 1, 2026 17:50
Quest-Claim: 823a51bc1a53f93e
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
…'s E2BIG limit

Each nested dev shell prepended another ~10 KB copy of its store paths. mbx
1.25-1.29 copies PATH into MBX_SESSION_PATH and MBX_SESSION_RESTORE at ~3.6x
its size, so four nested shells pushed both past Linux's 131,072-byte
per-string limit and every Cargo launch failed with E2BIG. Keeping each
directory's first entry leaves lookups unchanged and holds PATH at ~10.8 KB.

The repo-side goal is met and the maintainer chose not to report upstream,
so the quest is deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title docs(quest): isolate mbx E2BIG to upstream session metadata fix(nix): dedupe the dev shell's PATH so mbx stays under E2BIG Nov 2, 2026
@kixelated
kixelated marked this pull request as ready for review November 2, 2026 06:03
@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 4 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0bae0865-8b1d-4334-ab10-f11c609d1cfe
Commits

Reviewing files that changed from the base of the PR and between bcaaef1 and cb7b0e5.

Files selected for processing: 3
  • flake.nix
  • quest/m0/ci/README.md
  • quest/m0/ci/mbx-arg-max.md
Finishing Touches
Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: cb7b0e5

No actionable bugs found. Deduplicating PATH at the shared dev-shell boundary is a proportionate repo-side fix for repeated shell prefixes, and doing it after the mbx shim insertion preserves that shim's priority. It is simpler than changing each caller or bypassing mbx. The documented removal of a trailing empty PATH entry is a deliberate lookup change; this does not solve arbitrarily large, unique incoming PATHs or the upstream metadata expansion itself.

Verification: static review of the three-file diff, shell context, recent flake history, and the affected recipe test. No published prior review was present. I did not run Nix, tests, or the E2BIG reproduction; the PR's reported measurements and gate results were not independently reproduced.

(Written by OpenAI)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • PATH dedup in the dev shell — flake.nix's shellHook now drops later duplicate PATH entries after prepending the mbx Cargo shim, keeping each directory's first occurrence (awk -v RS=: '!seen[$0]++'). PATH lookup resolves against the first match, so removing later exact duplicates cannot change what any command resolves to.
  • Quest cleanup — deletes quest/m0/ci/mbx-arg-max.md and its line in quest/m0/ci/README.md; the test it tracked now passes and the maintainer declined the upstream mbx report, both recorded in the PR body and commit message.

I verified the awk dedup keeps first occurrences, preserves leading/middle empty entries, drops only an implicit-cwd trailing entry, and that the same !seen[...]++ idiom already runs in this repo (test/smoke/relays.sh:125, sh/rs/select.sh), so awk is a settled dev-shell dependency. No behavioral or portability concerns.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 04e9b36 Nov 2, 2026
7 checks passed
@kixelated
kixelated deleted the quest/m0/ci/mbx-arg-max branch November 2, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant