Repository navigation
test(runner): check the Claude Code sandbox's boundary nightly - #543
Conversation
Quest-Claim: 735171672e806ee8
A scripted Anthropic Messages endpoint, set as the account's endpoint in its settings, drives the pinned Claude Code through Claude.launch in a hostile checkout: credential reads by path and symlink, writes outside the checkout, a committed project hook and CLAUDE.md, the Nix daemon's and a host Unix socket, the network directly and through the proxy, and the file tools outside the checkout. The devShell names the pinned Claude Code so `just runner-sandbox` and the nightly workflow run it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
changed the title
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 40 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
kixelated
left a comment
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed head 7d38907 against base 1c3cf6a.
No actionable findings. The scripted endpoint exercises the real Claude launch path, checks returned tool results and host-side effects, and pairs the shell escape attempts with unconfined controls. The Linux Nix shell supplies the pinned Claude package and its sandbox dependencies; the existing live-test selection includes this file without adding it to the merge gate. Direction: the focused nightly regression test fits the planned boundary check; no revision suggested.
Verification: GitHub-only static inspection of the changed files, Claude adapter, sandbox policy, test selection and pinned Nix packaging. No Nix build, live sandbox test or security exploit was executed, and the reported container/mutation runs were not independently reproduced. quest context and quest guide were read from base-revision sources rather than run. This is not a comprehensive security audit.
(Written by review (OpenAI))
Finishes quest/m0/kix/sandbox/claude-boundary.md: Claude Code jobs get the same checked-in hostile-repository boundary test opencode jobs got in moq-dev/quest#404.
Summary
test/live/runner-sandbox-claude.test.ts: a scripted Anthropic Messages endpoint, set as the account's endpoint in$CLAUDE_CONFIG_DIR/settings.json(env.ANTHROPIC_BASE_URL), drives the pinned Claude Code throughClaude.launch, with the runner's realopenSandboxpolicy, in a checkout that tries to:.claude/settings.jsonhook to run (SessionStart and PreToolUse) and a committedCLAUDE.mdto reach the model;It also checks that a command's
HOMEis the attempt's, itsTMPDIRis inside the attempt's private directory and writable, its environment holds neither the credential nor the kix tools' token, it inherits no socket beyond its stdio, and nothing is left in the checkout. Each shell attempt first escapes unconfined, and the test fails rather than skips on Linux when the sandbox can't start.flake.nix: the devShell setsQUEST_RUNNER_CLAUDEto the runner package's pinned Claude Code (Linux only, as for opencode and srt), sojust runner-sandboxand the nightlysandbox.ymlrun it. nixpkgs' Claude Code already wraps bubblewrap and socat on itsPATH, so the workflow needs no change beyond its comment.Public API and wire impact: none (tests, devShell, docs).
Choices made without asking
runner-sandbox.test.ts, per the coordinator (a sibling quest owns that file). Its small helpers (shell,listen,listenUnix,git) are copied rather than moved to a shared module, also per the coordinator.ls -l /proc/self/fd/, not the shell's own fds. Claude Code wraps a command without its own stdin redirect ineval '…' < /dev/null, so bash keeps a close-on-exec copy of its original stdin (Claude Code's stdin socketpair) on fd 10. That copy is stdio, not a new channel, and no exec'd command inherits it; listing an exec'd program's fds checks exactly what a command inherits.dontAskwith an allowlist (Bash,Edit(//<checkout>/**),WebSearch,mcp__kix), so a new tool is refused unless the runner allows it. Pinning its ~25 tools would fail every bump without guarding anything.env, which Claude Code passes to commands, so the "unreadable secret" is a planted.credentials.jsonin the account home, as the quest planned.Validation
just check,just test: pass.bwrap-userns-restrictAppArmor profile breaks bwrap's loopback setup (bwrap: loopback: Failed RTM_NEWADDR), for opencode's test on main too. The nightly workflow sets the sysctl instead. I ranjust runner-sandbox's vitest command in a privileged rootless podman container (ubuntu:24.04, host/nixmounted): the Claude test passed 3 of 3 runs.sandbox.enabled: falseinclaudeSettings, the test fails (reads the account's credentials by absolute path)..claude/settings.jsonhook does fire when Claude Code loads project settings (--setting-sources user,project), so its refusal under--setting-sources useris not vacuous.socatreportssocket(1, 1, 0): Operation not permitted, so the seccomp filter, not chance, refuses them.Suggested follow-ups
opencode_session_move,opencode_session_rename,opencode_worktree_create/list/remove,opencode_models, MCP resource tools), most likely from the nixpkgs bump in moq-dev/quest#502. Session move/rename in the shared account database is exactly what opencode-data assumes jobs can't reach, so it needs a review and a deny injobConfig, not just an updated list. No open PR fixes it.Models
Size: M
Implementer: Claude Code, claude-opus-5-5, medium effort
Review
Rewritten by
quest reviewafter each round.01a1b768-c8d8-7d92-bc52-9f663ee838c9, round 1Code review found no actionable defects; boundary coverage and nightly integration look consistent, but runtime validation remains unverified because builds and tests were not run.