From c6f904947890e3216d8288a2bdae15d38eff6ab8 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Wed, 16 Sep 2026 20:14:57 +0000 Subject: [PATCH 1/3] test(groom): prove name resolution is dead in the sandbox, non-vacuously MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The egress-isolation proofs in sandbox-tests.sh (7a/7b/7c) all dial IP literals on purpose, so not one of them ever touches the resolver: they prove routing is dead and leave name resolution untested. Add that leg. The obvious one-liner ("DNS lookup fails") is a trap, and measuring it says so: on a systemd-resolved host /etc/resolv.conf is a symlink into /run, and the jail mounts /etc but deliberately not /run, so the resolver has no nameserver configured whatever the netns looks like. A bare resolution-failure check therefore stays GREEN under --share-net and would not notice a future change that put the jail back on a shared network. So assert both halves: resolution fails (on the resolver- specific exit codes getent 2 / curl 6, which a missing binary's 127 cannot satisfy), AND a hardcoded nameserver address is unroutable (curl 7, CURLE_COULDNT_CONNECT — a reachable netns returns 52/56/28 here, never 7). Verified against a shared-netns negative control, which the combined check correctly fails. Renumbers the --uds fail-loud check 7d -> 7e; nothing references it. --- .github/groom/README.md | 32 ++++++++++++----- .github/groom/tests/sandbox-tests.sh | 51 +++++++++++++++++++++++++++- 2 files changed, 74 insertions(+), 9 deletions(-) diff --git a/.github/groom/README.md b/.github/groom/README.md index eca038f0..2a244812 100644 --- a/.github/groom/README.md +++ b/.github/groom/README.md @@ -748,12 +748,19 @@ request-handling contract is identical on both transports. with only loopback up, so the broker — reached over the unix socket bind-mounted at `/run/broker.sock` via the in-jail `jail-shim.mjs` TCP→UDS forwarder — is the *only* thing the agent can talk to. Host network, host loopback services, and -cloud metadata (`169.254.169.254` / `168.63.129.16`) are all unreachable. Two -consequences for callers: set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` in the -agent env so the agent doesn't stall on telemetry endpoints that can never be -reached; and because there is no egress, in-jail `git fetch` / `npm install` -cannot work — anything the agent needs must already be in the clone before it is -sandboxed. +cloud metadata (`169.254.169.254` / `168.63.129.16`) are all unreachable. So is +**name resolution**, and twice over: the netns has no route to any nameserver (the +runner's stub resolver at `127.0.0.53` sits on the *host's* loopback, not the +jail's), and on a systemd-resolved runner `/etc/resolv.conf` is a symlink into +`/run` — which the jail mounts `/etc` but deliberately not — so the resolver has +no nameserver configured either way. A hostname the read-only `/etc/hosts` does +not already answer cannot be resolved at all. + +Two consequences for callers: set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` +in the agent env so the agent doesn't stall on telemetry endpoints that can +never be reached; and because there is no egress, in-jail `git fetch` / `npm +install` cannot work — anything the agent needs must already be in the clone +before it is sandboxed. ### The loud-preflight guarantee @@ -777,8 +784,17 @@ broker at a local fake upstream ([`tests/fake-upstream.mjs`](tests/fake-upstream *over the bind-mounted unix socket + in-jail `jail-shim.mjs`* to prove key injection/stripping, the `/healthz` + non-`/v1` behavior, and SSE pass-through. It also proves the BE-4369 egress isolation: host loopback, cloud metadata, and an -arbitrary external IP are all unreachable from the jail. No `claude`, no API key, -no spend. +arbitrary external IP are all unreachable from the jail, and name resolution is +dead. Every one of those reads a *failure* as the proof, so each is guarded +against false-passing on a missing tool — the IP-literal checks by asserting +`curl` is on the jail `PATH` first, the resolution check by asserting exact, +cause-specific exit codes (`getent` 2 = key not found, `curl` 6 = +could-not-resolve), which a missing binary's 127 cannot satisfy. The resolution +check also carries a second assertion on top — a no-route connect to a hardcoded +nameserver address, `curl` 7 — because a resolution failure ALONE would still +pass under a *shared* network namespace (the dangling `/etc/resolv.conf` above +breaks resolution regardless of routing), and a proof that cannot go red is not a +proof. No `claude`, no API key, no spend. ```bash shellcheck -x .github/groom/agent-sandbox.sh .github/groom/tests/sandbox-tests.sh diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh index bcfaf6e8..7d60d4e3 100755 --- a/.github/groom/tests/sandbox-tests.sh +++ b/.github/groom/tests/sandbox-tests.sh @@ -291,7 +291,56 @@ if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" -- bash -c '; then fail "arbitrary external IP reachable from jail"; fi pass "arbitrary external IP unreachable from jail" -# 7d. --uds fail-loud: a nonexistent socket path must exit non-zero BEFORE the cmd. +# 7d. Name resolution is dead inside the jail, and no nameserver is routable. +# +# 7a/7b/7c all use IP literals ON PURPOSE, so not one of them ever touches the +# resolver — routing is proven while name resolution is left untested. This section +# covers it, in TWO parts, because the obvious one-liner is a trap. +# +# (i) Resolution fails. Asserted on resolver-SPECIFIC exit codes rather than a bare +# non-zero, which is what stops the same false-pass the 7a/7b/7c tool-presence +# check exists to block: `getent` 2 is "key not found" specifically (a missing +# binary is 127) and `curl` 6 is CURLE_COULDNT_RESOLVE_HOST specifically (a +# connect-stage failure is 7, a timeout 28, a missing binary 127). The name is a +# real, permanently-resolvable one, so neither branch can pass merely because the +# hostname was bogus — and the host-side control below says so out loud. +# +# (ii) THE NETNS PROOF — do NOT collapse this into (i). Part (i) on its own is +# VACUOUS as evidence of network isolation: it passes under a SHARED netns too +# (measured, not theorized). On a systemd-resolved host /etc/resolv.conf is a +# symlink into /run, and the jail mounts /etc but deliberately NOT /run (mounting +# it would be a confinement regression in its own right), so the resolver has no +# nameserver configured whatever the netns looks like — meaning a future change +# that put the jail back on a shared network would NOT turn part (i) red. What +# only an isolated netns can produce is an immediate no-route to a hardcoded +# nameserver address, so assert that as well, again on the exact code: `curl` 7 +# is CURLE_COULDNT_CONNECT, which a shared netns cannot return here — it reaches +# the host stack and comes back 52/56 when 1.1.1.1:53 answers, or 28 when egress +# to it is filtered, never 7. +if getent hosts api.anthropic.com >/dev/null 2>&1; then + echo "note: this host resolves api.anthropic.com — the in-jail failure below is the sandbox, not a dead name" +else + echo "note: this host cannot resolve api.anthropic.com either (offline?) — the in-jail assertions are still enforced" +fi +if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" -- bash -c ' + # (i) name resolution fails + rc=0; getent hosts api.anthropic.com >/dev/null 2>&1 || rc=$? + if [ "$rc" = 0 ]; then echo "api.anthropic.com RESOLVED in jail — name resolution is not closed off (an /etc/hosts entry on the host would do this, served through the read-only /etc bind)"; exit 1; fi + [ "$rc" = 2 ] || { echo "getent exit $rc, want 2 (key not found) — getent missing from the jail PATH?"; exit 1; } + # --max-time is generous so a resolver that retried before giving up still reports + # 6 (could-not-resolve) rather than being clipped into an ambiguous 28 (timeout). + rc=0; curl -s --max-time 10 http://api.anthropic.com/ >/dev/null 2>&1 || rc=$? + if [ "$rc" = 0 ]; then echo "curl reached api.anthropic.com from jail — egress is NOT closed"; exit 1; fi + [ "$rc" = 6 ] || { echo "curl exit $rc, want 6 (CURLE_COULDNT_RESOLVE_HOST) — failed past the resolver stage instead of at it"; exit 1; } + # (ii) and no nameserver is routable either — the half a shared netns would fail + rc=0; curl -s --max-time 10 http://1.1.1.1:53/ >/dev/null 2>&1 || rc=$? + if [ "$rc" = 0 ]; then echo "nameserver 1.1.1.1:53 answered from jail — netns is not isolated"; exit 1; fi + [ "$rc" = 7 ] || { echo "curl exit $rc to 1.1.1.1:53, want 7 (CURLE_COULDNT_CONNECT/no route) — a reachable netns returns 52/56/28 here, so this jail still has one"; exit 1; } + exit 0 +'; then fail "jail name resolution / nameserver routing is not closed off"; fi +pass "name resolution dead in jail (getent 2, curl 6) and no nameserver routable (curl 7 to 1.1.1.1:53)" + +# 7e. --uds fail-loud: a nonexistent socket path must exit non-zero BEFORE the cmd. if "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ --uds /nonexistent.sock -- true 2>/dev/null; then fail "--uds /nonexistent.sock was accepted (must fail loud before running the command)" From 977b7cc11b3e0e73ba5d2580a4d59f8222a19469 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Wed, 16 Sep 2026 20:52:00 +0000 Subject: [PATCH 2/3] test(groom): key the 7d netns proof on the jail's own /proc, not a curl exit code MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review found the netns half of §7d rested on an invariant that does not hold: `curl` 7 is CURLE_COULDNT_CONNECT, which equally covers ECONNREFUSED, ENETUNREACH and a firewall REJECT, so a FULLY SHARED netns on a filtered or offline host returns 7 as well — the exact-7 assertion went green on precisely the confinement regression it existed to catch. It was also TCP-only, saying nothing about a routable UDP/53 path. Key (ii) on the netns itself instead, read out of the jail's fresh `--proc /proc` mount: `/proc/net/dev` must list `lo` and nothing else, and `/proc/net/route` must carry no default route. Those facts cannot be faked by host network state, need no egress of any kind, and cover any protocol — no route is no route. The 1.1.1.1:53 probe stays as behavioral confirmation but is relaxed to "did not succeed"; its exact code is no longer load-bearing. Also from review: - the host-side resolvability control now GATES part (i) instead of printing a note on both branches: when the host cannot resolve the name either, part (i) is reported SKIPPED rather than counted as proof (it still runs and must hold). - bound that host-side `getent` with `timeout 5`, so a blackholed resolver cannot stall a suite documented as hermetic in a job with no `timeout-minutes`. - correct the resolver wording in the (ii) comment and the README: with no readable `nameserver` line glibc falls back to 127.0.0.1 per resolv.conf(5), and the jail's `lo` carries all of 127.0.0.0/8 — so a resolver IS configured and routable in there, and lookups fail only because nothing listens on the jail's 127.0.0.1:53. Noted concretely because the jail already runs in-jail loopback listeners, so a future bind to :53 would become the agent's resolver. - soften the `--max-time 10` comment: it is a backstop, not margin over a retrying resolver (glibc timeout:5 attempts:2 burns exactly 10s on one blackholing nameserver). Verified: positive control (real isolated netns) exit 0; negative control (identical block, `--share-net`) exit 1, naming ens4/docker0/tailscale0. The old exact-7 assertion was separately shown false-passable — a fully shared, fully routable netns returns 7 on any closed port. Co-Authored-By: Claude Opus 5 --- .github/groom/README.md | 32 ++++++--- .github/groom/tests/sandbox-tests.sh | 101 +++++++++++++++++++++------ 2 files changed, 100 insertions(+), 33 deletions(-) diff --git a/.github/groom/README.md b/.github/groom/README.md index 2a244812..52afc7c5 100644 --- a/.github/groom/README.md +++ b/.github/groom/README.md @@ -749,12 +749,19 @@ with only loopback up, so the broker — reached over the unix socket bind-mount at `/run/broker.sock` via the in-jail `jail-shim.mjs` TCP→UDS forwarder — is the *only* thing the agent can talk to. Host network, host loopback services, and cloud metadata (`169.254.169.254` / `168.63.129.16`) are all unreachable. So is -**name resolution**, and twice over: the netns has no route to any nameserver (the -runner's stub resolver at `127.0.0.53` sits on the *host's* loopback, not the -jail's), and on a systemd-resolved runner `/etc/resolv.conf` is a symlink into -`/run` — which the jail mounts `/etc` but deliberately not — so the resolver has -no nameserver configured either way. A hostname the read-only `/etc/hosts` does -not already answer cannot be resolved at all. +**name resolution**, and twice over. First, the netns has no route off-box at all +— one interface (`lo`), no default route — so the runner's stub resolver at +`127.0.0.53`, which sits on the *host's* loopback and not the jail's, cannot be +reached. Second, on a systemd-resolved runner `/etc/resolv.conf` is a symlink into +`/run` — which the jail mounts `/etc` but deliberately not — so no `nameserver` +line is readable and glibc falls back to the local machine (`127.0.0.1`, per +resolv.conf(5)). That fallback *is* configured and routable: the jail's own `lo` +carries all of `127.0.0.0/8`. Lookups fail because nothing is listening on the +jail's `127.0.0.1:53` — which is worth knowing concretely, because the jail +already runs in-jail loopback listeners (`jail-shim.mjs` on `127.0.0.1:8790`), so +a future in-jail bind to `:53` would silently become the agent's resolver. Either +way, a hostname the read-only `/etc/hosts` does not already answer cannot be +resolved. Two consequences for callers: set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` in the agent env so the agent doesn't stall on telemetry endpoints that can @@ -790,11 +797,14 @@ against false-passing on a missing tool — the IP-literal checks by asserting `curl` is on the jail `PATH` first, the resolution check by asserting exact, cause-specific exit codes (`getent` 2 = key not found, `curl` 6 = could-not-resolve), which a missing binary's 127 cannot satisfy. The resolution -check also carries a second assertion on top — a no-route connect to a hardcoded -nameserver address, `curl` 7 — because a resolution failure ALONE would still -pass under a *shared* network namespace (the dangling `/etc/resolv.conf` above -breaks resolution regardless of routing), and a proof that cannot go red is not a -proof. No `claude`, no API key, no spend. +check also carries a second assertion on top, because a resolution failure ALONE +would still pass under a *shared* network namespace (the dangling +`/etc/resolv.conf` above breaks resolution regardless of routing), and a proof +that cannot go red is not a proof. That second assertion reads the jail's own +netns out of its fresh `/proc` — `/proc/net/dev` must list `lo` and nothing else, +`/proc/net/route` must carry no default route — rather than keying on a connect +exit code, which could not tell an isolated netns from a shared one behind a +firewall REJECT or on an offline host. No `claude`, no API key, no spend. ```bash shellcheck -x .github/groom/agent-sandbox.sh .github/groom/tests/sandbox-tests.sh diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh index 7d60d4e3..362f2dc1 100755 --- a/.github/groom/tests/sandbox-tests.sh +++ b/.github/groom/tests/sandbox-tests.sh @@ -58,6 +58,10 @@ pass() { echo "PASS: $*" } +skip() { + echo "SKIP: $*" +} + # --- fixtures ---------------------------------------------------------------- mkdir -p "$clone" @@ -291,7 +295,7 @@ if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" -- bash -c '; then fail "arbitrary external IP reachable from jail"; fi pass "arbitrary external IP unreachable from jail" -# 7d. Name resolution is dead inside the jail, and no nameserver is routable. +# 7d. Name resolution is dead inside the jail, and nothing off-box is routable. # # 7a/7b/7c all use IP literals ON PURPOSE, so not one of them ever touches the # resolver — routing is proven while name resolution is left untested. This section @@ -301,44 +305,97 @@ pass "arbitrary external IP unreachable from jail" # non-zero, which is what stops the same false-pass the 7a/7b/7c tool-presence # check exists to block: `getent` 2 is "key not found" specifically (a missing # binary is 127) and `curl` 6 is CURLE_COULDNT_RESOLVE_HOST specifically (a -# connect-stage failure is 7, a timeout 28, a missing binary 127). The name is a -# real, permanently-resolvable one, so neither branch can pass merely because the -# hostname was bogus — and the host-side control below says so out loud. +# connect-stage failure is 7, a timeout 28, a missing binary 127). Part (i) is +# only EVIDENCE about the sandbox if the name is live off-jail, so the host-side +# control below gates it: when this host cannot resolve the name either, the +# assertions still run and still have to hold, but part (i) is reported SKIPPED +# rather than counted as proof. # # (ii) THE NETNS PROOF — do NOT collapse this into (i). Part (i) on its own is # VACUOUS as evidence of network isolation: it passes under a SHARED netns too # (measured, not theorized). On a systemd-resolved host /etc/resolv.conf is a # symlink into /run, and the jail mounts /etc but deliberately NOT /run (mounting -# it would be a confinement regression in its own right), so the resolver has no -# nameserver configured whatever the netns looks like — meaning a future change -# that put the jail back on a shared network would NOT turn part (i) red. What -# only an isolated netns can produce is an immediate no-route to a hardcoded -# nameserver address, so assert that as well, again on the exact code: `curl` 7 -# is CURLE_COULDNT_CONNECT, which a shared netns cannot return here — it reaches -# the host stack and comes back 52/56 when 1.1.1.1:53 answers, or 28 when egress -# to it is filtered, never 7. -if getent hosts api.anthropic.com >/dev/null 2>&1; then +# it would be a confinement regression in its own right). With no readable +# `nameserver` line glibc falls back to the local machine — 127.0.0.1, per +# resolv.conf(5) — and the jail's own `lo` carries all of 127.0.0.0/8, so a +# resolver is both configured AND routable in there; lookups fail only because +# nothing is listening on the jail's 127.0.0.1:53. That holds whatever the netns +# looks like, so part (i) would NOT turn red if the jail went back on a shared +# network. (It also means a future in-jail bind to :53 would silently become the +# agent's resolver — the jail already runs in-jail loopback listeners, e.g. +# jail-shim.mjs on 127.0.0.1:8790.) +# +# So key (ii) on the netns ITSELF, read out of the jail's own /proc: the wrapper +# passes `--proc /proc`, a fresh procfs mount inside the new netns, so +# /proc/net/dev and /proc/net/route describe the JAIL's network and not the +# host's. An isolated netns has exactly one interface (`lo`) and an empty route +# table; a shared one shows the host's NICs and its default route. +# +# Do NOT key this on a connect exit code instead. `curl` 7 is +# CURLE_COULDNT_CONNECT, which equally covers ECONNREFUSED, ENETUNREACH and a +# firewall REJECT (--reject-with, ICMP admin-prohibited, or simply no default +# route), so a FULLY SHARED netns on a filtered or offline host returns 7 here +# too — an exact-7 assertion goes green on precisely the confinement regression +# it exists to catch. The /proc facts cannot be faked by host network state, need +# no egress of any kind, and cover a UDP/53 path as well as TCP: no route is no +# route, for any protocol. +host_resolves="" +if timeout 5 getent hosts api.anthropic.com >/dev/null 2>&1; then + host_resolves=1 echo "note: this host resolves api.anthropic.com — the in-jail failure below is the sandbox, not a dead name" -else - echo "note: this host cannot resolve api.anthropic.com either (offline?) — the in-jail assertions are still enforced" fi if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" -- bash -c ' # (i) name resolution fails rc=0; getent hosts api.anthropic.com >/dev/null 2>&1 || rc=$? if [ "$rc" = 0 ]; then echo "api.anthropic.com RESOLVED in jail — name resolution is not closed off (an /etc/hosts entry on the host would do this, served through the read-only /etc bind)"; exit 1; fi [ "$rc" = 2 ] || { echo "getent exit $rc, want 2 (key not found) — getent missing from the jail PATH?"; exit 1; } - # --max-time is generous so a resolver that retried before giving up still reports - # 6 (could-not-resolve) rather than being clipped into an ambiguous 28 (timeout). + # --max-time here is a BACKSTOP, not margin: with the dangling resolv.conf + # described above glibc falls back to 127.0.0.1 and the jail lo refuses the send + # instantly, so this returns in milliseconds. It is deliberately NOT sized to + # outlast a retrying resolver (glibc defaults timeout:5 attempts:2 burn 10s on a + # single blackholing nameserver); a clipped lookup exits 28 and fails loud on the + # next line with the code printed, rather than passing silently. rc=0; curl -s --max-time 10 http://api.anthropic.com/ >/dev/null 2>&1 || rc=$? if [ "$rc" = 0 ]; then echo "curl reached api.anthropic.com from jail — egress is NOT closed"; exit 1; fi [ "$rc" = 6 ] || { echo "curl exit $rc, want 6 (CURLE_COULDNT_RESOLVE_HOST) — failed past the resolver stage instead of at it"; exit 1; } - # (ii) and no nameserver is routable either — the half a shared netns would fail - rc=0; curl -s --max-time 10 http://1.1.1.1:53/ >/dev/null 2>&1 || rc=$? + + # (ii) the netns is EMPTY — the half a shared netns cannot fake, whatever the + # host network is doing. Read straight from the jail-local procfs; no tool + # beyond bash, so this cannot false-pass on a missing binary either. + [ -r /proc/net/dev ] || { echo "/proc/net/dev unreadable in jail — cannot verify the netns"; exit 1; } + [ -r /proc/net/route ] || { echo "/proc/net/route unreadable in jail — cannot verify the netns"; exit 1; } + extra_ifaces="" + while IFS= read -r line; do + case "$line" in *:*) ;; *) continue ;; esac # skip the two header rows + name="${line%%:*}" + name="${name// /}" + [ -n "$name" ] && [ "$name" != lo ] && extra_ifaces="$extra_ifaces $name" + done < /proc/net/dev + if [ -n "$extra_ifaces" ]; then + echo "jail netns has non-loopback interface(s):$extra_ifaces — the netns is SHARED with the host, so nothing here proves isolation"; exit 1 + fi + default_routes="" + while read -r iface dest _; do + [ "$iface" = Iface ] && continue + [ "$dest" = 00000000 ] && default_routes="$default_routes $iface" + done < /proc/net/route + if [ -n "$default_routes" ]; then + echo "jail netns has a default route via:$default_routes — a nameserver (and everything else off-box) is routable"; exit 1 + fi + + # Behavioral confirmation on top of those facts: nothing answers on a nameserver + # address. Asserted as "did not succeed" ONLY — the exact failure code is not + # load-bearing here, by design; see (ii). + rc=0; curl -s --max-time 5 http://1.1.1.1:53/ >/dev/null 2>&1 || rc=$? if [ "$rc" = 0 ]; then echo "nameserver 1.1.1.1:53 answered from jail — netns is not isolated"; exit 1; fi - [ "$rc" = 7 ] || { echo "curl exit $rc to 1.1.1.1:53, want 7 (CURLE_COULDNT_CONNECT/no route) — a reachable netns returns 52/56/28 here, so this jail still has one"; exit 1; } exit 0 -'; then fail "jail name resolution / nameserver routing is not closed off"; fi -pass "name resolution dead in jail (getent 2, curl 6) and no nameserver routable (curl 7 to 1.1.1.1:53)" +'; then fail "jail name resolution / netns isolation is not closed off"; fi +pass "jail netns is empty (lo only, no default route) and no nameserver answers — resolution cannot work, and the netns is why" +if [[ -n "$host_resolves" ]]; then + pass "name resolution dead in jail (getent 2, curl 6) against a name this host CAN resolve" +else + skip "7d part (i): this host cannot resolve api.anthropic.com either (offline?), so the in-jail resolution failure is not evidence about the sandbox — the assertions still ran and held, and part (ii) above is unaffected" +fi # 7e. --uds fail-loud: a nonexistent socket path must exit non-zero BEFORE the cmd. if "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ From c5a8cebd459a94222e9b6a6be44db2f024f8b490 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Wed, 16 Sep 2026 21:23:25 +0000 Subject: [PATCH 3/3] test(groom): key 7d part (ii) on netns IDENTITY, and close the round-2 gaps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 2 of the panel, all nine findings: - MEDIUM: on a host whose OWN netns is empty (a developer running this suite in a `--network=none` container) the interface/route facts are identical whether the jail unshares the netns or shares the host's, so a wrapper that stopped unsharing would still print PASS. Assert netns IDENTITY first — `readlink /proc/self/ns/net` in the jail must differ from the host's, passed in via `--env HOST_NETNS`. That discriminates on every host; the emptiness checks stay as the substantive claim (separate is not the same as empty). - bound the in-jail `getent` with `timeout 5`, matching its host-side twin: on the regression this section exists to catch, glibc would otherwise burn timeout x attempts x nameservers in a job with no `timeout-minutes`, and the run would die at the curl-6 assertion with a clipped 28 before part (ii) printed its precise diagnostic. A clipped lookup exits 124 and reds `= 2` loudly. - both loops must now OBSERVE their table rather than merely find no offending row: `saw_lo` for /proc/net/dev and a header check for /proc/net/route, so an empty or reformatted file cannot pass having read nothing. - also scan `/proc/net/ipv6_route` for a `::/0` default. The v4 FIB cannot see an IPv6 default, which contradicted the "no route for any protocol" claim. Count one only if it exits via a real device: every netns carries kernel-installed `unreachable` v6 defaults (RTF_REJECT, device lo). - drop the 1.1.1.1:53 probe. It could never go red — an HTTP GET to a DNS port cannot return 0 even on a routable netns (a DNS-over-TCP listener reads the request as a length-prefixed message and closes, giving 52/56) — and `rc = 0` was the only failing condition, so it spent 5s advertising an untested fact. This section's own standard: a proof that cannot go red is not a proof. - add `--noproxy "*"` to both in-jail curls: an exported http_proxy/HTTPS_PROXY/ ALL_PROXY would retarget curl at the proxy, exit 5/7, and red a correctly isolated jail. - correct the `--proc /proc` explanation. What makes these files jail-local is that /proc/net is a magic link to /proc/self/net, resolved against the READING task's netns — not the wrapper's `--proc` flag. As written it taught that part (ii) was coupled to that mount. - README: the "First" clause reintroduced the same mechanism error for `127.0.0.53`, which is inside the `127.0.0.0/8` the jail lo carries and so is routable in there. It contradicted the next two sentences and undercut the in-jail-bind hazard; the paragraph now names `127.0.0.53:53` explicitly. - qualify the final banner as `ALL SANDBOX TESTS PASSED (N skipped)` when part (i) is downgraded, so reduced coverage survives to the summary line. The existing prefix is preserved for anything grepping it. Verified: shellcheck clean; §1-§4 + §7a-§7e PASS locally (§5/§6 need node under /usr, pre-existing). Controls re-run after the rekey: real jail exit 0; `--share-net` jail exit 1 on the identity check, `jail netns id net:[...] is the HOST net namespace — the wrapper is not unsharing the netns at all`. Co-Authored-By: Claude Opus 5 --- .github/groom/README.md | 32 +++--- .github/groom/tests/sandbox-tests.sh | 144 +++++++++++++++++---------- 2 files changed, 111 insertions(+), 65 deletions(-) diff --git a/.github/groom/README.md b/.github/groom/README.md index 52afc7c5..3a77e11d 100644 --- a/.github/groom/README.md +++ b/.github/groom/README.md @@ -749,19 +749,22 @@ with only loopback up, so the broker — reached over the unix socket bind-mount at `/run/broker.sock` via the in-jail `jail-shim.mjs` TCP→UDS forwarder — is the *only* thing the agent can talk to. Host network, host loopback services, and cloud metadata (`169.254.169.254` / `168.63.129.16`) are all unreachable. So is -**name resolution**, and twice over. First, the netns has no route off-box at all -— one interface (`lo`), no default route — so the runner's stub resolver at -`127.0.0.53`, which sits on the *host's* loopback and not the jail's, cannot be -reached. Second, on a systemd-resolved runner `/etc/resolv.conf` is a symlink into +**name resolution**, and twice over. First, the jail is in its own network +namespace with no route off-box at all — one interface (`lo`), no IPv4 or IPv6 +default route — so nothing outside the jail is reachable, including any +nameserver on another host. Second, on a systemd-resolved runner `/etc/resolv.conf` is a symlink into `/run` — which the jail mounts `/etc` but deliberately not — so no `nameserver` line is readable and glibc falls back to the local machine (`127.0.0.1`, per resolv.conf(5)). That fallback *is* configured and routable: the jail's own `lo` carries all of `127.0.0.0/8`. Lookups fail because nothing is listening on the -jail's `127.0.0.1:53` — which is worth knowing concretely, because the jail -already runs in-jail loopback listeners (`jail-shim.mjs` on `127.0.0.1:8790`), so -a future in-jail bind to `:53` would silently become the agent's resolver. Either -way, a hostname the read-only `/etc/hosts` does not already answer cannot be -resolved. +jail's `127.0.0.1:53`. Note that the runner's stub resolver address +`127.0.0.53` is inside that same `127.0.0.0/8` the jail's `lo` carries, so it too +is *routable* inside the jail and fails only for want of a listener — which is +worth knowing concretely, because the jail already runs in-jail loopback +listeners (`jail-shim.mjs` on `127.0.0.1:8790`), so a future in-jail bind to +`127.0.0.1:53` or `127.0.0.53:53` would silently become the agent's resolver. +Either way, a hostname the read-only `/etc/hosts` does not already answer cannot +be resolved. Two consequences for callers: set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` in the agent env so the agent doesn't stall on telemetry endpoints that can @@ -801,10 +804,13 @@ check also carries a second assertion on top, because a resolution failure ALONE would still pass under a *shared* network namespace (the dangling `/etc/resolv.conf` above breaks resolution regardless of routing), and a proof that cannot go red is not a proof. That second assertion reads the jail's own -netns out of its fresh `/proc` — `/proc/net/dev` must list `lo` and nothing else, -`/proc/net/route` must carry no default route — rather than keying on a connect -exit code, which could not tell an isolated netns from a shared one behind a -firewall REJECT or on an offline host. No `claude`, no API key, no spend. +netns out of `/proc`: first its *identity* (`readlink /proc/self/ns/net` must +differ from the host's — the one fact that discriminates even on a host whose own +netns is empty), then its contents (`/proc/net/dev` must list `lo` and nothing +else, and neither `/proc/net/route` nor `/proc/net/ipv6_route` may carry a default +route). It deliberately does *not* key on a connect exit code, which could not +tell an isolated netns from a shared one behind a firewall REJECT or on an offline +host. No `claude`, no API key, no spend. ```bash shellcheck -x .github/groom/agent-sandbox.sh .github/groom/tests/sandbox-tests.sh diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh index 362f2dc1..cf528ee1 100755 --- a/.github/groom/tests/sandbox-tests.sh +++ b/.github/groom/tests/sandbox-tests.sh @@ -58,8 +58,10 @@ pass() { echo "PASS: $*" } +skips=0 skip() { echo "SKIP: $*" + skips=$(( skips + 1 )) } # --- fixtures ---------------------------------------------------------------- @@ -295,7 +297,7 @@ if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" -- bash -c '; then fail "arbitrary external IP reachable from jail"; fi pass "arbitrary external IP unreachable from jail" -# 7d. Name resolution is dead inside the jail, and nothing off-box is routable. +# 7d. Name resolution is dead inside the jail, and the jail is in its own netns. # # 7a/7b/7c all use IP literals ON PURPOSE, so not one of them ever touches the # resolver — routing is proven while name resolution is left untested. This section @@ -304,12 +306,12 @@ pass "arbitrary external IP unreachable from jail" # (i) Resolution fails. Asserted on resolver-SPECIFIC exit codes rather than a bare # non-zero, which is what stops the same false-pass the 7a/7b/7c tool-presence # check exists to block: `getent` 2 is "key not found" specifically (a missing -# binary is 127) and `curl` 6 is CURLE_COULDNT_RESOLVE_HOST specifically (a -# connect-stage failure is 7, a timeout 28, a missing binary 127). Part (i) is -# only EVIDENCE about the sandbox if the name is live off-jail, so the host-side -# control below gates it: when this host cannot resolve the name either, the -# assertions still run and still have to hold, but part (i) is reported SKIPPED -# rather than counted as proof. +# binary is 127, a clipped lookup 124) and `curl` 6 is CURLE_COULDNT_RESOLVE_HOST +# specifically (a connect-stage failure is 7, a timeout 28, a missing binary 127). +# Part (i) is only EVIDENCE about the sandbox if the name is live off-jail, so the +# host-side control below gates it: when this host cannot resolve the name either, +# the assertions still run and still have to hold, but part (i) is reported +# SKIPPED rather than counted as proof. # # (ii) THE NETNS PROOF — do NOT collapse this into (i). Part (i) on its own is # VACUOUS as evidence of network isolation: it passes under a SHARED netns too @@ -321,76 +323,110 @@ pass "arbitrary external IP unreachable from jail" # resolver is both configured AND routable in there; lookups fail only because # nothing is listening on the jail's 127.0.0.1:53. That holds whatever the netns # looks like, so part (i) would NOT turn red if the jail went back on a shared -# network. (It also means a future in-jail bind to :53 would silently become the -# agent's resolver — the jail already runs in-jail loopback listeners, e.g. -# jail-shim.mjs on 127.0.0.1:8790.) +# network. (It also means a future in-jail bind to 127.0.0.1:53 — or to +# 127.0.0.53:53, systemd-resolved's own address, which is in that same /8 — +# would silently become the agent's resolver. The jail already runs in-jail +# loopback listeners, e.g. jail-shim.mjs on 127.0.0.1:8790.) # -# So key (ii) on the netns ITSELF, read out of the jail's own /proc: the wrapper -# passes `--proc /proc`, a fresh procfs mount inside the new netns, so -# /proc/net/dev and /proc/net/route describe the JAIL's network and not the -# host's. An isolated netns has exactly one interface (`lo`) and an empty route -# table; a shared one shows the host's NICs and its default route. +# So key (ii) on the netns ITSELF, and key it FIRST on netns IDENTITY: compare +# `readlink /proc/self/ns/net` inside the jail against the host's. That is the one +# fact that discriminates on EVERY host — the interface/route facts below are +# identical for a shared and an unshared netns when the HOST itself has only `lo` +# and no default route (a developer running this suite inside a `--network=none` +# container), so on such a host they would pass a wrapper that had stopped +# unsharing the netns. The identity check cannot. # -# Do NOT key this on a connect exit code instead. `curl` 7 is +# The interface and route tables are then asserted as the substantive claim — +# the jail's network is EMPTY, not merely separate. `/proc/net` is a magic link +# to `/proc/self/net`, which the kernel resolves against the READING TASK's +# netns, so these describe the jail's network however /proc got mounted (this is +# NOT a property of the wrapper's `--proc /proc`; section 4 is what covers that). +# +# Do NOT key any of this on a connect exit code. `curl` 7 is # CURLE_COULDNT_CONNECT, which equally covers ECONNREFUSED, ENETUNREACH and a # firewall REJECT (--reject-with, ICMP admin-prohibited, or simply no default -# route), so a FULLY SHARED netns on a filtered or offline host returns 7 here -# too — an exact-7 assertion goes green on precisely the confinement regression -# it exists to catch. The /proc facts cannot be faked by host network state, need -# no egress of any kind, and cover a UDP/53 path as well as TCP: no route is no -# route, for any protocol. +# route), so a FULLY SHARED netns on a filtered or offline host returns 7 too — +# an exact-7 assertion goes green on precisely the confinement regression it +# exists to catch. The /proc facts need no egress of any kind, and cover a UDP/53 +# path as well as TCP: no route is no route, for any protocol. host_resolves="" if timeout 5 getent hosts api.anthropic.com >/dev/null 2>&1; then host_resolves=1 echo "note: this host resolves api.anthropic.com — the in-jail failure below is the sandbox, not a dead name" fi -if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" -- bash -c ' - # (i) name resolution fails - rc=0; getent hosts api.anthropic.com >/dev/null 2>&1 || rc=$? +host_netns="$(readlink /proc/self/ns/net || true)" +[[ -n "$host_netns" ]] || fail "cannot read the host's own netns id (/proc/self/ns/net) — 7d part (ii) needs it as the control" +if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ + --env "HOST_NETNS=$host_netns" -- bash -c ' + # (i) name resolution fails. `timeout 5` bounds the lookup: on exactly the + # regression this section exists to catch (jail back on a shared netns behind a + # blackholed resolver) glibc would otherwise burn timeout x attempts x + # nameservers here, and the sandbox-tests job sets no timeout-minutes. A clipped + # lookup exits 124 and reds the `= 2` assertion loudly — safe direction. + # --noproxy "*" keeps an exported http_proxy/HTTPS_PROXY/ALL_PROXY on the runner + # from retargeting curl at the proxy, which would exit 5/7 and red a correctly + # isolated jail. + rc=0; timeout 5 getent hosts api.anthropic.com >/dev/null 2>&1 || rc=$? if [ "$rc" = 0 ]; then echo "api.anthropic.com RESOLVED in jail — name resolution is not closed off (an /etc/hosts entry on the host would do this, served through the read-only /etc bind)"; exit 1; fi - [ "$rc" = 2 ] || { echo "getent exit $rc, want 2 (key not found) — getent missing from the jail PATH?"; exit 1; } - # --max-time here is a BACKSTOP, not margin: with the dangling resolv.conf - # described above glibc falls back to 127.0.0.1 and the jail lo refuses the send - # instantly, so this returns in milliseconds. It is deliberately NOT sized to - # outlast a retrying resolver (glibc defaults timeout:5 attempts:2 burn 10s on a - # single blackholing nameserver); a clipped lookup exits 28 and fails loud on the - # next line with the code printed, rather than passing silently. - rc=0; curl -s --max-time 10 http://api.anthropic.com/ >/dev/null 2>&1 || rc=$? + [ "$rc" = 2 ] || { echo "getent exit $rc, want 2 (key not found) — 124 is a clipped lookup, 127 getent missing from the jail PATH"; exit 1; } + # --max-time is a BACKSTOP, not margin: with the dangling resolv.conf described + # above glibc falls back to 127.0.0.1 and the jail lo refuses the send instantly, + # so this returns in milliseconds. It is deliberately NOT sized to outlast a + # retrying resolver (glibc defaults timeout:5 attempts:2 burn 10s on a single + # blackholing nameserver); a clipped lookup exits 28 and fails loud on the next + # line with the code printed, rather than passing silently. + rc=0; curl -s --noproxy "*" --max-time 10 http://api.anthropic.com/ >/dev/null 2>&1 || rc=$? if [ "$rc" = 0 ]; then echo "curl reached api.anthropic.com from jail — egress is NOT closed"; exit 1; fi [ "$rc" = 6 ] || { echo "curl exit $rc, want 6 (CURLE_COULDNT_RESOLVE_HOST) — failed past the resolver stage instead of at it"; exit 1; } - # (ii) the netns is EMPTY — the half a shared netns cannot fake, whatever the - # host network is doing. Read straight from the jail-local procfs; no tool - # beyond bash, so this cannot false-pass on a missing binary either. - [ -r /proc/net/dev ] || { echo "/proc/net/dev unreadable in jail — cannot verify the netns"; exit 1; } - [ -r /proc/net/route ] || { echo "/proc/net/route unreadable in jail — cannot verify the netns"; exit 1; } - extra_ifaces="" + # (ii-a) NETNS IDENTITY — the discriminator that works on every host. + jail_netns="$(readlink /proc/self/ns/net || true)" + [ -n "$jail_netns" ] || { echo "cannot read the jail netns id (/proc/self/ns/net) — cannot verify isolation"; exit 1; } + [ -n "${HOST_NETNS:-}" ] || { echo "HOST_NETNS was not passed into the jail — the netns-identity control is missing"; exit 1; } + if [ "$jail_netns" = "$HOST_NETNS" ]; then + echo "jail netns id $jail_netns is the HOST net namespace — the wrapper is not unsharing the netns at all"; exit 1 + fi + + # (ii-b) and that netns is EMPTY. Both loops must OBSERVE the table, not merely + # find no offending row: an unreadable-but-present or reformatted file would + # otherwise leave the collector empty and pass having seen nothing. + saw_lo=""; extra_ifaces="" while IFS= read -r line; do case "$line" in *:*) ;; *) continue ;; esac # skip the two header rows name="${line%%:*}" name="${name// /}" - [ -n "$name" ] && [ "$name" != lo ] && extra_ifaces="$extra_ifaces $name" + [ -z "$name" ] && continue + if [ "$name" = lo ]; then saw_lo=1; else extra_ifaces="$extra_ifaces $name"; fi done < /proc/net/dev + [ -n "$saw_lo" ] || { echo "/proc/net/dev listed no lo interface in the jail — read nothing, so it proves nothing"; exit 1; } if [ -n "$extra_ifaces" ]; then - echo "jail netns has non-loopback interface(s):$extra_ifaces — the netns is SHARED with the host, so nothing here proves isolation"; exit 1 + echo "jail netns has non-loopback interface(s):$extra_ifaces — the netns is not empty"; exit 1 fi - default_routes="" + # Default routes, v4 (/proc/net/route) AND v6 (/proc/net/ipv6_route) — the v4 FIB + # alone cannot see an IPv6 default, and "no route for any protocol" is the claim. + saw_v4_hdr=""; default_routes="" while read -r iface dest _; do - [ "$iface" = Iface ] && continue - [ "$dest" = 00000000 ] && default_routes="$default_routes $iface" + if [ "$iface" = Iface ]; then saw_v4_hdr=1; continue; fi + [ "$dest" = 00000000 ] && default_routes="$default_routes v4:$iface" done < /proc/net/route + [ -n "$saw_v4_hdr" ] || { echo "/proc/net/route had no header row in the jail — format changed, so this check proves nothing"; exit 1; } + # The v6 default rows present in EVERY netns are installed by the kernel as + # `unreachable` (flags RTF_REJECT, device lo) — count a v6 default only if it exits + # via a real device, which is what a shared netns would show. + if [ -r /proc/net/ipv6_route ]; then + while read -r dst dstlen _ _ _ _ _ _ _ dev _; do + [ "$dst" = 00000000000000000000000000000000 ] || continue + [ "$dstlen" = 00 ] || continue + [ "$dev" = lo ] && continue + default_routes="$default_routes v6:$dev" + done < /proc/net/ipv6_route + fi if [ -n "$default_routes" ]; then - echo "jail netns has a default route via:$default_routes — a nameserver (and everything else off-box) is routable"; exit 1 + echo "jail netns has a default route ($default_routes) — a nameserver (and everything else off-box) is routable"; exit 1 fi - - # Behavioral confirmation on top of those facts: nothing answers on a nameserver - # address. Asserted as "did not succeed" ONLY — the exact failure code is not - # load-bearing here, by design; see (ii). - rc=0; curl -s --max-time 5 http://1.1.1.1:53/ >/dev/null 2>&1 || rc=$? - if [ "$rc" = 0 ]; then echo "nameserver 1.1.1.1:53 answered from jail — netns is not isolated"; exit 1; fi exit 0 '; then fail "jail name resolution / netns isolation is not closed off"; fi -pass "jail netns is empty (lo only, no default route) and no nameserver answers — resolution cannot work, and the netns is why" +pass "jail is in its OWN netns (id != host) and that netns is empty (lo only, no v4/v6 default route) — resolution cannot work, and the netns is why" if [[ -n "$host_resolves" ]]; then pass "name resolution dead in jail (getent 2, curl 6) against a name this host CAN resolve" else @@ -404,4 +440,8 @@ if "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ fi pass "--uds fail-loud on a nonexistent socket path" -echo "ALL SANDBOX TESTS PASSED" +if [[ "$skips" -gt 0 ]]; then + echo "ALL SANDBOX TESTS PASSED ($skips skipped)" +else + echo "ALL SANDBOX TESTS PASSED" +fi