diff --git a/.github/groom/README.md b/.github/groom/README.md index eca038f0..3a77e11d 100644 --- a/.github/groom/README.md +++ b/.github/groom/README.md @@ -748,12 +748,29 @@ request-handling contract is identical on both transports. with only loopback up, so the broker — reached over the unix socket bind-mounted at `/run/broker.sock` via the in-jail `jail-shim.mjs` TCP→UDS forwarder — is the *only* thing the agent can talk to. Host network, host loopback services, and -cloud metadata (`169.254.169.254` / `168.63.129.16`) are all unreachable. Two -consequences for callers: set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` in the -agent env so the agent doesn't stall on telemetry endpoints that can never be -reached; and because there is no egress, in-jail `git fetch` / `npm install` -cannot work — anything the agent needs must already be in the clone before it is -sandboxed. +cloud metadata (`169.254.169.254` / `168.63.129.16`) are all unreachable. So is +**name resolution**, and twice over. First, the jail is in its own network +namespace with no route off-box at all — one interface (`lo`), no IPv4 or IPv6 +default route — so nothing outside the jail is reachable, including any +nameserver on another host. Second, on a systemd-resolved runner `/etc/resolv.conf` is a symlink into +`/run` — which the jail mounts `/etc` but deliberately not — so no `nameserver` +line is readable and glibc falls back to the local machine (`127.0.0.1`, per +resolv.conf(5)). That fallback *is* configured and routable: the jail's own `lo` +carries all of `127.0.0.0/8`. Lookups fail because nothing is listening on the +jail's `127.0.0.1:53`. Note that the runner's stub resolver address +`127.0.0.53` is inside that same `127.0.0.0/8` the jail's `lo` carries, so it too +is *routable* inside the jail and fails only for want of a listener — which is +worth knowing concretely, because the jail already runs in-jail loopback +listeners (`jail-shim.mjs` on `127.0.0.1:8790`), so a future in-jail bind to +`127.0.0.1:53` or `127.0.0.53:53` would silently become the agent's resolver. +Either way, a hostname the read-only `/etc/hosts` does not already answer cannot +be resolved. + +Two consequences for callers: set `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1` +in the agent env so the agent doesn't stall on telemetry endpoints that can +never be reached; and because there is no egress, in-jail `git fetch` / `npm +install` cannot work — anything the agent needs must already be in the clone +before it is sandboxed. ### The loud-preflight guarantee @@ -777,8 +794,23 @@ broker at a local fake upstream ([`tests/fake-upstream.mjs`](tests/fake-upstream *over the bind-mounted unix socket + in-jail `jail-shim.mjs`* to prove key injection/stripping, the `/healthz` + non-`/v1` behavior, and SSE pass-through. It also proves the BE-4369 egress isolation: host loopback, cloud metadata, and an -arbitrary external IP are all unreachable from the jail. No `claude`, no API key, -no spend. +arbitrary external IP are all unreachable from the jail, and name resolution is +dead. Every one of those reads a *failure* as the proof, so each is guarded +against false-passing on a missing tool — the IP-literal checks by asserting +`curl` is on the jail `PATH` first, the resolution check by asserting exact, +cause-specific exit codes (`getent` 2 = key not found, `curl` 6 = +could-not-resolve), which a missing binary's 127 cannot satisfy. The resolution +check also carries a second assertion on top, because a resolution failure ALONE +would still pass under a *shared* network namespace (the dangling +`/etc/resolv.conf` above breaks resolution regardless of routing), and a proof +that cannot go red is not a proof. That second assertion reads the jail's own +netns out of `/proc`: first its *identity* (`readlink /proc/self/ns/net` must +differ from the host's — the one fact that discriminates even on a host whose own +netns is empty), then its contents (`/proc/net/dev` must list `lo` and nothing +else, and neither `/proc/net/route` nor `/proc/net/ipv6_route` may carry a default +route). It deliberately does *not* key on a connect exit code, which could not +tell an isolated netns from a shared one behind a firewall REJECT or on an offline +host. No `claude`, no API key, no spend. ```bash shellcheck -x .github/groom/agent-sandbox.sh .github/groom/tests/sandbox-tests.sh diff --git a/.github/groom/tests/sandbox-tests.sh b/.github/groom/tests/sandbox-tests.sh index 4d8225a2..e5157df9 100755 --- a/.github/groom/tests/sandbox-tests.sh +++ b/.github/groom/tests/sandbox-tests.sh @@ -58,6 +58,12 @@ pass() { echo "PASS: $*" } +skips=0 +skip() { + echo "SKIP: $*" + skips=$(( skips + 1 )) +} + # --- fixtures ---------------------------------------------------------------- mkdir -p "$clone" @@ -291,7 +297,143 @@ if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" -- bash -c '; then fail "arbitrary external IP reachable from jail"; fi pass "arbitrary external IP unreachable from jail" -# 7d. --uds fail-loud: a nonexistent socket path must exit non-zero BEFORE the cmd. +# 7d. Name resolution is dead inside the jail, and the jail is in its own netns. +# +# 7a/7b/7c all use IP literals ON PURPOSE, so not one of them ever touches the +# resolver — routing is proven while name resolution is left untested. This section +# covers it, in TWO parts, because the obvious one-liner is a trap. +# +# (i) Resolution fails. Asserted on resolver-SPECIFIC exit codes rather than a bare +# non-zero, which is what stops the same false-pass the 7a/7b/7c tool-presence +# check exists to block: `getent` 2 is "key not found" specifically (a missing +# binary is 127, a clipped lookup 124) and `curl` 6 is CURLE_COULDNT_RESOLVE_HOST +# specifically (a connect-stage failure is 7, a timeout 28, a missing binary 127). +# Part (i) is only EVIDENCE about the sandbox if the name is live off-jail, so the +# host-side control below gates it: when this host cannot resolve the name either, +# the assertions still run and still have to hold, but part (i) is reported +# SKIPPED rather than counted as proof. +# +# (ii) THE NETNS PROOF — do NOT collapse this into (i). Part (i) on its own is +# VACUOUS as evidence of network isolation: it passes under a SHARED netns too +# (measured, not theorized). On a systemd-resolved host /etc/resolv.conf is a +# symlink into /run, and the jail mounts /etc but deliberately NOT /run (mounting +# it would be a confinement regression in its own right). With no readable +# `nameserver` line glibc falls back to the local machine — 127.0.0.1, per +# resolv.conf(5) — and the jail's own `lo` carries all of 127.0.0.0/8, so a +# resolver is both configured AND routable in there; lookups fail only because +# nothing is listening on the jail's 127.0.0.1:53. That holds whatever the netns +# looks like, so part (i) would NOT turn red if the jail went back on a shared +# network. (It also means a future in-jail bind to 127.0.0.1:53 — or to +# 127.0.0.53:53, systemd-resolved's own address, which is in that same /8 — +# would silently become the agent's resolver. The jail already runs in-jail +# loopback listeners, e.g. jail-shim.mjs on 127.0.0.1:8790.) +# +# So key (ii) on the netns ITSELF, and key it FIRST on netns IDENTITY: compare +# `readlink /proc/self/ns/net` inside the jail against the host's. That is the one +# fact that discriminates on EVERY host — the interface/route facts below are +# identical for a shared and an unshared netns when the HOST itself has only `lo` +# and no default route (a developer running this suite inside a `--network=none` +# container), so on such a host they would pass a wrapper that had stopped +# unsharing the netns. The identity check cannot. +# +# The interface and route tables are then asserted as the substantive claim — +# the jail's network is EMPTY, not merely separate. `/proc/net` is a magic link +# to `/proc/self/net`, which the kernel resolves against the READING TASK's +# netns, so these describe the jail's network however /proc got mounted (this is +# NOT a property of the wrapper's `--proc /proc`; section 4 is what covers that). +# +# Do NOT key any of this on a connect exit code. `curl` 7 is +# CURLE_COULDNT_CONNECT, which equally covers ECONNREFUSED, ENETUNREACH and a +# firewall REJECT (--reject-with, ICMP admin-prohibited, or simply no default +# route), so a FULLY SHARED netns on a filtered or offline host returns 7 too — +# an exact-7 assertion goes green on precisely the confinement regression it +# exists to catch. The /proc facts need no egress of any kind, and cover a UDP/53 +# path as well as TCP: no route is no route, for any protocol. +host_resolves="" +if timeout 5 getent hosts api.anthropic.com >/dev/null 2>&1; then + host_resolves=1 + echo "note: this host resolves api.anthropic.com — the in-jail failure below is the sandbox, not a dead name" +fi +host_netns="$(readlink /proc/self/ns/net || true)" +[[ -n "$host_netns" ]] || fail "cannot read the host's own netns id (/proc/self/ns/net) — 7d part (ii) needs it as the control" +if ! "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ + --env "HOST_NETNS=$host_netns" -- bash -c ' + # (i) name resolution fails. `timeout 5` bounds the lookup: on exactly the + # regression this section exists to catch (jail back on a shared netns behind a + # blackholed resolver) glibc would otherwise burn timeout x attempts x + # nameservers here, and the sandbox-tests job sets no timeout-minutes. A clipped + # lookup exits 124 and reds the `= 2` assertion loudly — safe direction. + # --noproxy "*" keeps an exported http_proxy/HTTPS_PROXY/ALL_PROXY on the runner + # from retargeting curl at the proxy, which would exit 5/7 and red a correctly + # isolated jail. + rc=0; timeout 5 getent hosts api.anthropic.com >/dev/null 2>&1 || rc=$? + if [ "$rc" = 0 ]; then echo "api.anthropic.com RESOLVED in jail — name resolution is not closed off (an /etc/hosts entry on the host would do this, served through the read-only /etc bind)"; exit 1; fi + [ "$rc" = 2 ] || { echo "getent exit $rc, want 2 (key not found) — 124 is a clipped lookup, 127 getent missing from the jail PATH"; exit 1; } + # --max-time is a BACKSTOP, not margin: with the dangling resolv.conf described + # above glibc falls back to 127.0.0.1 and the jail lo refuses the send instantly, + # so this returns in milliseconds. It is deliberately NOT sized to outlast a + # retrying resolver (glibc defaults timeout:5 attempts:2 burn 10s on a single + # blackholing nameserver); a clipped lookup exits 28 and fails loud on the next + # line with the code printed, rather than passing silently. + rc=0; curl -s --noproxy "*" --max-time 10 http://api.anthropic.com/ >/dev/null 2>&1 || rc=$? + if [ "$rc" = 0 ]; then echo "curl reached api.anthropic.com from jail — egress is NOT closed"; exit 1; fi + [ "$rc" = 6 ] || { echo "curl exit $rc, want 6 (CURLE_COULDNT_RESOLVE_HOST) — failed past the resolver stage instead of at it"; exit 1; } + + # (ii-a) NETNS IDENTITY — the discriminator that works on every host. + jail_netns="$(readlink /proc/self/ns/net || true)" + [ -n "$jail_netns" ] || { echo "cannot read the jail netns id (/proc/self/ns/net) — cannot verify isolation"; exit 1; } + [ -n "${HOST_NETNS:-}" ] || { echo "HOST_NETNS was not passed into the jail — the netns-identity control is missing"; exit 1; } + if [ "$jail_netns" = "$HOST_NETNS" ]; then + echo "jail netns id $jail_netns is the HOST net namespace — the wrapper is not unsharing the netns at all"; exit 1 + fi + + # (ii-b) and that netns is EMPTY. Both loops must OBSERVE the table, not merely + # find no offending row: an unreadable-but-present or reformatted file would + # otherwise leave the collector empty and pass having seen nothing. + saw_lo=""; extra_ifaces="" + while IFS= read -r line; do + case "$line" in *:*) ;; *) continue ;; esac # skip the two header rows + name="${line%%:*}" + name="${name// /}" + [ -z "$name" ] && continue + if [ "$name" = lo ]; then saw_lo=1; else extra_ifaces="$extra_ifaces $name"; fi + done < /proc/net/dev + [ -n "$saw_lo" ] || { echo "/proc/net/dev listed no lo interface in the jail — read nothing, so it proves nothing"; exit 1; } + if [ -n "$extra_ifaces" ]; then + echo "jail netns has non-loopback interface(s):$extra_ifaces — the netns is not empty"; exit 1 + fi + # Default routes, v4 (/proc/net/route) AND v6 (/proc/net/ipv6_route) — the v4 FIB + # alone cannot see an IPv6 default, and "no route for any protocol" is the claim. + saw_v4_hdr=""; default_routes="" + while read -r iface dest _; do + if [ "$iface" = Iface ]; then saw_v4_hdr=1; continue; fi + [ "$dest" = 00000000 ] && default_routes="$default_routes v4:$iface" + done < /proc/net/route + [ -n "$saw_v4_hdr" ] || { echo "/proc/net/route had no header row in the jail — format changed, so this check proves nothing"; exit 1; } + # The v6 default rows present in EVERY netns are installed by the kernel as + # `unreachable` (flags RTF_REJECT, device lo) — count a v6 default only if it exits + # via a real device, which is what a shared netns would show. + if [ -r /proc/net/ipv6_route ]; then + while read -r dst dstlen _ _ _ _ _ _ _ dev _; do + [ "$dst" = 00000000000000000000000000000000 ] || continue + [ "$dstlen" = 00 ] || continue + [ "$dev" = lo ] && continue + default_routes="$default_routes v6:$dev" + done < /proc/net/ipv6_route + fi + if [ -n "$default_routes" ]; then + echo "jail netns has a default route ($default_routes) — a nameserver (and everything else off-box) is routable"; exit 1 + fi + exit 0 +'; then fail "jail name resolution / netns isolation is not closed off"; fi +pass "jail is in its OWN netns (id != host) and that netns is empty (lo only, no v4/v6 default route) — resolution cannot work, and the netns is why" +if [[ -n "$host_resolves" ]]; then + pass "name resolution dead in jail (getent 2, curl 6) against a name this host CAN resolve" +else + skip "7d part (i): this host cannot resolve api.anthropic.com either (offline?), so the in-jail resolution failure is not evidence about the sandbox — the assertions still ran and held, and part (ii) above is unaffected" +fi + +# 7e. --uds fail-loud: a nonexistent socket path must exit non-zero BEFORE the cmd. if "$SANDBOX" --clone "$clone" --clone-mode ro --out-dir "$outdir" \ --uds /nonexistent.sock -- true 2>/dev/null; then fail "--uds /nonexistent.sock was accepted (must fail loud before running the command)" @@ -341,4 +483,8 @@ if PATH="$failbin:$PATH" "$SANDBOX" --preflight-only >/dev/null 2>&1; then fi pass "--preflight-only fails loud when the sandbox self-test cannot pass" -echo "ALL SANDBOX TESTS PASSED" +if [[ "$skips" -gt 0 ]]; then + echo "ALL SANDBOX TESTS PASSED ($skips skipped)" +else + echo "ALL SANDBOX TESTS PASSED" +fi