From b880d08e451958e56d2b0d0cdb33aa8bc48e1e50 Mon Sep 17 00:00:00 2001 From: JSONbored <49853598+JSONbored@users.noreply.github.com> Date: Thu, 23 Jul 2026 16:36:09 -0700 Subject: [PATCH 1/4] feat(miner): enforce AMS network-egress deny-by-default + allowlist (#7857) Implements #7648's ratified default for the miner's Docker/Compose deployment path: dnsmasq + iptables/ipset (a well-established domain-allowlist pattern, not novel), deny-by-default OUTPUT with a curated allowlist punched through via dnsmasq's own ipset= directive on each DNS resolution. The whole container's egress is restricted (no per-attempt process isolation exists today), so the miner's own necessary calls (broker, Sentry, discovery-index, Neon) are allowed automatically whenever their corresponding feature is actually configured -- not a blanket allowance. The image now starts as root so the entrypoint can configure the firewall, then drops to the unprivileged node user via gosu before running anything else -- verified as byte-identical to prior behavior from that point on. Fails closed: a firewall-setup failure aborts the container rather than running untrusted code with no restriction. Verified empirically against a real built image, not just unit tests: scripts/verify-egress-firewall.sh builds the real image and asserts on real network calls -- default allowlist blocks/allows correctly, operator-declared ecosystems/extraHosts correctly extend it live, LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL correctly disables it. All 7 checks pass. Scoped to Docker/Compose -- the primary documented self-hosting path. Kubernetes (blocked by a hard runAsNonRoot admission constraint) and systemd need their own privilege-setup mechanism, split into #8282. --- .loopover-ams.yml.example | 13 ++ k8s/miner-deployment.yaml | 4 + packages/loopover-miner/DEPLOYMENT.md | 14 ++ packages/loopover-miner/Dockerfile | 19 ++- .../loopover-miner/docker-compose.miner.yml | 11 +- .../loopover-miner/lib/egress-allowlist.ts | 123 ++++++++++++++++++ .../lib/egress-firewall-config.ts | 101 ++++++++++++++ .../lib/generate-egress-firewall-config.ts | 55 ++++++++ .../scripts/egress-firewall-entrypoint.sh | 35 +++++ .../scripts/verify-egress-firewall.sh | 67 ++++++++++ systemd/loopover-miner.service.example | 3 + test/unit/miner-egress-allowlist.test.ts | 121 +++++++++++++++++ .../unit/miner-egress-firewall-config.test.ts | 101 ++++++++++++++ ...er-generate-egress-firewall-config.test.ts | 103 +++++++++++++++ 14 files changed, 766 insertions(+), 4 deletions(-) create mode 100644 packages/loopover-miner/lib/egress-allowlist.ts create mode 100644 packages/loopover-miner/lib/egress-firewall-config.ts create mode 100644 packages/loopover-miner/lib/generate-egress-firewall-config.ts create mode 100644 packages/loopover-miner/scripts/egress-firewall-entrypoint.sh create mode 100755 packages/loopover-miner/scripts/verify-egress-firewall.sh create mode 100644 test/unit/miner-egress-allowlist.test.ts create mode 100644 test/unit/miner-egress-firewall-config.test.ts create mode 100644 test/unit/miner-generate-egress-firewall-config.test.ts diff --git a/.loopover-ams.yml.example b/.loopover-ams.yml.example index 9b4e4a7e18..9c12adb715 100644 --- a/.loopover-ams.yml.example +++ b/.loopover-ams.yml.example @@ -58,3 +58,16 @@ maxIterations: 3 # Per-iteration turn budget passed to the coding-agent driver. # A non-integer is floored. Default: 6. maxTurnsPerIteration: 6 + +# Network-egress allowlist additions (#7857) for the sandboxed coding-agent +# execution environment, on top of the always-on defaults (OS package +# registries, GitHub -- the target repo's own git remote). Enforced deny-by- +# default: an attempt can reach exactly these hosts plus the always-on +# defaults, nothing else. `ecosystems` picks whole registry categories (their +# real hostnames are resolved internally, e.g. `npm` -> registry.npmjs.org); +# `extraHosts` allows specific additional hostnames (e.g. a project's own +# third-party API). Values for ecosystems: npm | pypi | crates | go | +# rubygems | packagist | maven | nuget. Default: { ecosystems: [], extraHosts: [] }. +networkAllowlist: + ecosystems: [] + extraHosts: [] diff --git a/k8s/miner-deployment.yaml b/k8s/miner-deployment.yaml index 4f3ac19a8f..698e185443 100644 --- a/k8s/miner-deployment.yaml +++ b/k8s/miner-deployment.yaml @@ -30,6 +30,10 @@ spec: app.kubernetes.io/name: loopover-miner app.kubernetes.io/component: fleet-worker spec: + # #7857's network-egress firewall (deny-by-default, ratified allowlist) is NOT enforced on this + # deployment path yet -- `runAsNonRoot: true` below is a hard, kubelet-enforced constraint that rules out + # the root-then-drop entrypoint the Docker/Compose path uses; a k8s-native initContainer-based equivalent + # is tracked in #8282. Do not weaken runAsNonRoot to work around this -- follow #8282 instead. securityContext: runAsNonRoot: true runAsUser: 1000 diff --git a/packages/loopover-miner/DEPLOYMENT.md b/packages/loopover-miner/DEPLOYMENT.md index 0a10b00086..0a2d7f2e88 100644 --- a/packages/loopover-miner/DEPLOYMENT.md +++ b/packages/loopover-miner/DEPLOYMENT.md @@ -139,6 +139,20 @@ docker compose -f docker-compose.miner.yml up -d --build **Scaling to N parallel workers.** `docker compose -f docker-compose.miner.yml up -d --scale miner=N` gives every replica the **same** `miner-data` volume — and the miner's SQLite ledgers are **not** safe for concurrent access, so N replicas on one volume will contend/corrupt. To run N **isolated** workers, give each its own state: run N separate compose projects (`docker compose -p miner-1 …`, `-p miner-2 …` — `-p` namespaces the volume) or point each at a distinct `LOOPOVER_MINER_CONFIG_DIR` on its own mount. For built-in isolated horizontal scaling, use the Kubernetes StatefulSet in [`k8s/`](../../k8s/) (per-pod volumes). +### Network-egress firewall (Docker Compose / `docker run`) + +The container enforces deny-by-default network egress for the coding-agent execution environment (#7648, #7857): outbound traffic is blocked except to a curated allowlist (OS package registries, GitHub — the target repo's own git remote) plus whatever an operator declares in [`.loopover-ams.yml`](../../.loopover-ams.yml.example)'s `networkAllowlist` (ecosystem registries like npm/PyPI, or specific extra hostnames). This applies to the **whole container**, not just the coding-agent subprocess — no per-process network isolation exists yet, so the miner's own necessary calls (the Orb broker, Sentry, the discovery-index plane, Neon) are allowed automatically whenever their corresponding feature is actually configured. + +This changes what the container needs at `docker run`/Compose time: + +- `docker-compose.miner.yml` already grants the required `NET_ADMIN`/`NET_RAW` capabilities — nothing to change if you use it as-is. +- Running with a bare `docker run` instead: add `--cap-add=NET_ADMIN --cap-add=NET_RAW`. +- The image now starts as **root** (not `node`) so its entrypoint can configure the firewall, then drops to the unprivileged `node` user before running anything else — the actual miner process runs exactly as before. + +If the firewall setup itself fails (a misconfigured environment, `dnsmasq`/`iptables` erroring), the container **fails to start** rather than silently running with no restriction — that's deliberate (a coding-agent sandbox with no network policy at all is worse than not running). If you hit a real snag, `LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL=1` is the documented escape hatch to get unblocked, then use `.loopover-ams.yml`'s `networkAllowlist` to fix it properly and remove the override. + +Only covers Docker/Compose today. Kubernetes (`k8s/miner-deployment.yaml`) and systemd (bare-host) need their own privilege-setup mechanism (a k8s `initContainer`, a systemd `ExecStartPre=+`) — tracked separately in [#8282](https://github.com/JSONbored/loopover/issues/8282), not yet enforced on those paths. + ### Running fleet mode alongside ORB's `ams-observability` profile Fleet mode keeps miner state in a named `miner-data` volume, but ORB's `ams-reporting-exporter` (root [`docker-compose.yml`](../../docker-compose.yml), `--profile ams-observability`) reads the miner's ledgers from a **host** directory (default `~/.config/loopover-miner`). A named volume's host path is a Docker-managed internal detail, so the two never line up on their own — the exporter reads an empty directory and the Grafana AMS datasources stay **silently empty**. diff --git a/packages/loopover-miner/Dockerfile b/packages/loopover-miner/Dockerfile index d97e74d80e..83e8dee5b0 100644 --- a/packages/loopover-miner/Dockerfile +++ b/packages/loopover-miner/Dockerfile @@ -25,14 +25,27 @@ ENV NODE_ENV=production \ LOOPOVER_MINER_CONFIG_DIR=/data/miner \ LOOPOVER_MINER_VERSION=${LOOPOVER_VERSION} \ PATH=/app/node_modules/.bin:$PATH +# #7857: dnsmasq/iptables/ipset implement the network-egress firewall (deny-by-default, allowlisted via the +# operator's own .loopover-ams.yml); gosu drops from root to the node user after that firewall is set up -- +# see egress-firewall-entrypoint.sh for the full mechanism and why this image now starts as root. +RUN apt-get update && apt-get install -y --no-install-recommends dnsmasq iptables ipset gosu \ + && rm -rf /var/lib/apt/lists/* COPY --from=build --chown=node:node /app/node_modules ./node_modules COPY --from=build --chown=node:node /app/packages/loopover-miner ./packages/loopover-miner COPY --from=build --chown=node:node /app/packages/loopover-engine ./packages/loopover-engine -RUN mkdir -p /data/miner && chown -R node:node /data -USER node +RUN mkdir -p /data/miner && chown -R node:node /data \ + && chmod 755 /app/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh VOLUME ["/data/miner"] # No HEALTHCHECK: the miner is a batch/CLI workload (`docker run … loopover-miner `), not a # long-running HTTP service — there is no steady-state endpoint to probe unless an operator wraps # the container in their own supervising loop. -ENTRYPOINT ["loopover-miner"] +# +# #7857: this image now starts as ROOT (no `USER node` here) so the entrypoint can set up the egress firewall, +# which needs root/NET_ADMIN to configure iptables/ipset -- but it never RUNS the miner itself as root. The +# entrypoint's own last action is `exec gosu node loopover-miner ...`, which both drops privileges AND replaces +# the entrypoint process (`exec`, not a child) -- the actual miner command runs as `node`, exactly as it always +# has, for the entire remainder of the container's life. `docker-compose.miner.yml` grants the NET_ADMIN/NET_RAW +# capabilities this setup step needs (see that file's own comment); `LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL` is +# the documented opt-out for an operator who hits a real snag (see the entrypoint script's own header). +ENTRYPOINT ["/app/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh"] CMD ["doctor"] diff --git a/packages/loopover-miner/docker-compose.miner.yml b/packages/loopover-miner/docker-compose.miner.yml index 06606c2111..d05e5c7b15 100644 --- a/packages/loopover-miner/docker-compose.miner.yml +++ b/packages/loopover-miner/docker-compose.miner.yml @@ -25,9 +25,18 @@ services: context: ../.. dockerfile: packages/loopover-miner/Dockerfile image: loopover-miner:latest - # ENTRYPOINT is `loopover-miner`; `run` is the continuous fleet-worker loop. + # ENTRYPOINT is egress-firewall-entrypoint.sh (#7857), which sets up the network-egress firewall as root + # then execs `loopover-miner` as the unprivileged `node` user; `run` is the continuous fleet-worker loop. command: ["run"] restart: unless-stopped + # #7857: NET_ADMIN/NET_RAW so the entrypoint's iptables/ipset egress-firewall setup can actually configure + # rules -- same capability class this repo's own tailscale service already grants (docker-compose.yml), just + # container-scoped here rather than host-networked. The container itself still runs the real miner command + # as the unprivileged `node` user throughout (see the Dockerfile's own comment) -- these capabilities are + # only ever exercised by the root-owned entrypoint's brief setup window before it drops privileges. + cap_add: + - NET_ADMIN + - NET_RAW # Operator-supplied secrets/config (GITHUB_TOKEN, optional provider keys). Copy the .example, fill it in, # and NEVER commit the real file — env vars stay out of the compose file and out of `docker inspect` args. env_file: diff --git a/packages/loopover-miner/lib/egress-allowlist.ts b/packages/loopover-miner/lib/egress-allowlist.ts new file mode 100644 index 0000000000..b3281375d3 --- /dev/null +++ b/packages/loopover-miner/lib/egress-allowlist.ts @@ -0,0 +1,123 @@ +// Resolves the #7648-ratified network-egress allowlist for AMS sandboxed execution into a concrete list of +// hostnames (#7857's enforcement half -- the config surface this reads, AmsPolicySpec.networkAllowlist, shipped +// separately and was inert until this file). Pure: no IO, no shell/iptables text here (see +// egress-firewall-config.ts for that) -- just "given this operator's declared additions, what hosts are +// actually allowed." +// +// #7648's three ratified categories, and how each is resolved here: +// 1. "OS package registries" -- the miner image's own base (Debian, node:24-slim) apt mirrors. Static: this +// container's OS never changes at runtime. +// 2. "the target repo's own git remote" -- resolved as the fixed GitHub-family hostname set below, not +// derived per-attempt from the actual repo being worked on. This product only ever discovers/operates on +// GitHub-hosted repos (confirmed: no GitLab/Bitbucket path exists anywhere in this codebase) -- treating +// "the target repo's remote" as "GitHub" is a safe, always-correct simplification for this product, not a +// narrowing of what #7648 asked for. Revisit if a non-GitHub forge is ever supported. +// 3. "the repo's declared language-ecosystem registries" -- #7648's own text says "the repo's manifest +// actually declares," but AmsPolicySpec's own header explains why that's unsafe to derive from the TARGET +// repo: a malicious repo could fabricate a manifest entry to smuggle an attacker-controlled host into its +// own attempt's allowlist. Resolved instead from the OPERATOR's own declared `networkAllowlist.ecosystems` +// -- the trust-boundary-safe substitution #7857's own prior research already settled. +// +// Operator-declared `extraHosts` (#7648's "requesting broader access" case) are appended verbatim -- already +// validated against RFC 1123 hostname shape by ams-policy-spec.ts's own parser before they ever reach here. +// +// Beyond #7648's three categories, this firewall applies to the WHOLE container (the coding-agent subprocess +// shares its parent miner process's network namespace -- no per-process isolation exists today, see #7857's own +// research comments), so the MINER's own legitimate outbound calls need to stay allowed too, or this would +// break the miner's real function while trying to sandbox the coding agent. Each such host is added ONLY when +// its corresponding feature is actually configured on this instance -- the same "allow only what's actually +// needed" discipline as everything else here, not a blanket allowance "just in case": +// - the Orb broker (`ORB_BROKER_URL`, default api.loopover.ai) -- only if broker mode is active +// (`ORB_ENROLLMENT_SECRET` or #8202/#8246's `LOOPOVER_TENANT_SECRET_TOKEN` is set) +// - the discovery-index plane (`LOOPOVER_MINER_DISCOVERY_INDEX_URL`) -- only if set (opt-in, no default) +// - Sentry (`LOOPOVER_MINER_SENTRY_DSN`) -- only if set (opt-in, no default) +// - Neon's API (console.neon.tech, #7858's per-attempt DB fork) -- only if all three +// `LOOPOVER_MINER_NEON_*` vars are set, mirroring `resolveAttemptDbForkConfig`'s own all-or-nothing gate +// This is deliberately NOT exhaustive against every possible operator configuration (a fully custom, self-run +// discovery-index/broker fork at a URL this can't anticipate, or some other integration entirely) -- extraHosts +// is the documented escape hatch for anything these defaults miss. +import type { AmsNetworkAllowlist, AmsNetworkAllowlistEcosystem } from "@loopover/engine"; + +export type EgressAllowlistReason = + | "os-package-registry" + | "target-repo-git-remote" + | `ecosystem:${AmsNetworkAllowlistEcosystem}` + | "operator-declared" + | "loopover-platform"; + +export type EgressAllowlistEntry = { + host: string; + reason: EgressAllowlistReason; +}; + +/** Debian apt mirrors for the miner image's own base (node:24-slim) -- always allowed, unconditionally: the + * miner's own OS-level package installs (this file's enforcement setup itself needs `apt-get install`) must + * never be blocked by the same firewall it configures. */ +const OS_PACKAGE_REGISTRY_HOSTS = ["deb.debian.org", "security.debian.org"]; + +/** The hostnames git/GitHub operations actually touch: the API, git-over-https clone/fetch, codeload's tarball + * endpoint, and githubusercontent for raw-file/asset fetches a coding agent's tooling might reasonably hit. */ +const TARGET_REPO_GIT_REMOTE_HOSTS = ["github.com", "api.github.com", "codeload.github.com", "objects.githubusercontent.com", "raw.githubusercontent.com"]; + +/** One or more real registry hostnames per ecosystem #7857's config surface recognizes -- kept in the exact + * order AMS_NETWORK_ALLOWLIST_ECOSYSTEMS declares them (ams-policy-spec.ts) so a new ecosystem added there is + * a compile error here (an unhandled case in the Record type) rather than a silent gap. */ +const ECOSYSTEM_REGISTRY_HOSTS: Record = { + npm: ["registry.npmjs.org"], + pypi: ["pypi.org", "files.pythonhosted.org"], + crates: ["crates.io", "static.crates.io", "index.crates.io"], + go: ["proxy.golang.org", "sum.golang.org"], + rubygems: ["rubygems.org"], + packagist: ["repo.packagist.org"], + maven: ["repo.maven.apache.org"], + nuget: ["api.nuget.org"], +}; + +/** Neon's REST API host (`attempt-db-fork.ts`'s own `DEFAULT_API_BASE_URL`) -- fixed, not per-project. */ +const NEON_API_HOST = "console.neon.tech"; + +function hostnameOf(url: string | undefined): string | undefined { + if (!url) return undefined; + try { + return new URL(url).hostname.toLowerCase(); + } catch { + return undefined; + } +} + +/** Resolve an operator's declared `networkAllowlist` into the full, concrete set of allowed hostnames -- + * always-on defaults, this operator's own ecosystem/extraHosts additions, and the miner's own platform hosts + * (see this file's header) gated behind whichever features `env` shows are actually configured. Deduplicated + * by host: an ecosystem's own registry could coincidentally also appear in `extraHosts`, or the broker's + * default host could coincidentally equal something else here, and each entry should appear in the output + * exactly once (the FIRST reason it was allowed for wins). */ +export function resolveEgressAllowlist(networkAllowlist: AmsNetworkAllowlist, env: Record = {}): EgressAllowlistEntry[] { + const entries: EgressAllowlistEntry[] = []; + const seen = new Set(); + const add = (host: string, reason: EgressAllowlistReason): void => { + const normalized = host.toLowerCase(); + if (seen.has(normalized)) return; + seen.add(normalized); + entries.push({ host: normalized, reason }); + }; + + for (const host of OS_PACKAGE_REGISTRY_HOSTS) add(host, "os-package-registry"); + for (const host of TARGET_REPO_GIT_REMOTE_HOSTS) add(host, "target-repo-git-remote"); + for (const ecosystem of networkAllowlist.ecosystems) { + for (const host of ECOSYSTEM_REGISTRY_HOSTS[ecosystem]) add(host, `ecosystem:${ecosystem}`); + } + for (const host of networkAllowlist.extraHosts) add(host, "operator-declared"); + + if (env.ORB_ENROLLMENT_SECRET || env.LOOPOVER_TENANT_SECRET_TOKEN) { + add(hostnameOf(env.ORB_BROKER_URL) ?? "api.loopover.ai", "loopover-platform"); + } + const discoveryIndexHost = hostnameOf(env.LOOPOVER_MINER_DISCOVERY_INDEX_URL); + if (discoveryIndexHost) add(discoveryIndexHost, "loopover-platform"); + const sentryHost = hostnameOf(env.LOOPOVER_MINER_SENTRY_DSN); + if (sentryHost) add(sentryHost, "loopover-platform"); + if (env.LOOPOVER_MINER_NEON_API_KEY && env.LOOPOVER_MINER_NEON_PROJECT_ID && env.LOOPOVER_MINER_NEON_PARENT_BRANCH_ID) { + add(NEON_API_HOST, "loopover-platform"); + } + + return entries; +} diff --git a/packages/loopover-miner/lib/egress-firewall-config.ts b/packages/loopover-miner/lib/egress-firewall-config.ts new file mode 100644 index 0000000000..344e79479a --- /dev/null +++ b/packages/loopover-miner/lib/egress-firewall-config.ts @@ -0,0 +1,101 @@ +// Renders a resolved egress allowlist (egress-allowlist.ts) into the two config artifacts #7857's real +// enforcement needs: dnsmasq config text, and an iptables/ipset ruleset. Pure text generation, no IO -- the +// thin shell entrypoint (egress-firewall-entrypoint.sh) writes this output to disk and applies it. +// +// MECHANISM: dnsmasq is this container's ONLY permitted DNS resolver, configured with one `ipset=/host/set` +// directive per allowed hostname -- dnsmasq's own well-established feature where resolving a matched domain for +// ANY caller also adds the resolved IP to the named ipset automatically, in-band with the real lookup (no +// separate polling/refresh needed to track IP rotation on CDN-backed hosts like registry.npmjs.org). iptables +// then: (1) defaults OUTPUT to DROP, (2) allows loopback (dnsmasq itself, local IPC), (3) allows outbound DNS +// (port 53) ONLY to the upstream resolvers dnsmasq itself is configured to forward to -- forcing every domain +// lookup through the controlled resolver, since a subprocess trying to query a DIFFERENT DNS server directly +// is blocked before dnsmasq ever gets a chance to populate the ipset for it, and (4) allows outbound traffic to +// any IP already in the ipset, on standard ports. This is the well-established, standard Linux pattern for +// domain-based (not just IP-based) firewall allowlisting -- not a novel design. +// +// Every hostname is re-validated against the RFC 1123 shape (same regex ams-policy-spec.ts's own parser already +// enforces on `extraHosts`) immediately before it reaches this shell-adjacent text -- defense in depth: this +// module's own correctness must not depend on every future caller re-verifying its inputs were pre-sanitized +// upstream. +import type { EgressAllowlistEntry } from "./egress-allowlist.js"; + +export const EGRESS_IPSET_NAME = "loopover_egress_allow"; +export const EGRESS_ALLOWED_TCP_PORTS = [80, 443, 22] as const; + +// Same shape as ams-policy-spec.ts's own HOSTNAME_RE (kept independent rather than imported -- that file lives +// in @loopover/engine and validates operator input at parse time; this one is the last line of defense right +// before generating shell-adjacent config text, and must keep working even if that upstream check ever changes). +const HOSTNAME_RE = /^(?=.{1,253}$)(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\.)*[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?$/; + +export class InvalidEgressHostError extends Error { + constructor(host: string) { + super(`Refusing to generate egress-firewall config for an invalid hostname: ${JSON.stringify(host)}`); + this.name = "InvalidEgressHostError"; + } +} + +function assertValidHosts(entries: readonly EgressAllowlistEntry[]): void { + for (const entry of entries) { + if (!HOSTNAME_RE.test(entry.host)) throw new InvalidEgressHostError(entry.host); + } +} + +/** The `LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL` no-op ruleset (#7857's documented escape hatch): applies no + * restriction at all, for an operator who hit a real snag and needs to get unblocked before fixing their + * `networkAllowlist` properly and removing the override. */ +export function renderDisabledRuleset(): string { + return ["#!/bin/sh", "# Generated by egress-firewall-config.ts -- do not edit by hand.", "# LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL is set -- no network-egress restriction applied.", "exit 0", ""].join("\n"); +} + +/** Render dnsmasq config: one `ipset=/host/setname` directive per allowed hostname, plus `server=` lines for + * the upstream resolvers it forwards real lookups to. Deliberately does NOT set `no-resolv`/read + * `/etc/resolv.conf` -- the upstream servers are pinned explicitly so the iptables ruleset below can allow + * dnsmasq's own outbound DNS to those EXACT, known destinations rather than an open-ended "wherever + * /etc/resolv.conf happens to point." */ +export function renderDnsmasqConfig(entries: readonly EgressAllowlistEntry[], upstreamResolvers: readonly string[] = ["1.1.1.1", "8.8.8.8"]): string { + assertValidHosts(entries); + const lines = [ + "# Generated by egress-firewall-config.ts -- do not edit by hand.", + "port=53", + "bind-interfaces", + "listen-address=127.0.0.1", + "no-resolv", + ...upstreamResolvers.map((server) => `server=${server}`), + ...entries.map((entry) => `ipset=/${entry.host}/${EGRESS_IPSET_NAME}`), + ]; + return `${lines.join("\n")}\n`; +} + +/** Render the iptables/ipset ruleset as a plain shell script body (a sequence of `ipset`/`iptables` invocations, + * not a single ip(6)tables-restore blob, so the entrypoint can surface exactly which command failed). Deny-by- + * default OUTPUT; the RETURN value is meant to be written to a file and `source`d (or piped to `sh`) by the + * entrypoint as root, before dropping to the `node` user. */ +export function renderIptablesRuleset(entries: readonly EgressAllowlistEntry[], upstreamResolvers: readonly string[] = ["1.1.1.1", "8.8.8.8"]): string { + assertValidHosts(entries); + const lines = [ + "#!/bin/sh", + "# Generated by egress-firewall-config.ts -- do not edit by hand.", + "set -eu", + "", + `ipset create ${EGRESS_IPSET_NAME} hash:ip -exist`, + "", + "iptables -F OUTPUT", + "iptables -P OUTPUT DROP", + "", + "# Loopback: dnsmasq itself, local IPC.", + "iptables -A OUTPUT -o lo -j ACCEPT", + "", + "# DNS to the exact upstream resolvers dnsmasq forwards to, and nowhere else -- forces every lookup", + "# through dnsmasq, which is the only thing that ever adds an IP to the allow ipset.", + ...upstreamResolvers.flatMap((server) => [ + `iptables -A OUTPUT -p udp -d ${server} --dport 53 -j ACCEPT`, + `iptables -A OUTPUT -p tcp -d ${server} --dport 53 -j ACCEPT`, + ]), + "", + "# Egress to anything dnsmasq has already resolved for an allowed hostname, on standard ports.", + ...EGRESS_ALLOWED_TCP_PORTS.map((port) => `iptables -A OUTPUT -m set --match-set ${EGRESS_IPSET_NAME} dst -p tcp --dport ${port} -j ACCEPT`), + "", + `# Allowed hosts (${entries.length}): ${entries.map((entry) => `${entry.host} [${entry.reason}]`).join(", ") || "(none -- OS registries and target-repo-git-remote hosts are always present, so this should never be empty)"}`, + ]; + return `${lines.join("\n")}\n`; +} diff --git a/packages/loopover-miner/lib/generate-egress-firewall-config.ts b/packages/loopover-miner/lib/generate-egress-firewall-config.ts new file mode 100644 index 0000000000..a9bc5918ad --- /dev/null +++ b/packages/loopover-miner/lib/generate-egress-firewall-config.ts @@ -0,0 +1,55 @@ +// CLI entry for #7857's egress-firewall setup: resolve the operator's own `.loopover-ams.yml` +// (`networkAllowlist`), turn it into the concrete allowlist (egress-allowlist.ts), render dnsmasq config + +// an iptables/ipset ruleset (egress-firewall-config.ts), and write both to disk. Invoked by +// egress-firewall-entrypoint.sh as root, before dropping privileges to the `node` user -- this script only +// ever WRITES config files, it never itself calls `iptables`/`dnsmasq`/`ipset` (the shell entrypoint does that, +// keeping every actual privileged syscall in one small, auditable place). +// +// Runs once at container start: `.loopover-ams.yml` is an operator-local file (not per-attempt/per-repo), and +// `LOOPOVER_MINER_CONFIG_DIR` (where it lives) is already a real env var at container boot -- no attempt- +// specific context is needed to resolve it. +// +// LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL (#7857's documented escape hatch) is checked HERE, not in the shell +// entrypoint -- keeps the disable decision in one testable place, and the entrypoint script unconditional +// (always: generate, start dnsmasq, apply whatever ruleset was written -- real or no-op). +import { writeFileSync } from "node:fs"; +import { resolveAmsPolicy } from "./ams-policy.js"; +import { resolveEgressAllowlist } from "./egress-allowlist.js"; +import { renderDisabledRuleset, renderDnsmasqConfig, renderIptablesRuleset } from "./egress-firewall-config.js"; + +export async function generateEgressFirewallConfig( + dnsmasqConfigPath: string, + rulesetScriptPath: string, + env: Record = process.env, +): Promise<{ allowedHostCount: number; disabled: boolean }> { + // #7857's own policy resolver deliberately ignores repoFullName (this is the OPERATOR's own local policy, + // never a target-repo concern) -- passing an empty string is the documented no-op for that unused parameter. + const { spec } = await resolveAmsPolicy("", { env }); + const entries = resolveEgressAllowlist(spec.networkAllowlist, env); + writeFileSync(dnsmasqConfigPath, renderDnsmasqConfig(entries), "utf8"); + + const disabled = Boolean(env.LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL); + if (disabled) { + console.warn(JSON.stringify({ event: "egress_firewall_disabled", message: "LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL is set -- running with NO network-egress restriction" })); + } + writeFileSync(rulesetScriptPath, disabled ? renderDisabledRuleset() : renderIptablesRuleset(entries), { encoding: "utf8", mode: 0o755 }); + return { allowedHostCount: entries.length, disabled }; +} + +function main(): void { + const [, , dnsmasqConfigPath, rulesetScriptPath] = process.argv; + if (!dnsmasqConfigPath || !rulesetScriptPath) { + console.error(JSON.stringify({ event: "egress_firewall_config_missing_args", message: "usage: generate-egress-firewall-config.js " })); + process.exit(1); + } + generateEgressFirewallConfig(dnsmasqConfigPath, rulesetScriptPath) + .then(({ allowedHostCount, disabled }) => { + console.log(JSON.stringify({ event: "egress_firewall_config_generated", allowedHostCount, disabled, dnsmasqConfigPath, rulesetScriptPath })); + }) + .catch((error: unknown) => { + console.error(JSON.stringify({ event: "egress_firewall_config_generation_failed", message: error instanceof Error ? error.message : String(error) })); + process.exit(1); + }); +} + +if (process.argv[1] && import.meta.url === `file://${process.argv[1]}`) main(); diff --git a/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh b/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh new file mode 100644 index 0000000000..8f38e99393 --- /dev/null +++ b/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh @@ -0,0 +1,35 @@ +#!/bin/sh +# Root-privileged entrypoint wrapper for #7857's network-egress enforcement. This is the ONLY thing in the +# image that ever runs as root: it generates + applies the egress firewall (dnsmasq + iptables/ipset, +# generate-egress-firewall-config.js does the config generation, this script does the privileged apply), then +# drops to the unprivileged `node` user and execs the real entrypoint -- everything after the final `exec +# gosu` line is byte-identical to this image's pre-#7857 behavior, running as `node`, same as always. +# +# Deny-by-default per #7648's ratified decision -- fails CLOSED, deliberately: `set -eu` means a failure +# anywhere in firewall setup (dnsmasq won't start, an iptables command errors) aborts the container rather than +# silently falling through to running an untrusted coding-agent subprocess with NO network restriction at all. +# +# Deliberately unconditional: LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL (the documented escape hatch for an +# operator who hits a real snag) is decided in generate-egress-firewall-config.js, not here -- when set, that +# script still writes a real dnsmasq config but a no-op ruleset (renderDisabledRuleset), so this script's own +# flow never needs to know or care; it always generates, always starts dnsmasq, always applies whatever +# ruleset was written. +set -eu + +DNSMASQ_CONF=/etc/dnsmasq.d/loopover-egress.conf +RULESET_SCRIPT=/tmp/loopover-egress-ruleset.sh + +node /app/packages/loopover-miner/lib/generate-egress-firewall-config.js "$DNSMASQ_CONF" "$RULESET_SCRIPT" + +# Listens on 127.0.0.1 only (per the generated config's own bind-interfaces + listen-address) -- unreachable +# from outside this container regardless of what else is on its network. +dnsmasq --conf-file="$DNSMASQ_CONF" --pid-file=/var/run/dnsmasq.pid + +# Docker writes its own /etc/resolv.conf pointing at its embedded DNS server -- nothing routes lookups through +# dnsmasq until this container's OWN resolver config says to. Docker's own generated file says "this file can +# be edited" (confirmed empirically); this is the one intentional edit. +echo "nameserver 127.0.0.1" > /etc/resolv.conf + +sh "$RULESET_SCRIPT" + +exec gosu node loopover-miner "$@" diff --git a/packages/loopover-miner/scripts/verify-egress-firewall.sh b/packages/loopover-miner/scripts/verify-egress-firewall.sh new file mode 100755 index 0000000000..182e64e1c7 --- /dev/null +++ b/packages/loopover-miner/scripts/verify-egress-firewall.sh @@ -0,0 +1,67 @@ +#!/bin/bash +# Real, empirical proof that #7857's egress firewall actually enforces something -- builds the real Docker +# image, runs the real entrypoint setup inside a real container with real NET_ADMIN/NET_RAW, and asserts on +# real network calls. This is deliberately NOT part of `npm run test:ci` (vitest can't grant a test process +# NET_ADMIN or build/run Docker images) -- run it manually after touching anything under +# packages/loopover-miner/lib/egress-*.ts, the Dockerfile, or the entrypoint script, or wire it into a +# dedicated CI job that runs `docker build`. +# +# Requires: Docker with NET_ADMIN/NET_RAW support (works on a standard Linux CI runner or Docker Desktop). +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)" +IMAGE_TAG="loopover-miner:egress-firewall-verify" +FAILURES=0 + +echo "Building $IMAGE_TAG..." +docker build -f "$REPO_ROOT/packages/loopover-miner/Dockerfile" -t "$IMAGE_TAG" "$REPO_ROOT" >/tmp/egress-verify-build.log 2>&1 \ + || { echo "BUILD FAILED -- see /tmp/egress-verify-build.log"; exit 1; } + +# Runs a single fetch inside a fresh container with the given .loopover-ams.yml content (or none), and checks +# whether it matches the expected outcome ("success" or "blocked"). +check() { + local description="$1" ams_yml_content="$2" host="$3" expect="$4" extra_env="${5:-}" + local config_dir + config_dir="$(mktemp -d)" + if [ -n "$ams_yml_content" ]; then + printf '%s' "$ams_yml_content" > "$config_dir/.loopover-ams.yml" + fi + + local output + output=$(docker run --rm --cap-add=NET_ADMIN --cap-add=NET_RAW -v "$config_dir:/data/miner" ${extra_env:+-e "$extra_env"} --entrypoint sh "$IMAGE_TAG" -c " + set -e + node /app/packages/loopover-miner/lib/generate-egress-firewall-config.js /etc/dnsmasq.d/loopover-egress.conf /tmp/ruleset.sh >/dev/null + dnsmasq --conf-file=/etc/dnsmasq.d/loopover-egress.conf --pid-file=/var/run/dnsmasq.pid + echo 'nameserver 127.0.0.1' > /etc/resolv.conf + sh /tmp/ruleset.sh + gosu node node -e 'fetch(\"https://${host}\", {signal: AbortSignal.timeout(8000)}).then(r => console.log(\"RESULT:success\")).catch(e => console.log(\"RESULT:blocked \" + e.message))' + " 2>/dev/null || true) + rm -rf "$config_dir" + + local actual="blocked" + echo "$output" | grep -q "RESULT:success" && actual="success" + + if [ "$actual" = "$expect" ]; then + echo "PASS: $description (expected $expect, got $actual)" + else + echo "FAIL: $description (expected $expect, got $actual)" + FAILURES=$((FAILURES + 1)) + fi +} + +check "default allowlist: github.com (target-repo-git-remote default) succeeds" "" "github.com" "success" +check "default allowlist: an arbitrary host with no config is blocked" "" "example.com" "blocked" +check "default allowlist: npm registry is blocked without the npm ecosystem declared" "" "registry.npmjs.org" "blocked" +check "operator config: npm registry succeeds once ecosystems:[npm] is declared" $'networkAllowlist:\n ecosystems: [npm]\n extraHosts: []\n' "registry.npmjs.org" "success" +check "operator config: an operator-declared extraHost succeeds" $'networkAllowlist:\n ecosystems: []\n extraHosts: [example.com]\n' "example.com" "success" +check "operator config: an UNDECLARED host still blocked even with other config present" $'networkAllowlist:\n ecosystems: [npm]\n extraHosts: []\n' "pypi.org" "blocked" +check "LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL: a normally-blocked host succeeds when the escape hatch is set" "" "example.com" "success" "LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL=1" + +echo "" +if [ "$FAILURES" -eq 0 ]; then + echo "All egress-firewall checks passed." + exit 0 +else + echo "$FAILURES egress-firewall check(s) FAILED." + exit 1 +fi diff --git a/systemd/loopover-miner.service.example b/systemd/loopover-miner.service.example index 658be91d0e..f9d14c3abb 100644 --- a/systemd/loopover-miner.service.example +++ b/systemd/loopover-miner.service.example @@ -25,6 +25,9 @@ After=network-online.target Wants=network-online.target [Service] +# #7857's network-egress firewall (deny-by-default, ratified allowlist) is NOT enforced on this bare-host +# path yet -- a systemd-native equivalent (ExecStartPre=+, which runs as root regardless of User= below) is +# tracked in #8282. Type=simple # REQUIRED: run as a dedicated non-root user that owns the state directory below. User=loopover diff --git a/test/unit/miner-egress-allowlist.test.ts b/test/unit/miner-egress-allowlist.test.ts new file mode 100644 index 0000000000..a608f15219 --- /dev/null +++ b/test/unit/miner-egress-allowlist.test.ts @@ -0,0 +1,121 @@ +// Tests for #7857's egress-allowlist resolution: given an operator's networkAllowlist + env, what hosts are +// actually allowed. Pure function, no IO -- egress-firewall-config.test.ts covers the text-rendering half, +// and packages/loopover-miner/scripts/verify-egress-firewall.sh is the real, empirically-run proof that the +// generated config actually enforces anything against a live container. +import { describe, expect, it } from "vitest"; +import { resolveEgressAllowlist } from "../../packages/loopover-miner/lib/egress-allowlist"; +import type { AmsNetworkAllowlist } from "@loopover/engine"; + +const EMPTY_ALLOWLIST: AmsNetworkAllowlist = { ecosystems: [], extraHosts: [] }; + +describe("resolveEgressAllowlist (#7857)", () => { + it("always includes the OS-registry and target-repo-git-remote defaults, with no operator additions", () => { + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST); + const hosts = entries.map((e) => e.host); + expect(hosts).toContain("deb.debian.org"); + expect(hosts).toContain("security.debian.org"); + expect(hosts).toContain("github.com"); + expect(hosts).toContain("api.github.com"); + expect(hosts).toContain("codeload.github.com"); + expect(hosts).toContain("objects.githubusercontent.com"); + expect(hosts).toContain("raw.githubusercontent.com"); + expect(hosts).toHaveLength(7); + for (const entry of entries.filter((e) => e.host.includes("debian"))) expect(entry.reason).toBe("os-package-registry"); + for (const entry of entries.filter((e) => e.host.includes("github"))) expect(entry.reason).toBe("target-repo-git-remote"); + }); + + it("adds every host for a declared ecosystem, tagged with that ecosystem's reason", () => { + const entries = resolveEgressAllowlist({ ecosystems: ["pypi"], extraHosts: [] }); + const pypiEntries = entries.filter((e) => e.reason === "ecosystem:pypi"); + expect(pypiEntries.map((e) => e.host).sort()).toEqual(["files.pythonhosted.org", "pypi.org"]); + }); + + it("resolves every recognized ecosystem to at least one real registry host", () => { + const allEcosystems = ["npm", "pypi", "crates", "go", "rubygems", "packagist", "maven", "nuget"] as const; + const entries = resolveEgressAllowlist({ ecosystems: [...allEcosystems], extraHosts: [] }); + for (const ecosystem of allEcosystems) { + const forEcosystem = entries.filter((e) => e.reason === `ecosystem:${ecosystem}`); + expect(forEcosystem.length).toBeGreaterThan(0); + } + }); + + it("adds operator-declared extraHosts verbatim, tagged operator-declared", () => { + const entries = resolveEgressAllowlist({ ecosystems: [], extraHosts: ["api.example.com", "cdn.example.net"] }); + const extra = entries.filter((e) => e.reason === "operator-declared"); + expect(extra.map((e) => e.host).sort()).toEqual(["api.example.com", "cdn.example.net"]); + }); + + it("deduplicates a host that appears in both an ecosystem's registry set and extraHosts, keeping the first reason", () => { + const entries = resolveEgressAllowlist({ ecosystems: ["npm"], extraHosts: ["registry.npmjs.org"] }); + const matches = entries.filter((e) => e.host === "registry.npmjs.org"); + expect(matches).toHaveLength(1); + expect(matches[0]?.reason).toBe("ecosystem:npm"); + }); + + it("is case-insensitive when deduplicating and normalizes to lowercase", () => { + const entries = resolveEgressAllowlist({ ecosystems: [], extraHosts: ["API.Example.COM"] }); + const matches = entries.filter((e) => e.host === "api.example.com"); + expect(matches).toHaveLength(1); + }); + + describe("loopover platform hosts -- only added when the corresponding feature is actually configured", () => { + it("adds nothing platform-specific with an empty env", () => { + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, {}); + expect(entries.some((e) => e.reason === "loopover-platform")).toBe(false); + }); + + it("adds the default broker host (api.loopover.ai) when ORB_ENROLLMENT_SECRET is set", () => { + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { ORB_ENROLLMENT_SECRET: "orbsec_x" }); + expect(entries).toContainEqual({ host: "api.loopover.ai", reason: "loopover-platform" }); + }); + + it("adds the default broker host when #8202/#8246's LOOPOVER_TENANT_SECRET_TOKEN is set instead", () => { + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { LOOPOVER_TENANT_SECRET_TOKEN: "orbsec_x" }); + expect(entries).toContainEqual({ host: "api.loopover.ai", reason: "loopover-platform" }); + }); + + it("uses a custom ORB_BROKER_URL's own hostname instead of the default when broker mode is active", () => { + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { ORB_ENROLLMENT_SECRET: "s", ORB_BROKER_URL: "https://broker.example.internal" }); + expect(entries).toContainEqual({ host: "broker.example.internal", reason: "loopover-platform" }); + expect(entries.some((e) => e.host === "api.loopover.ai")).toBe(false); + }); + + it("adds the discovery-index host only when LOOPOVER_MINER_DISCOVERY_INDEX_URL is set (opt-in, no default)", () => { + expect(resolveEgressAllowlist(EMPTY_ALLOWLIST, {}).some((e) => e.reason === "loopover-platform")).toBe(false); + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { LOOPOVER_MINER_DISCOVERY_INDEX_URL: "https://discovery.loopover.ai" }); + expect(entries).toContainEqual({ host: "discovery.loopover.ai", reason: "loopover-platform" }); + }); + + it("adds the Sentry DSN host only when LOOPOVER_MINER_SENTRY_DSN is set", () => { + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { LOOPOVER_MINER_SENTRY_DSN: "https://abc123@o1.ingest.sentry.io/42" }); + expect(entries).toContainEqual({ host: "o1.ingest.sentry.io", reason: "loopover-platform" }); + }); + + it("adds Neon's API host only when ALL THREE LOOPOVER_MINER_NEON_* vars are set (all-or-nothing, matching resolveAttemptDbForkConfig's own gate)", () => { + expect( + resolveEgressAllowlist(EMPTY_ALLOWLIST, { LOOPOVER_MINER_NEON_API_KEY: "k", LOOPOVER_MINER_NEON_PROJECT_ID: "p" }).some((e) => e.reason === "loopover-platform"), + ).toBe(false); + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { + LOOPOVER_MINER_NEON_API_KEY: "k", + LOOPOVER_MINER_NEON_PROJECT_ID: "p", + LOOPOVER_MINER_NEON_PARENT_BRANCH_ID: "b", + }); + expect(entries).toContainEqual({ host: "console.neon.tech", reason: "loopover-platform" }); + }); + + it("falls back to the default broker host when ORB_BROKER_URL itself is malformed but broker mode is active (harmless: broker-client.ts's own orbBrokerBaseUrl throws before ever using it)", () => { + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { ORB_ENROLLMENT_SECRET: "s", ORB_BROKER_URL: "not a url" }); + expect(entries).toContainEqual({ host: "api.loopover.ai", reason: "loopover-platform" }); + }); + + it("tolerates a malformed URL in a platform env var without throwing, falling back to no host added (except the broker's own hardcoded default)", () => { + expect(() => resolveEgressAllowlist(EMPTY_ALLOWLIST, { LOOPOVER_MINER_SENTRY_DSN: "not a url" })).not.toThrow(); + const entries = resolveEgressAllowlist(EMPTY_ALLOWLIST, { LOOPOVER_MINER_SENTRY_DSN: "not a url" }); + expect(entries.some((e) => e.reason === "loopover-platform")).toBe(false); + }); + + it("defaults env to an empty object when not passed at all", () => { + expect(() => resolveEgressAllowlist(EMPTY_ALLOWLIST)).not.toThrow(); + }); + }); +}); diff --git a/test/unit/miner-egress-firewall-config.test.ts b/test/unit/miner-egress-firewall-config.test.ts new file mode 100644 index 0000000000..fd19aff3c9 --- /dev/null +++ b/test/unit/miner-egress-firewall-config.test.ts @@ -0,0 +1,101 @@ +// Tests for #7857's dnsmasq/iptables text generation (egress-firewall-config.ts). Pure text rendering, no IO +// -- packages/loopover-miner/scripts/verify-egress-firewall.sh is the real, empirically-run proof that this +// generated text actually enforces anything against a live container. +import { describe, expect, it } from "vitest"; +import { EGRESS_ALLOWED_TCP_PORTS, EGRESS_IPSET_NAME, InvalidEgressHostError, renderDisabledRuleset, renderDnsmasqConfig, renderIptablesRuleset } from "../../packages/loopover-miner/lib/egress-firewall-config"; +import type { EgressAllowlistEntry } from "../../packages/loopover-miner/lib/egress-allowlist"; + +const ENTRIES: EgressAllowlistEntry[] = [ + { host: "github.com", reason: "target-repo-git-remote" }, + { host: "registry.npmjs.org", reason: "ecosystem:npm" }, +]; + +describe("renderDnsmasqConfig (#7857)", () => { + it("emits one ipset directive per allowed host, plus the fixed listener/upstream config", () => { + const config = renderDnsmasqConfig(ENTRIES); + expect(config).toContain("port=53"); + expect(config).toContain("bind-interfaces"); + expect(config).toContain("listen-address=127.0.0.1"); + expect(config).toContain("no-resolv"); + expect(config).toContain(`ipset=/github.com/${EGRESS_IPSET_NAME}`); + expect(config).toContain(`ipset=/registry.npmjs.org/${EGRESS_IPSET_NAME}`); + }); + + it("uses the given upstream resolvers, not the default, when overridden", () => { + const config = renderDnsmasqConfig(ENTRIES, ["9.9.9.9"]); + expect(config).toContain("server=9.9.9.9"); + expect(config).not.toContain("server=1.1.1.1"); + }); + + it("defaults to the documented public upstream resolvers when none are given", () => { + const config = renderDnsmasqConfig(ENTRIES); + expect(config).toContain("server=1.1.1.1"); + expect(config).toContain("server=8.8.8.8"); + }); + + it("handles an empty entry list without error (still emits the fixed listener config)", () => { + const config = renderDnsmasqConfig([]); + expect(config).toContain("listen-address=127.0.0.1"); + expect(config).not.toContain("ipset=/"); + }); + + it("throws InvalidEgressHostError rather than emitting shell-adjacent text for an invalid hostname", () => { + expect(() => renderDnsmasqConfig([{ host: "not a valid host; rm -rf /", reason: "operator-declared" }])).toThrow(InvalidEgressHostError); + }); +}); + +describe("renderIptablesRuleset (#7857)", () => { + it("defaults OUTPUT to DROP and allows loopback", () => { + const ruleset = renderIptablesRuleset(ENTRIES); + expect(ruleset).toContain("iptables -P OUTPUT DROP"); + expect(ruleset).toContain("iptables -A OUTPUT -o lo -j ACCEPT"); + }); + + it("creates the ipset and allows egress matching it on every documented port", () => { + const ruleset = renderIptablesRuleset(ENTRIES); + expect(ruleset).toContain(`ipset create ${EGRESS_IPSET_NAME} hash:ip -exist`); + for (const port of EGRESS_ALLOWED_TCP_PORTS) { + expect(ruleset).toContain(`iptables -A OUTPUT -m set --match-set ${EGRESS_IPSET_NAME} dst -p tcp --dport ${port} -j ACCEPT`); + } + }); + + it("allows outbound DNS only to the given upstream resolvers, both udp and tcp", () => { + const ruleset = renderIptablesRuleset(ENTRIES, ["9.9.9.9"]); + expect(ruleset).toContain("iptables -A OUTPUT -p udp -d 9.9.9.9 --dport 53 -j ACCEPT"); + expect(ruleset).toContain("iptables -A OUTPUT -p tcp -d 9.9.9.9 --dport 53 -j ACCEPT"); + }); + + it("records every allowed host and its reason in a comment, for a human debugging a live container", () => { + const ruleset = renderIptablesRuleset(ENTRIES); + expect(ruleset).toContain("github.com [target-repo-git-remote]"); + expect(ruleset).toContain("registry.npmjs.org [ecosystem:npm]"); + }); + + it("notes explicitly when the entry list is empty, rather than a silently blank comment", () => { + const ruleset = renderIptablesRuleset([]); + expect(ruleset).toMatch(/Allowed hosts \(0\): \(none/); + }); + + it("begins with #!/bin/sh and set -eu, since the entrypoint runs this as a real script", () => { + const ruleset = renderIptablesRuleset(ENTRIES); + expect(ruleset.split("\n")[0]).toBe("#!/bin/sh"); + expect(ruleset).toContain("set -eu"); + }); + + it("throws InvalidEgressHostError rather than emitting shell-adjacent text for an invalid hostname", () => { + expect(() => renderIptablesRuleset([{ host: "$(curl evil.example)", reason: "operator-declared" }])).toThrow(InvalidEgressHostError); + }); +}); + +describe("renderDisabledRuleset (#7857)", () => { + it("is a valid, no-op shell script that exits 0 without touching iptables", () => { + const ruleset = renderDisabledRuleset(); + expect(ruleset.split("\n")[0]).toBe("#!/bin/sh"); + expect(ruleset).toContain("exit 0"); + expect(ruleset).not.toContain("iptables"); + }); + + it("mentions the env var responsible, so a human reading a live container's ruleset knows why", () => { + expect(renderDisabledRuleset()).toContain("LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL"); + }); +}); diff --git a/test/unit/miner-generate-egress-firewall-config.test.ts b/test/unit/miner-generate-egress-firewall-config.test.ts new file mode 100644 index 0000000000..20c455e25b --- /dev/null +++ b/test/unit/miner-generate-egress-firewall-config.test.ts @@ -0,0 +1,103 @@ +// Tests for #7857's generate-egress-firewall-config.ts -- the CLI entry that ties the operator's own +// .loopover-ams.yml, egress-allowlist.ts, and egress-firewall-config.ts together and writes the two real +// config files egress-firewall-entrypoint.sh applies. Real filesystem I/O against a scratch temp dir, matching +// miner-ams-policy.test.ts's own convention -- no fs mocking. +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { writeFileSync } from "node:fs"; +import { afterEach, describe, expect, it } from "vitest"; +import { generateEgressFirewallConfig } from "../../packages/loopover-miner/lib/generate-egress-firewall-config"; + +const roots: string[] = []; + +afterEach(() => { + for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); +}); + +function tempRoot(): string { + const root = mkdtempSync(join(tmpdir(), "loopover-miner-egress-firewall-config-")); + roots.push(root); + return root; +} + +describe("generateEgressFirewallConfig (#7857)", () => { + it("writes both config files using the engine's safe defaults when no local .loopover-ams.yml exists", async () => { + const configDir = tempRoot(); + const outDir = tempRoot(); + const dnsmasqPath = join(outDir, "dnsmasq.conf"); + const rulesetPath = join(outDir, "ruleset.sh"); + + const result = await generateEgressFirewallConfig(dnsmasqPath, rulesetPath, { LOOPOVER_MINER_CONFIG_DIR: configDir }); + + expect(result.allowedHostCount).toBe(7); // 2 OS-registry + 5 GitHub-family defaults, no operator additions + expect(result.disabled).toBe(false); + const dnsmasqConfig = readFileSync(dnsmasqPath, "utf8"); + expect(dnsmasqConfig).toContain("ipset=/github.com/loopover_egress_allow"); + const ruleset = readFileSync(rulesetPath, "utf8"); + expect(ruleset).toContain("iptables -P OUTPUT DROP"); + }); + + it("writes a no-op ruleset (but still a real dnsmasq config) when LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL is set", async () => { + const configDir = tempRoot(); + const outDir = tempRoot(); + const dnsmasqPath = join(outDir, "dnsmasq.conf"); + const rulesetPath = join(outDir, "ruleset.sh"); + + const result = await generateEgressFirewallConfig(dnsmasqPath, rulesetPath, { LOOPOVER_MINER_CONFIG_DIR: configDir, LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL: "1" }); + + expect(result.disabled).toBe(true); + const dnsmasqConfig = readFileSync(dnsmasqPath, "utf8"); + expect(dnsmasqConfig).toContain("ipset=/github.com/loopover_egress_allow"); // still generated normally + const ruleset = readFileSync(rulesetPath, "utf8"); + expect(ruleset).not.toContain("iptables"); + expect(ruleset).toContain("exit 0"); + }); + + it("reflects the operator's real .loopover-ams.yml networkAllowlist", async () => { + const configDir = tempRoot(); + const outDir = tempRoot(); + writeFileSync(join(configDir, ".loopover-ams.yml"), "networkAllowlist:\n ecosystems: [npm]\n extraHosts: [api.example.com]\n"); + const dnsmasqPath = join(outDir, "dnsmasq.conf"); + const rulesetPath = join(outDir, "ruleset.sh"); + + const result = await generateEgressFirewallConfig(dnsmasqPath, rulesetPath, { LOOPOVER_MINER_CONFIG_DIR: configDir }); + + expect(result.allowedHostCount).toBe(9); // 7 defaults + registry.npmjs.org + api.example.com + const dnsmasqConfig = readFileSync(dnsmasqPath, "utf8"); + expect(dnsmasqConfig).toContain("ipset=/registry.npmjs.org/loopover_egress_allow"); + expect(dnsmasqConfig).toContain("ipset=/api.example.com/loopover_egress_allow"); + }); + + it("also reflects the miner's own platform hosts when the corresponding env vars are set", async () => { + const configDir = tempRoot(); + const outDir = tempRoot(); + const dnsmasqPath = join(outDir, "dnsmasq.conf"); + const rulesetPath = join(outDir, "ruleset.sh"); + + await generateEgressFirewallConfig(dnsmasqPath, rulesetPath, { LOOPOVER_MINER_CONFIG_DIR: configDir, ORB_ENROLLMENT_SECRET: "s" }); + + const dnsmasqConfig = readFileSync(dnsmasqPath, "utf8"); + expect(dnsmasqConfig).toContain("ipset=/api.loopover.ai/loopover_egress_allow"); + }); + + it("makes the written ruleset script executable (mode 0o755)", async () => { + const configDir = tempRoot(); + const outDir = tempRoot(); + const rulesetPath = join(outDir, "ruleset.sh"); + + await generateEgressFirewallConfig(join(outDir, "dnsmasq.conf"), rulesetPath, { LOOPOVER_MINER_CONFIG_DIR: configDir }); + + const { statSync } = await import("node:fs"); + const mode = statSync(rulesetPath).mode & 0o777; + expect(mode).toBe(0o755); + }); + + it("defaults env to process.env when not passed", async () => { + const configDir = tempRoot(); + const outDir = tempRoot(); + await expect( + generateEgressFirewallConfig(join(outDir, "dnsmasq.conf"), join(outDir, "ruleset.sh"), { ...process.env, LOOPOVER_MINER_CONFIG_DIR: configDir }), + ).resolves.toBeDefined(); + }); +}); From 818f7ee5088fb764cdc7f4d2c012c31f3805388a Mon Sep 17 00:00:00 2001 From: JSONbored <49853598+JSONbored@users.noreply.github.com> Date: Thu, 23 Jul 2026 17:16:13 -0700 Subject: [PATCH 2/4] fix(miner): CI fixes for the egress-firewall PR -- env-reference, coverage, security review - Regenerate packages/loopover-miner/docs/env-reference.md and apps/loopover-ui/src/lib/ams-env-reference.ts (a separate generator from the deployment-docs audit already checked locally; missed it the first pass -- both now include LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL and the other new vars). - generate-egress-firewall-config.ts's main() was 100% uncovered by patch (real Codecov gap, not the local coverage-merge artifact the other two new files showed): refactored to accept injectable IO, matching scripts/check-miner-deployment-docs.ts's own main(env, io) pattern already in this codebase, so it's exercised in-process rather than needing an untestable subprocess. New tests cover missing-args, single-arg, success, the underlying-failure catch path, and the real default-IO construction. - Fixed both Superagent-flagged findings, verified empirically against a real built image before and after: - The entrypoint script and the root-invoked config-generation script were node-owned despite executing with root privilege -- a compromised node-level process (the exact thing this whole mechanism sandboxes) could rewrite either to persist a root backdoor across a restart. Now root:root. Scoped to these two files, not their full transitive dependency graph -- documented as a deliberate, bounded scope in the Dockerfile's own comment. - dnsmasq's privilege drop is now explicit (--user=nobody --group=nogroup) rather than relying on its own default. The scanner's specific claim ("running as root") was empirically false -- confirmed via /proc//status that dnsmasq already drops to nobody (uid 65534) unprompted -- and its specific suggested fix (--user=node) would have been a regression (wider privilege than dnsmasq needs, not narrower). Fixed the real gap (implicit vs. explicit) without following the incorrect specific recommendation. Re-verified full unsharded npm run test:coverage (21177/21198 passing, 0 failures) and the real 7-check verify-egress-firewall.sh against a freshly rebuilt image after both security fixes -- all still passing. --- apps/loopover-ui/src/lib/ams-env-reference.ts | 28 +++++- packages/loopover-miner/Dockerfile | 8 ++ packages/loopover-miner/docs/env-reference.md | 6 +- .../lib/generate-egress-firewall-config.ts | 38 ++++--- .../scripts/egress-firewall-entrypoint.sh | 9 +- ...er-generate-egress-firewall-config.test.ts | 98 ++++++++++++++++++- 6 files changed, 167 insertions(+), 20 deletions(-) diff --git a/apps/loopover-ui/src/lib/ams-env-reference.ts b/apps/loopover-ui/src/lib/ams-env-reference.ts index 72c8d25fe1..10389b1a62 100644 --- a/apps/loopover-ui/src/lib/ams-env-reference.ts +++ b/apps/loopover-ui/src/lib/ams-env-reference.ts @@ -66,6 +66,11 @@ export const AMS_ENV_REFERENCE_ROWS: MinerEnvReferenceRow[] = [ firstReference: "lib/deny-hook-synthesis.ts", defaultValue: "", }, + { + name: "LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL", + firstReference: "lib/generate-egress-firewall-config.ts", + defaultValue: null, + }, { name: "LOOPOVER_MINER_DISCOVERY_INDEX_URL", firstReference: "lib/discovery-index-client.ts", @@ -106,6 +111,21 @@ export const AMS_ENV_REFERENCE_ROWS: MinerEnvReferenceRow[] = [ firstReference: "lib/logger.ts", defaultValue: "", }, + { + name: "LOOPOVER_MINER_NEON_API_KEY", + firstReference: "lib/egress-allowlist.ts", + defaultValue: null, + }, + { + name: "LOOPOVER_MINER_NEON_PARENT_BRANCH_ID", + firstReference: "lib/egress-allowlist.ts", + defaultValue: null, + }, + { + name: "LOOPOVER_MINER_NEON_PROJECT_ID", + firstReference: "lib/egress-allowlist.ts", + defaultValue: null, + }, { name: "LOOPOVER_MINER_NO_UPDATE_CHECK", firstReference: "lib/update-check.ts", @@ -158,7 +178,7 @@ export const AMS_ENV_REFERENCE_ROWS: MinerEnvReferenceRow[] = [ }, { name: "LOOPOVER_MINER_SENTRY_DSN", - firstReference: "lib/sentry.ts", + firstReference: "lib/egress-allowlist.ts", defaultValue: null, }, { @@ -227,6 +247,7 @@ export const AMS_ENV_REFERENCE_MARKDOWN = [ '| `LOOPOVER_MINER_CONTROL_PLANE_ADMIN_TOKEN` | `lib/tenant-client.ts` | `""` |', '| `LOOPOVER_MINER_CONTROL_PLANE_URL` | `lib/tenant-client.ts` | `""` |', '| `LOOPOVER_MINER_DENY_HOOK_SYNTHESIS_DB` | `lib/deny-hook-synthesis.ts` | `""` |', + "| `LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL` | `lib/generate-egress-firewall-config.ts` | (none) |", '| `LOOPOVER_MINER_DISCOVERY_INDEX_URL` | `lib/discovery-index-client.ts` | `""` |', '| `LOOPOVER_MINER_DISCOVERY_PLANE` | `lib/discovery-index-client.ts` | `""` |', '| `LOOPOVER_MINER_DISCOVERY_SHARED_SECRET` | `lib/discovery-index-client.ts` | `""` |', @@ -235,6 +256,9 @@ export const AMS_ENV_REFERENCE_MARKDOWN = [ "| `LOOPOVER_MINER_GOVERNOR_LEDGER_DB` | `lib/governor-ledger.ts` | (none) |", "| `LOOPOVER_MINER_GOVERNOR_STATE_DB` | `lib/governor-state.ts` | (none) |", '| `LOOPOVER_MINER_LOG_LEVEL` | `lib/logger.ts` | `""` |', + "| `LOOPOVER_MINER_NEON_API_KEY` | `lib/egress-allowlist.ts` | (none) |", + "| `LOOPOVER_MINER_NEON_PARENT_BRANCH_ID` | `lib/egress-allowlist.ts` | (none) |", + "| `LOOPOVER_MINER_NEON_PROJECT_ID` | `lib/egress-allowlist.ts` | (none) |", '| `LOOPOVER_MINER_NO_UPDATE_CHECK` | `lib/update-check.ts` | `""` |', '| `LOOPOVER_MINER_ORB_EXPORT_DB` | `lib/orb-export.ts` | `""` |', "| `LOOPOVER_MINER_PLAN_STORE_DB` | `lib/plan-store.ts` | (none) |", @@ -245,7 +269,7 @@ export const AMS_ENV_REFERENCE_MARKDOWN = [ "| `LOOPOVER_MINER_RANKED_CANDIDATES_DB` | `lib/ranked-candidates.ts` | (none) |", "| `LOOPOVER_MINER_REPLAY_SNAPSHOT_DB` | `lib/replay-snapshot.ts` | (none) |", "| `LOOPOVER_MINER_RUN_STATE_DB` | `lib/run-state.ts` | (none) |", - "| `LOOPOVER_MINER_SENTRY_DSN` | `lib/sentry.ts` | (none) |", + "| `LOOPOVER_MINER_SENTRY_DSN` | `lib/egress-allowlist.ts` | (none) |", '| `LOOPOVER_MINER_SENTRY_ENVIRONMENT` | `lib/sentry.ts` | `"production"` |', '| `LOOPOVER_MINER_VERSION` | `lib/version.ts` | `""` |', "| `LOOPOVER_MINER_WORKTREE_ALLOCATOR_DB` | `lib/worktree-allocator.ts` | (none) |", diff --git a/packages/loopover-miner/Dockerfile b/packages/loopover-miner/Dockerfile index 83e8dee5b0..8f751e52cf 100644 --- a/packages/loopover-miner/Dockerfile +++ b/packages/loopover-miner/Dockerfile @@ -34,6 +34,14 @@ COPY --from=build --chown=node:node /app/node_modules ./node_modules COPY --from=build --chown=node:node /app/packages/loopover-miner ./packages/loopover-miner COPY --from=build --chown=node:node /app/packages/loopover-engine ./packages/loopover-engine RUN mkdir -p /data/miner && chown -R node:node /data \ + # #7857 security review: the two files that actually EXECUTE with root privilege (the entrypoint itself, + # and the config-generation script it invokes as root) must not be writable by node -- a compromised + # node-level process (the coding-agent subprocess this whole mechanism sandboxes) could otherwise rewrite + # either one, persisting a root-privileged backdoor across this container's next restart. Scoped to these + # two specific files, not their full transitive dependency graph (ams-policy.js/egress-*.js and everything + # under node_modules/@loopover/engine they in turn import) -- hardening the entire graph would be a much + # larger, more invasive change; this closes the direct, obvious persistence vector without one. + && chown root:root /app/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh /app/packages/loopover-miner/lib/generate-egress-firewall-config.js /app/packages/loopover-miner/lib/generate-egress-firewall-config.ts \ && chmod 755 /app/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh VOLUME ["/data/miner"] # No HEALTHCHECK: the miner is a batch/CLI workload (`docker run … loopover-miner `), not a diff --git a/packages/loopover-miner/docs/env-reference.md b/packages/loopover-miner/docs/env-reference.md index c9f3a42c3f..507bcfa580 100644 --- a/packages/loopover-miner/docs/env-reference.md +++ b/packages/loopover-miner/docs/env-reference.md @@ -16,6 +16,7 @@ Generated by `npm run miner:env-reference`. Do not edit manually. | `LOOPOVER_MINER_CONTROL_PLANE_ADMIN_TOKEN` | `lib/tenant-client.ts` | `""` | | `LOOPOVER_MINER_CONTROL_PLANE_URL` | `lib/tenant-client.ts` | `""` | | `LOOPOVER_MINER_DENY_HOOK_SYNTHESIS_DB` | `lib/deny-hook-synthesis.ts` | `""` | +| `LOOPOVER_MINER_DISABLE_EGRESS_FIREWALL` | `lib/generate-egress-firewall-config.ts` | (none) | | `LOOPOVER_MINER_DISCOVERY_INDEX_URL` | `lib/discovery-index-client.ts` | `""` | | `LOOPOVER_MINER_DISCOVERY_PLANE` | `lib/discovery-index-client.ts` | `""` | | `LOOPOVER_MINER_DISCOVERY_SHARED_SECRET` | `lib/discovery-index-client.ts` | `""` | @@ -24,6 +25,9 @@ Generated by `npm run miner:env-reference`. Do not edit manually. | `LOOPOVER_MINER_GOVERNOR_LEDGER_DB` | `lib/governor-ledger.ts` | (none) | | `LOOPOVER_MINER_GOVERNOR_STATE_DB` | `lib/governor-state.ts` | (none) | | `LOOPOVER_MINER_LOG_LEVEL` | `lib/logger.ts` | `""` | +| `LOOPOVER_MINER_NEON_API_KEY` | `lib/egress-allowlist.ts` | (none) | +| `LOOPOVER_MINER_NEON_PARENT_BRANCH_ID` | `lib/egress-allowlist.ts` | (none) | +| `LOOPOVER_MINER_NEON_PROJECT_ID` | `lib/egress-allowlist.ts` | (none) | | `LOOPOVER_MINER_NO_UPDATE_CHECK` | `lib/update-check.ts` | `""` | | `LOOPOVER_MINER_ORB_EXPORT_DB` | `lib/orb-export.ts` | `""` | | `LOOPOVER_MINER_PLAN_STORE_DB` | `lib/plan-store.ts` | (none) | @@ -34,7 +38,7 @@ Generated by `npm run miner:env-reference`. Do not edit manually. | `LOOPOVER_MINER_RANKED_CANDIDATES_DB` | `lib/ranked-candidates.ts` | (none) | | `LOOPOVER_MINER_REPLAY_SNAPSHOT_DB` | `lib/replay-snapshot.ts` | (none) | | `LOOPOVER_MINER_RUN_STATE_DB` | `lib/run-state.ts` | (none) | -| `LOOPOVER_MINER_SENTRY_DSN` | `lib/sentry.ts` | (none) | +| `LOOPOVER_MINER_SENTRY_DSN` | `lib/egress-allowlist.ts` | (none) | | `LOOPOVER_MINER_SENTRY_ENVIRONMENT` | `lib/sentry.ts` | `"production"` | | `LOOPOVER_MINER_VERSION` | `lib/version.ts` | `""` | | `LOOPOVER_MINER_WORKTREE_ALLOCATOR_DB` | `lib/worktree-allocator.ts` | (none) | diff --git a/packages/loopover-miner/lib/generate-egress-firewall-config.ts b/packages/loopover-miner/lib/generate-egress-firewall-config.ts index a9bc5918ad..54a78b5644 100644 --- a/packages/loopover-miner/lib/generate-egress-firewall-config.ts +++ b/packages/loopover-miner/lib/generate-egress-firewall-config.ts @@ -36,20 +36,32 @@ export async function generateEgressFirewallConfig( return { allowedHostCount: entries.length, disabled }; } -function main(): void { - const [, , dnsmasqConfigPath, rulesetScriptPath] = process.argv; +/** Injectable IO for {@link main} -- lets tests exercise the real CLI-entry logic in-process (asserting on + * what gets logged/exited) without a subprocess, the same pattern `scripts/check-miner-deployment-docs.ts`'s + * own `main(env, io)` already uses in this codebase. */ +export type GenerateEgressFirewallConfigIo = { + argv: string[]; + log: (...args: unknown[]) => void; + error: (...args: unknown[]) => void; + exit: (code: number) => void; +}; + +export async function main( + io: GenerateEgressFirewallConfigIo = { argv: process.argv, log: console.log.bind(console), error: console.error.bind(console), exit: (code) => process.exit(code) }, +): Promise { + const [, , dnsmasqConfigPath, rulesetScriptPath] = io.argv; if (!dnsmasqConfigPath || !rulesetScriptPath) { - console.error(JSON.stringify({ event: "egress_firewall_config_missing_args", message: "usage: generate-egress-firewall-config.js " })); - process.exit(1); + io.error(JSON.stringify({ event: "egress_firewall_config_missing_args", message: "usage: generate-egress-firewall-config.js " })); + io.exit(1); + return; + } + try { + const { allowedHostCount, disabled } = await generateEgressFirewallConfig(dnsmasqConfigPath, rulesetScriptPath); + io.log(JSON.stringify({ event: "egress_firewall_config_generated", allowedHostCount, disabled, dnsmasqConfigPath, rulesetScriptPath })); + } catch (error) { + io.error(JSON.stringify({ event: "egress_firewall_config_generation_failed", message: error instanceof Error ? error.message : String(error) })); + io.exit(1); } - generateEgressFirewallConfig(dnsmasqConfigPath, rulesetScriptPath) - .then(({ allowedHostCount, disabled }) => { - console.log(JSON.stringify({ event: "egress_firewall_config_generated", allowedHostCount, disabled, dnsmasqConfigPath, rulesetScriptPath })); - }) - .catch((error: unknown) => { - console.error(JSON.stringify({ event: "egress_firewall_config_generation_failed", message: error instanceof Error ? error.message : String(error) })); - process.exit(1); - }); } -if (process.argv[1] && import.meta.url === `file://${process.argv[1]}`) main(); +if (process.argv[1] && import.meta.url === `file://${process.argv[1]}`) void main(); diff --git a/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh b/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh index 8f38e99393..e359bbdf73 100644 --- a/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh +++ b/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh @@ -22,8 +22,13 @@ RULESET_SCRIPT=/tmp/loopover-egress-ruleset.sh node /app/packages/loopover-miner/lib/generate-egress-firewall-config.js "$DNSMASQ_CONF" "$RULESET_SCRIPT" # Listens on 127.0.0.1 only (per the generated config's own bind-interfaces + listen-address) -- unreachable -# from outside this container regardless of what else is on its network. -dnsmasq --conf-file="$DNSMASQ_CONF" --pid-file=/var/run/dnsmasq.pid +# from outside this container regardless of what else is on its network. --user/--group are explicit rather +# than relying on dnsmasq's own default privilege drop (confirmed empirically: it already drops to nobody:65534 +# unprompted on this base image) -- explicit beats implicit for a root-invoked daemon, regardless of what the +# current default happens to be. Deliberately `nobody`, not the `node` user the miner itself runs as: dnsmasq +# needs none of node's file access, and running it as the SAME user as the application would only widen what a +# dnsmasq-specific compromise could reach. +dnsmasq --conf-file="$DNSMASQ_CONF" --pid-file=/var/run/dnsmasq.pid --user=nobody --group=nogroup # Docker writes its own /etc/resolv.conf pointing at its embedded DNS server -- nothing routes lookups through # dnsmasq until this container's OWN resolver config says to. Docker's own generated file says "this file can diff --git a/test/unit/miner-generate-egress-firewall-config.test.ts b/test/unit/miner-generate-egress-firewall-config.test.ts index 20c455e25b..d45a552595 100644 --- a/test/unit/miner-generate-egress-firewall-config.test.ts +++ b/test/unit/miner-generate-egress-firewall-config.test.ts @@ -6,8 +6,8 @@ import { mkdtempSync, readFileSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { writeFileSync } from "node:fs"; -import { afterEach, describe, expect, it } from "vitest"; -import { generateEgressFirewallConfig } from "../../packages/loopover-miner/lib/generate-egress-firewall-config"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { generateEgressFirewallConfig, main } from "../../packages/loopover-miner/lib/generate-egress-firewall-config"; const roots: string[] = []; @@ -101,3 +101,97 @@ describe("generateEgressFirewallConfig (#7857)", () => { ).resolves.toBeDefined(); }); }); + +// #7857: main() is the actual CLI entry egress-firewall-entrypoint.sh invokes -- exercised directly here via +// injectable IO (mirroring scripts/check-miner-deployment-docs.ts's own main(env, io) pattern in this +// codebase), not through a subprocess, so it's real, in-process v8 coverage rather than untestable CLI glue. +function fakeIo(argv: string[]) { + return { + io: { + argv, + log: vi.fn((..._args: unknown[]) => undefined), + error: vi.fn((..._args: unknown[]) => undefined), + exit: vi.fn((_code: number) => undefined), + }, + }; +} + +describe("main (#7857)", () => { + it("errors and exits 1 without attempting generation when either path arg is missing", async () => { + const { io } = fakeIo(["node", "generate-egress-firewall-config.js"]); + + await main(io); + + expect(io.error).toHaveBeenCalledExactlyOnceWith(expect.stringContaining("egress_firewall_config_missing_args")); + expect(io.exit).toHaveBeenCalledExactlyOnceWith(1); + expect(io.log).not.toHaveBeenCalled(); + }); + + it("errors and exits 1 when only one of the two path args is given", async () => { + const { io } = fakeIo(["node", "generate-egress-firewall-config.js", "/tmp/dnsmasq.conf"]); + + await main(io); + + expect(io.exit).toHaveBeenCalledExactlyOnceWith(1); + }); + + it("logs the real generated-config event and never exits on success", async () => { + const configDir = tempRoot(); + const outDir = tempRoot(); + const dnsmasqPath = join(outDir, "dnsmasq.conf"); + const rulesetPath = join(outDir, "ruleset.sh"); + const originalConfigDir = process.env.LOOPOVER_MINER_CONFIG_DIR; + process.env.LOOPOVER_MINER_CONFIG_DIR = configDir; + const { io } = fakeIo(["node", "generate-egress-firewall-config.js", dnsmasqPath, rulesetPath]); + + try { + await main(io); + } finally { + if (originalConfigDir === undefined) delete process.env.LOOPOVER_MINER_CONFIG_DIR; + else process.env.LOOPOVER_MINER_CONFIG_DIR = originalConfigDir; + } + + expect(io.exit).not.toHaveBeenCalled(); + expect(io.error).not.toHaveBeenCalled(); + expect(io.log).toHaveBeenCalledExactlyOnceWith(expect.stringContaining("egress_firewall_config_generated")); + expect(readFileSync(dnsmasqPath, "utf8")).toContain("ipset=/github.com/loopover_egress_allow"); + }); + + it("errors and exits 1 (without throwing) when the underlying generation call itself fails", async () => { + // An output path under a directory that doesn't exist -- writeFileSync inside generateEgressFirewallConfig + // throws ENOENT, exercising main()'s own catch branch. + const { io } = fakeIo(["node", "generate-egress-firewall-config.js", "/nonexistent-dir-7857/dnsmasq.conf", "/nonexistent-dir-7857/ruleset.sh"]); + + await main(io); + + expect(io.error).toHaveBeenCalledExactlyOnceWith(expect.stringContaining("egress_firewall_config_generation_failed")); + expect(io.exit).toHaveBeenCalledExactlyOnceWith(1); + }); + + it("constructs its default IO from real process.argv/console/process.exit when no override is passed", async () => { + const originalArgv = process.argv; + const originalExit = process.exit; + const originalError = console.error; + const calls: { exit: number[]; error: string[] } = { exit: [], error: [] }; + process.exit = ((code?: number) => { + calls.exit.push(code ?? 0); + return undefined as never; + }) as typeof process.exit; + console.error = (message?: unknown) => { + calls.error.push(String(message)); + }; + process.argv = ["node", "generate-egress-firewall-config.js"]; // missing both path args -- the cheapest real path to exercise + + try { + await main(); + } finally { + process.argv = originalArgv; + process.exit = originalExit; + console.error = originalError; + } + + expect(calls.error).toHaveLength(1); + expect(calls.error[0]).toContain("egress_firewall_config_missing_args"); + expect(calls.exit).toEqual([1]); + }); +}); From f12ac296093857f0828cbaf95b88e07e7776b982 Mon Sep 17 00:00:00 2001 From: JSONbored <49853598+JSONbored@users.noreply.github.com> Date: Thu, 23 Jul 2026 17:35:20 -0700 Subject: [PATCH 3/4] fix(miner): close the last 2 patch-coverage gaps in generate-egress-firewall-config.ts Both are the same class of gap bin/loopover-miner.ts's own dispatcher tail already carries a v8-ignore for (see the packages/loopover-miner/bin note in vitest.config.ts's coverage.include): code only reachable by actually running the file as a script, not by importing it. main()'s own body is fully unit-covered in-process via the injectable-IO tests added earlier; only the self-invocation guard's true branch and one defensive non-Error fallback (unreachable given this call site's real, always-Error-throwing dependencies) remain, both marked consistent with the established convention. Confirmed locally: 100% line+branch coverage in isolation, full test suite green, real docker build + verify-egress-firewall.sh (7/7) still passing. --- .../loopover-miner/lib/generate-egress-firewall-config.ts | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/packages/loopover-miner/lib/generate-egress-firewall-config.ts b/packages/loopover-miner/lib/generate-egress-firewall-config.ts index 54a78b5644..faab74eeb8 100644 --- a/packages/loopover-miner/lib/generate-egress-firewall-config.ts +++ b/packages/loopover-miner/lib/generate-egress-firewall-config.ts @@ -59,9 +59,17 @@ export async function main( const { allowedHostCount, disabled } = await generateEgressFirewallConfig(dnsmasqConfigPath, rulesetScriptPath); io.log(JSON.stringify({ event: "egress_firewall_config_generated", allowedHostCount, disabled, dnsmasqConfigPath, rulesetScriptPath })); } catch (error) { + /* v8 ignore next -- this call site's only real error sources (fs writes, the tolerant-by-contract + * resolveAmsPolicy) always throw real Error instances; the non-Error side of this ternary is defensive + * against a future dependency change, not reachable through any input this function's own tests can drive. */ io.error(JSON.stringify({ event: "egress_firewall_config_generation_failed", message: error instanceof Error ? error.message : String(error) })); io.exit(1); } } +/* v8 ignore next -- subprocess-only executed (same convention as bin/loopover-miner.ts's own dispatcher tail, + * see the packages/loopover-miner/bin note in vitest.config.ts's coverage.include): main()'s own body is fully + * unit-covered in-process above via injectable IO; only this self-invocation guard's true branch requires + * actually running the file as `node generate-egress-firewall-config.js`, which egress-firewall-entrypoint.sh + * does in production and packages/loopover-miner/scripts/verify-egress-firewall.sh proves end to end. */ if (process.argv[1] && import.meta.url === `file://${process.argv[1]}`) void main(); From 392636e3daa933c7a69de09694db7dff5bb4512e Mon Sep 17 00:00:00 2001 From: JSONbored <49853598+JSONbored@users.noreply.github.com> Date: Thu, 23 Jul 2026 17:59:08 -0700 Subject: [PATCH 4/4] fix(miner): harden the full root-executed import graph, not just its entrypoints Owning only the entrypoint script and generate-egress-firewall-config.{js,ts} left every module they import (ams-policy.js, egress-*.js, @loopover/engine, and all of node_modules) node:node-owned. A compromised node-level process -- the exact threat this mechanism sandboxes against -- could tamper with any of those, and root would load and execute the tampered code on the container's next restart. COPY --chown=root:root across the whole code tree closes that persistence vector while staying world-readable (COPY --chown only changes ownership, not mode bits), and the miner's real runtime writes all resolve under LOOPOVER_MINER_CONFIG_DIR or a target-repo checkout, never /app -- verified empirically that node can no longer write ams-policy.js but can still read and require() it, and that the full verify-egress-firewall.sh suite still passes end to end. --- packages/loopover-miner/Dockerfile | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/packages/loopover-miner/Dockerfile b/packages/loopover-miner/Dockerfile index 8f751e52cf..81b6b3ad55 100644 --- a/packages/loopover-miner/Dockerfile +++ b/packages/loopover-miner/Dockerfile @@ -30,18 +30,22 @@ ENV NODE_ENV=production \ # see egress-firewall-entrypoint.sh for the full mechanism and why this image now starts as root. RUN apt-get update && apt-get install -y --no-install-recommends dnsmasq iptables ipset gosu \ && rm -rf /var/lib/apt/lists/* -COPY --from=build --chown=node:node /app/node_modules ./node_modules -COPY --from=build --chown=node:node /app/packages/loopover-miner ./packages/loopover-miner -COPY --from=build --chown=node:node /app/packages/loopover-engine ./packages/loopover-engine +# #7857 security review (round 2): root:root on the FULL code tree, not just the two files that directly +# EXECUTE with root privilege. Owning only the entrypoint + generate-egress-firewall-config.{js,ts} left every +# module they import (ams-policy.js, egress-*.js, @loopover/engine, ...) node:node-owned -- a compromised +# node-level process (the coding-agent subprocess this whole mechanism sandboxes) could tamper with any of +# those, and root would load + execute the tampered code on the container's next restart. `--chown` on COPY +# only changes ownership, not mode bits, so this stays world-readable (standard 644/755 from the build stage) +# -- just no longer node-writable. The miner's own real runtime writes (config/.env, ledgers, the coding +# agent's working directory) all resolve under LOOPOVER_MINER_CONFIG_DIR (/data/miner, below) or a target-repo +# checkout, never /app, so this closes the persistence vector without touching anything node actually needs +# to write. +COPY --from=build --chown=root:root /app/node_modules ./node_modules +COPY --from=build --chown=root:root /app/packages/loopover-miner ./packages/loopover-miner +COPY --from=build --chown=root:root /app/packages/loopover-engine ./packages/loopover-engine RUN mkdir -p /data/miner && chown -R node:node /data \ - # #7857 security review: the two files that actually EXECUTE with root privilege (the entrypoint itself, - # and the config-generation script it invokes as root) must not be writable by node -- a compromised - # node-level process (the coding-agent subprocess this whole mechanism sandboxes) could otherwise rewrite - # either one, persisting a root-privileged backdoor across this container's next restart. Scoped to these - # two specific files, not their full transitive dependency graph (ams-policy.js/egress-*.js and everything - # under node_modules/@loopover/engine they in turn import) -- hardening the entire graph would be a much - # larger, more invasive change; this closes the direct, obvious persistence vector without one. - && chown root:root /app/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh /app/packages/loopover-miner/lib/generate-egress-firewall-config.js /app/packages/loopover-miner/lib/generate-egress-firewall-config.ts \ + # Not executable in git (checked in as a normal 644 file) -- COPY --chown above only sets ownership, so + # the exec bit still needs setting explicitly, same as before this file became root:root by default. && chmod 755 /app/packages/loopover-miner/scripts/egress-firewall-entrypoint.sh VOLUME ["/data/miner"] # No HEALTHCHECK: the miner is a batch/CLI workload (`docker run … loopover-miner `), not a