diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a55c2d38..13f30dcc 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -247,6 +247,17 @@ jobs: env: PS_REQUIRE_CANARY: '1' + # After the build, because it drives the built CLI against a project that links the built package: + # the listener reporter is COPIED into `dist/` rather than bundled, so a rename or a missed copy + # step is invisible to every source test and shows up only here. + # + # `PS_REQUIRE_RUNTIME_CHECK` turns a skipped run into a failure, for the same reason as the canary: + # a verification check that skips reads exactly like one that passed. + - name: The runtime traversal check works through the built CLI + run: npx vitest run tests/protect/runtime-check-built.test.ts + env: + PS_REQUIRE_RUNTIME_CHECK: '1' + - name: Verify package contents run: npm pack --dry-run diff --git a/AGENT-INSTALL.md b/AGENT-INSTALL.md index c3fafa0e..d7aaa469 100644 --- a/AGENT-INSTALL.md +++ b/AGENT-INSTALL.md @@ -140,6 +140,46 @@ It is server-only. Never put it in the widget tag, client bundles, or public env `setup` performs both steps automatically. The explicit commands are for manual setup or repair. If verification reports a generic or existing framework seam, complete the printed source edit and re-run `--check`; do not report protection as active until it exits successfully. + `--check` reads the app's source. It can establish that the guard is imported and called on a request + path; it cannot establish that a request ever reaches it — an app can wire the guard onto one server + and serve traffic from another, and that passes. To settle the difference there is an opt-in check + that **starts the application**: + + ``` + npx @patchstack/connect protect --check --runtime + ``` + + It launches the project's entry with `node`, moves the HTTP listeners **that process** opens to an + ephemeral loopback port, sends one request per listener carrying a per-run challenge, and reports + whether the scaffolded guard seam answered it. Exit `0` runtime traversal reached the seam, `1` a + listener answered and the seam did not, `2` it could not be established — neither a pass nor a + failure, with the structural checks still standing on their own. + + Exit `2` is the answer for everything this cannot speak for, and the reason is always printed. The + common one is an entry that needs the project's own toolchain (a TypeScript entry, a framework + launcher, a watcher, another runtime, anything reached through a package manager), which this never + installs, builds or invents. The others are about scope: **the run answers for one process, one + thread, and one discovery window.** If the app attempts to start another process, the launch is + refused and the answer is `2`. A child can daemonize after it starts without declaring that in its + launch options, so allowing it would make the end-of-run process-group cleanup a claim the verifier + cannot establish. The app sees `EPERM`. A worker thread is also `2`: it inherits the listener + handling, but it cannot report back, so its listeners can be neither counted nor asked. So is a + listener that bound an address other than loopback, one that cannot be probed, and anything the app + opens after the discovery window has closed — the app is asked to stop and its acknowledgement is + what closes that window, so a run that never gets one is `2` as well. An inherited `NODE_OPTIONS` + that would run code before the listener handling is in place — a `--require` or `--import` in your + environment — is `2` too, and is refused before the app is launched rather than after. + + A worker handed a replacement environment that does not preserve the propagated `NODE_OPTIONS` is + refused outright, because it would not load the listener handling. The app sees `EPERM`, and the run + reports `2`. + + What a pass says is exactly: **runtime traversal reached the scaffolded guard seam.** It does not say + rules were delivered, that the deployed app is wired, or that ordinary traffic is blocked. + + Nothing else runs the application. `protect`, `protect --check`, `setup`, `guide`, `scan`, `status` + and `mark-build` only read and write files. + 5. **Commit** `.patchstackrc.json`, the updated `package.json`, the guard/framework source changes, and the layout/HTML file carrying the widget tag (and the production marker, when `scan` wrote one into a JSX root), so every developer and CI run reports to the same site. **Do not commit `.patchstackrc.local.json`.** That file holds the API key issued at provision; the scan writes it and adds it to `.gitignore`, and tells you if it could not. `.patchstackrc.json` holds only the site UUID and settings, and the UUID is public by design — it ships in the widget tag in served HTML. @@ -388,7 +428,8 @@ Two more endpoints the package can call, for completeness: ## Verifying the install - `npx @patchstack/connect status` re-prints the site UUID and dashboard URL, and checks whether the site still exists on Patchstack (`Site status: active / removed / could not be verified`). -- `npx @patchstack/connect protect --check` verifies the runtime guard is connected to the request path. +- `npx @patchstack/connect protect --check` verifies from the source that the runtime guard is connected to the request path. It does not run the app. +- `npx @patchstack/connect protect --check --runtime` additionally **starts the app** on a loopback port and sends it one request, to establish that a request reaches the guard seam. Opt-in, and the only command that runs the application; exit `0`/`1`/`2` as described in step 4. - Load the site in a browser — the "Report a vulnerability" button should appear. Refresh a page that was already open before the tag was added: the button only loads with the page. - On the deployed site, the button appears only after a deploy that includes these source changes. diff --git a/CLAUDE.md b/CLAUDE.md index f7d9b029..ebaa632b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -61,10 +61,12 @@ The one-line install prompt in `README.md` ("Install prompt (for AI coding tools Invariants when touching it: - The prompt appears in three places that must stay identical: `README.md`, `GETTING-STARTED.md` (the teammate-facing flow), and `field-test/prompt.txt` — `prompt.txt` is the artifact the harness tests. -- Any change to the prompt, the `guide` checklist output (`src/guide.ts`), or `AGENT-INSTALL.md` must pass `node field-test/run.mjs --persona hostile --rounds 3` before shipping. Agents audit the shipped docs, so inaccuracies in `AGENT-INSTALL.md` cost trust and cause refusals. **A round that never unpacked the tarball is VOID, not a pass and not a failure**: the shipped docs were never on disk, so no audit of them can have happened, and its scorecard is identical to a doc regression's. Unpacked means a non-empty `node_modules/@patchstack/connect/AGENT-INSTALL.md` — a dependency declaration in `package.json` is not an install. The harness retries void rounds within a bounded budget and exits 2 when every round was void — read that as "re-run", never as "the docs are fine". A doc change also wants a re-run after publication, when the published tarball actually carries it. +- **When the hostile run is the gate depends on what changed, because the fixture installs the PUBLISHED tarball.** A change to the prompt itself must pass `node field-test/run.mjs --persona hostile --rounds 3` *before shipping*: the prompt the harness sends comes from `field-test/prompt.txt` in this checkout, so a pre-publication run tests the real artifact. A change to `AGENT-INSTALL.md` or the `guide` checklist output (`src/guide.ts`) cannot be gated that way — the docs the agent audits are unpacked from the registry, so a pre-publication run audits the *previous* text and a green result says nothing about the change. Those changes require the same hostile run *immediately after the release that carries them*, and until then the local deterministic gates (the disclosure tests, `capabilities:check`) are what stand behind them. Ship such a change with the outstanding run named explicitly rather than reporting the gate as met. This split exists only because the harness has no local-registry mode; give it one and both become pre-publication gates. +- Agents audit the shipped docs, so inaccuracies in `AGENT-INSTALL.md` cost trust and cause refusals. **A round that never unpacked the tarball is VOID, not a pass and not a failure**: the shipped docs were never on disk, so no audit of them can have happened, and its scorecard is identical to a doc regression's. Unpacked means a non-empty `node_modules/@patchstack/connect/AGENT-INSTALL.md` — a dependency declaration in `package.json` is not an install. The harness retries void rounds within a bounded budget and exits 2 when every round was void — read that as "re-run", never as "the docs are fine". +- `hostile` measures whether the PROMPT survives pressure. It is a poor instrument for doc accuracy — void rounds are its modal outcome — so a docs-only verification wants a persona that reliably installs (`standard`, or `lovable`). Neither persona escapes the published-tarball problem above. - Don't add reassurance language ("it's safe", "nothing is executed remotely") — agents flag it as a manipulation signal. Don't ask the agent to "follow the guide/instructions it prints" unbounded — name the concrete steps instead. - A new real-world refusal report becomes a persona in `field-test/personas/`, written in your own words as a synthetic reconstruction, so the regression stays covered without publishing someone else's material. -- The fixture installs the *published* package, so an unpublished `guide`/CLI change can't be exercised end-to-end — publish first, or accept the run validates only the prompt shape. +- The fixture installs the *published* package, so an unpublished `guide`/CLI change can't be exercised end-to-end — publish first, or accept the run validates only the prompt shape. The free stubs (`field-test/README.md`) exercise the harness's own paths without an agent, and are worth running when the real gate is deferred: they show the harness is sound, and nothing about the docs. ## Comments and Explanations diff --git a/MAINTAINING.md b/MAINTAINING.md index 1d36eeac..8b93d588 100644 --- a/MAINTAINING.md +++ b/MAINTAINING.md @@ -22,9 +22,9 @@ The deep "why" — the AI-agent refusal modes each clause guards against — liv The prompt deliberately contains no model-authored verification step. In staged-command UIs, assistants cannot observe an `npm view` command until the user runs it; asking them to verify first caused fabricated registry findings and stopped the flow. Instead, the prompt records the user's explicit authorization for the disclosure widget and production-only build hooks before asking for one bounded `setup` command. Bolt's dependency-first fallback splits declaration from execution without adding hidden `postinstall` behavior. Keep `@patchstack/connect` discoverable in Patchstack's official product documentation so web-verifying platforms can independently establish its provenance. -## Before shipping a 🔴 or 🟠 change +## Verification timing for a 🔴 or 🟠 change -Run the gate: +For a prompt change, run the gate before shipping: ```bash node field-test/run.mjs --persona hostile --rounds 3 @@ -32,9 +32,17 @@ node field-test/run.mjs --persona hostile --rounds 3 It runs a real AI agent through the full install in a throwaway fixture against a mocked API and scores the outcome. Read each round's `report.md` (the HESITATIONS / DECISION ANALYSIS sections) — anything an agent pauses on is a future refusal. See [`field-test/README.md`](field-test/README.md) for the improve-and-retest loop, the safety model, and what to do when the gate is red for environmental reasons (stale published docs, release freshness). +For an `AGENT-INSTALL.md` or `src/guide.ts` change, the fixture installs the published tarball and +therefore cannot exercise unpublished text. Run the deterministic disclosure and capability checks +before merge, ship with the hostile field test recorded as outstanding, and run it immediately after the +release carries the change. That remains a prompt-survival gate. Add `standard` or `lovable` when the +question is document accuracy; those personas install more reliably than `hostile`. + What it establishes is **structural regression coverage**: the prompt still survives pressures that once broke it. The personas are synthetic — written from our own analysis, not from any platform's policy text — so a green run is not evidence that a live platform accepts the prompt, and should not be reported as though it were. -Caveat: the fixture installs the **published** package, so an unpublished `guide`/CLI change can't be exercised end-to-end — publish first, or accept that the run validates only the prompt shape. +This split is a limitation of the current harness. A local-registry mode would let every artifact be +tested before publication; until one exists, do not describe a run against the previous tarball as a +gate on unpublished docs. ## Don'ts (these are refusal triggers, not style nits) diff --git a/README.md b/README.md index 02261a71..9c9b54d3 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,13 @@ patchstack-connect protect Install/reconcile the always- guard. Auto-wires supported server stacks; use --check to verify or --demo for local rules. Also run by setup; never run by scan/guide/mark-build. + --check reads your source and never runs the app. + --check --runtime STARTS THE APP on a loopback + port and sends it one request, to establish that + a request reaches the guard seam. Opt-in, and the + only mode that runs the app. Exit 0 traversed, + 1 a listener answered instead of the guard, + 2 could not be established (see below). patchstack-connect map [--dir p] [--out f] [--upload] Print a JSON map of this project's attack surface: server entry points, the inputs each @@ -103,6 +110,69 @@ Options (for demo and demo-guide): (default: http://localhost:3000/api/tasks) ``` +### Verifying the guard at runtime (opt-in) + +`protect --check` reads the app's source. That establishes the guard is imported and called on a +request path — not that a request ever reaches it. An app can wire the guard onto one server and serve +its traffic from another, and the structural check passes. + +`protect --check --runtime` settles that one question by **starting the application**: + +``` +npx @patchstack/connect protect --check --runtime +``` + +It runs the project's entry with `node`, moves the HTTP listeners **that process** opens to an ephemeral +loopback port (so a port already in use is not a failure), sends one request per listener carrying a +challenge generated for that run, and reports whether the scaffolded guard seam answered it. The child +is started in its own process group and killed with it — including if you interrupt the command. A +listener it cannot probe — a Unix socket, a file descriptor, a handed-over handle, an HTTP/2 server — is +reported and prevented from binding at all, rather than opened on the verifier's behalf. + +**One process is the scope — one thread of it, and one discovery window.** If the app attempts to start +another process, the launch is refused and the answer is `2`, whatever that process is. A child can +daemonize after it starts without declaring that in its launch options, so allowing it would make the +end-of-run process-group cleanup a claim the verifier cannot establish. The app sees the launch fail +with `EPERM`. A **worker thread** is also `2`: it inherits the listener handling, but a worker has no +channel back, so its listeners can be neither counted nor asked. A worker handed a replacement +environment that does not preserve the propagated `NODE_OPTIONS` is refused, because it would not load +that handling at all. + +Everything else the run finds also ends it this way: a listener that bound an address other than +loopback, a listener it cannot probe, and anything the app opens **after the discovery window closes** — +a second listener appearing while the first is still being asked cannot join a set that is already being +answered from. Closing that window is a handshake: the app is asked to stop opening listeners and its +acknowledgement is what proves nothing is still in flight, so a run that never gets one reports `2` +rather than passing. An inherited `NODE_OPTIONS` is checked before anything is launched, too: Node reads +that variable ahead of the command line, so a `--require` or `--import` sitting in your environment would +run before the listener handling was in place, and the run reports `2` rather than starting the app with +less containment than it claims. Recognised flags are passed through, with the reporter first. + +A pass says exactly this: **runtime traversal reached the scaffolded guard seam.** It does not say +rules were delivered, that the deployed app is wired, or that ordinary traffic is blocked. The +challenge is generated per run, so a fixed response or a reflected header cannot answer it — but the +challenge does reach the whole app process, so this establishes traversal in a cooperating app rather +than against an app written to answer for itself. + +| Exit | Meaning | +|---|---| +| `0` | A request reached the scaffolded guard seam. | +| `1` | A listener answered and the seam did not — or the structural checks failed, in which case the app is not started at all. | +| `2` | It could not be established. Neither a pass nor a failure; the structural checks still stand. | + +Exit `2` is the common answer for entries this deliberately will not start. It runs `node ` on a +file the project already has, and nothing else — no package-manager scripts, no `node_modules/.bin`, no +build, no install. So a TypeScript entry, a framework launcher (`next start`), a watcher (`nodemon`), +another runtime (`bun`), or a script wrapped in an environment shim all report unavailable with the +reason printed. To make such a project verifiable, point a `start` script at a built, directly loadable +file — the check reports which entry it used and where it came from. + +Windows reports exit `2` without starting anything: the cleanup this relies on is a POSIX process +group, and a verification that can leave a server running is worse than an unanswered question. + +No other command runs your application. `protect`, `protect --check`, `setup`, `guide`, `scan`, +`status` and `mark-build` only read and write files. + ## Configuration Precedence (highest wins): diff --git a/scripts/copy-protect-templates.mjs b/scripts/copy-protect-templates.mjs index adc5c7ca..cc7714e2 100644 --- a/scripts/copy-protect-templates.mjs +++ b/scripts/copy-protect-templates.mjs @@ -18,3 +18,10 @@ copyFileSync('src/protect/protect.d.ts', 'dist/protect.d.ts'); // so there is nothing per-format to express. copyFileSync('src/protect/protect.d.ts', 'dist/protect.d.cts'); console.log('copied protect types -> dist/protect.d.ts, dist/protect.d.cts'); + +// The listener reporter `protect --check --runtime` preloads into the app it starts. Copied rather than +// bundled: it is loaded by path into ANOTHER process, as CommonJS, so it has to exist as a file that +// `node --require` can take. The built-CLI runtime test asserts the CLI can find it after a build. +mkdirSync('dist/protect/runtime', { recursive: true }); +copyFileSync('src/protect/install/runtime/report-listeners.cjs', 'dist/protect/runtime/report-listeners.cjs'); +console.log('copied listener reporter -> dist/protect/runtime/report-listeners.cjs'); diff --git a/src/cli.ts b/src/cli.ts index 9a44a63c..705aaade 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -54,6 +54,7 @@ import { } from './guide.js'; import { login, readPendingLogin, redeemIfApproved, startLogin, waitForApproval } from './login.js'; import { runProtect, runVerify } from './protect/install/index.js'; +import { formatRuntimeCheck, runRuntimeCheck, runtimeExitCode } from './protect/install/runtime/check.js'; import { runMap } from './map-command.js'; import { getStringFlag } from './flags.js'; import { setupProtection, wireBuildScripts } from './setup.js'; @@ -112,6 +113,15 @@ Usage: demonstrations, not production). --check verifies the guard is wired (exit 1 if not) — for the wire-then-verify loop. + It reads the app's source and never runs it. + --check --runtime additionally STARTS THE APP + on a loopback port and sends it one request, + to establish that a request reaches the + guard. Opt-in, and the only mode that runs + the app. Exit 0 traversed, 1 a listener + answered instead of the guard, 2 could not + be established (e.g. a TypeScript or + framework entry this must not start). patchstack-connect demo node-serialize [--url URL] Run the production-backed node-serialize walkthrough: verify the vulnerable package, scan it, wait for live rule 18843, install + @@ -627,6 +637,14 @@ function reportSourceMarker(framework: string | null): void { } async function runProtectCommand(args: ParsedArgs): Promise { + const runtime = args.flags.get('runtime') === true; + // A stray `--runtime` would otherwise scaffold quietly while the caller believed their app had been + // started and probed — a false green about the one check that exists to prevent false greens. + if (runtime && args.flags.get('check') !== true) { + console.error('patchstack protect: --runtime only applies to --check. Run `protect --check --runtime`.'); + return 1; + } + // `--check`: verify the guard is wired (for the agent/CI loop). Non-zero exit if not. if (args.flags.get('check') === true) { const report = runVerify(process.cwd()); @@ -653,7 +671,16 @@ async function runProtectCommand(args: ParsedArgs): Promise { if (report.checks.some((c) => c.unverifiable)) { console.log('One or more checks could not be answered from here — see the `?` lines above.'); } - return report.wired ? 0 : 1; + // The structural verdict comes first either way: an app whose guard is not wired has nothing to + // gain from being started, and its runtime result would only be a second way of saying the same no. + if (!report.wired) return 1; + if (!runtime) return 0; + + console.log(''); + const runtimeReport = await runRuntimeCheck(process.cwd()); + for (const line of formatRuntimeCheck(runtimeReport)) console.log(line); + + return runtimeExitCode(runtimeReport); } // Best-effort: like mark-build, this runs during builds and must never fail one. const demo = args.flags.get('demo') === true; diff --git a/src/guide.ts b/src/guide.ts index 25c5effc..1a99cdbb 100644 --- a/src/guide.ts +++ b/src/guide.ts @@ -586,6 +586,8 @@ export function renderGuideChecklist(state: GuideState, useColor: boolean): stri lines.push(detail(`${check.label}${check.hint ? ` — ${check.hint}` : ''}`)); } lines.push(detail('Verify → npx @patchstack/connect protect --check')); + // Named here rather than run: `guide` is read-only and must never start the application. + lines.push(detail('Prove a request reaches the guard (starts your app) → npx @patchstack/connect protect --check --runtime')); } // 7. Dashboard access — always keep the URL prominent. diff --git a/src/protect/install/node-flags.ts b/src/protect/install/node-flags.ts new file mode 100644 index 00000000..886a366c --- /dev/null +++ b/src/protect/install/node-flags.ts @@ -0,0 +1,174 @@ +// The Node flags this package is willing to understand, in one place. +// +// Three callers need the same inventory and would otherwise each keep their own. The runtime check's +// entry resolver reads flags out of a project's start script; the probe decides whether an inherited +// `NODE_OPTIONS` can be carried into the child it launches; the structural parse retains only known-safe +// options in the environment of a `node --check`. A flag classified generously in one of those places +// and strictly in another is a hole, so the classification lives here. +// +// The list is an ALLOWLIST, and deliberately short. An unrecognised flag is refused rather than passed +// on, because the two things that make a flag safe to carry — it does not run code, and its meaning does +// not depend on the token after it — cannot be decided by looking at a name nobody wrote down. + +/** `--name=value` and `--name` both name the same flag. */ +export const flagName = (token: string): string => token.split('=')[0]!; + +/** + * Flags that take no operand, so the token after one is not its value. + * + * This is what makes them safe for a reader that splits on whitespace: nothing about the meaning of the + * next token depends on them. + */ +export const NO_OPERAND_FLAGS = new Set([ + '--disallow-code-generation-from-strings', + '--enable-source-maps', + '--experimental-import-meta-resolve', + '--experimental-json-modules', + '--experimental-vm-modules', + '--frozen-intrinsics', + '--no-deprecation', + '--no-warnings', + '--pending-deprecation', + '--preserve-symlinks', + '--preserve-symlinks-main', + '--throw-deprecation', + '--trace-deprecation', + '--trace-exit', + '--trace-uncaught', + '--trace-warnings', + '--use-strict', + '--zero-fill-buffers', +]); + +/** + * Flags accepted only in their self-contained `--name=value` form. + * + * Each one sets a limit or a preference. None of them loads a module, evaluates a string, opens a port, + * or changes which file Node treats as the program — which is the whole test for being on this list. + * `--env-file` is deliberately absent: a file it reads can set `NODE_OPTIONS` itself. + */ +export const VALUED_FLAGS = new Set([ + '--dns-result-order', + '--max-http-header-size', + '--max-old-space-size', + '--max-semi-space-size', + '--stack-size', + '--title', + '--unhandled-rejections', +]); + +/** + * Flags refused in every form, including `--name=value`. + * + * The first group runs code — a module preloaded, a loader installed, a string evaluated — which for the + * runtime check means code running before the listener reporter is in place, the one window in which an + * app could open a listener nothing screened. `--eval` and `--print` also take the program itself out of + * the file named on the command line, so a resolver that passed one on would report on a file that never + * ran. The second group opens a debugger port, which is a listener of its own that a verification has no + * business opening on someone's machine. + */ +export const CODE_LOADING_FLAGS = new Set([ + '--require', + '-r', + '--import', + '--loader', + '--experimental-loader', + '--eval', + '-e', + '--print', + '-p', + '--inspect', + '--inspect-brk', + '--inspect-wait', + '--inspect-port', + '--debug', + '--debug-brk', +]); + +/** Of those, the ones whose value is the NEXT token, so dropping the flag has to drop its operand too. */ +const TAKES_NEXT_TOKEN = new Set(['--require', '-r', '--import', '--loader', '--experimental-loader', '--eval', '-e', '--print', '-p', '--inspect-port']); + +/** + * Split a `NODE_OPTIONS` value the way Node's own parser does: on whitespace, honouring double quotes. + * + * Returns null for an unbalanced quote. Node would split such a value somehow, but not necessarily the + * way this reads it, and a classification of tokens that are not the tokens Node will see is worth + * nothing. + */ +export function tokenizeNodeOptions(value: string): string[] | null { + const tokens: string[] = []; + let current = ''; + let quoted = false; + let started = false; + + for (const ch of value) { + if (ch === '"') { + quoted = !quoted; + started = true; + continue; + } + if (!quoted && /\s/.test(ch)) { + if (started) tokens.push(current); + current = ''; + started = false; + continue; + } + current += ch; + started = true; + } + if (quoted) return null; + if (started) tokens.push(current); + + return tokens; +} + +export type NodeOptionsVerdict = { kind: 'safe' } | { kind: 'refused'; why: string }; + +/** Whether every flag in a `NODE_OPTIONS` value is one of the two allowlists above. */ +export function classifyNodeOptions(value: string): NodeOptionsVerdict { + const tokens = tokenizeNodeOptions(value); + if (tokens === null) return { kind: 'refused', why: 'NODE_OPTIONS has an unbalanced double quote, so this cannot read it the way Node will' }; + + for (const token of tokens) { + const name = flagName(token); + if (CODE_LOADING_FLAGS.has(name)) { + return { kind: 'refused', why: `NODE_OPTIONS carries ${name}, which runs code or opens a port before the listener reporter is in place` }; + } + if (NO_OPERAND_FLAGS.has(token)) continue; + if (token.includes('=') && VALUED_FLAGS.has(name)) continue; + + return { kind: 'refused', why: `NODE_OPTIONS carries ${name}, which this does not recognise well enough to say what it does` }; + } + + return { kind: 'safe' }; +} + +/** + * The known-safe part of the same value, for a child that must not evaluate anything. + * + * Unlike the classification above this keeps going rather than refusing: the caller is a structural + * parse whose answer is useful even when the environment carries something odd. Only the two explicit + * allowlists survive. That also drops compact or future code-loading forms this package does not know by + * name; keeping an unknown flag would make the claim that the child only parses depend on what a newer + * Node assigns that flag to mean. A value that cannot be tokenised is dropped whole. + */ +export function withoutCodeLoading(value: string | undefined): string { + if (value === undefined || value.trim() === '') return ''; + const tokens = tokenizeNodeOptions(value); + if (tokens === null) return ''; + + const kept: string[] = []; + for (let i = 0; i < tokens.length; i++) { + const token = tokens[i]!; + const name = flagName(token); + if (CODE_LOADING_FLAGS.has(name)) { + if (!token.includes('=') && TAKES_NEXT_TOKEN.has(name)) i++; // its value is the next token + continue; + } + if (NO_OPERAND_FLAGS.has(token) || (token.includes('=') && VALUED_FLAGS.has(name))) kept.push(token); + } + + // Re-quoted on the way out, so a kept path with a space survives the round trip. A token cannot + // contain a double quote: the tokeniser consumes those. + return kept.map((token) => (/\s/.test(token) ? `"${token}"` : token)).join(' '); +} diff --git a/src/protect/install/runtime/check.ts b/src/protect/install/runtime/check.ts new file mode 100644 index 00000000..70c67bbc --- /dev/null +++ b/src/protect/install/runtime/check.ts @@ -0,0 +1,102 @@ +// `protect --check --runtime`: the opt-in half of the wiring check. +// +// The default `--check` reads the app's source and can say the guard is wired. It cannot say a request +// ever reached it. This adds that one fact, and it is opt-in because the only way to establish it is to +// start the application — which the default check, `setup` and `guide` must never do. +// +// Three answers, never two, and each one keeps its own exit code. `proven` means a request traversed the +// scaffolded seam; `not-traversed` means a listener answered and the seam did not; `unavailable` means +// this could not be established from here, which is neither a pass nor a failure. `unavailable` is the +// answer whenever the run cannot speak for the whole app — an entry it may not launch, a listener it +// cannot probe, an attempted process launch, a worker whose listeners cannot report back, or anything +// the app opened after discovery closed. + +import { relative } from 'node:path'; + +import { resolveEntry } from './entry.js'; +import { probeRuntimeTraversal, type ListenerResult } from './probe.js'; + +export interface RuntimeCheckReport { + outcome: 'proven' | 'not-traversed' | 'unavailable'; + /** Why the run was `unavailable`. */ + reason?: string; + /** The entry that was launched, repo-relative, and where it came from. Absent if nothing was. */ + entry?: { file: string; from: string }; + listeners: ListenerResult[]; + /** Bounded child output, kept for the failure lines. */ + output: string; +} + +export interface RuntimeCheckOptions { + timeoutMs?: number; + settleMs?: number; + platform?: string; +} + +export async function runRuntimeCheck(cwd: string, options: RuntimeCheckOptions = {}): Promise { + const entry = resolveEntry(cwd); + if (entry.kind === 'unavailable') return { outcome: 'unavailable', reason: entry.reason, listeners: [], output: '' }; + + const result = await probeRuntimeTraversal({ + cwd, + entry: entry.file, + nodeArgs: entry.nodeArgs, + appArgs: entry.appArgs, + timeoutMs: options.timeoutMs, + settleMs: options.settleMs, + platform: options.platform, + }); + + return { ...result, entry: { file: relative(cwd, entry.file).replace(/\\/g, '/'), from: entry.from } }; +} + +/** + * 0 proven, 1 observed and not traversed, 2 could not be established. + * + * `unavailable` is deliberately its own code rather than folded into either neighbour: a caller that + * treats it as success ships an unverified app believing it verified one, and a caller that treats it as + * failure fails apps whose entry this cannot start — which is most TypeScript projects. + */ +export function runtimeExitCode(report: RuntimeCheckReport): 0 | 1 | 2 { + if (report.outcome === 'proven') return 0; + + return report.outcome === 'not-traversed' ? 1 : 2; +} + +/** The lines the CLI prints. Kept here so what the command says is covered by tests. */ +export function formatRuntimeCheck(report: RuntimeCheckReport): string[] { + const lines = ['runtime traversal (--runtime):']; + if (report.entry) lines.push(` entry: ${report.entry.file} (from ${report.entry.from})`); + + for (const listener of report.listeners) { + const mark = listener.outcome === 'traversed' ? '✓' : listener.outcome === 'answered-without-sentinel' ? '✗' : '?'; + const note = + listener.outcome === 'traversed' + ? 'runtime traversal reached the scaffolded guard seam' + : listener.outcome === 'answered-without-sentinel' + ? 'the listener answered and the guard seam did not' + : (listener.detail ?? 'could not be reached'); + // The address that was actually bound, not the one this hoped for: a line that prints a fixed + // loopback address would read the same whatever the app did. + const authority = listener.host.includes(':') ? `[${listener.host}]` : listener.host; + lines.push(` ${mark} ${listener.scheme}://${authority}:${listener.port} — ${note}`); + } + + if (report.outcome === 'proven') { + lines.push('runtime traversal reached the scaffolded guard seam ✓'); + } else if (report.outcome === 'not-traversed') { + lines.push('runtime traversal did NOT reach the scaffolded guard seam ✗'); + } else { + // Neither verdict. The reason is always printed, because "could not be established" without a cause + // is the kind of line people learn to scroll past. + lines.push(`? runtime traversal could not be established — ${report.reason ?? 'no reason given'}`); + lines.push(' This is not a failure. The structural checks above still stand on their own.'); + } + + if (report.outcome !== 'proven' && report.output.trim() !== '') { + lines.push(' the app printed:'); + for (const line of report.output.trim().split('\n').slice(0, 20)) lines.push(` ${line}`); + } + + return lines; +} diff --git a/src/protect/install/runtime/entry.ts b/src/protect/install/runtime/entry.ts new file mode 100644 index 00000000..73911d5f --- /dev/null +++ b/src/protect/install/runtime/entry.ts @@ -0,0 +1,194 @@ +// What `protect --check --runtime` is allowed to launch. +// +// One rule decides everything here: the verifier runs `node ` on a file this project already has, +// and nothing else. It does not run package scripts through a package manager, does not run a binary out +// of `node_modules/.bin`, does not build, and does not install. A verification command that installs or +// builds is a verification command that changes the thing it is verifying — and on an unfamiliar repo it +// is arbitrary code execution dressed as a check. +// +// The cost of that rule is that TypeScript entries, framework dev servers and monorepo launchers come +// back unavailable rather than verified. That is the honest answer: those entries cannot be started +// without the project's own toolchain, and a check that guesses at one would report on something the app +// never runs. + +import { existsSync, readFileSync, realpathSync, statSync } from 'node:fs'; +import { join, resolve, sep } from 'node:path'; + +import { CODE_LOADING_FLAGS, NO_OPERAND_FLAGS, VALUED_FLAGS, flagName } from '../node-flags.js'; + +export type EntryResolution = + | { + kind: 'entry'; + file: string; + /** Node flags from the project's own start script. */ + nodeArgs: string[]; + /** Arguments the start script passes to the app itself. */ + appArgs: string[]; + /** Where the entry came from, for the line the CLI prints. */ + from: string; + } + | { kind: 'unavailable'; reason: string }; + +/** Files a project conventionally starts, in the order a reader would try them. */ +const CONVENTIONAL = [ + 'server.js', 'server.mjs', 'server.cjs', + 'app.js', 'app.mjs', 'app.cjs', + 'index.js', 'index.mjs', 'index.cjs', + 'src/server.js', 'src/server.mjs', 'src/server.cjs', + 'src/index.js', 'src/index.mjs', 'src/index.cjs', + 'dist/server.js', 'dist/index.js', 'dist/main.js', + 'build/index.js', 'build/server.js', +]; + +/** Node loads these directly. `.ts` and `.tsx` need the project's own loader, whatever it is. */ +const LOADABLE = /\.(?:js|mjs|cjs)$/; + +function readPackage(cwd: string): Record | null { + try { + const parsed: unknown = JSON.parse(readFileSync(join(cwd, 'package.json'), 'utf8')); + + return parsed !== null && typeof parsed === 'object' ? (parsed as Record) : null; + } catch { + return null; + } +} + +function isFile(path: string): boolean { + try { + return statSync(path).isFile(); + } catch { + return false; + } +} + +/** + * Inside `cwd`, and a file Node can load on its own. Nothing else is launchable. + * + * Containment is checked on the REAL paths of both sides, not the written ones. Two things would slip + * past a comparison of path text: a project at `/srv/app` accepting `/srv/app-staging/server.js`, which + * a bare prefix test allows, and a `server.js` inside the project that is a symlink to a file outside + * it — `resolve` normalises text and never looks at the filesystem, while the thing that would actually + * be executed is the link's target. + */ +function loadable(cwd: string, candidate: string): string | null { + const written = resolve(cwd, candidate); + if (!LOADABLE.test(written) || !isFile(written)) return null; + + let root: string; + let path: string; + try { + root = realpathSync(cwd); + path = realpathSync(written); + } catch { + return null; // a path that cannot be resolved is not a path this will run + } + if (path !== root && !path.startsWith(root + sep)) return null; + + return written; +} + +/** + * Node flags this understands well enough to pass on, deliberately few. + * + * The grammar is the problem, not the flags. A start script is written for a shell and for Node's own + * argument parser, and this reads it with neither: it splits on whitespace. So the only tokens accepted + * are ones whose meaning cannot depend on what follows them — one of the operand-free flags, or a + * self-contained `--name=value` whose name is on the valued allowlist. Every other flag is refused, + * an unrecognised `--name=value` included: a flag that takes its operand as the NEXT token turns the + * token after it into a flag value, and `--eval=…` and `--print=…` take the program out of the file + * named on the command line altogether. Reading either as the entry would launch a different program + * than the script describes. + * + * The inventory itself lives in `../node-flags.js`, shared with the probe and the structural parse. + */ +function passable(token: string): boolean { + if (CODE_LOADING_FLAGS.has(flagName(token))) return false; + if (NO_OPERAND_FLAGS.has(token)) return true; + + return token.includes('=') && VALUED_FLAGS.has(flagName(token)); +} + +/** + * A start script this may run itself, which means: the word `node`, flags from the bounded set above, a + * file it can load — and, after that, whatever arguments the project passes its own app. + * + * Anything else is somebody else's program, or a program this cannot read with confidence. `next start` + * needs Next, `tsx server.ts` and `node server.ts` need a loader, `bun server.js` and `nodemon + * server.js` are not this runtime, `cross-env NODE_ENV=x node .` needs cross-env, and `npm run build && + * node .` builds first. Quoting, `$VARIABLE`, globs and `--` are refused for a plainer reason: they mean + * something to the shell that runs the real script and nothing to the whitespace split done here, so a + * script containing them is not a script this can claim to have reproduced. + * + * Nothing here reaches a shell — the launch is an argument array, so a metacharacter is a misreading + * risk rather than an injection one — but reading a script wrongly is exactly how a verification ends up + * reporting on a program nobody runs. + */ +function fromScript(cwd: string, script: string): { file: string; nodeArgs: string[]; appArgs: string[] } | null { + // This is not a shell parser. Refuse every shell form that can change the words the argument-array + // launch receives: operators and comments, quoting and escapes, substitutions, glob forms, tilde + // expansion, and a newline that starts another command. Spaces and tabs remain the only separators. + if (/[&|;><()#$`'"*?\[\]{}\\~\r\n]/.test(script)) return null; + const parts = script.trim().split(/\s+/); + if (parts[0] !== 'node') return null; + + const nodeArgs: string[] = []; + let rest = parts.slice(1); + while (rest.length > 0 && rest[0]!.startsWith('-')) { + const token = rest[0]!; + if (!passable(token)) return null; + nodeArgs.push(token); + rest = rest.slice(1); + } + if (rest.length === 0) return null; // `node` with no file: a REPL, not an app + + const file = loadable(cwd, rest[0]!); + + return file === null ? null : { file, nodeArgs, appArgs: rest.slice(1) }; +} + +/** + * The file the runtime check will start, or why it will not start anything. + * + * Order: a start script that is a plain `node` invocation, then `package.json` `main`, then the + * conventional entries. The script comes first because it is the project SAYING what it runs; the rest + * are inference, and inference that lands on the wrong file would verify a program nobody serves. + */ +export function resolveEntry(cwd: string): EntryResolution { + const pkg = readPackage(cwd); + const scripts = (pkg?.scripts ?? null) as Record | null; + + // A declared script is the end of the search either way: where the project says what it runs, a + // fallback to some other file would verify a program nobody serves. + const declared = ['start', 'serve'] + .map((name) => ({ name, script: scripts?.[name] })) + .find((entry): entry is { name: string; script: string } => typeof entry.script === 'string' && entry.script.trim() !== ''); + + if (declared !== undefined) { + const resolved = fromScript(cwd, declared.script); + + return resolved === null + ? { + kind: 'unavailable', + reason: `the "${declared.name}" script is outside the bounded \`node \` form this check can launch without the project's own toolchain`, + } + : { kind: 'entry', ...resolved, from: `the "${declared.name}" script` }; + } + + const main = pkg?.main; + if (typeof main === 'string') { + const file = loadable(cwd, main); + if (file !== null) return { kind: 'entry', file, nodeArgs: [], appArgs: [], from: 'the package "main" field' }; + } + + for (const candidate of CONVENTIONAL) { + const file = loadable(cwd, candidate); + if (file !== null) return { kind: 'entry', file, nodeArgs: [], appArgs: [], from: candidate }; + } + + return { + kind: 'unavailable', + reason: existsSync(join(cwd, 'package.json')) + ? 'no directly loadable entry was found — declare one as a "start" script of the form `node `, or run the check against a built output' + : 'there is no package.json here', + }; +} diff --git a/src/protect/install/runtime/probe.ts b/src/protect/install/runtime/probe.ts new file mode 100644 index 00000000..110d2016 --- /dev/null +++ b/src/protect/install/runtime/probe.ts @@ -0,0 +1,640 @@ +// `protect --check --runtime`: start the app, send one request per HTTP listener it opens, and report +// whether that request reached the scaffolded guard seam. +// +// Every other wiring check reads the app's source. This one is the only check that can say a request +// arrived — and it is opt-in for a reason, because answering it means running the application. +// +// What a proven result says, and the only wording for it: runtime traversal reached the scaffolded guard +// seam. Not that rules were delivered, not that the deployed app is wired, and not that ordinary traffic +// is blocked. +// +// ## What it answers for, exactly +// +// One process: the entry this started. Its listeners are moved to an ephemeral loopback port, one that +// cannot be probed is stopped from binding, and each one that binds is asked. An attempt to start another +// process is refused: a child can daemonize after launch, so its reachability cannot be established from +// launch options and a process-group kill alone. +// +// An app that attempts to start ANOTHER process gets `unavailable`, whatever that process is. A worker +// THREAD is also `unavailable`: it loads the reporter, but a worker has no `process.send`, so its listener +// reports cannot leave it. +// +// Discovery is a window, and it is frozen when it closes. The set of listeners the probes are drawn from +// is the set observed before that moment; anything the app reports afterwards — a second listener while +// the first is still being asked, a process started late — cannot join a set already being answered +// from, so it makes the run `unavailable` instead. +// +// Closing that window is a handshake, not a moment in this process's own timeline. A listener that opens +// as the last response finishes has already sent its report, and that report can still be in the pipe +// while the promise chain here runs to its answer — so the app is asked to stop, and its confirmation is +// what proves the pipe is empty, because IPC is ordered. A run that gets no confirmation says it could +// not tell rather than passing. +// +// The inherited `NODE_OPTIONS` is part of what this can answer for. Node parses it ahead of the command +// line, so a `--require` sitting in the environment would run before the reporter and could open a +// listener nothing screened; such a value makes the run `unavailable` before anything is launched. +import { spawn, type ChildProcess } from 'node:child_process'; +import { createHash, randomBytes } from 'node:crypto'; +import { existsSync } from 'node:fs'; +import { request as httpRequest } from 'node:http'; +import { request as httpsRequest, Agent as HttpsAgent } from 'node:https'; +import { fileURLToPath } from 'node:url'; + +import { classifyNodeOptions } from '../node-flags.js'; + +/** Long enough that nothing in the app can guess it, short enough to sit in an environment variable. */ +const CHALLENGE_BYTES = 32; + +/** + * The two names the seam and this harness have to agree on, restated here rather than imported: the + * seam is edge-safe runtime JavaScript and this is the CLI's TypeScript. A test asserts they match. + */ +export const VERIFY_HEADER_NAME = 'x-patchstack-verify'; +export const CHALLENGE_ENV = 'PATCHSTACK_VERIFY_CHALLENGE'; + +/** How long to wait for the app to open its listeners, restarted whenever a new one appears. */ +const SETTLE_MS = 1_500; + +/** The whole run: startup, settling, every request, and cleanup. Not a per-phase budget. */ +const RUN_MS = 20_000; + +/** The most any single request may take, still bounded by whatever is left of the run. */ +const PROBE_MS = 5_000; + +/** + * How long to wait for the app to confirm it has stopped reporting. + * + * The confirmation is what makes the final read of what the app opened trustworthy, so a run that does + * not get one does not pass — it reports that it could not tell. Still bounded by the run's deadline. + */ +const FREEZE_MS = 500; + +/** Retained child output, per stream. Draining continues past this; only the kept portion is bounded. */ +const OUTPUT_LIMIT = 4_096; + +/** + * Kept past the bound, then thrown away after redaction. + * + * The challenge and its answer are 64 characters each. Bounding the text first and redacting second + * would leave the first half of a value that straddles the bound sitting in output this promises not to + * carry, so the overlap is retained long enough to match against and dropped again afterwards. + */ +const REDACTION_OVERLAP = 256; + +/** + * How much of a listener's answer is read. + * + * The answer is a 64-character digest, so this is generous. It exists because the thing being read is an + * unverified application: a listener that streams without end must not be able to grow this process. + */ +const ANSWER_LIMIT = 8_192; + +/** The only address a verification may leave the app reachable on. */ +const LOOPBACK = ['127.0.0.1', '::1']; + +/** + * Signals that mean this command is ending, and the app it started has to end with it. + * + * Ctrl-C, a `kill`, and a closed terminal. `SIGKILL` is not here because it cannot be: a parent killed + * outright runs nothing, which is why the child is also bounded by the run's own deadline. + */ +const ENDING_SIGNALS: NodeJS.Signals[] = ['SIGINT', 'SIGTERM', 'SIGHUP', 'SIGQUIT']; + +export type ListenerOutcome = 'traversed' | 'answered-without-sentinel' | 'unreachable'; + +export interface ListenerResult { + scheme: 'http' | 'https'; + /** The address actually bound, as the app process reported it. */ + host: string; + port: number; + outcome: ListenerOutcome; + detail?: string; +} + +export interface RuntimeProbeResult { + /** `proven` only when every observed listener traversed; see `verdictOf`. */ + outcome: 'proven' | 'not-traversed' | 'unavailable'; + /** Why the run could not answer, when it could not. */ + reason?: string; + /** One entry per HTTP(S) listener observed while starting up. */ + listeners: ListenerResult[]; + /** Bounded, and stripped of the challenge and its answer. Not otherwise sanitised. */ + output: string; + /** + * The process this started, when it started one. + * + * Reported because it is a fact about what the check DID, and the only way for a caller — or a test — + * to confirm afterwards that nothing it launched survived it. + */ + pid?: number; +} + +export interface ProbeOptions { + cwd: string; + /** The entry to run. The caller decides it is directly loadable; this runs whatever it is given. */ + entry: string; + /** Node flags the project's own start script carries, passed through unchanged. */ + nodeArgs?: string[]; + /** Arguments the start script passes to the app, passed through unchanged. */ + appArgs?: string[]; + /** The whole run's budget, cleanup included. */ + timeoutMs?: number; + settleMs?: number; + /** For tests: the platform to answer for, defaulting to this one. */ + platform?: string; + /** For tests: the reporter to preload, defaulting to the one shipped beside this file. */ + preload?: string; +} + +/** + * The answer a seam derives from the challenge. + * + * Computed here with `node:crypto` because the seam's own copy runs in the edge-safe graph and uses web + * crypto. Two implementations of one format, so a test asserts they agree — drift between them would + * read as an app that never traversed. + */ +export function expectedAnswerFor(challenge: string): string { + return createHash('sha256').update(`patchstack-verify:${challenge}`).digest('hex'); +} + +/** Where the preload lives, in a build and in this repository. */ +export function preloadPath(): string | null { + const candidates = [ + new URL('./protect/runtime/report-listeners.cjs', import.meta.url), // beside the built CLI + new URL('./report-listeners.cjs', import.meta.url), // beside this file, in the repository + ]; + for (const candidate of candidates) { + const path = fileURLToPath(candidate); + if (existsSync(path)) return path; + } + + return null; +} + +export type NodeOptionsPropagation = { kind: 'options'; value: string } | { kind: 'unavailable'; reason: string }; + +/** + * `NODE_OPTIONS` carrying the reporter, so the entry loads it before inherited options and a worker + * using the process environment loads it too. + * + * Three things have to hold, and where one does not the run declines rather than launching with weaker + * containment than it claims. + * + * The reporter has to be nameable. The value is space-separated and double-quote-aware, which a path + * with a space needs and a path with a double quote cannot survive. + * + * Anything already in the environment has to be a flag this recognises. Node parses `NODE_OPTIONS` + * ahead of the command line, so an inherited `--require` or `--import` loads before the reporter does + * and sees `net.Server.prototype.listen` as the app's own code would — the one window the start-script + * reader refuses a preload in order to close. + * + * The reporter goes FIRST. Order within the value is order of execution, and a recognised flag that + * follows the reporter cannot undo it. + */ +export function propagatedNodeOptions(preload: string, existing: string | undefined): NodeOptionsPropagation { + if (preload.includes('"')) { + return { + kind: 'unavailable', + reason: 'the path to the listener reporter contains a double quote, which NODE_OPTIONS cannot carry — so a worker thread could not inherit the listener containment', + }; + } + const ours = `--require "${preload}"`; + if (existing === undefined || existing.trim() === '') return { kind: 'options', value: ours }; + + const verdict = classifyNodeOptions(existing); + + return verdict.kind === 'refused' ? { kind: 'unavailable', reason: verdict.why } : { kind: 'options', value: `${ours} ${existing}` }; +} + +/** Bounded, and with the verification values removed. Nothing else about it is claimed. */ +function keptOutput(chunks: string[], challenge: string, answer: string): string { + const text = chunks.join('').split(challenge).join('').split(answer).join(''); + + return text.slice(0, OUTPUT_LIMIT); +} + +/** Ask one listener, and say what came back. */ +async function askListener( + listener: { scheme: 'http' | 'https'; host: string; port: number }, + challenge: string, + answer: string, + budgetMs: number, +): Promise { + if (budgetMs <= 0) { + return { ...listener, outcome: 'unreachable', detail: 'the run ran out of time before it could be asked' }; + } + const send = listener.scheme === 'https' ? httpsRequest : httpRequest; + const options: Record = { + host: listener.host, + port: listener.port, + path: '/', + method: 'GET', + headers: { [VERIFY_HEADER_NAME]: challenge }, + timeout: budgetMs, + }; + // Scoped to this request alone: a loopback listener the app just created has whatever certificate it + // has, and a process-wide switch would change what the app and this CLI trust for everything else. + if (listener.scheme === 'https') options.agent = new HttpsAgent({ rejectUnauthorized: false }); + + return new Promise((resolve) => { + const req = send(options as never, (res) => { + const body: Buffer[] = []; + let read = 0; + res.on('data', (chunk: Buffer) => { + read += chunk.length; + if (read <= ANSWER_LIMIT) { + body.push(chunk); + return; + } + // Past the budget the answer cannot be the digest whatever else arrives, so stop reading. + res.destroy(); + }); + const finish = () => { + const text = Buffer.concat(body).toString('utf8').trim(); + resolve({ + ...listener, + outcome: text === answer ? 'traversed' : 'answered-without-sentinel', + detail: text === answer ? undefined : `answered with ${res.statusCode ?? '?'} and no sentinel`, + }); + }; + res.on('end', finish); + res.on('close', finish); // a body this stopped reading ends here rather than at `end` + }); + req.on('timeout', () => req.destroy(new Error('timed out'))); + req.on('error', (err) => resolve({ ...listener, outcome: 'unreachable', detail: (err as Error).message })); + req.end(); + }); +} + +/** + * The aggregate, in order: anything unavailable outranks a failure, and a failure outranks a pass. + * + * No listener observed is `unavailable`, not a pass: nothing observed is nothing proven. + */ +export function verdictOf(listeners: ListenerResult[]): 'proven' | 'not-traversed' | 'unavailable' { + if (listeners.length === 0 || listeners.some((l) => l.outcome === 'unreachable')) return 'unavailable'; + if (listeners.some((l) => l.outcome === 'answered-without-sentinel')) return 'not-traversed'; + + return 'proven'; +} + +/** + * Kill the entry's process group, once, and stay killed. + * + * The entry is launched as the leader of a dedicated group, so the cleanup target does not depend on a + * later read of the process tree. Idempotent because this is called from the normal path, from `finally`, + * and from signal and exit handlers, which can happen in any order. + */ +function terminator(child: ChildProcess): () => void { + let done = false; + + return () => { + if (done) return; + done = true; + try { + if (child.pid !== undefined) process.kill(-child.pid, 'SIGKILL'); + } catch { + try { + child.kill('SIGKILL'); + } catch { + // Already gone. + } + } + }; +} + +export async function probeRuntimeTraversal(options: ProbeOptions): Promise { + const platform = options.platform ?? process.platform; + const challenge = randomBytes(CHALLENGE_BYTES).toString('hex'); + const answer = expectedAnswerFor(challenge); + const nothing = (reason: string): RuntimeProbeResult => ({ outcome: 'unavailable', reason, listeners: [], output: '' }); + + // Before launching, not after: without a way to kill a process tree, a run that times out can leave + // the application running — and a server left behind by a verification command is worse than an + // unanswered question. + if (platform === 'win32') return nothing('process-tree cleanup is not implemented on Windows'); + + const preload = options.preload ?? preloadPath(); + if (preload === null) return nothing('the listener reporter is missing from this installation'); + + const deadline = Date.now() + (options.timeoutMs ?? RUN_MS); + const remaining = () => Math.max(0, deadline - Date.now()); + + // Before launching, like the platform check: containment the run cannot guarantee is not something to + // discover after the app is already running. + const propagation = propagatedNodeOptions(preload, process.env.NODE_OPTIONS); + if (propagation.kind === 'unavailable') return nothing(propagation.reason); + + let detachMessages = (): void => {}; + const child = spawn(process.execPath, [...(options.nodeArgs ?? []), '--require', preload, options.entry, ...(options.appArgs ?? [])], { + cwd: options.cwd, + detached: true, // its own process group, so cleanup has one stable target + stdio: ['ignore', 'pipe', 'pipe', 'ipc'], + // The app's own environment, plus the challenge and the propagated reporter. `NODE_ENV` is + // deliberately left alone: choosing one here would change which code path the app takes, and the + // point is to verify the app as it is. + env: { + ...process.env, + [CHALLENGE_ENV]: challenge, + // Replaced rather than added to: what may be inherited was decided above, and the reporter is + // first in the value that goes out. + NODE_OPTIONS: propagation.value, + }, + }); + + const terminate = terminator(child); + // Installed now, immediately after the spawn: between here and the end of this function the child is a + // detached process group that outlives an interrupted parent. Ctrl-C, a `kill`, a closed terminal, an + // exception in this function and an ordinary return must all reach the same cleanup. They are removed + // again once it has run, so a session that verifies twice accumulates nothing. + const onSignal = (signal: NodeJS.Signals) => { + terminate(); + detachHandlers(); + // Whether anyone else is listening, which decides whether re-sending is safe. + // + // Counting is only meaningful because this handler is PREPENDED below and so runs before any of + // them: a handler the caller registered with `once` removes itself as it runs, and had it gone + // first this count would read zero whether nobody was listening or somebody was and has already + // dealt with the signal. + // + // With nobody else listening, removing ours restores the default disposition and re-sending ends + // this process the way the signal would have if nothing here had listened for it. With another + // listener present there is no default to restore and it owns what happens next, so re-sending + // would either call it a second time or kill a process it meant to keep alive. + if (process.listenerCount(signal) === 0) process.kill(process.pid, signal); + }; + const onExit = () => terminate(); + const detachHandlers = () => { + for (const signal of ENDING_SIGNALS) process.removeListener(signal, onSignal); + process.removeListener('exit', onExit); + }; + // Prepended, so cleanup happens before the caller's own handler runs and so the count above sees + // every listener that is going to participate in this signal. + for (const signal of ENDING_SIGNALS) process.prependListener(signal, onSignal); + process.on('exit', onExit); + + try { + const chunks: string[] = []; + let kept = 0; + const drain = (stream: NodeJS.ReadableStream | null) => { + // Drained for the life of the child whether or not anything is kept: a full pipe stops the app + // starting, which would read as a timeout this verifier caused. + stream?.setEncoding('utf8'); + stream?.on('data', (chunk: string) => { + if (kept < OUTPUT_LIMIT + REDACTION_OVERLAP) { + chunks.push(chunk); + kept += chunk.length; + } + }); + }; + drain(child.stdout); + drain(child.stderr); + + const observed: Array<{ scheme: 'http' | 'https'; host: string; port: number }> = []; + const unsupported: string[] = []; + const created: string[] = []; + const workers: string[] = []; + const late: string[] = []; + const escaped: string[] = []; + let loaded = false; + let startFailure: string | null = null; + let exitHow: string | null = null; + // Open while listeners may still join the set the probes are drawn from, and false the moment that + // set is answered from. Nothing reported after it closes can be part of the answer. + let discovering = true; + let onFrozen: (() => void) | null = null; + + /** + * Ask the app to stop reporting, and wait for it to say it has. + * + * Returns whether it confirmed. IPC is ordered, so the confirmation is proof that every report sent + * before it has been delivered and handled here — which is the only thing that makes the final read + * of `late` mean anything. Without it, a listener that opened as the last response finished is a + * message still in the pipe when this forms a verdict. + */ + const freeze = async (): Promise => { + if (!child.connected) return false; + + return new Promise((resolve) => { + const settle = (confirmed: boolean) => { + clearTimeout(timer); + onFrozen = null; + child.removeListener('exit', onGone); + resolve(confirmed); + }; + const onGone = () => settle(false); + const timer = setTimeout(() => settle(false), Math.min(FREEZE_MS, remaining())); + onFrozen = () => settle(true); + child.once('exit', onGone); + try { + child.send({ patchstackVerify: 'freeze' }); + } catch { + settle(false); // the channel is gone, so nothing more can arrive over it either + } + }); + }; + + await new Promise((resolve) => { + let settleTimer: NodeJS.Timeout | null = null; + const done = () => { + discovering = false; + clearTimeout(runTimer); + if (settleTimer) clearTimeout(settleTimer); + resolve(); + }; + // Through `done` as well, so the overall deadline leaves no pending settling timer behind holding + // the event loop open after the answer is already known. + const runTimer = setTimeout(done, remaining()); + const settle = () => { + if (settleTimer) clearTimeout(settleTimer); + // Never past the run's deadline: settling is a way to finish EARLY, not an extension. + settleTimer = setTimeout(done, Math.min(options.settleMs ?? SETTLE_MS, remaining())); + }; + + const onMessage = (message: unknown) => { + const note = message as { + patchstackVerify?: string; + scheme?: string; + host?: string; + port?: number; + why?: string; + via?: string; + what?: string; + node?: boolean; + escape?: string; + }; + if (note?.patchstackVerify === 'frozen') { + onFrozen?.(); + + return; + } + if (note?.patchstackVerify === 'process-created' && typeof note.escape === 'string') escaped.push(`${note.what ?? 'a process'} ${note.escape}`); + if (note?.patchstackVerify === 'worker-created' && typeof note.escape === 'string') escaped.push(`a worker thread that ${note.escape}`); + if (!discovering) { + // Discovery is closed and this arrived after it. Pushing it into the observed set would add to + // an array the probes are already being run over, and dropping it would let something nobody + // asked stand behind a pass. Recorded, and the run declines below. + const kind = note?.patchstackVerify; + if (kind === 'listener') late.push(`an ${note.scheme === 'https' ? 'HTTPS' : 'HTTP'} listener on port ${Number(note.port)}`); + else if (kind === 'unsupported-listener') late.push(`a listener this cannot probe (${note.why ?? 'an unsupported transport'})`); + else if (kind === 'process-created') late.push(`another process (${note.what ?? 'a process'} via ${note.via ?? 'child_process'})`); + else if (kind === 'worker-created') late.push(`a worker thread (${note.what ?? 'a worker'})`); + + return; + } + if (note?.patchstackVerify === 'listener' && (note.scheme === 'http' || note.scheme === 'https')) { + observed.push({ scheme: note.scheme, host: String(note.host), port: Number(note.port) }); + settle(); // a newly observed listener restarts the interval + } + if (note?.patchstackVerify === 'unsupported-listener') { + // Discovery ends here. The run cannot speak for the whole app once a listener was refused, and + // the app is stopped rather than left to carry on being started for an answer nobody will get. + unsupported.push(note.why ?? 'an unsupported transport'); + done(); + } + if (note?.patchstackVerify === 'process-created') { + // Discovery ends here too. The reporter refuses process creation, because a launch cannot be + // proven to remain inside the process group this run can clean up. + created.push( + `${note.what ?? 'a process'} (via ${note.via ?? 'child_process'})${note.node === true ? '' : ', which the listener reporter cannot reach'}`, + ); + done(); + } + if (note?.patchstackVerify === 'worker-created') { + // Discovery ends here too, and for the containment reason rather than a reachability one: a + // worker loads the reporter and its listeners are moved to loopback, but a worker has no + // `process.send`, so a listener it opens can be neither counted nor asked. + workers.push(note.what ?? 'a worker'); + done(); + } + // `ready` starts nothing. It only distinguishes a reporter that never loaded from an app that + // never listened — an app that takes seconds to boot must be bounded by the overall deadline, + // not by an interval that started before it had opened anything. + if (note?.patchstackVerify === 'ready') loaded = true; + }; + child.on('message', onMessage); + detachMessages = () => child.removeListener('message', onMessage); + child.on('error', (err) => { + startFailure = `the app could not be started (${err.message})`; + done(); + }); + child.on('exit', (code, signal) => { + // Only HOW it exited is recorded here. What that means — never listened, or listened and was gone + // before it could be asked — depends on what had been observed by the end of the run, and the + // exit can be delivered before the listener report that preceded it. + exitHow = signal ?? `code ${code}`; + done(); + }); + }); + + // A listener on any other address is one this verification put on an interface it must not have. It + // is not probed and it is not a finding about the guard: the run declines to answer. + const offLoopback = observed.filter((l) => !LOOPBACK.includes(l.host)); + + // Nothing is probed once any listener was refused, the app attempted another process or worker, or a + // listener turned out to be off-loopback: each means the run cannot speak for the whole app, and + // asking the rest would produce a partial pass that reads like a whole one. + const contained = unsupported.length === 0 && created.length === 0 && workers.length === 0 && offLoopback.length === 0; + const listeners = contained ? await Promise.all(observed.map((l) => askListener(l, challenge, answer, Math.min(PROBE_MS, remaining())))) : []; + + // Only now is what the app opened knowable. Everything below reads sets this can no longer change. + const confirmed = await freeze(); + + const output = keptOutput(chunks, challenge, answer); + const pid = child.pid; + const exitNote = + startFailure ?? + (exitHow === null + ? null + : observed.length === 0 + ? `the app exited before it listened (${exitHow})` + : `the app exited while it was being probed (${exitHow})`); + + if (escaped.length > 0) { + return { + outcome: 'unavailable', + reason: `the app tried to start something outside this run's one-process scope, and it was refused: ${escaped.join('; ')}`, + listeners: [], + output, + pid, + }; + } + if (unsupported.length > 0) { + return { outcome: 'unavailable', reason: `a listener this cannot probe: ${unsupported.join('; ')}`, listeners: [], output, pid }; + } + if (created.length > 0) { + return { + outcome: 'unavailable', + reason: `the app attempted to start another process, so this run cannot answer for the whole app: ${created.join('; ')}`, + listeners: [], + output, + pid, + }; + } + if (workers.length > 0) { + return { + outcome: 'unavailable', + reason: `the app started a worker thread, whose listeners cannot report back to this run: ${workers.join('; ')}`, + listeners: [], + output, + pid, + }; + } + if (offLoopback.length > 0) { + return { + outcome: 'unavailable', + reason: `a listener bound an address this verification must not open: ${offLoopback.map((l) => l.host).join('; ')}`, + listeners: [], + output, + pid, + }; + } + // Checked after the probes and after the app confirmed it had stopped, because these are the + // refusals that can arrive DURING the probing. + if (late.length > 0) { + return { + outcome: 'unavailable', + reason: `the app was still opening things after discovery had ended, so this run cannot answer for the whole app: ${late.join('; ')}`, + listeners: [], + output, + pid, + }; + } + if (observed.length === 0) { + if (exitNote !== null) return { outcome: 'unavailable', reason: exitNote, listeners: [], output, pid }; + if (!loaded) { + return { outcome: 'unavailable', reason: 'the listener reporter did not load in the app process', listeners: [], output, pid }; + } + + return { outcome: 'unavailable', reason: 'no HTTP listener was observed while starting up', listeners: [], output, pid }; + } + + // A pass is the one answer the confirmation is load-bearing for. Every other outcome rests on + // something already observed, and an undelivered report could only ever have made it worse — but a + // pass is a claim that nothing else was open, which is exactly what an undrained channel cannot + // support. + // + // Two things count as settling the question instead. The reporter never loading is its own finding + // above. And an app that has EXITED cannot still be holding a listener nobody asked about, which is + // the whole thing the confirmation establishes — an exit ends the question rather than leaving it + // open, so it is not treated as a missing answer. + const unconfirmed = !confirmed && loaded && exitHow === null; + const outcome = unconfirmed && verdictOf(listeners) === 'proven' ? 'unavailable' : verdictOf(listeners); + + // An unavailable verdict always says why. "Could not be established" with no cause is a line people + // learn to scroll past, and the exit code it produces is the one that most needs reading. + const reason = + outcome === 'unavailable' + ? (exitNote ?? + (unconfirmed + ? 'the app never confirmed it had stopped opening listeners, so one it opened last cannot be ruled out' + : (listeners.find((l) => l.outcome === 'unreachable')?.detail ?? 'a listener could not be reached'))) + : undefined; + + return { outcome, reason, listeners, output, pid }; + } finally { + terminate(); + detachHandlers(); + detachMessages(); + } +} diff --git a/src/protect/install/runtime/report-listeners.cjs b/src/protect/install/runtime/report-listeners.cjs new file mode 100644 index 00000000..7dc20299 --- /dev/null +++ b/src/protect/install/runtime/report-listeners.cjs @@ -0,0 +1,360 @@ +// Preloaded into the verification child by `protect --check --runtime`, before the app's own entry. +// +// It answers what the parent cannot see for itself: which HTTP listeners does starting this app produce, +// on what address, and did this app attempt to start another process. It also contains the run: a +// probeable listener is moved to loopback, an unsupported listener is stopped from binding, and another +// process is refused before launch. The child exists only because the verifier started it, so a server +// or process it leaves behind would be the verifier's doing. +// +// The scope of that containment is one process. Every attempt to start another process is refused. A +// child can replace its environment, create a process group in its launch options, or daemonize after it +// starts; the parent cannot establish from the call that its reporter and group kill will still reach +// the resulting process. The runtime check already has to decline when another process is involved, so +// it does not start one it cannot safely leave behind. +// +// A worker THREAD does load this file, and its listeners are moved to loopback like any other, but a +// worker has no `process.send`, so its reports cannot leave it. Containment holds and observation does +// not — which would make a pass drawn from the main thread's listeners a pass for part of an app. So +// worker creation is reported and the run declines. A worker given a replacement `env` does not load +// this file at all (a worker's `execArgv` does not carry a preload, so `NODE_OPTIONS` is the only route +// in), and is refused rather than allowed to bind outside the reporter. +// +// What is reported is bounded and names no content. An argument to one of these calls can be a whole +// shell command line, and a startup command commonly carries a token or a password; a verifier that +// copied one into its own diagnostics would put a secret in terminal or CI output that the app never +// printed itself. So a launch is described by its launcher and the basename of its executable, and a +// form whose argument is a command line is described by the fact that it is one. +// +// Reported over IPC rather than stdout: the app owns stdout, and a report that has to be parsed out of +// arbitrary application output is a report that breaks the first time an app prints something similar. +'use strict'; + +const net = require('node:net'); +const http = require('node:http'); +const https = require('node:https'); +const childProcess = require('node:child_process'); +const workerThreads = require('node:worker_threads'); +const { basename } = require('node:path'); + +const LOOPBACK = '127.0.0.1'; + +/** + * Set once the parent says it has stopped listening. + * + * After this point a report would not be read, so a listener is refused rather than bound: a listener + * nobody is going to ask about must not exist, and the parent is about to end this process anyway. + */ +let frozen = false; + +function report(message) { + try { + if (typeof process.send === 'function') process.send(message); + } catch { + // The channel is the parent's to keep open. If it is gone there is nothing to tell and nothing to do + // about it, and this must not be what breaks the app it was preloaded into. + } +} + +/** + * What transport a `listen()` call asks for, read from the ARGUMENTS. + * + * The call decides this, not the class of the server: an `http.Server` listening on a Unix path or a + * file descriptor is not a TCP listener, and rewriting it to one would run a different program than the + * app. So the arguments are classified first, and the class is consulted only for a TCP listen. + * + * @returns {{ kind: 'tcp' } | { kind: 'unsupported', why: string }} + */ +function transportOf(args) { + const [first] = args; + + const validPort = (value) => { + if (typeof value !== 'number' && typeof value !== 'string') return false; + if (typeof value === 'string' && value.trim() === '') return false; + const number = Number(value); + + return Number.isInteger(number) && number >= 0 && number <= 0xffff; + }; + + // Node's rule (`isPipeName` in `net`): a string is a path only when it is not a non-negative + // number. `'3000'`, `' 3000'` and `'3e3'` are all TCP port 3000 — which is also what + // `listen(process.env.PORT)` normally hands over. + if (typeof first === 'string') { + if (Number(first) < 0 || Number.isNaN(Number(first))) return { kind: 'unsupported', why: 'a Unix socket path' }; + + return validPort(first) ? { kind: 'tcp' } : { kind: 'unsupported', why: 'an invalid TCP port' }; + } + if (typeof first === 'number') return validPort(first) ? { kind: 'tcp' } : { kind: 'unsupported', why: 'an invalid TCP port' }; + + if (first && typeof first === 'object') { + // This is Node's precedence: a brought handle, then a usable fd, then a port, then a path. Reading + // `path` first would refuse `{ path, port }` even though Node uses its port; reading `port` before a + // handle would rewrite an options object whose connection Node takes from somewhere else. + if (first._handle || first.handle) return { kind: 'unsupported', why: 'a handle' }; + if (typeof first.fd === 'number' && first.fd >= 0) return { kind: 'unsupported', why: 'a file descriptor' }; + + const hasPort = 'port' in first; + const port = (hasPort && (first.port === undefined || first.port === null)) ? 0 : first.port; + if (typeof port === 'number' || typeof port === 'string') { + return validPort(port) ? { kind: 'tcp' } : { kind: 'unsupported', why: 'an invalid TCP port' }; + } + if (typeof first.path === 'string' && (Number(first.path) < 0 || Number.isNaN(Number(first.path)))) { + return { kind: 'unsupported', why: 'a Unix socket path' }; + } + + return { kind: 'unsupported', why: 'invalid listen options' }; + } + if (first === undefined || first === null || typeof first === 'function') return { kind: 'tcp' }; // `listen()` / `listen(null)` / `listen(cb)` + + return { kind: 'unsupported', why: `an argument of type ${typeof first}` }; +} + +/** + * The scheme a server serves, or null when it is not one this can probe. + * + * HTTP/2 is not one: a cleartext `Http2Server` is a `net.Server` and not an `http.Server`, and a secure + * one extends `tls.Server` rather than `https.Server`, so neither matches — which is the right answer, + * because an HTTP/1 request is not how you ask an h2 server anything. + */ +function schemeOf(server) { + if (server instanceof https.Server) return 'https'; + if (server instanceof http.Server) return 'http'; + + return null; +} + +/** + * Rewrite a TCP listen to an ephemeral loopback port, preserving the overload. + * + * Two changes, both load-bearing. The PORT becomes ephemeral because the app's own port may already be + * held by whatever the developer is running, and a verification that fails on EADDRINUSE says nothing + * about wiring. The HOST becomes loopback because a verification must not put the app on an address + * other machines can reach; the parent checks the address that was actually bound and refuses anything + * else, so this rewrite is stated rather than trusted. + */ +function onLoopback(args) { + const [first, ...rest] = args; + + if (first === undefined || typeof first === 'function') return [{ port: 0, host: LOOPBACK }, ...args]; + if (typeof first === 'number' || typeof first === 'string') { + // `listen(port[, host][, backlog][, cb])`: the host, if any, is replaced; anything after it that is + // not a host is kept. + const tail = rest.filter((a) => typeof a !== 'string'); + + return [{ port: 0, host: LOOPBACK }, ...tail]; + } + + return [{ ...first, port: 0, host: LOOPBACK }, ...rest]; +} + +const originalListen = net.Server.prototype.listen; +let pendingListens = 0; +let freezeAcknowledged = false; + +/** A freeze is acknowledged only after every listen already admitted has either reported or failed. */ +function acknowledgeFreeze() { + if (!frozen || pendingListens !== 0 || freezeAcknowledged) return; + freezeAcknowledged = true; + report({ patchstackVerify: 'frozen' }); +} + +net.Server.prototype.listen = function patchstackVerifyListen(...args) { + const transport = transportOf(args); + const scheme = schemeOf(this); + + if (transport.kind === 'unsupported' || scheme === null || frozen) { + const why = frozen + ? 'opens after the verification stopped reading reports' + : transport.kind === 'unsupported' + ? `listens on ${transport.why}` + : `is not an HTTP or HTTPS server (${this?.constructor?.name ?? 'unknown'})`; + report({ patchstackVerify: 'unsupported-listener', why }); + + // Not bound. The verification cannot speak for this listener, and leaving it to open a port while a + // verifier holds the process would be the verifier opening it. The error is the app's to see, so it + // arrives the way a refused bind would. + const error = Object.assign(new Error(`Patchstack runtime verification does not support a server that ${why}`), { + code: 'EPERM', + syscall: 'listen', + }); + setImmediate(() => this.emit('error', error)); + + return this; + } + + pendingListens++; + const onListening = () => { + const address = this.address(); + if (address && typeof address === 'object') { + report({ patchstackVerify: 'listener', scheme, host: address.address, port: address.port }); + } + pendingListens--; + acknowledgeFreeze(); + }; + this.once('listening', onListening); + + let result; + try { + result = originalListen.apply(this, onLoopback(args)); + } catch (error) { + this.removeListener('listening', onListening); + pendingListens--; + acknowledgeFreeze(); + throw error; + } + + return result; +}; + +/** Bounded, so an unbounded argument cannot become an unbounded diagnostic. */ +const WHAT_LIMIT = 80; + +/** A launched program named by its executable alone — never by its arguments, which carry the secrets. */ +function executableName(command) { + if (typeof command !== 'string') return `a ${typeof command}`; + const name = basename(command).replace(/\.exe$/i, ''); + + return name === '' ? 'a program' : name.slice(0, WHAT_LIMIT); +} + +const isNodeExecutable = (command) => + typeof command === 'string' && (command === process.execPath || basename(command).replace(/\.exe$/i, '') === 'node'); + +/** + * Whether the call asked for a shell, which makes its command argument a command LINE. + * + * `exec` and `execSync` always do. Every other helper does it through `shell` in an options object, and + * `exec` reaches `execFile` that way internally — so the option is looked for wherever it sits in the + * argument list rather than at a fixed position. A command line is the argument most likely to carry a + * credential and the one least possible to judge, so it is never described by its content. + */ +const shellRequested = (args) => + args.some((arg) => arg !== null && typeof arg === 'object' && !Array.isArray(arg) && Boolean(arg.shell)); + +/** Whether a replacement environment preserves the exact reporter options this process received. */ +const carriesReporter = (env) => { + try { + return ( + env !== null && + typeof env === 'object' && + typeof process.env.NODE_OPTIONS === 'string' && + env.NODE_OPTIONS === process.env.NODE_OPTIONS + ); + } catch { + return false; + } +}; + +/** Refused the way an impossible bind is: reported, and handed to the app as its own failed call. */ +function refuse(what, why) { + throw Object.assign(new Error(`Patchstack runtime verification does not start ${what} that ${why}`), { code: 'EPERM' }); +} + +/** + * Report and refuse every process the app attempts to start. + * + * Starting another process is outside the check's one-process contract. The launch is reported and + * refused before delegation. This is intentionally broader than inspecting `detached` and `env`: a + * program that starts normally may create its own session after launch, beyond the process-group cleanup + * the parent relies on. + * + * Two layers, because the exported helpers are not the only way in. They are wrapped for what they can + * say — which helper was called, and whether the command is this runtime — and `ChildProcess.prototype + * .spawn` is wrapped underneath them because every asynchronous launch goes through it however it was + * reached: `new ChildProcess().spawn(…)` directly, `cluster.fork()`, or a helper captured as a value. + * Because the exported wrapper refuses before it delegates, a helper produces one report. The low-level + * wrapper covers calls that bypass the helpers entirely; the synchronous helpers do not pass through it. + */ +const PROCESS_REFUSAL = 'leaves the one-process scope of this runtime verification'; + +for (const name of ['spawn', 'spawnSync', 'exec', 'execSync', 'execFile', 'execFileSync', 'fork']) { + if (typeof childProcess[name] !== 'function') continue; + childProcess[name] = function patchstackVerifyLaunch(...args) { + // `fork` always runs this runtime. For the rest the first argument is the command, unless a shell + // was asked for, in which case it is a whole command line and cannot be judged. + const shellForm = name === 'exec' || name === 'execSync' || shellRequested(args); + const command = name === 'fork' ? process.execPath : args[0]; + report({ + patchstackVerify: 'process-created', + via: name, + what: shellForm ? 'a shell command line' : executableName(command), + node: !shellForm && isNodeExecutable(command), + escape: PROCESS_REFUSAL, + }); + refuse('a process', PROCESS_REFUSAL); + }; +} + +const ChildProcessClass = childProcess.ChildProcess; +if (typeof ChildProcessClass === 'function' && typeof ChildProcessClass.prototype.spawn === 'function') { + ChildProcessClass.prototype.spawn = function patchstackVerifyChildSpawn(...args) { + const options = args[0] && typeof args[0] === 'object' ? args[0] : {}; + report({ + patchstackVerify: 'process-created', + via: 'ChildProcess.spawn', + what: executableName(options.file), + node: isNodeExecutable(options.file), + escape: PROCESS_REFUSAL, + }); + refuse('a process', PROCESS_REFUSAL); + }; +} + +/** + * Report every worker thread the app starts. + * + * Reported rather than screened, for the reason at the top of this file: a worker loads this and its + * listeners are contained, but it has no channel back, so a listener it opens can be neither counted + * nor asked. The name is the worker's file, or the fact that its source was given inline — the source + * itself is the app's, and not this file's to copy anywhere. + */ +const OriginalWorker = workerThreads.Worker; +if (typeof OriginalWorker === 'function') { + const workerName = (filename) => { + try { + if (typeof filename === 'string') return basename(filename).slice(0, WHAT_LIMIT) || 'a worker'; + if (filename && typeof filename.pathname === 'string') return basename(filename.pathname).slice(0, WHAT_LIMIT) || 'a worker'; + } catch { + // A `filename` of some other shape is still a worker, and saying so is the whole report. + } + + return 'a worker'; + }; + + // Sharing the parent's environment is the default and is also stated explicitly with this symbol; + // either way the reporter is carried in. + const shared = workerThreads.SHARE_ENV; + const inherits = (options) => options === null || typeof options !== 'object' || options.env === undefined || options.env === shared; + + workerThreads.Worker = class Worker extends OriginalWorker { + constructor(filename, options) { + const escape = inherits(options) || carriesReporter(options.env) ? null : 'replaces the environment that carries the listener reporter'; + report({ + patchstackVerify: 'worker-created', + what: options && options.eval === true ? 'inline source' : workerName(filename), + ...(escape === null ? {} : { escape }), + }); + if (escape !== null) refuse('a worker thread', escape); + super(filename, options); + } + }; +} + +/** + * Answer the parent's freeze request, and stop letting anything new open. + * + * The acknowledgement is what makes the parent's final read safe. IPC is ordered, so by the time this + * reply arrives every report sent before it has already been delivered — including one from a listener + * that opened while the parent was still reading the last response. Nothing may bind after this point, + * because nothing would be read. + */ +process.on('message', (message) => { + if (message === null || typeof message !== 'object' || message.patchstackVerify !== 'freeze') return; + frozen = true; + acknowledgeFreeze(); +}); + +// The channel is the parent's to hold open. Listening on it must not be what keeps an app alive that +// would otherwise have exited on its own. +if (process.channel && typeof process.channel.unref === 'function') process.channel.unref(); + +report({ patchstackVerify: 'ready', pid: process.pid }); diff --git a/src/protect/install/source-scope.ts b/src/protect/install/source-scope.ts index 745c29e8..c3f409cd 100644 --- a/src/protect/install/source-scope.ts +++ b/src/protect/install/source-scope.ts @@ -10,6 +10,8 @@ import { execFileSync } from 'node:child_process'; import { existsSync } from 'node:fs'; import { dirname, join, resolve } from 'node:path'; +import { withoutCodeLoading } from './node-flags.js'; + /** An import statement, or a `const x = require(...)` binding. Matched only at the start of a line. */ const IMPORT_LINE = /^\s*(?:import\b|export\s+(?:\*|\{)|(?:const|let|var)\s+[^=]+=\s*require\()/; @@ -149,13 +151,21 @@ function advanceDepth(line: string, inBlockComment: boolean, add: (delta: number * is not the same as passing: TypeScript needs a compiler this package must not require of a consumer * project, so a `.ts` entry is edited on the strength of the scope check alone and says so. * + * `--check` parses and exits without evaluating the file, but it does honour `NODE_OPTIONS`, so a + * code-loading flag in the environment would run in this child. The environment passed below retains + * only flags on the shared safe allowlist; that is what keeps the claim above true for compact and + * future flag forms too — this process evaluates nobody's code, the target's or anyone else's. + * * @returns true when it parses, false when it does not, null when nothing could be checked */ export function parses(filePath: string): boolean | null { if (!/\.(?:js|cjs|mjs)$/.test(filePath)) return null; try { - execFileSync(process.execPath, ['--check', filePath], { stdio: 'pipe' }); + execFileSync(process.execPath, ['--check', filePath], { + stdio: 'pipe', + env: { ...process.env, NODE_OPTIONS: withoutCodeLoading(process.env.NODE_OPTIONS) }, + }); return true; } catch (error) { diff --git a/src/protect/protect.d.ts b/src/protect/protect.d.ts index 40f49fbf..211f8b89 100644 --- a/src/protect/protect.d.ts +++ b/src/protect/protect.d.ts @@ -117,6 +117,20 @@ export interface Protection { }; } +/** + * The header a runtime wiring check sends, and the value a scaffolded guard answers it with. + * + * `protect --check --runtime` starts the app with a fresh challenge carried for that verification run, + * then asks for a response derived from it. The whole app process can read the challenge; the response + * rules out an accidental match, but does not prove which module answered. + * + * The seam reads the header itself and passes the value here; this decides whether it is the challenge + * and, if so, what to answer with. In a process not launched for runtime verification there is no + * challenge to match, so the answer is null and the request is screened exactly as it would have been. + */ +export declare const VERIFY_HEADER: "x-patchstack-verify"; +export declare function sentinelAnswer(offered: unknown): Promise; + export interface CreateProtectionOptions { /** * Fallback when the Pulse rules API does not send `enforcement`. diff --git a/src/protect/runtime.js b/src/protect/runtime.js index 9ff9e460..f1ce2e74 100644 --- a/src/protect/runtime.js +++ b/src/protect/runtime.js @@ -41,6 +41,9 @@ import { createFirewallLogReporter, resolveApiBase, telemetryEnabled } from './f // Supabase-tunnel guard for AI-builder apps (Lovable / TanStack Start + Supabase). export { createSupabaseGuard, GUARD_PATH } from './supabase-guard.js'; +// The seam side of `protect --check --runtime`. Exported because a scaffolded guard is a file in the +// app, and it has to be able to answer a verification request without carrying the logic itself. +export { sentinelAnswer, VERIFY_HEADER } from './verify-sentinel.js'; // Per-site live rule client (Pulse). Re-exported for callers/tests that want to use it directly. export { PulseRuleClient }; diff --git a/src/protect/templates/express-guard.cjs b/src/protect/templates/express-guard.cjs index 42ebfb1d..d5e772ba 100644 --- a/src/protect/templates/express-guard.cjs +++ b/src/protect/templates/express-guard.cjs @@ -1,5 +1,5 @@ // Patchstack runtime guard for CommonJS Express apps. Managed by `patchstack-connect protect`. -const { createProtection } = require("@patchstack/connect/protect"); +const { createProtection, sentinelAnswer, VERIFY_HEADER } = require("@patchstack/connect/protect"); // The fallback bundle is optional at RUNTIME. This file is imported on the app's own module path, so a // throw here is the app failing to boot rather than protection failing open — and a rules file can be // absent for ordinary reasons: a bundler that copied no JSON, a partial deploy, a half-written edit. @@ -78,19 +78,37 @@ function patchstackMiddleware(req, res, next) { passedOn = true; next(err); }; - getProtection().then( - (active) => active.express()(req, res, carryOn), - (err) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered here, + // before the protection is asked for and before the request is passed on, so no handler of the app's + // ever sees a verification request. The answer is derived from a challenge the verifying process mints + // per run, which rules out an app matching it by accident; without a challenge in the environment + // there is nothing to answer and the request is screened as normal. + const screen = () => { + getProtection().then( + (active) => active.express()(req, res, carryOn), + (err) => { + psStepAside(err); + carryOn(); + }, + ).catch((err) => { + // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the + // request is carried on only if it never was — an error here must not take the process down and + // must not answer twice. psStepAside(err); carryOn(); - }, - ).catch((err) => { - // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the - // request is carried on only if it never was — an error here must not take the process down and - // must not answer twice. - psStepAside(err); - carryOn(); - }); + }); + }; + + sentinelAnswer(req.headers?.[VERIFY_HEADER]).then((answered) => { + if (answered) { + res.statusCode = 200; + res.setHeader("content-type", "text/plain"); + res.end(answered); + + return; + } + screen(); + }, screen); } module.exports = { patchstackMiddleware }; diff --git a/src/protect/templates/express-guard.js b/src/protect/templates/express-guard.js index b6840874..454cd645 100644 --- a/src/protect/templates/express-guard.js +++ b/src/protect/templates/express-guard.js @@ -1,6 +1,6 @@ // Patchstack runtime guard for ESM Express apps. Managed by `patchstack-connect protect`. import { readFileSync } from "node:fs"; -import { createProtection } from "@patchstack/connect/protect"; +import { createProtection, sentinelAnswer, VERIFY_HEADER } from "@patchstack/connect/protect"; // The fallback bundle is optional at RUNTIME. This file is imported on the app's own module path, so a // throw here is the app failing to boot rather than protection failing open — and a rules file can be @@ -79,17 +79,35 @@ export function patchstackMiddleware(req, res, next) { passedOn = true; next(err); }; - getProtection().then( - (active) => active.express()(req, res, carryOn), - (err) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered here, + // before the protection is asked for and before the request is passed on, so no handler of the app's + // ever sees a verification request. The answer is derived from a challenge the verifying process mints + // per run, which rules out an app matching it by accident; without a challenge in the environment + // there is nothing to answer and the request is screened as normal. + const screen = () => { + getProtection().then( + (active) => active.express()(req, res, carryOn), + (err) => { + psStepAside(err); + carryOn(); + }, + ).catch((err) => { + // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the + // request is carried on only if it never was — an error here must not take the process down and + // must not answer twice. psStepAside(err); carryOn(); - }, - ).catch((err) => { - // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the - // request is carried on only if it never was — an error here must not take the process down and - // must not answer twice. - psStepAside(err); - carryOn(); - }); + }); + }; + + sentinelAnswer(req.headers?.[VERIFY_HEADER]).then((answered) => { + if (answered) { + res.statusCode = 200; + res.setHeader("content-type", "text/plain"); + res.end(answered); + + return; + } + screen(); + }, screen); } diff --git a/src/protect/templates/express-guard.ts b/src/protect/templates/express-guard.ts index ad494aee..9e7ab843 100644 --- a/src/protect/templates/express-guard.ts +++ b/src/protect/templates/express-guard.ts @@ -1,6 +1,6 @@ // Patchstack runtime guard for Express. Managed by `patchstack-connect protect`. // Register after body parsing and before routes: app.use(patchstackMiddleware). -import { createProtection } from "@patchstack/connect/protect"; +import { createProtection, sentinelAnswer, VERIFY_HEADER } from "@patchstack/connect/protect"; import fallbackRules from "./rules.json"; const PS_SITE_UUID = "__PATCHSTACK_SITE_UUID__"; @@ -75,17 +75,40 @@ export function patchstackMiddleware(req: unknown, res: unknown, next: (err?: un passedOn = true; next(err); }; - getProtection().then( - (protection) => (protection.express() as (a: unknown, b: unknown, c: (e?: unknown) => void) => void)(req, res, carryOn), - (err) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered here, + // before the protection is asked for and before the request is passed on, so no handler of the app's + // ever sees a verification request. The answer is derived from a challenge the verifying process mints + // per run, which rules out an app matching it by accident; without a challenge in the environment + // there is nothing to answer and the request is screened as normal. + const screen = () => { + getProtection().then( + (protection) => (protection.express() as (a: unknown, b: unknown, c: (e?: unknown) => void) => void)(req, res, carryOn), + (err) => { + psStepAside(err); + carryOn(); + }, + ).catch((err) => { + // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the + // request is carried on only if it never was — an error here must not take the process down and + // must not answer twice. psStepAside(err); carryOn(); - }, - ).catch((err) => { - // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the - // request is carried on only if it never was — an error here must not take the process down and - // must not answer twice. - psStepAside(err); - carryOn(); - }); + }); + }; + + // The two shapes this seam touches, named rather than asserted wholesale: a request whose headers it + // reads, and a response it answers on. + const inbound = req as { headers?: Record }; + const outbound = res as { statusCode: number; setHeader(name: string, value: string): void; end(body?: string): void }; + + sentinelAnswer(inbound.headers?.[VERIFY_HEADER]).then((answered) => { + if (answered) { + outbound.statusCode = 200; + outbound.setHeader("content-type", "text/plain"); + outbound.end(answered); + + return; + } + screen(); + }, screen); } diff --git a/src/protect/templates/fastify-plugin.cjs b/src/protect/templates/fastify-plugin.cjs index 2d111433..6ce12701 100644 --- a/src/protect/templates/fastify-plugin.cjs +++ b/src/protect/templates/fastify-plugin.cjs @@ -1,6 +1,6 @@ // Patchstack runtime guard for CommonJS Fastify apps. Managed by `patchstack-connect protect`. // Register once: app.register(patchstackFastify) -const { createProtection } = require("@patchstack/connect/protect"); +const { createProtection, sentinelAnswer, VERIFY_HEADER } = require("@patchstack/connect/protect"); // The fallback bundle is optional at RUNTIME. This file is imported on the app's own module path, so a // throw here is the app failing to boot rather than protection failing open — and a rules file can be // absent for ordinary reasons: a bundler that copied no JSON, a partial deploy, a half-written edit. @@ -73,6 +73,19 @@ async function patchstackFastify(fastify) { await getProtection().catch(psStepAside); fastify.addHook("preHandler", async (request, reply) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered before + // the route and before this request's screening — not before the protection was ever asked for, + // which registration above already did, deliberately, so startup egress is screened. The answer is + // derived from a challenge the verifying process mints per run, which rules out an app matching it + // by accident; without a challenge in the environment this is a header read. + const answered = await sentinelAnswer(request.headers?.[VERIFY_HEADER]); + if (answered) { + reply.code(200); + reply.header("content-type", "text/plain"); + reply.send(answered); + + return reply; + } const protection = await getProtection().catch(psStepAside); if (!protection) return; // the route answers this request, unscreened const guard = protection.fetchGuard(); diff --git a/src/protect/templates/fastify-plugin.js b/src/protect/templates/fastify-plugin.js index 619f2f3c..91f584ab 100644 --- a/src/protect/templates/fastify-plugin.js +++ b/src/protect/templates/fastify-plugin.js @@ -1,7 +1,7 @@ // Patchstack runtime guard for ESM Fastify apps. Managed by `patchstack-connect protect`. // Register once: app.register(patchstackFastify) import { readFileSync } from "node:fs"; -import { createProtection } from "@patchstack/connect/protect"; +import { createProtection, sentinelAnswer, VERIFY_HEADER } from "@patchstack/connect/protect"; // The fallback bundle is optional at RUNTIME. This file is imported on the app's own module path, so a // throw here is the app failing to boot rather than protection failing open — and a rules file can be @@ -74,6 +74,19 @@ export async function patchstackFastify(fastify) { await getProtection().catch(psStepAside); fastify.addHook("preHandler", async (request, reply) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered before + // the route and before this request's screening — not before the protection was ever asked for, + // which registration above already did, deliberately, so startup egress is screened. The answer is + // derived from a challenge the verifying process mints per run, which rules out an app matching it + // by accident; without a challenge in the environment this is a header read. + const answered = await sentinelAnswer(request.headers?.[VERIFY_HEADER]); + if (answered) { + reply.code(200); + reply.header("content-type", "text/plain"); + reply.send(answered); + + return reply; + } const protection = await getProtection().catch(psStepAside); if (!protection) return; // the route answers this request, unscreened const guard = protection.fetchGuard(); diff --git a/src/protect/templates/fastify-plugin.ts b/src/protect/templates/fastify-plugin.ts index 7c5d5d25..efc2a468 100644 --- a/src/protect/templates/fastify-plugin.ts +++ b/src/protect/templates/fastify-plugin.ts @@ -2,7 +2,7 @@ // Register it once (`app.register(patchstackFastify)`); it adds a preHandler hook that runs the // request-phase WAF (+ egress SSRF) on every request. Fastify's request/reply aren't Web-Fetch // shaped, so we reconstruct a Request from the parsed fastify request and run the fetch guard. -import { createProtection } from "@patchstack/connect/protect"; +import { createProtection, sentinelAnswer, VERIFY_HEADER } from "@patchstack/connect/protect"; import fallbackRules from "./rules.json"; const PS_SITE_UUID = "__PATCHSTACK_SITE_UUID__"; @@ -72,6 +72,19 @@ export async function patchstackFastify(fastify: any) { await getProtection().catch(psStepAside); fastify.addHook("preHandler", async (request: any, reply: any) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered before + // the route and before this request's screening — not before the protection was ever asked for, + // which registration above already did, deliberately, so startup egress is screened. The answer is + // derived from a challenge the verifying process mints per run, which rules out an app matching it + // by accident; without a challenge in the environment this is a header read. + const answered = await sentinelAnswer(request.headers?.[VERIFY_HEADER]); + if (answered) { + reply.code(200); + reply.header("content-type", "text/plain"); + reply.send(answered); + + return reply; + } const protection = await getProtection().catch(psStepAside); if (!protection) return; // the route answers this request, unscreened const guard = protection.fetchGuard(); diff --git a/src/protect/templates/generic-guard.cjs b/src/protect/templates/generic-guard.cjs index 00bb2038..2a0d5509 100644 --- a/src/protect/templates/generic-guard.cjs +++ b/src/protect/templates/generic-guard.cjs @@ -1,6 +1,6 @@ // Patchstack runtime protection — GENERIC guard (CommonJS). Managed by `patchstack-connect protect`. // Wire whichever helper fits your server into your request path, then run `protect --check`. -const { createProtection } = require("@patchstack/connect/protect"); +const { createProtection, sentinelAnswer, VERIFY_HEADER } = require("@patchstack/connect/protect"); // The fallback bundle is optional at RUNTIME. This file is imported on the app's own module path, so a // throw here is the app failing to boot rather than protection failing open — and a rules file can be // absent for ordinary reasons: a bundler that copied no JSON, a partial deploy, a half-written edit. @@ -66,6 +66,12 @@ function psStepAside(err) { } function protectFetch(handler) { return async (request, ...rest) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. The answer is + // derived from a challenge the verifying process mints per run, which rules out an app matching it + // by accident, and there is nothing to answer unless that process started this one. Without a + // challenge in the environment this is a header read. + const answered = await sentinelAnswer(request.headers.get(VERIFY_HEADER)); + if (answered) return new Response(answered, { status: 200, headers: { "content-type": "text/plain" } }); const active = await getProtection().catch(psStepAside); // The handler, and nothing around it: an exception the app throws is the app's, not a protection // failure, and must not be read as one or answered twice. @@ -92,19 +98,37 @@ function patchstackMiddleware(req, res, next) { passedOn = true; next(err); }; - getProtection().then( - (active) => active.node()(req, res, carryOn), - (err) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered here, + // before the protection is asked for and before the request is passed on, so no handler of the app's + // ever sees a verification request. The answer is derived from a challenge the verifying process mints + // per run, which rules out an app matching it by accident; without a challenge in the environment + // there is nothing to answer and the request is screened as normal. + const screen = () => { + getProtection().then( + (active) => active.node()(req, res, carryOn), + (err) => { + psStepAside(err); + carryOn(); + }, + ).catch((err) => { + // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the + // request is carried on only if it never was — an error here must not take the process down and + // must not answer twice. psStepAside(err); carryOn(); - }, - ).catch((err) => { - // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the - // request is carried on only if it never was — an error here must not take the process down and - // must not answer twice. - psStepAside(err); - carryOn(); - }); + }); + }; + + sentinelAnswer(req.headers?.[VERIFY_HEADER]).then((answered) => { + if (answered) { + res.statusCode = 200; + res.setHeader("content-type", "text/plain"); + res.end(answered); + + return; + } + screen(); + }, screen); } module.exports = { getProtection, protectFetch, patchstackMiddleware }; diff --git a/src/protect/templates/generic-guard.js b/src/protect/templates/generic-guard.js index 6927cd93..1b66850a 100644 --- a/src/protect/templates/generic-guard.js +++ b/src/protect/templates/generic-guard.js @@ -1,7 +1,7 @@ // Patchstack runtime protection — GENERIC guard (ESM). Managed by `patchstack-connect protect`. // Wire whichever helper fits your server into your request path, then run `protect --check`. import { readFileSync } from "node:fs"; -import { createProtection } from "@patchstack/connect/protect"; +import { createProtection, sentinelAnswer, VERIFY_HEADER } from "@patchstack/connect/protect"; // The fallback bundle is optional at RUNTIME. This file is imported on the app's own module path, so a // throw here is the app failing to boot rather than protection failing open — and a rules file can be @@ -68,6 +68,12 @@ function psStepAside(err) { // Web-Fetch: export default { fetch: protectFetch(originalFetch) } export function protectFetch(handler) { return async (request, ...rest) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. The answer is + // derived from a challenge the verifying process mints per run, which rules out an app matching it + // by accident, and there is nothing to answer unless that process started this one. Without a + // challenge in the environment this is a header read. + const answered = await sentinelAnswer(request.headers.get(VERIFY_HEADER)); + if (answered) return new Response(answered, { status: 200, headers: { "content-type": "text/plain" } }); const active = await getProtection().catch(psStepAside); // The handler, and nothing around it: an exception the app throws is the app's, not a protection // failure, and must not be read as one or answered twice. @@ -94,17 +100,35 @@ export function patchstackMiddleware(req, res, next) { passedOn = true; next(err); }; - getProtection().then( - (active) => active.node()(req, res, carryOn), - (err) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered here, + // before the protection is asked for and before the request is passed on, so no handler of the app's + // ever sees a verification request. The answer is derived from a challenge the verifying process mints + // per run, which rules out an app matching it by accident; without a challenge in the environment + // there is nothing to answer and the request is screened as normal. + const screen = () => { + getProtection().then( + (active) => active.node()(req, res, carryOn), + (err) => { + psStepAside(err); + carryOn(); + }, + ).catch((err) => { + // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the + // request is carried on only if it never was — an error here must not take the process down and + // must not answer twice. psStepAside(err); carryOn(); - }, - ).catch((err) => { - // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the - // request is carried on only if it never was — an error here must not take the process down and - // must not answer twice. - psStepAside(err); - carryOn(); - }); + }); + }; + + sentinelAnswer(req.headers?.[VERIFY_HEADER]).then((answered) => { + if (answered) { + res.statusCode = 200; + res.setHeader("content-type", "text/plain"); + res.end(answered); + + return; + } + screen(); + }, screen); } diff --git a/src/protect/templates/generic-guard.ts b/src/protect/templates/generic-guard.ts index adef440f..34c7189d 100644 --- a/src/protect/templates/generic-guard.ts +++ b/src/protect/templates/generic-guard.ts @@ -4,7 +4,7 @@ // whichever helper below fits your server into your request path (see the plan the CLI printed), // then run `patchstack-connect protect --check` to confirm it's hooked up. The engine ships inside // @patchstack/connect — nothing else to install. -import { createProtection } from "@patchstack/connect/protect"; +import { createProtection, sentinelAnswer, VERIFY_HEADER } from "@patchstack/connect/protect"; import fallbackRules from "./rules.json"; // Baked by `patchstack-connect protect` from .patchstackrc.json when available. @@ -76,6 +76,12 @@ function psStepAside(err: unknown) { // Wrap your fetch handler: export default { fetch: protectFetch(originalFetch) } export function protectFetch unknown>(handler: H): H { return (async (request: Request, ...rest: unknown[]) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. The answer is + // derived from a challenge the verifying process mints per run, which rules out an app matching it + // by accident, and there is nothing to answer unless that process started this one. Without a + // challenge in the environment this is a header read. + const answered = await sentinelAnswer(request.headers.get(VERIFY_HEADER)); + if (answered) return new Response(answered, { status: 200, headers: { "content-type": "text/plain" } }); const protection = await getProtection().catch(psStepAside); // The handler, and nothing around it: an exception the app throws is the app's, not a protection // failure, and must not be read as one or answered twice. @@ -103,17 +109,40 @@ export function patchstackMiddleware(req: unknown, res: unknown, next: (err?: un passedOn = true; next(err); }; - getProtection().then( - (protection) => (protection.node() as (a: unknown, b: unknown, c: (e?: unknown) => void) => void)(req, res, carryOn), - (err) => { + // `protect --check --runtime` asks whether a request actually reaches this seam. Answered here, + // before the protection is asked for and before the request is passed on, so no handler of the app's + // ever sees a verification request. The answer is derived from a challenge the verifying process mints + // per run, which rules out an app matching it by accident; without a challenge in the environment + // there is nothing to answer and the request is screened as normal. + const screen = () => { + getProtection().then( + (protection) => (protection.node() as (a: unknown, b: unknown, c: (e?: unknown) => void) => void)(req, res, carryOn), + (err) => { + psStepAside(err); + carryOn(); + }, + ).catch((err) => { + // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the + // request is carried on only if it never was — an error here must not take the process down and + // must not answer twice. psStepAside(err); carryOn(); - }, - ).catch((err) => { - // Not a failed build: the guard, or the app's own chain, threw after this point. Reported, and the - // request is carried on only if it never was — an error here must not take the process down and - // must not answer twice. - psStepAside(err); - carryOn(); - }); + }); + }; + + // The two shapes this seam touches, named rather than asserted wholesale: a request whose headers it + // reads, and a response it answers on. + const inbound = req as { headers?: Record }; + const outbound = res as { statusCode: number; setHeader(name: string, value: string): void; end(body?: string): void }; + + sentinelAnswer(inbound.headers?.[VERIFY_HEADER]).then((answered) => { + if (answered) { + outbound.statusCode = 200; + outbound.setHeader("content-type", "text/plain"); + outbound.end(answered); + + return; + } + screen(); + }, screen); } diff --git a/src/protect/verify-sentinel.js b/src/protect/verify-sentinel.js new file mode 100644 index 00000000..2f2189aa --- /dev/null +++ b/src/protect/verify-sentinel.js @@ -0,0 +1,77 @@ +// The seam side of `protect --check --runtime`. Every other wiring check reads the app's source and +// answers "is the edit present"; this one answers "did a request arrive at the seam". +// +// It is inert unless a verification is running. There is no sentinel without a challenge in the +// environment, and the harness mints a fresh random one per run and passes it only to the child it +// starts. No challenge, or a request offering the wrong value, and this returns null — the seam then +// does what it always does with that request. +// +// The answer is a digest of the challenge rather than an echo of it or a fixed string, which rules out +// the two ways an app could match by accident: a route that happens to return a fixed body, and an app +// that reflects request headers. It is not proof of which module answered. The challenge is an +// environment variable the whole child process can read and `sentinelAnswer` is exported, so any code +// in that process could compute the same value; what the digest establishes is that something holding +// the challenge answered, not that this file did. +export const VERIFY_HEADER = 'x-patchstack-verify'; + +/** The environment variable that carries the challenge for a runtime verification. */ +const CHALLENGE = 'PATCHSTACK_VERIFY_CHALLENGE'; + +/** Read the challenge without assuming a `process` (this file is in the edge-safe graph). */ +function configuredChallenge() { + const value = typeof process !== 'undefined' ? process.env?.[CHALLENGE] : undefined; + + return typeof value === 'string' && value.length >= 32 ? value : null; +} + +/** + * Compare two values without revealing where they first differ. + * + * A type or length mismatch is rejected before the loop; equal-length values are scanned in full. This + * is defence in depth rather than a requirement: the challenge is not a credential, it authorises + * nothing, and it expires with the process. + */ +function sameValue(a, b) { + if (typeof a !== 'string' || typeof b !== 'string' || a.length !== b.length) return false; + let differs = 0; + for (let i = 0; i < a.length; i += 1) differs |= a.charCodeAt(i) ^ b.charCodeAt(i); + + return differs === 0; +} + +/** + * The answer to a verification request, or null when there is nothing to answer. + * + * Null is the answer for every case that is not a live verification: no challenge configured, nothing + * offered, or the wrong value offered. + * + * Web crypto is relied on rather than worked around. A verification only happens in a process the + * harness started, the harness only starts what Node can load directly, and every runtime at this + * package's supported floor (Node 20) has `crypto.subtle`. The guard on an edge runtime never takes + * this branch, because nothing there sets a challenge. A build with no web crypto at all would read to + * the harness as a seam the request did not reach — stated here rather than dressed as a third outcome + * the caller cannot act on. + * + * @param {unknown} offered the value the seam read from the verify header + * @returns {Promise} the body to answer with, or null to carry on as normal + */ +export async function sentinelAnswer(offered) { + const challenge = configuredChallenge(); + if (challenge === null || !sameValue(challenge, typeof offered === 'string' ? offered : null)) return null; + + // Guarded, not because a supported runtime can be missing this, but because a seam on the request + // path may not throw for anything — including a runtime that surprises us. + const subtle = globalThis.crypto?.subtle; + if (!subtle || typeof subtle.digest !== 'function' || typeof TextEncoder === 'undefined') return null; + + try { + const bytes = new TextEncoder().encode(`patchstack-verify:${challenge}`); + const digest = await subtle.digest('SHA-256', bytes); + + return Array.from(new Uint8Array(digest), (b) => b.toString(16).padStart(2, '0')).join(''); + } catch { + // A runtime that has `subtle.digest` and refuses to use it. Nothing to answer with, so nothing is + // claimed: the request goes on to the app exactly as it would have. + return null; + } +} diff --git a/tests/execution-disclosure.test.ts b/tests/execution-disclosure.test.ts new file mode 100644 index 00000000..a672a4e6 --- /dev/null +++ b/tests/execution-disclosure.test.ts @@ -0,0 +1,503 @@ +// Anything in this package that starts a process has to be classified — call site by call site — and +// anything that runs the PROJECT'S OWN code has to be disclosed in the shipped docs. +// +// Agents `npm pack` the tarball and audit it before installing. "This CLI executes your application" is +// the most consequential capability the package has, and one that appeared in `dist/` without a mention +// would read as misrepresentation: it gets installs refused, and the refusal would be correct. +// +// The polarity is inverted deliberately, following the endpoint disclosure test: an exact inventory of +// launch sites is discovered from the SYNTAX, and it must equal the inventory declared below. A new +// `spawn` — including a second one in a file that already has a classified launch — changes the +// inventory and stops the suite until someone writes down what it launches. Files are also gated: a +// file may import `node:child_process` only if it appears in the table, which catches the call forms a +// walk over syntax could still miss (a computed member, a name passed around as a value). +import { describe, expect, it } from 'vitest'; +import { readFileSync, readdirSync, statSync } from 'node:fs'; +import { join, dirname, relative } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import ts from 'typescript'; + +const root = join(dirname(fileURLToPath(import.meta.url)), '..'); + +/** Every function `node:child_process` exports that starts something. */ +const LAUNCHERS = new Set(['spawn', 'spawnSync', 'exec', 'execSync', 'execFile', 'execFileSync', 'fork']); + +/** + * Both specifiers load the same module. A scan that knew only the prefixed one would miss the other, + * and `child_process` is what a lot of existing code is written with. + */ +const MODULE_SPECIFIERS = ['node:child_process', 'child_process']; + +/** The substring every specifier contains, for the cheap reject before the syntax is parsed. */ +const MODULE = 'child_process'; + +interface Site { + file: string; + line: number; + /** The exported name being called, however it was bound locally. */ + call: string; +} + +function sourceFiles(dir: string): string[] { + return readdirSync(dir).flatMap((name) => { + const path = join(dir, name); + if (statSync(path).isDirectory()) return sourceFiles(path); + + return /\.(?:ts|js|cjs|mjs)$/.test(name) ? [path] : []; + }); +} + +/** + * Launch sites in one file, from its syntax. + * + * Both halves matter. First the local names for the module's launchers are collected — a named import + * with or without an alias, a namespace import, `require` destructured with either quote style, a + * dynamic `import()` awaited or `.then`-ed, `require(...).spawn` used inline. Then every call whose + * callee is one of those names, or a property of a namespace bound to the module, is a site. + */ +function launchSitesIn(file: string, source: string): Site[] { + return analyse(file, source).sites; +} + +/** Whether a file really imports the module, as opposed to naming it in a string. */ +export function importsModuleIn(file: string, source: string): boolean { + return analyse(file, source).imports; +} + +function analyse(file: string, source: string): { imports: boolean; sites: Site[] } { + const kind = file.endsWith('.cjs') ? ts.ScriptKind.JS : undefined; + const tree = ts.createSourceFile(file, source, ts.ScriptTarget.Latest, true, kind); + const direct = new Map(); // local name -> exported launcher name + const namespaces = new Set(); // local name bound to the whole module + const sites: Site[] = []; + let imports = false; + const isModule = (node: ts.Node): boolean => + (ts.isStringLiteral(node) || ts.isNoSubstitutionTemplateLiteral(node)) && MODULE_SPECIFIERS.includes(node.text); + + /** `{ spawn, exec: run }` in an import clause or a destructuring pattern. */ + const bindFromPattern = (node: ts.Node): void => { + if (ts.isObjectBindingPattern(node)) { + for (const element of node.elements) { + const exported = element.propertyName ?? element.name; + if (ts.isIdentifier(exported) && ts.isIdentifier(element.name)) direct.set(element.name.text, exported.text); + } + return; + } + if (ts.isNamedImports(node)) { + for (const element of node.elements) direct.set(element.name.text, (element.propertyName ?? element.name).text); + return; + } + if (ts.isIdentifier(node)) namespaces.add(node.text); + }; + + const collectBindings = (node: ts.Node): void => { + // The module reached at all, in any of the forms that actually load it. A string that merely spells + // its name — a fixture, a message, a pattern the map looks for in OTHER people's code — is not this. + if (ts.isImportDeclaration(node) && isModule(node.moduleSpecifier)) imports = true; + if ( + ts.isCallExpression(node) && + node.arguments.length === 1 && + isModule(node.arguments[0]!) && + ((ts.isIdentifier(node.expression) && node.expression.text === 'require') || node.expression.kind === ts.SyntaxKind.ImportKeyword) + ) { + imports = true; + } + // import … from 'node:child_process' + if (ts.isImportDeclaration(node) && isModule(node.moduleSpecifier) && node.importClause) { + const { name, namedBindings } = node.importClause; + if (name) namespaces.add(name.text); + if (namedBindings && ts.isNamespaceImport(namedBindings)) namespaces.add(namedBindings.name.text); + if (namedBindings && ts.isNamedImports(namedBindings)) bindFromPattern(namedBindings); + } + // const … = require('node:child_process') / await import('node:child_process') + if (ts.isVariableDeclaration(node) && node.initializer) { + const init = ts.isAwaitExpression(node.initializer) ? node.initializer.expression : node.initializer; + if (ts.isCallExpression(init) && init.arguments.length === 1 && isModule(init.arguments[0]!)) { + const callee = init.expression; + const requiring = + (ts.isIdentifier(callee) && callee.text === 'require') || callee.kind === ts.SyntaxKind.ImportKeyword; + if (requiring) bindFromPattern(node.name); + } + } + // import('node:child_process').then(({ spawn }) => …) and .then((cp) => …) + if ( + ts.isCallExpression(node) && + ts.isPropertyAccessExpression(node.expression) && + node.expression.name.text === 'then' && + ts.isCallExpression(node.expression.expression) && + node.expression.expression.arguments.length === 1 && + isModule(node.expression.expression.arguments[0]!) + ) { + const [callback] = node.arguments; + if (callback !== undefined && (ts.isArrowFunction(callback) || ts.isFunctionExpression(callback))) { + const [parameter] = callback.parameters; + if (parameter !== undefined) bindFromPattern(parameter.name); + } + } + ts.forEachChild(node, collectBindings); + }; + collectBindings(tree); + + const lineOf = (node: ts.Node): number => tree.getLineAndCharacterOfPosition(node.getStart(tree)).line + 1; + + /** The launcher a member expression names, written either way: `cp.spawn` or `cp['spawn']`. */ + const launcherMember = (callee: ts.Expression): { name: string; receiver: ts.Expression } | null => { + if (ts.isPropertyAccessExpression(callee) && LAUNCHERS.has(callee.name.text)) { + return { name: callee.name.text, receiver: callee.expression }; + } + if (ts.isElementAccessExpression(callee)) { + const key = callee.argumentExpression; + const literal = ts.isStringLiteral(key) || ts.isNoSubstitutionTemplateLiteral(key) ? key.text : null; + if (literal !== null && LAUNCHERS.has(literal)) return { name: literal, receiver: callee.expression }; + } + + return null; + }; + + /** Whether a member expression's receiver is the module itself — a namespace, or an inline require. */ + const throughModule = (receiver: ts.Expression): boolean => + (ts.isIdentifier(receiver) && namespaces.has(receiver.text)) || + (ts.isCallExpression(receiver) && receiver.arguments.length === 1 && isModule(receiver.arguments[0]!)); + + const collectCalls = (node: ts.Node): void => { + if (ts.isCallExpression(node)) { + const callee = node.expression; + const member = launcherMember(callee); + if (ts.isIdentifier(callee) && direct.has(callee.text)) { + sites.push({ file, line: lineOf(node), call: direct.get(callee.text)! }); + } else if (member !== null && throughModule(member.receiver)) { + // `childProcess.spawn(…)`, `childProcess['spawn'](…)`, and the inline `require(…).spawn(…)`. + sites.push({ file, line: lineOf(node), call: member.name }); + } + } + ts.forEachChild(node, collectCalls); + }; + collectCalls(tree); + + /** + * A launcher binding that is not being called, which the walk above cannot classify. + * + * `const launch = spawn` puts a launcher somewhere this scan will not follow, and so does passing one + * as an argument. Neither is a launch by itself, and either can become one out of sight of the + * syntax — so an unclassified use is recorded like a launch is, and has to be written down before the + * suite will pass. The binding site itself is not a use. + */ + const collectEscapes = (node: ts.Node): void => { + const { parent } = node; + const called = parent !== undefined && ts.isCallExpression(parent) && parent.expression === node; + const binding = + parent !== undefined && + ((ts.isImportClause(parent) && parent.name === node) || + (ts.isNamespaceImport(parent) && parent.name === node) || + (ts.isVariableDeclaration(parent) && parent.name === node) || + (ts.isParameter(parent) && parent.name === node)); + const namedMember = + parent !== undefined && + ((ts.isPropertyAccessExpression(parent) && parent.expression === node) || + (ts.isElementAccessExpression(parent) && parent.expression === node)); + + if (ts.isIdentifier(node) && LAUNCHERS.has(direct.get(node.text) ?? '')) { + const declaring = + parent !== undefined && (ts.isImportSpecifier(parent) || ts.isBindingElement(parent) || ts.isNamespaceImport(parent)); + if (!called && !declaring) sites.push({ file, line: lineOf(node), call: `${direct.get(node.text)!} (not called)` }); + } else if (ts.isIdentifier(node) && namespaces.has(node.text) && !binding) { + // A named property can be classified by the member walk below. Handing the whole module away, or + // indexing it dynamically, can put any launcher beyond this scan and must change the inventory. + const member = namedMember ? launcherMember(parent as ts.Expression) : null; + const staticallyNonLauncher = + namedMember && + member === null && + (ts.isPropertyAccessExpression(parent!) || + (ts.isElementAccessExpression(parent!) && + (ts.isStringLiteral(parent!.argumentExpression) || ts.isNoSubstitutionTemplateLiteral(parent!.argumentExpression)))); + if (!namedMember || (!staticallyNonLauncher && member === null)) { + sites.push({ file, line: lineOf(node), call: 'child_process namespace (not called)' }); + } + } else if ( + ts.isCallExpression(node) && + node.arguments.length === 1 && + isModule(node.arguments[0]!) && + ((ts.isIdentifier(node.expression) && node.expression.text === 'require') || node.expression.kind === ts.SyntaxKind.ImportKeyword) + ) { + // The call itself may bind the module, select one statically named member, or feed a `.then` + // callback; those forms are classified elsewhere. Passing the whole result directly, or reaching + // through a dynamic key, can hide any launcher and therefore changes the inventory. + const throughAwait = parent !== undefined && ts.isAwaitExpression(parent) ? parent.parent : parent; + const binds = throughAwait !== undefined && ts.isVariableDeclaration(throughAwait) && throughAwait.initializer !== undefined; + const member = + parent !== undefined && + ((ts.isPropertyAccessExpression(parent) && parent.expression === node) || + (ts.isElementAccessExpression(parent) && parent.expression === node)) + ? parent + : null; + const thenCallback = member !== null && ts.isPropertyAccessExpression(member) && member.name.text === 'then'; + const staticMember = + member !== null && + (ts.isPropertyAccessExpression(member) || + (ts.isElementAccessExpression(member) && + (ts.isStringLiteral(member.argumentExpression) || ts.isNoSubstitutionTemplateLiteral(member.argumentExpression)))); + if (!binds && !thenCallback && !staticMember) { + sites.push({ file, line: lineOf(node), call: 'child_process namespace (not called)' }); + } + } else if (!called && (ts.isPropertyAccessExpression(node) || ts.isElementAccessExpression(node))) { + // The same recognition the call walk uses, applied where the launcher is NOT being called: + // `register(cp.spawn)`, `const launch = cp['spawn']`, `cp.spawn.call(…)`. A launcher reached + // through the module and then handed somewhere else is as unclassifiable as one reached through + // a binding, and a namespace is how an already-allowed file most naturally reaches it. + const member = launcherMember(node); + if (member !== null && throughModule(member.receiver)) { + sites.push({ file, line: lineOf(node), call: `${member.name} (not called)` }); + } + } + ts.forEachChild(node, collectEscapes); + }; + collectEscapes(tree); + + return { imports, sites: sites.sort((a, b) => a.line - b.line) }; +} + +/** Files that reference the module at all, and every launch site in them. */ +function scan(): { importers: string[]; sites: Site[] } { + const importers: string[] = []; + const sites: Site[] = []; + for (const path of sourceFiles(join(root, 'src'))) { + const source = readFileSync(path, 'utf8'); + // A cheap reject first, then the syntax decides: a file that never spells the name cannot import it. + if (!source.includes(MODULE)) continue; + const file = relative(root, path).replace(/\\/g, '/'); + const { imports, sites: found } = analyse(file, source); + if (!imports) continue; + importers.push(file); + sites.push(...found); + } + + return { importers, sites }; +} + +/** + * What each launch site does, and whether it runs code the project wrote. + * + * Keyed by file and by the launcher called, with the number of times. A file may hold more than one + * entry; what it may not do is hold an unlisted launch. + */ +const DECLARED: Array<{ file: string; call: string; times: number; executesProjectCode: boolean; what: string }> = [ + { + file: 'src/protect/install/source-scope.ts', + call: 'execFileSync', + times: 1, + executesProjectCode: false, + what: '`node --check ` parses a file and exits; it never evaluates it', + }, + { + file: 'src/protect/install/runtime/probe.ts', + call: 'spawn', + times: 1, + executesProjectCode: true, + what: "`protect --check --runtime` starts the project's entry to see whether a request reaches the guard", + }, + { + file: 'src/protect/install/runtime/report-listeners.cjs', + call: 'child_process namespace (not called)', + times: 2, + executesProjectCode: false, + what: 'indexes the fixed launcher list to replace each function with the runtime verifier refusal', + }, +]; + +/** Files allowed to reference the module, including ones that launch nothing themselves. */ +const ALLOWED_IMPORTERS: Record = { + 'src/protect/install/source-scope.ts': 'the syntax check above', + 'src/protect/install/runtime/probe.ts': 'the runtime check above', + 'src/protect/install/runtime/report-listeners.cjs': + 'wraps the module in the verification child so a process the app starts is reported; every launch through it is the app’s own call, made with the app’s own arguments', +}; + +const inventory = (sites: Site[]): Record => + sites.reduce>((acc, site) => ({ ...acc, [`${site.file} ${site.call}`]: (acc[`${site.file} ${site.call}`] ?? 0) + 1 }), {}); + +describe('every process this package can start', () => { + it('is discovered by the scan at all — the scan itself has to work', () => { + // Both known sites, so a walk that quietly stops recognising a call form cannot leave the + // assertions below standing on nothing. + expect(inventory(scan().sites)).toMatchObject({ + 'src/protect/install/runtime/probe.ts spawn': 1, + 'src/protect/install/source-scope.ts execFileSync': 1, + }); + }); + + it('matches the declared inventory exactly, site for site', () => { + const declared = DECLARED.reduce>((acc, entry) => ({ ...acc, [`${entry.file} ${entry.call}`]: entry.times }), {}); + expect(inventory(scan().sites)).toEqual(declared); + }); + + it('is only reachable from a file that was allowed to reach it', () => { + const unexpected = scan().importers.filter((file) => ALLOWED_IMPORTERS[file] === undefined); + expect(unexpected).toEqual([]); + }); + + it('has no allowance standing for a file that no longer references it', () => { + // Otherwise the table becomes a list of things that used to be true. + const importers = new Set(scan().importers); + expect(Object.keys(ALLOWED_IMPORTERS).filter((file) => !importers.has(file))).toEqual([]); + }); + + it('says of exactly one site that it runs the project’s own code', () => { + expect(DECLARED.filter((entry) => entry.executesProjectCode).map((entry) => entry.file)).toEqual([ + 'src/protect/install/runtime/probe.ts', + ]); + }); +}); + +describe('the scan recognises the forms a launch can take', () => { + const found = (source: string) => launchSitesIn('probe.ts', source).map((site) => site.call); + + it.each([ + ['a named import', `import { spawn } from 'node:child_process';\nspawn('x');`], + ['an aliased named import', `import { spawn as go } from 'node:child_process';\ngo('x');`], + ['a namespace import', `import * as cp from 'node:child_process';\ncp.execFile('x');`], + ['a default import', `import cp from 'node:child_process';\ncp.fork('x');`], + ['a destructured require', `const { execSync } = require('node:child_process');\nexecSync('x');`], + ['a destructured require in double quotes', `const { execSync } = require("node:child_process");\nexecSync('x');`], + ['an aliased destructured require', `const { spawn: launch } = require('node:child_process');\nlaunch('x');`], + ['a whole-module require', `const cp = require('node:child_process');\ncp.spawnSync('x');`], + ['an inline require', `require('node:child_process').spawn('x');`], + ['a dynamic import', `const { spawn } = await import('node:child_process');\nspawn('x');`], + ['a namespace dynamic import', `const cp = await import('node:child_process');\ncp.spawn('x');`], + ['a call split over lines', `import { spawn } from 'node:child_process';\nspawn(\n 'x',\n [],\n);`], + ['the specifier without the node: prefix', `import { spawn } from 'child_process';\nspawn('x');`], + ['a destructured require without the prefix', `const { execSync } = require('child_process');\nexecSync('x');`], + ['a computed member with a literal key', `import * as cp from 'node:child_process';\ncp['spawn']('x');`], + ['a computed member on an inline require', `require('node:child_process')['exec']('x');`], + ['a destructuring .then callback', `import('node:child_process').then(({ spawn }) => spawn('x'));`], + ['an aliasing .then callback', `import('node:child_process').then(({ spawn: go }) => go('x'));`], + ['a namespace .then callback', `import('node:child_process').then((cp) => cp.fork('x'));`], + ])('finds %s', (_label, source) => { + expect(found(source)).toHaveLength(1); + }); + + it('finds a second launch in a file that already has one', () => { + expect(found(`import { spawn, execFile } from 'node:child_process';\nspawn('a');\nexecFile('b');`)).toEqual(['spawn', 'execFile']); + }); + + it.each([ + ['a computed member', `import * as cp from 'node:child_process';\ncp.spawn('a');\ncp['exec']('b');`, ['spawn', 'exec']], + ['a .then callback', `import { spawn } from 'node:child_process';\nspawn('a');\nimport('node:child_process').then(({ fork }) => fork('b'));`, ['spawn', 'fork']], + ['the unprefixed specifier', `import { spawn } from 'node:child_process';\nspawn('a');\nconst { exec } = require('child_process');\nexec('b');`, ['spawn', 'exec']], + ])('finds a second launch written as %s in a file that already has one', (_label, source, expected) => { + // The file-level allowance lets an already-classified file import the module, so a form the walk + // misses in one of those files is an extra launch that changes no inventory and stops no suite. + expect(found(source)).toEqual(expected); + }); + + it.each([ + ['assigned to another name', `import { spawn } from 'node:child_process';\nconst launch = spawn;\nlaunch('x');`], + ['passed as an argument', `import { spawn } from 'node:child_process';\nregister(spawn);`], + ['put in an object', `const { spawn } = require('node:child_process');\nmodule.exports = { launch: spawn };`], + ])('records a launcher %s, which it cannot classify', (_label, source) => { + // Not a launch by itself, and not something this scan can follow either. Recorded like a launch, so + // it has to be written down before the suite passes. + expect(found(source)).toEqual(['spawn (not called)']); + }); + + it.each([ + ['a namespace member passed as an argument', `import * as cp from 'node:child_process';\nregister(cp.spawn);`], + ['a namespace member assigned away', `import * as cp from 'node:child_process';\nconst launch = cp['spawn'];`], + ['a namespace member invoked through call', `import * as cp from 'node:child_process';\ncp.spawn.call(null, 'x');`], + ['a namespace member invoked through apply', `const cp = require('node:child_process');\ncp.spawn.apply(null, ['x']);`], + ['an inline require member passed along', `register(require('child_process').spawn);`], + ])('records %s, which it cannot classify either', (_label, source) => { + // A namespace is how a file that is already allowed to import the module reaches a launcher, so a + // non-call use through one is exactly the shape that could add a launch without changing the table. + expect(found(source)).toEqual(['spawn (not called)']); + }); + + it.each([ + ['a namespace assigned away', `import * as cp from 'node:child_process';\nconst other = cp;\nother.spawn('x');`], + ['a namespace passed as an argument', `const cp = require('node:child_process');\nregister(cp);`], + ['a namespace indexed dynamically', `import cp from 'node:child_process';\ncp[name]('x');`], + ['an inline module passed as an argument', `register(require('node:child_process'));`], + ])('records %s, which can hide any launcher', (_label, source) => { + expect(found(source)).toEqual(['child_process namespace (not called)']); + }); + + it('does not mistake the binding itself for an unclassified use', () => { + expect(found(`import { spawn } from 'node:child_process';\nspawn('x');`)).toEqual(['spawn']); + expect(found(`import { spawn as go } from 'node:child_process';\ngo('x');`)).toEqual(['spawn']); + expect(found(`const { spawn: go } = require('node:child_process');\ngo('x');`)).toEqual(['spawn']); + }); + + it('does not mistake a same-named function from somewhere else', () => { + expect(found(`import { spawn } from './my-pool.js';\nspawn('x');`)).toEqual([]); + expect(found(`const { exec } = require('./sql.js');\nexec('select 1');`)).toEqual([]); + }); + + it('does not mistake a regular expression for a launch', () => { + expect(found(`import { spawn } from 'node:child_process';\nconst m = /x(y)/.exec('xy');\nspawn('a');`)).toEqual(['spawn']); + }); +}); + +describe('the docs', () => { + const readme = readFileSync(join(root, 'README.md'), 'utf8'); + const agentInstall = readFileSync(join(root, 'AGENT-INSTALL.md'), 'utf8'); + + it('disclose that the runtime check starts the application', () => { + for (const [name, text] of [ + ['README.md', readme], + ['AGENT-INSTALL.md', agentInstall], + ] as const) { + expect(text, `${name} does not mention the flag`).toContain('--check --runtime'); + expect(text, `${name} does not say it starts the app`).toMatch(/START(S)? THE APP|start(s|ing)? the app/i); + } + }); + + it('say which commands do not start it, since that is the question an auditor asks', () => { + expect(readme).toMatch(/No other command runs your application/); + expect(agentInstall).toMatch(/Nothing else runs the application/); + }); + + it('claim only traversal for a pass, in both places', () => { + for (const text of [readme, agentInstall]) { + expect(text).toContain('runtime traversal reached the scaffolded guard seam'); + expect(text).toMatch(/does not say\s+rules were delivered|It does not say\s*\n?rules were delivered/); + } + }); + + it('do not claim the run covers listeners it cannot see', () => { + // The run answers for one process. A claim about "every listener the app opens" would cover a + // listener held by a process it can neither count nor ask. + for (const text of [readme, agentInstall]) { + expect(text).not.toMatch(/every (?:HTTP )?listener/i); + expect(text).toMatch(/one process is the scope|answers for one process/i); + } + }); + + it('name the other two ways a listener can fall outside that scope', () => { + // A worker thread and a late listener are both inside "one process", so the process claim alone + // reads as covering them. Each one ends a run at `2`, and a reader deciding whether to trust a pass + // has to be able to find that out from the documents rather than from the source. + for (const text of [readme, agentInstall]) { + expect(text, 'a worker thread is not mentioned').toMatch(/worker thread/i); + expect(text, 'the discovery window is not mentioned').toMatch(/discovery window/i); + } + }); + + it('disclose that another process is refused rather than started for a partial answer', () => { + for (const [name, text] of [ + ['README.md', readme], + ['AGENT-INSTALL.md', agentInstall], + ] as const) { + expect(text, `${name} does not state the process refusal`).toMatch( + /attempts? to start\s+another process[\s\S]{0,120}(?:launch is\s+)?refused/i, + ); + expect(text, `${name} does not state what the app observes`).toContain('EPERM'); + } + }); + + it('disclose that an inherited NODE_OPTIONS decides whether the run happens at all', () => { + // Node reads it before the command line, so a preload in the environment runs before the listener + // handling. Refusing that is a behaviour of the command, and one an auditor would want stated. + for (const text of [readme, agentInstall]) { + expect(text).toContain('NODE_OPTIONS'); + } + }); +}); diff --git a/tests/protect/a-guard-without-its-fallback-file.test.ts b/tests/protect/a-guard-without-its-fallback-file.test.ts index 3304c7a9..7ba9f2f2 100644 --- a/tests/protect/a-guard-without-its-fallback-file.test.ts +++ b/tests/protect/a-guard-without-its-fallback-file.test.ts @@ -17,6 +17,8 @@ import { pathToFileURL } from 'node:url'; * console that the file was not read. */ const TEMPLATE_DIR = new URL('../../src/protect/templates/', import.meta.url); +/** The seam imports the verification sentinel from the package, so a stubbed package has to carry it. */ +const INERT = "sentinelAnswer: async () => null, VERIFY_HEADER: 'x-patchstack-verify'"; const RUNTIME_TEMPLATES = readdirSync(new URL(TEMPLATE_DIR)).filter((name) => /\.(?:js|cjs)$/.test(name)); const WHOLE_FILE = readFileSync(new URL('rules.json', TEMPLATE_DIR), 'utf8'); @@ -56,7 +58,9 @@ async function loadGuard(name: string, rules: string | null): Promise null;\nexport const VERIFY_HEADER = 'x-patchstack-verify';\n`, ); const source = readFileSync(new URL(name, TEMPLATE_DIR), 'utf8').replace( diff --git a/tests/protect/every-seam-steps-aside.test.ts b/tests/protect/every-seam-steps-aside.test.ts index 3fffa631..27322d47 100644 --- a/tests/protect/every-seam-steps-aside.test.ts +++ b/tests/protect/every-seam-steps-aside.test.ts @@ -149,8 +149,8 @@ async function seamWith(name: string, factory = REJECTS): Promise {}, createServerFnGuard: () => {}, GUARD_PATH: "/x" };\n` - : `export const createProtection = ${factory};\nexport const createSupabaseGuard = () => {};\nexport const createServerFnGuard = () => {};\nexport const GUARD_PATH = "/x";\n`, + ? `module.exports = { createProtection: ${factory}, createSupabaseGuard: () => {}, createServerFnGuard: () => {}, GUARD_PATH: "/x", sentinelAnswer: async () => null, VERIFY_HEADER: "x-patchstack-verify" };\n` + : `export const createProtection = ${factory};\nexport const createSupabaseGuard = () => {};\nexport const createServerFnGuard = () => {};\nexport const GUARD_PATH = "/x";\nexport const sentinelAnswer = async () => null;\nexport const VERIFY_HEADER = "x-patchstack-verify";\n`, ); // The runtime templates READ the file beside them; the TypeScript ones IMPORT it, and a JSON import // needs an attribute Node would demand of the compiled output. So both are provided. diff --git a/tests/protect/fastify-scope.test.ts b/tests/protect/fastify-scope.test.ts index 0c71a921..578d63d0 100644 --- a/tests/protect/fastify-scope.test.ts +++ b/tests/protect/fastify-scope.test.ts @@ -65,6 +65,10 @@ async function loadPlugin(): Promise<(fastify: unknown) => Promise> { 'async function getProtection() { return globalThis.__psTestProtection; }', ); + // The imports this strips include the verification sentinel, so it is supplied inert: none of these + // requests is a verification, and the seam has to behave as it does for ordinary traffic. + const preamble = 'const sentinelAnswer = async () => null;\nconst VERIFY_HEADER = "x-patchstack-verify";\n'; + const protection = await createProtection({ mode: 'block', rules: RULES as never }); protections.push(protection); (globalThis as Record).__psTestProtection = protection; @@ -72,7 +76,7 @@ async function loadPlugin(): Promise<(fastify: unknown) => Promise> { const dir = mkdtempSync(join(tmpdir(), 'ps-fastify-')); dirs.push(dir); const file = join(dir, 'plugin.mjs'); - writeFileSync(file, source); + writeFileSync(file, preamble + source); const mod = (await import(pathToFileURL(file).href)) as { patchstackFastify: (fastify: unknown) => Promise }; diff --git a/tests/protect/install-scope.test.ts b/tests/protect/install-scope.test.ts index 838dedbd..a4d97318 100644 --- a/tests/protect/install-scope.test.ts +++ b/tests/protect/install-scope.test.ts @@ -1,5 +1,5 @@ import { describe, it, expect, afterEach } from 'vitest'; -import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from 'node:fs'; +import { existsSync, mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { @@ -129,6 +129,58 @@ describe('checking the file after editing it', () => { expect(parses(join(dir, 'entry.ts'))).toBeNull(); }); + + it('does not run a preload the environment carries', async () => { + // `node --check` parses and exits without evaluating the file, but it honours `NODE_OPTIONS` like + // any other Node process. A `--require` in the environment would run here, in a process this + // package classifies as one that never evaluates anyone's code — so the code-loading flags are + // stripped out of the environment this child gets. + const dir = project({ 'ok.js': 'const a = 1;\n' }); + const marker = join(dir, 'preload-ran'); + writeFileSync(join(dir, 'boot.cjs'), `require('node:fs').writeFileSync(${JSON.stringify(marker)}, 'yes');\n`); + + const before = process.env.NODE_OPTIONS; + process.env.NODE_OPTIONS = `--require "${join(dir, 'boot.cjs')}"`; + try { + expect(parses(join(dir, 'ok.js'))).toBe(true); + expect(existsSync(marker), 'the structural parse ran a preload').toBe(false); + } finally { + if (before === undefined) delete process.env.NODE_OPTIONS; + else process.env.NODE_OPTIONS = before; + } + }); + + it('does not run a compact preload flag it does not recognise by exact name', () => { + // Short Node flags may carry their operand in the same token. Keeping unknown tokens while removing + // only an exact `-r` would let `-r/path/to/file` execute during a check described as parse-only. + const dir = project({ 'ok.js': 'const a = 1;\n' }); + const marker = join(dir, 'compact-preload-ran'); + writeFileSync(join(dir, 'compact.cjs'), `require('node:fs').writeFileSync(${JSON.stringify(marker)}, 'yes');\n`); + + const before = process.env.NODE_OPTIONS; + process.env.NODE_OPTIONS = `-r${join(dir, 'compact.cjs')}`; + try { + expect(parses(join(dir, 'ok.js'))).toBe(true); + expect(existsSync(marker), 'the structural parse ran a compact preload').toBe(false); + } finally { + if (before === undefined) delete process.env.NODE_OPTIONS; + else process.env.NODE_OPTIONS = before; + } + }); + + it('keeps a recognised flag the environment carries', () => { + // Known-safe options may affect parsing or diagnostics without running code. Unknown options are + // dropped because this package cannot make the same statement about them. + const dir = project({ 'ok.js': 'const a = 1;\n' }); + const before = process.env.NODE_OPTIONS; + process.env.NODE_OPTIONS = '--no-warnings --max-old-space-size=512'; + try { + expect(parses(join(dir, 'ok.js'))).toBe(true); + } finally { + if (before === undefined) delete process.env.NODE_OPTIONS; + else process.env.NODE_OPTIONS = before; + } + }); }); describe('wiring an Express entry', () => { diff --git a/tests/protect/runtime-check-built.test.ts b/tests/protect/runtime-check-built.test.ts new file mode 100644 index 00000000..b8e08922 --- /dev/null +++ b/tests/protect/runtime-check-built.test.ts @@ -0,0 +1,241 @@ +// `protect --check --runtime` driven the way a consumer drives it: the BUILT CLI, in a project whose +// guard came from the built scaffolder and whose seam is the built `@patchstack/connect/protect`. +// +// The source tests cover the probe, the launch contract and the seam. None of them can see the questions +// this answers: is the listener reporter where the built CLI looks for it, does the scaffolded guard's +// seam answer through the published module, and does the command exit with the code its documentation +// claims. A rename or a missed copy step leaves every source test green and this check is what fails. +// +// The middle case is the reason the whole feature exists. Its app imports the guard, calls it, and passes +// the structural check — while the server that actually listens has no guard on it. `--check` says wired; +// only a request can say otherwise. +import { execFileSync, spawn, spawnSync } from 'node:child_process'; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { afterEach, describe, expect, it } from 'vitest'; + +// Every case here starts a real process, so every case declares a timeout that fits the run it performs. +// The default five seconds is shorter than the runs below allow, and a test vitest abandons mid-run is +// not just a failure: the awaited promise is dropped, its cleanup never happens, and the app it started +// is left on the machine. Long enough for a loaded CI runner, and each run bounds itself anyway. +const SLOW = { timeout: 60_000 }; + + +const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..'); +const cliPath = path.join(root, 'dist', 'cli.js'); + +// `dist/` is gitignored and built on publish, so a plain checkout has nothing to drive. In CI the +// opposite holds: the post-build step sets `PS_REQUIRE_RUNTIME_CHECK` and a skip there would read exactly +// like a pass — which is the same defect this feature exists to remove. +const built = existsSync(cliPath); +const required = process.env.PS_REQUIRE_RUNTIME_CHECK === '1'; + +if (required && !built) { + throw new Error( + `PS_REQUIRE_RUNTIME_CHECK=1 but ${cliPath} does not exist — this check runs after the build. ` + + 'Refusing to skip, because a skipped verification check reads exactly like a passing one.', + ); +} + +let dirs: string[] = []; +afterEach(() => { + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + dirs = []; +}); + +/** A project with the built package linked in, scaffolded by the built CLI. */ +function scaffolded(files: Record): string { + const dir = mkdtempSync(path.join(tmpdir(), 'ps-runtime-built-')); + dirs.push(dir); + mkdirSync(path.join(dir, 'node_modules', '@patchstack'), { recursive: true }); + // The repository itself: `dist/protect.js` is what the scaffolded guard imports, and what a consumer + // would have installed. + symlinkSync(root, path.join(dir, 'node_modules', '@patchstack', 'connect'), 'dir'); + for (const [name, body] of Object.entries(files)) { + const file = path.join(dir, name); + mkdirSync(path.dirname(file), { recursive: true }); + writeFileSync(file, body); + } + execFileSync(process.execPath, [cliPath, 'protect'], { cwd: dir, stdio: 'ignore' }); + + return dir; +} + +const check = (cwd: string, ...flags: string[]) => { + const result = spawnSync(process.execPath, [cliPath, 'protect', '--check', ...flags], { cwd, encoding: 'utf8' }); + + return { code: result.status, out: `${result.stdout}${result.stderr}` }; +}; + +const manifest = JSON.stringify({ name: 'app', private: true, type: 'module', scripts: { start: 'node server.js' } }); + +/** A wired app that records the fact it was loaded, so "did this command run it" is answerable. */ +const STARTS_AND_MARKS = `import { writeFileSync } from 'node:fs'; +import { createServer } from 'node:http'; +import { patchstackMiddleware } from './patchstack/guard.js'; + +writeFileSync(new URL('./started', import.meta.url), 'the app ran'); +createServer((req, res) => { + patchstackMiddleware(req, res, () => { res.writeHead(200); res.end('app'); }); +}).listen(3000);`; + +describe.skipIf(!built)('the built CLI', SLOW, () => { + it('proves traversal for an app whose serving path holds the guard', () => { + const dir = scaffolded({ + 'package.json': manifest, + 'server.js': `import { createServer } from 'node:http'; +import { patchstackMiddleware } from './patchstack/guard.js'; + +createServer((req, res) => { + patchstackMiddleware(req, res, () => { res.writeHead(200); res.end('app'); }); +}).listen(3000);`, + }); + expect(check(dir).code).toBe(0); // structurally wired + + const runtime = check(dir, '--runtime'); + expect(runtime.out).toContain('runtime traversal reached the scaffolded guard seam'); + expect(runtime.out).toContain('entry: server.js (from the "start" script)'); + expect(runtime.code).toBe(0); + }); + + it('catches an app that passes the structural check and never routes a request through the guard', () => { + const dir = scaffolded({ + 'package.json': manifest, + 'server.js': `import { createServer } from 'node:http'; +import { patchstackMiddleware } from './patchstack/guard.js'; + +// Imported, called, and on a server that never listens. +const guarded = createServer((req, res) => { + patchstackMiddleware(req, res, () => { res.writeHead(200); res.end('guarded'); }); +}); + +// The server that actually serves traffic, wired by nobody. +createServer((req, res) => { res.writeHead(200); res.end('unguarded'); }).listen(3000);`, + }); + expect(check(dir).code).toBe(0); // the source says wired, and it is not + + const runtime = check(dir, '--runtime'); + expect(runtime.out).toContain('the listener answered and the guard seam did not'); + expect(runtime.code).toBe(1); + }); + + it('reports an entry it must not start as neither passed nor failed', () => { + const dir = scaffolded({ + 'package.json': JSON.stringify({ name: 'app', private: true, type: 'module', scripts: { start: 'tsx server.ts' } }), + 'server.ts': `import { createServer } from 'node:http'; +import { patchstackMiddleware } from './patchstack/guard.js'; + +createServer((req: unknown, res: unknown) => { + patchstackMiddleware(req, res, () => {}); +}).listen(3000);`, + }); + const runtime = check(dir, '--runtime'); + expect(runtime.out).toContain('runtime traversal could not be established'); + expect(runtime.out).toContain('This is not a failure.'); + expect(runtime.code).toBe(2); + }); + + it('does not start an app whose guard is not wired at all', () => { + // The structural verdict comes first: there is nothing to learn from starting an app that has no + // guard on any path, and starting one to say so would run the app for no reason. + const dir = scaffolded({ + 'package.json': manifest, + 'server.js': `import { writeFileSync } from 'node:fs'; +import { createServer } from 'node:http'; + +writeFileSync(new URL('./started', import.meta.url), 'the app ran'); +createServer((req, res) => { res.writeHead(200); res.end('unguarded'); }).listen(3000);`, + }); + const structural = check(dir); + expect(structural.code).toBe(1); + + const runtime = check(dir, '--runtime'); + expect(runtime.code).toBe(1); + expect(runtime.out).not.toContain('runtime traversal'); + expect(existsSync(path.join(dir, 'started'))).toBe(false); + }); + + it('does not leave the app running when the command is interrupted', async () => { + // The app is a detached process-group leader, so a Ctrl-C that ends the CLI mid-run must explicitly + // end that group rather than leave the app running on the machine. + const dir = scaffolded({ + 'package.json': manifest, + 'server.js': `import { writeFileSync } from 'node:fs'; +import { createServer } from 'node:http'; +import { patchstackMiddleware } from './patchstack/guard.js'; + +const server = createServer((req, res) => { + patchstackMiddleware(req, res, () => { res.writeHead(200); res.end('app'); }); +}); +server.listen(3000, () => { + // Its own process group id: the CLI put it in one, and it is what must be gone afterwards. + writeFileSync(new URL('./started', import.meta.url), String(process.pid)); +});`, + }); + + const cli = spawn(process.execPath, [cliPath, 'protect', '--check', '--runtime'], { cwd: dir, stdio: 'ignore' }); + const pidFile = path.join(dir, 'started'); + const deadline = Date.now() + 20_000; + while (!existsSync(pidFile) && Date.now() < deadline) await new Promise((resolve) => setTimeout(resolve, 25)); + expect(existsSync(pidFile), 'the app never started, so nothing was interrupted').toBe(true); + const appPid = Number(readFileSync(pidFile, 'utf8')); + + cli.kill('SIGINT'); + await new Promise((resolve) => cli.on('exit', () => resolve())); + + // Both the app and its group, polled: whoever reaps it decides when the pid disappears. + const gone = async (pid: number) => { + const until = Date.now() + 5_000; + for (;;) { + try { + process.kill(pid, 0); + } catch { + return true; + } + if (Date.now() > until) return false; + await new Promise((resolve) => setTimeout(resolve, 25)); + } + }; + expect(await gone(appPid)).toBe(true); + expect(await gone(-appPid)).toBe(true); + }); + + it('refuses --runtime without --check rather than scaffolding quietly', () => { + const dir = scaffolded({ 'package.json': manifest, 'server.js': '' }); + const result = spawnSync(process.execPath, [cliPath, 'protect', '--runtime'], { cwd: dir, encoding: 'utf8' }); + expect(`${result.stdout}${result.stderr}`).toContain('--runtime only applies to --check'); + expect(result.status).toBe(1); + }); + + it('does not start the app for any other command', () => { + // Same marker as above. `setup` and `guide` may fail here for want of a credential, and that is not + // what is being asserted: whatever they do, they must not have run the application to do it. + const dir = scaffolded({ 'package.json': manifest, 'server.js': STARTS_AND_MARKS }); + for (const command of ['setup', 'guide']) { + // A dead endpoint: `setup` posts a manifest, and this test is about what the command runs + // locally, not about reaching Patchstack. It fails fast, which is fine — the assertion is that + // whatever it did, it did not start the application. + spawnSync(process.execPath, [cliPath, command, '--endpoint', 'http://127.0.0.1:1'], { + cwd: dir, + encoding: 'utf8', + timeout: 60_000, + }); + expect(existsSync(path.join(dir, 'started')), `\`${command}\` started the application`).toBe(false); + } + }); + + it('does not start the app for a default check', () => { + // The app writes a file the moment it is loaded. A default `--check` that ever runs it would leave + // that file behind, and this is the assertion that the opt-in stays opt-in. + const dir = scaffolded({ 'package.json': manifest, 'server.js': STARTS_AND_MARKS }); + expect(check(dir).code).toBe(0); + expect(existsSync(path.join(dir, 'started'))).toBe(false); + + // And the same app under `--runtime` does start, so the assertion above is about the flag and not + // about an app that never runs at all. + expect(check(dir, '--runtime').code).toBe(0); + expect(existsSync(path.join(dir, 'started'))).toBe(true); + }); +}); diff --git a/tests/protect/runtime-check.test.ts b/tests/protect/runtime-check.test.ts new file mode 100644 index 00000000..8557c9ca --- /dev/null +++ b/tests/protect/runtime-check.test.ts @@ -0,0 +1,144 @@ +// The three answers the runtime check gives, the codes they exit with, and the lines they print. +// +// The middle answer is the one under test everywhere here: `unavailable` is neither a pass nor a +// failure, and the whole check is worth nothing if a caller can read it as either. So the exit code is +// its own, the printed lines say so in words, and the reason is never omitted. +import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, describe, expect, it } from 'vitest'; + +// Every case here starts a real process, so every case declares a timeout that fits the run it performs. +// The default five seconds is shorter than the runs below allow, and a test vitest abandons mid-run is +// not just a failure: the awaited promise is dropped, its cleanup never happens, and the app it started +// is left on the machine. Long enough for a loaded CI runner, and each run bounds itself anyway. +const SLOW = { timeout: 60_000 }; + + +import { + formatRuntimeCheck, + runRuntimeCheck, + runtimeExitCode, + type RuntimeCheckReport, +} from '../../src/protect/install/runtime/check.js'; + +let dirs: string[] = []; +afterEach(() => { + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + dirs = []; +}); + +function project(files: Record): string { + const dir = mkdtempSync(join(tmpdir(), 'ps-runtime-check-')); + dirs.push(dir); + for (const [name, body] of Object.entries(files)) { + const path = join(dir, name); + mkdirSync(join(path, '..'), { recursive: true }); + writeFileSync(path, body); + } + + return dir; +} + +const report = (over: Partial): RuntimeCheckReport => ({ outcome: 'proven', listeners: [], output: '', ...over }); + +describe('an entry the check must not start', SLOW, () => { + it('is unavailable, with the resolver’s reason and nothing launched', async () => { + const dir = project({ + 'package.json': JSON.stringify({ name: 'app', scripts: { start: 'tsx server.ts' } }), + 'server.ts': 'throw new Error("this must never run");', + }); + const result = await runRuntimeCheck(dir); + expect(result).toEqual({ outcome: 'unavailable', reason: expect.stringContaining('outside the bounded'), listeners: [], output: '' }); + expect(result.reason).not.toContain('tsx server.ts'); + expect(result.entry).toBeUndefined(); + }); + + it('names the entry it did use, and where it came from', async () => { + const dir = project({ + 'package.json': JSON.stringify({ name: 'app', type: 'module', scripts: { start: 'node quiet.mjs' } }), + 'quiet.mjs': 'console.log("no listener here");', + }); + const result = await runRuntimeCheck(dir, { timeoutMs: 4_000, settleMs: 300 }); + expect(result.entry).toEqual({ file: 'quiet.mjs', from: 'the "start" script' }); + expect(result.outcome).toBe('unavailable'); + }); +}); + +describe('the exit code', SLOW, () => { + it('is 0 only for a proven traversal', () => { + expect(runtimeExitCode(report({ outcome: 'proven' }))).toBe(0); + }); + + it('is 1 when a listener answered and the seam did not', () => { + expect(runtimeExitCode(report({ outcome: 'not-traversed' }))).toBe(1); + }); + + it('is 2 when nothing could be established — its own code, not either neighbour', () => { + expect(runtimeExitCode(report({ outcome: 'unavailable' }))).toBe(2); + }); +}); + +describe('what the command prints', SLOW, () => { + const printed = (over: Partial) => formatRuntimeCheck(report(over)).join('\n'); + + it('states the claim in the one form it is allowed to take', () => { + const out = printed({ + entry: { file: 'dist/server.js', from: 'the "start" script' }, + listeners: [{ scheme: 'http', host: '127.0.0.1', port: 5000, outcome: 'traversed' }], + }); + expect(out).toContain('entry: dist/server.js (from the "start" script)'); + expect(out).toContain('✓ http://127.0.0.1:5000 — runtime traversal reached the scaffolded guard seam'); + expect(out).toContain('runtime traversal reached the scaffolded guard seam ✓'); + // Not a claim this check can make. + expect(out).not.toMatch(/protected|blocked|rules/i); + }); + + it('prints the address that was actually bound', () => { + const out = printed({ listeners: [{ scheme: 'http', host: '::1', port: 7000, outcome: 'traversed' }] }); + expect(out).toContain('http://[::1]:7000'); + }); + + it('says which listener answered instead of the guard', () => { + const out = printed({ + outcome: 'not-traversed', + listeners: [ + { scheme: 'http', host: '127.0.0.1', port: 5000, outcome: 'traversed' }, + { scheme: 'https', host: '127.0.0.1', port: 5001, outcome: 'answered-without-sentinel' }, + ], + }); + expect(out).toContain('✓ http://127.0.0.1:5000'); + expect(out).toContain('✗ https://127.0.0.1:5001 — the listener answered and the guard seam did not'); + expect(out).toContain('did NOT reach the scaffolded guard seam ✗'); + }); + + it('marks an unreachable listener with its detail, not as a failure to traverse', () => { + const out = printed({ + outcome: 'unavailable', + reason: 'a listener could not be reached', + listeners: [{ scheme: 'https', host: '127.0.0.1', port: 5001, outcome: 'unreachable', detail: 'socket hang up' }], + }); + expect(out).toContain('? https://127.0.0.1:5001 — socket hang up'); + }); + + it('always gives a reason for an answer it could not establish, and says it is not a failure', () => { + const out = printed({ outcome: 'unavailable', reason: 'the "start" script runs `next start`' }); + expect(out).toContain('? runtime traversal could not be established — the "start" script runs `next start`'); + expect(out).toContain('This is not a failure.'); + }); + + it('shows what the app printed when the answer was not a pass', () => { + expect(printed({ outcome: 'unavailable', reason: 'x', output: 'Error: EADDRINUSE' })).toContain(' Error: EADDRINUSE'); + }); + + it('does not pad a pass with the app’s own log output', () => { + const out = printed({ listeners: [{ scheme: 'http', host: '127.0.0.1', port: 1, outcome: 'traversed' }], output: 'listening on 3000' }); + expect(out).not.toContain('listening on 3000'); + }); + + it('caps how much of the app’s output it repeats', () => { + const out = printed({ outcome: 'unavailable', reason: 'x', output: Array.from({ length: 200 }, (_, i) => `line ${i}`).join('\n') }); + expect(out).toContain(' line 19'); + expect(out).not.toContain(' line 20'); + }); +}); diff --git a/tests/protect/runtime-entry.test.ts b/tests/protect/runtime-entry.test.ts new file mode 100644 index 00000000..6b3943f2 --- /dev/null +++ b/tests/protect/runtime-entry.test.ts @@ -0,0 +1,285 @@ +// What the runtime check will and will not launch. +// +// The negatives are the point. This resolver decides what a verification command executes in somebody +// else's repository, so every case that must come back `unavailable` is written down: TypeScript +// entries, framework launchers, anything reached through a package manager, and anything outside the +// project. A resolver that reached for one more of those to raise its hit rate would be running code +// the author never asked it to run. +import { mkdtempSync, mkdirSync, symlinkSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { basename, join } from 'node:path'; +import { afterEach, describe, expect, it } from 'vitest'; + +import { resolveEntry } from '../../src/protect/install/runtime/entry.js'; + +let dirs: string[] = []; +afterEach(() => { + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + dirs = []; +}); + +function project(files: Record): string { + const dir = mkdtempSync(join(tmpdir(), 'ps-runtime-entry-')); + dirs.push(dir); + for (const [name, body] of Object.entries(files)) { + const path = join(dir, name); + mkdirSync(join(path, '..'), { recursive: true }); + writeFileSync(path, body); + } + + return dir; +} + +const pkg = (fields: Record) => JSON.stringify({ name: 'app', ...fields }); + +describe('an entry it will launch', () => { + it('takes the start script when it is a plain node invocation', () => { + const dir = project({ 'package.json': pkg({ scripts: { start: 'node dist/server.js' } }), 'dist/server.js': '', 'server.js': '' }); + expect(resolveEntry(dir)).toEqual({ + kind: 'entry', + file: join(dir, 'dist/server.js'), + nodeArgs: [], + appArgs: [], + from: 'the "start" script', + }); + }); + + it('passes the script’s own node flags through', () => { + const dir = project({ 'package.json': pkg({ scripts: { start: 'node --enable-source-maps app.js' } }), 'app.js': '' }); + expect(resolveEntry(dir)).toMatchObject({ kind: 'entry', nodeArgs: ['--enable-source-maps'] }); + }); + + it('falls back to the package main field', () => { + const dir = project({ 'package.json': pkg({ main: 'lib/entry.mjs' }), 'lib/entry.mjs': '' }); + expect(resolveEntry(dir)).toMatchObject({ kind: 'entry', file: join(dir, 'lib/entry.mjs'), from: 'the package "main" field' }); + }); + + it('falls back to a conventional entry, naming which one', () => { + const dir = project({ 'package.json': pkg({}), 'src/index.js': '' }); + expect(resolveEntry(dir)).toMatchObject({ kind: 'entry', file: join(dir, 'src/index.js'), from: 'src/index.js' }); + }); + + it('passes a self-contained flag through', () => { + const dir = project({ 'package.json': pkg({ scripts: { start: 'node --max-old-space-size=4096 app.js' } }), 'app.js': '' }); + expect(resolveEntry(dir)).toMatchObject({ kind: 'entry', file: join(dir, 'app.js'), nodeArgs: ['--max-old-space-size=4096'] }); + }); + + it('follows a symlink that stays inside the project', () => { + const dir = project({ 'package.json': pkg({ scripts: { start: 'node server.js' } }), 'src/real.js': '' }); + symlinkSync(join(dir, 'src/real.js'), join(dir, 'server.js')); + expect(resolveEntry(dir)).toMatchObject({ kind: 'entry', file: join(dir, 'server.js') }); + }); + + it('passes the app’s own arguments through, in order', () => { + const dir = project({ 'package.json': pkg({ scripts: { start: 'node server.js --port 8080' } }), 'server.js': '' }); + expect(resolveEntry(dir)).toMatchObject({ kind: 'entry', file: join(dir, 'server.js'), appArgs: ['--port', '8080'] }); + }); + + it('prefers start over serve when a project has both', () => { + const dir = project({ + 'package.json': pkg({ scripts: { start: 'node start.js', serve: 'node serve.js' } }), + 'start.js': '', + 'serve.js': '', + }); + expect(resolveEntry(dir)).toMatchObject({ file: join(dir, 'start.js'), from: 'the "start" script' }); + }); + + it('reads serve only when there is no start script', () => { + const dir = project({ 'package.json': pkg({ scripts: { serve: 'node run.cjs' } }), 'run.cjs': '' }); + expect(resolveEntry(dir)).toMatchObject({ kind: 'entry', from: 'the "serve" script' }); + }); +}); + +describe('what it refuses to launch', () => { + const refuses = (files: Record, matching: RegExp) => { + const result = resolveEntry(project(files)); + expect(result.kind).toBe('unavailable'); + expect(result.kind === 'unavailable' && result.reason).toMatch(matching); + }; + + it('will not run a framework launcher', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'next start' } }), 'server.js': '' }, /outside the bounded/); + }); + + it('will not run a TypeScript entry through a loader', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'tsx server.ts' } }), 'server.ts': '' }, /own toolchain/); + }); + + it('will not run a TypeScript entry named as node would not load it', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node server.ts' } }), 'server.ts': '' }, /own toolchain/); + }); + + it('will not run another runtime', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'bun server.js' } }), 'server.js': '' }, /outside the bounded/); + }); + + it('will not run a watcher that would restart the app under it', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'nodemon server.js' } }), 'server.js': '' }, /outside the bounded/); + }); + + it('will not half-run a compound command', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node server.js && echo started' } }), 'server.js': '' }, /own toolchain/); + }); + + it('will not start a REPL', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node --experimental-repl-await' } }) }, /own toolchain/); + }); + + it('will not reach a sibling directory whose name starts with the project’s', () => { + // The sibling is named so that a bare prefix comparison against the project root accepts it, and it + // holds a real file, so nothing else in the resolver would refuse it. + const dir = project({}); + const sibling = `${dir}-staging`; + mkdirSync(sibling, { recursive: true }); + writeFileSync(join(sibling, 'server.js'), ''); + writeFileSync(join(dir, 'package.json'), pkg({ scripts: { start: `node ../${basename(sibling)}/server.js` } })); + try { + expect(resolveEntry(dir).kind).toBe('unavailable'); + } finally { + rmSync(sibling, { recursive: true, force: true }); + } + }); + + it('will not read a flag’s operand as the entry file', () => { + // `--require` takes the NEXT token, so a whitespace split reads `boot.cjs` as the entry and would + // launch a different program than the script describes. Both forms are refused, and for the same + // reason a preload runs project code before the listener reporter is in place. + for (const start of ['node --require ./boot.cjs server.js', 'node -r ./boot.cjs server.js', 'node --require=./boot.cjs server.js']) { + const result = resolveEntry(project({ 'package.json': pkg({ scripts: { start } }), 'boot.cjs': '', 'server.js': '' })); + expect(result.kind, start).toBe('unavailable'); + } + }); + + it('will not let a flag replace the entry it is about to report', () => { + // `--eval` takes the program out of the file named on the command line: node runs the string and + // leaves `server.js` as an ordinary argument. So this script starts `alternate.js` while the file + // the resolver would report is `server.js`. A pass reported against an entry that never ran is the + // worst answer this command can give, so a flag that can do this is not passable in any form. + const start = 'node --eval=import(process.argv[2]) server.js ./alternate.js'; + const result = resolveEntry(project({ 'package.json': pkg({ scripts: { start } }), 'server.js': '', 'alternate.js': '' })); + expect(result.kind).toBe('unavailable'); + }); + + it('will not run a valued flag that evaluates or prints a program', () => { + for (const start of [ + 'node --eval=0 server.js', + 'node --print=0 server.js', + 'node -e=0 server.js', + 'node -p=0 server.js', + 'node --import=./boot.mjs server.js', + 'node --experimental-loader=./loader.mjs server.js', + ]) { + const result = resolveEntry(project({ 'package.json': pkg({ scripts: { start } }), 'server.js': '', 'boot.mjs': '', 'loader.mjs': '' })); + expect(result.kind, start).toBe('unavailable'); + } + }); + + it('will not pass on a valued flag it does not recognise', () => { + // The allowlist is the whole guard: a `--name=value` this has never heard of may do anything, + // including changing which file Node treats as the program. + for (const start of ['node --experimental-policy=./p.json server.js', 'node --env-file=.env server.js', 'node --made-up=1 server.js']) { + const result = resolveEntry(project({ 'package.json': pkg({ scripts: { start } }), 'server.js': '', '.env': '', 'p.json': '' })); + expect(result.kind, start).toBe('unavailable'); + } + }); + + it('will not run a flag it cannot tell takes an operand', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node --experimental-specifier-resolution node server.js' } }), 'server.js': '' }, /own toolchain/); + }); + + it('will not open a debugger port', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node --inspect server.js' } }), 'server.js': '' }, /own toolchain/); + }); + + it('will not read past a bare `--`', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node -- server.js' } }), 'server.js': '' }, /own toolchain/); + }); + + it('will not guess at shell syntax it is not evaluating', () => { + for (const start of [ + 'node "my server.js"', + 'node $ENTRY', + 'node dist/*.js', + 'node dist/[ab].js', + 'node ${ENTRY}', + 'node server.js # the shell ignores this', + 'node server.js one\\ argument', + 'node server.js (one)', + 'node server.js ~/config.json', + 'node server.js\nnode alternate.js', + ]) { + const result = resolveEntry( + project({ 'package.json': pkg({ scripts: { start } }), 'server.js': '', 'alternate.js': '', 'my server.js': '' }), + ); + expect(result.kind, start).toBe('unavailable'); + } + }); + + it('will not follow a symlink out of the project', () => { + // The written path is inside the project; the file that would run is not. Only the real paths say so. + const dir = project({ 'package.json': pkg({ scripts: { start: 'node server.js' } }) }); + const outside = mkdtempSync(join(tmpdir(), 'ps-runtime-entry-outside-')); + dirs.push(outside); + writeFileSync(join(outside, 'target.js'), ''); + symlinkSync(join(outside, 'target.js'), join(dir, 'server.js')); + expect(resolveEntry(dir).kind).toBe('unavailable'); + }); + + it('will not go through a package manager', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'npm run serve' } }), 'server.js': '' }, /outside the bounded/); + }); + + it('does not copy an unavailable start command into its diagnostic', () => { + const secret = 'ps-secret-5d2a9f'; + const dir = project({ + 'package.json': pkg({ scripts: { start: `node --made-up=${secret} server.js` } }), + 'server.js': '', + }); + const result = resolveEntry(dir); + expect(result.kind).toBe('unavailable'); + expect(result.kind === 'unavailable' && result.reason).not.toContain(secret); + }); + + it('will not run a script that builds first', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'npm run build && node dist/index.js' } }), 'dist/index.js': '' }, /own toolchain/); + }); + + it('will not run a script wrapped in an environment shim', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'cross-env NODE_ENV=production node app.js' } }), 'app.js': '' }, /outside the bounded/); + }); + + it('will not fall back past a declared script it cannot run', () => { + // `server.js` exists and would otherwise be launched. The project said it runs Next; that is the answer. + refuses({ 'package.json': pkg({ scripts: { start: 'next start' } }), 'server.js': '', 'src/index.js': '' }, /outside the bounded/); + }); + + it('will not follow a start script out of the project', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node ../elsewhere/server.js' } }) }, /own toolchain/); + }); + + it('will not follow an absolute path', () => { + refuses({ 'package.json': pkg({ scripts: { start: 'node /usr/local/bin/thing.js' } }) }, /own toolchain/); + }); + + it('will not launch a directory that only looks like an entry', () => { + // `main: "."` and `node .` both resolve through package semantics rather than to a file. + refuses({ 'package.json': pkg({ main: '.', scripts: { start: 'node .' } }) }, /own toolchain/); + }); + + it('says so when the project has no loadable entry at all', () => { + refuses({ 'package.json': pkg({}), 'src/index.ts': '' }, /declare one as a "start" script/); + }); + + it('reads a directory with no package.json as the wrong directory', () => { + refuses({ 'src/index.ts': '' }, /no package.json/); + }); + + it('still launches a loadable entry when there is no package.json', () => { + // The manifest is how a project DECLARES its entry; it is not what makes a file loadable. + expect(resolveEntry(project({ 'server.js': '' }))).toMatchObject({ kind: 'entry', from: 'server.js' }); + }); + + it('says so when package.json cannot be read as JSON', () => { + refuses({ 'package.json': '{ not json', 'src/index.ts': '' }, /declare one as a "start" script/); + }); +}); diff --git a/tests/protect/runtime-listener-reporter.test.ts b/tests/protect/runtime-listener-reporter.test.ts new file mode 100644 index 00000000..ee24486b --- /dev/null +++ b/tests/protect/runtime-listener-reporter.test.ts @@ -0,0 +1,459 @@ +// The listener reporter, on its own, in a real child process. +// +// It is the half of the runtime check that changes the application: it decides which listeners may bind +// and where, and it is the only thing standing between a verification run and a server opened on a +// public interface. So its boundary is pinned here rather than through the harness — a run driven from +// the parent kills the child as soon as it hears a refusal, which leaves no time to observe whether the +// refusal held. Here the child outlives the answer and says what happened. +import { spawn } from 'node:child_process'; +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, describe, expect, it } from 'vitest'; + +// Every case here starts a real process, so every case declares a timeout that fits the run it performs. +// The default five seconds is shorter than the runs below allow, and a test vitest abandons mid-run is +// not just a failure: the awaited promise is dropped, its cleanup never happens, and the app it started +// is left on the machine. Long enough for a loaded CI runner, and each run bounds itself anyway. +const SLOW = { timeout: 60_000 }; + +import { preloadPath } from '../../src/protect/install/runtime/probe.js'; + + +const preload = preloadPath(); + +let dirs: string[] = []; +afterEach(() => { + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + dirs = []; +}); + +interface Note { + patchstackVerify?: string; + scheme?: string; + host?: string; + port?: number; + why?: string; + via?: string; + what?: string; + node?: boolean; + escape?: string; + pid?: number; +} + +/** Run one script under the reporter and collect what it reported and what it printed. */ +async function under(script: string, alongside: Record = {}): Promise<{ notes: Note[]; out: string; dir: string }> { + const dir = mkdtempSync(join(tmpdir(), 'ps-reporter-')); + dirs.push(dir); + writeFileSync(join(dir, 'case.cjs'), script); + for (const [name, body] of Object.entries(alongside)) writeFileSync(join(dir, name), body); + + return new Promise((resolve) => { + const child = spawn(process.execPath, ['--require', preload!, join(dir, 'case.cjs')], { + cwd: dir, + detached: true, + stdio: ['ignore', 'pipe', 'pipe', 'ipc'], + // The reporter in `NODE_OPTIONS` as well as on the command line, which is how the harness + // launches an app. The reporter reads that variable to decide whether a worker the app starts + // would still load it, so a case run without it would not be running the real thing. + env: { ...process.env, NODE_OPTIONS: `--require "${preload!}"` }, + }); + const notes: Note[] = []; + let out = ''; + child.on('message', (note) => notes.push(note as Note)); + child.stdout?.setEncoding('utf8'); + child.stderr?.setEncoding('utf8'); + child.stdout?.on('data', (chunk: string) => (out += chunk)); + child.stderr?.on('data', (chunk: string) => (out += chunk)); + // Each case prints its verdict and exits; the timer is only so a hung case cannot hang the suite. + const timer = setTimeout(() => { + try { + process.kill(-child.pid!, 'SIGKILL'); + } catch { + /* already gone */ + } + }, 6_000); + child.on('exit', () => { + clearTimeout(timer); + resolve({ notes, out, dir }); + }); + }); +} + +/** The shape every unsupported case uses: bind, then report what became of the server. */ +const REFUSAL_CASE = (create: string, listen: string) => `'use strict'; +const server = ${create}; +let listening = false; +server.on('listening', () => { listening = true; }); +server.on('error', (err) => { console.log('error ' + err.code); }); +${listen}; +// Long enough for a bind to have completed if the refusal had let it. +setTimeout(() => { console.log('listening=' + listening); process.exit(0); }, 400); +`; + +const firstOf = (notes: Note[], kind: string) => notes.find((n) => n.patchstackVerify === kind); + +describe('the reporter is present', SLOW, () => { + it('is where the harness looks for it', () => { + expect(preload).not.toBeNull(); + }); +}); + +describe('a listener it cannot probe', SLOW, () => { + const cases: Array<[string, string, string, RegExp]> = [ + ['a raw TCP server', "require('node:net').createServer()", 'server.listen(0)', /is not an HTTP or HTTPS server \(Server\)/], + ['an HTTP/2 server', "require('node:http2').createServer()", 'server.listen(0)', /not an HTTP or HTTPS server \(Http2Server\)/], + [ + 'a secure HTTP/2 server', + "require('node:http2').createSecureServer()", + 'server.listen(0)', + /not an HTTP or HTTPS server \(Http2SecureServer\)/, + ], + ['an HTTP server on a Unix path', "require('node:http').createServer()", "server.listen('./app.sock')", /listens on a Unix socket path/], + ['an HTTP server on a negative string path', "require('node:http').createServer()", "server.listen('-1')", /listens on a Unix socket path/], + [ + 'an HTTP server on a Unix path given as an option', + "require('node:http').createServer()", + "server.listen({ path: './app.sock' })", + /listens on a Unix socket path/, + ], + ['an HTTP server on a file descriptor', "require('node:http').createServer()", 'server.listen({ fd: 3 })', /listens on a file descriptor/], + ['an HTTP server handed a handle', "require('node:http').createServer()", 'server.listen({ handle: {} })', /listens on a handle/], + ['an HTTP server given invalid options', "require('node:http').createServer()", 'server.listen({})', /invalid listen options/], + ['an HTTP server given an empty string port', "require('node:http').createServer()", "server.listen('')", /invalid TCP port/], + ['an HTTP server given an out-of-range port', "require('node:http').createServer()", 'server.listen(65536)', /invalid TCP port/], + ]; + + it.each(cases)('refuses %s, and it never becomes listening', async (_label, create, listen, why) => { + const { notes, out } = await under(REFUSAL_CASE(create, listen)); + expect(firstOf(notes, 'unsupported-listener')?.why).toMatch(why); + // Reported AND refused: the app saw a refused bind, and nothing ever bound. + expect(out).toContain('error EPERM'); + expect(out).toContain('listening=false'); + expect(firstOf(notes, 'listener')).toBeUndefined(); + }); + + it('does not create the socket file it was asked for', async () => { + const { out, dir } = await under( + `${REFUSAL_CASE("require('node:http').createServer()", "server.listen('./app.sock')")}`.replace( + "console.log('listening=' + listening)", + "console.log('listening=' + listening); console.log('socket=' + require('node:fs').existsSync('./app.sock'))", + ), + ); + expect(out).toContain('socket=false'); + expect(dir).toBeTruthy(); + }); +}); + +describe('a listener it can probe', SLOW, () => { + // The app's own `listening` handler runs before the reporter's, so the exit waits a turn: a process + // that calls `process.exit` from inside that handler cuts off its own report, which no server does but + // a fixture easily can. + const LISTENS = (listen: string) => `'use strict'; +const server = require('node:http').createServer((req, res) => res.end('x')); +server.on('listening', () => { + console.log('bound ' + JSON.stringify(server.address())); + setTimeout(() => process.exit(0), 200); +}); +${listen}; +`; + + it.each([ + ['a bare listen', 'server.listen()'], + ['a port', 'server.listen(3000)'], + ['a port as a string', "server.listen('3000')"], + ['a whitespace-padded port as a string', "server.listen(' 3000')"], + ['a port in exponential notation as a string', "server.listen('3e3')"], + ['a hexadecimal port as a string', "server.listen('0x10')"], + ['a port read from the environment', "process.env.PORT = '3000'; server.listen(process.env.PORT)"], + ['a null port', 'server.listen(null)'], + ['options whose port takes precedence over a path', "server.listen({ port: 3000, path: './ignored.sock' })"], + ['options whose unusable fd leaves the port in force', 'server.listen({ fd: -1, port: 3000 })'], + ['a port and a public host', "server.listen(3000, '0.0.0.0')"], + ['options naming a public host', "server.listen({ port: 3000, host: '0.0.0.0' })"], + ])('moves %s onto an ephemeral loopback port', async (_label, listen) => { + const { notes, out } = await under(LISTENS(listen)); + const listener = firstOf(notes, 'listener'); + expect(listener).toMatchObject({ scheme: 'http', host: '127.0.0.1' }); + expect(listener!.port).not.toBe(3000); + expect(listener!.port).toBeGreaterThan(0); + // What the app itself sees, so the report cannot claim an address the app did not get. + expect(out).toContain(`"address":"127.0.0.1"`); + }); + + it('reports an HTTPS server as https', async () => { + const { notes } = await under(`'use strict'; +const server = require('node:https').createServer(); +server.on('listening', () => setTimeout(() => process.exit(0), 200)); +server.listen(0); +`); + expect(firstOf(notes, 'listener')).toMatchObject({ scheme: 'https', host: '127.0.0.1' }); + }); +}); + +describe('a process the app attempts to start', SLOW, () => { + it('is reported, recognised as this runtime, and refused before launch', async () => { + const { notes, out } = await under(`'use strict'; +try { require('node:child_process').spawnSync(process.execPath, ['-e', '0']); } +catch (err) { console.log('refused ' + err.code); } +`); + expect(firstOf(notes, 'process-created')).toMatchObject({ via: 'spawnSync', node: true }); + expect(out).toContain('refused EPERM'); + }); + + it('is reported and refused when it is not this runtime', async () => { + const { notes, out } = await under(`'use strict'; +try { require('node:child_process').spawnSync('/bin/sh', ['-c', 'exit 0']); } +catch (err) { console.log('refused ' + err.code); } +`); + // Named by its executable alone. The arguments are the app's, and one of them is commonly a secret. + expect(firstOf(notes, 'process-created')).toMatchObject({ via: 'spawnSync', what: 'sh', node: false }); + expect(out).toContain('refused EPERM'); + }); + + it('describes a shell launch as a command line, which cannot be judged as Node', async () => { + const { notes } = await under(`'use strict'; +try { require('node:child_process').execSync('node -e 0'); } catch {} +`); + // The command line happens to name node; a shell can run anything, so it is not credited as node. + expect(firstOf(notes, 'process-created')).toMatchObject({ via: 'execSync', node: false }); + }); + + it('does not copy the command line of a shell launch into the report', async () => { + // A startup command routinely carries a token or a password. The report says a shell command line + // was run and nothing about what was in it, so a verifier cannot put a secret into someone's + // terminal or CI log that the app never printed itself. + const { notes } = await under(`'use strict'; +try { require('node:child_process').execSync('echo ps-secret-4f8a21 >/dev/null'); } catch {} +`); + expect(firstOf(notes, 'process-created')).toMatchObject({ via: 'execSync', what: 'a shell command line' }); + expect(JSON.stringify(notes)).not.toContain('ps-secret-4f8a21'); + }); + + it.each([ + ['spawn', "spawn(process.execPath, ['-e', '0'], { stdio: 'ignore' })"], + ['exec', "exec('exit 0')"], + ['execFile', "execFile('/bin/echo', ['x'])"], + ['fork', "fork(require('node:path').join(__dirname, 'noop.cjs'), { stdio: 'ignore' })"], + ['spawnSync', "spawnSync('/bin/echo', ['x'])"], + ['execSync', "execSync('exit 0')"], + ])('reports one refusal for %s', async (_label, call) => { + // The exported wrapper reports and refuses before delegation. The low-level wrapper exists for a + // direct `ChildProcess.spawn` call, not as a second report for one helper invocation. + const { notes } = await under( + `'use strict'; +const { spawn, exec, execFile, fork, spawnSync, execSync } = require('node:child_process'); +try { ${call}; } catch {} +setTimeout(() => process.exit(0), 400); +`, + { 'noop.cjs': "'use strict';\n" }, + ); + expect(notes.filter((note) => note.patchstackVerify === 'process-created')).toHaveLength(1); + }); + + it.each([ + ['spawn', "spawn('echo ps-secret-2ba90f', { shell: true })"], + ['spawn with an args array', "spawn('echo', ['ps-secret-2ba90f'], { shell: true })"], + ['execFile', "execFile('echo ps-secret-2ba90f', { shell: true })"], + ])('describes %s asking for a shell as a command line, not by its contents', async (_label, call) => { + // `shell: true` turns the command argument into a command line on any helper, not only `exec`. + const { notes } = await under(`'use strict'; +const { spawn, execFile } = require('node:child_process'); +try { ${call}; } catch {} +setTimeout(() => process.exit(0), 400); +`); + expect(firstOf(notes, 'process-created')).toMatchObject({ what: 'a shell command line', node: false }); + expect(JSON.stringify(notes)).not.toContain('ps-secret-2ba90f'); + }); + + it('is reported and refused when it goes around the helpers entirely', async () => { + // A direct call to the low-level launch method has the same refusal as an exported helper. + const { notes, out } = await under(`'use strict'; +const { ChildProcess } = require('node:child_process'); +const child = new ChildProcess(); +try { child.spawn({ file: '/bin/echo', args: ['echo', 'went-around'], stdio: 'inherit' }); } +catch (err) { console.log('refused ' + err.code); } +`); + expect(out).toContain('refused EPERM'); + expect(out).not.toContain('went-around'); + expect(firstOf(notes, 'process-created')).toMatchObject({ via: 'ChildProcess.spawn', what: 'echo', node: false }); + }); + + it('is reported and refused for a cluster worker, which reaches the launch path its own way', async () => { + const { notes, out } = await under(`'use strict'; +const cluster = require('node:cluster'); +if (cluster.isPrimary) { + try { cluster.fork(); } catch (err) { console.log('refused ' + err.code); } +} else { + process.exit(0); +} +`); + const created = firstOf(notes, 'process-created'); + expect(created?.node).toBe(true); + expect(created?.via).toMatch(/^(?:fork|ChildProcess\.spawn)$/); + expect(out).toContain('refused EPERM'); + }); +}); + +describe('the one-process boundary', SLOW, () => { + it('refuses a launch even when its options do not reveal how the child could escape later', async () => { + // A program launched in the current group may daemonize after it starts. No launch is delegated, + // because option inspection cannot make the end-of-run cleanup a process-tree guarantee. + const { notes, out } = await under(`'use strict'; +const { spawn } = require('node:child_process'); +try { + spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { stdio: 'ignore' }); + console.log('launched'); +} catch (err) { + console.log('refused ' + err.code); +} +setTimeout(() => process.exit(0), 200); +`); + expect(firstOf(notes, 'process-created')?.escape).toMatch(/one-process scope/); + expect(out).toContain('refused EPERM'); + expect(out).not.toContain('launched'); + }); + + it('also refuses a launch whose environment preserves the reporter', async () => { + const { notes, out } = await under(`'use strict'; +const { spawnSync } = require('node:child_process'); +try { spawnSync(process.execPath, ['-e', 'console.log("child ran")'], { env: { ...process.env, EXTRA: '1' }, encoding: 'utf8' }); } +catch (err) { console.log('refused ' + err.code); } +`); + expect(firstOf(notes, 'process-created')?.escape).toMatch(/one-process scope/); + expect(out).toContain('refused EPERM'); + expect(out).not.toContain('child ran'); + }); +}); + +describe('a worker thread the app starts', SLOW, () => { + it('is reported, because a worker cannot report its own listeners', async () => { + // The worker loads this file and its listeners are contained like any other. What it does not have + // is `process.send`, so a listener it opens can be neither counted nor asked — which is why the + // creation is reported and the run declines rather than passing on the main thread alone. + const { notes } = await under( + `'use strict'; +const { Worker } = require('node:worker_threads'); +const worker = new Worker(require('node:path').join(__dirname, 'inside.cjs')); +worker.on('exit', () => process.exit(0)); +`, + { 'inside.cjs': "'use strict';\n" }, + ); + expect(firstOf(notes, 'worker-created')).toMatchObject({ what: 'inside.cjs' }); + }); + + it('names inline source as inline rather than quoting it', async () => { + const { notes } = await under(`'use strict'; +const { Worker } = require('node:worker_threads'); +const worker = new Worker('/* ps-secret-9c2b7 */', { eval: true }); +worker.on('exit', () => process.exit(0)); +`); + expect(firstOf(notes, 'worker-created')).toMatchObject({ what: 'inline source' }); + expect(JSON.stringify(notes)).not.toContain('ps-secret-9c2b7'); + }); + + it.each([ + ['a replaced environment', "{ env: {} }"], + ['a replaced environment and no execArgv', "{ execArgv: [], env: {} }"], + ])('is refused when it has %s, which is the only route the reporter has in', async (_label, options) => { + // A worker's `execArgv` does not carry a preload, so `NODE_OPTIONS` is the only way this file + // reaches one. A worker without it patches nothing and binds exactly what it asks for. + const { notes, out } = await under(`'use strict'; +const { Worker } = require('node:worker_threads'); +try { + new Worker('0;', { eval: true, ...${options} }); + console.log('started'); +} catch (err) { + console.log('refused ' + err.code); +} +setTimeout(() => process.exit(0), 200); +`); + expect(firstOf(notes, 'worker-created')?.escape).toMatch(/replaces the environment/); + expect(out).toContain('refused EPERM'); + expect(out).not.toContain('started'); + }); + + it.each([ + ['nothing set', ''], + ['only execArgv dropped', ', execArgv: []'], + ['the shared environment named explicitly', ', env: require("node:worker_threads").SHARE_ENV'], + ['an environment that still carries the reporter', ', env: { NODE_OPTIONS: process.env.NODE_OPTIONS }'], + ])('is allowed with %s, and is screened', async (_label, extra) => { + const { out } = await under(`'use strict'; +const { Worker } = require('node:worker_threads'); +const w = new Worker("console.log('worker listen is ' + (require('node:net').Server.prototype.listen.name || '(anon)'));", { eval: true${extra} }); +w.on('exit', () => process.exit(0)); +`); + expect(out).toContain('worker listen is patchstackVerifyListen'); + }); + + it('refuses a replacement NODE_OPTIONS value that merely contains the reporter path', async () => { + // Presence as a substring is not a preload. The exact propagated value must survive replacement. + const { notes, out } = await under(`'use strict'; +const { Worker } = require('node:worker_threads'); +const misleading = process.env.NODE_OPTIONS.replace(/"$/, '.missing"'); +try { + new Worker('0;', { eval: true, env: { NODE_OPTIONS: misleading } }); + console.log('started'); +} catch (err) { + console.log('refused ' + err.code); +} +setTimeout(() => process.exit(0), 200); +`); + expect(firstOf(notes, 'worker-created')?.escape).toMatch(/replaces the environment/); + expect(out).toContain('refused EPERM'); + expect(out).not.toContain('started'); + }); + + it('confirms a worker really does load the reporter without a way to report', async () => { + // The premise of the case above, stated on its own: the containment is there and the channel is not. + const { out } = await under(`'use strict'; +const { Worker } = require('node:worker_threads'); +const worker = new Worker("require('node:http').createServer().listen(3000, () => { console.log('worker send=' + typeof process.send); process.exit(0); });", { eval: true }); +worker.on('exit', () => process.exit(0)); +`); + expect(out).toContain('worker send=undefined'); + }); + + it('does not run a child-process call it observed', async () => { + const { out } = await under(`'use strict'; +try { require('node:child_process').spawnSync(process.execPath, ['-e', 'console.log("grandchild ran")'], { encoding: 'utf8' }); } +catch (err) { console.log('refused ' + err.code); } +`); + expect(out).toContain('refused EPERM'); + expect(out).not.toContain('grandchild ran'); + }); +}); + +describe('closing discovery', SLOW, () => { + it('acknowledges freeze only after a listen already admitted has reported', async () => { + const { notes } = await under(`'use strict'; +const server = require('node:http').createServer((req, res) => res.end('x')); +const emit = server.emit; +server.emit = function (event, ...args) { + if (event === 'listening') { + setTimeout(() => emit.call(this, event, ...args), 150); + return true; + } + return emit.call(this, event, ...args); +}; +server.listen(0); +process.emit('message', { patchstackVerify: 'freeze' }); +setTimeout(() => process.exit(0), 400); +`); + const kinds = notes.map((note) => note.patchstackVerify); + expect(kinds.indexOf('listener')).toBeGreaterThanOrEqual(0); + expect(kinds.indexOf('frozen')).toBeGreaterThan(kinds.indexOf('listener')); + }); +}); + +describe('what it says about itself', SLOW, () => { + it('reports that it loaded, with the process it loaded into', async () => { + const { notes, out } = await under(`'use strict'; +console.log('pid=' + process.pid); +`); + const ready = firstOf(notes, 'ready'); + expect(ready).toBeDefined(); + expect(out).toContain(`pid=${ready!.pid}`); + }); +}); diff --git a/tests/protect/runtime-probe.test.ts b/tests/protect/runtime-probe.test.ts new file mode 100644 index 00000000..e49166fe --- /dev/null +++ b/tests/protect/runtime-probe.test.ts @@ -0,0 +1,934 @@ +// The opt-in runtime check: start an app, ask each listener it opens, and report whether the request +// reached the scaffolded seam. +// +// Every case here is a real child process on a real loopback port, because the whole value of this check +// is that it observes a running app rather than reading its source. The controls matter more than the +// passes: an app that answers 200 without the sentinel, one that never listens, and one that cannot be +// loaded must all be distinguishable from a traversal. +import { execFileSync } from 'node:child_process'; +import { existsSync, mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; +import { afterEach, describe, expect, it } from 'vitest'; + +// Every case here starts a real process, so every case declares a timeout that fits the run it performs. +// The default five seconds is shorter than the runs below allow, and a test vitest abandons mid-run is +// not just a failure: the awaited promise is dropped, its cleanup never happens, and the app it started +// is left on the machine. Long enough for a loaded CI runner, and each run bounds itself anyway. +const SLOW = { timeout: 60_000 }; + +import { VERIFY_HEADER, sentinelAnswer } from '../../src/protect/verify-sentinel.js'; +import { + CHALLENGE_ENV, + VERIFY_HEADER_NAME, + expectedAnswerFor, + preloadPath, + probeRuntimeTraversal, + propagatedNodeOptions, + verdictOf, +} from '../../src/protect/install/runtime/probe.js'; + +const SENTINEL = pathToFileURL(fileURLToPath(new URL('../../src/protect/verify-sentinel.js', import.meta.url))).href; + +let dirs: string[] = []; + +afterEach(() => { + for (const dir of dirs) rmSync(dir, { recursive: true, force: true }); + dirs = []; +}); + +/** A project directory with the given files, cleaned up after each test. */ +function project(files: Record): string { + const dir = mkdtempSync(join(tmpdir(), 'ps-runtime-probe-')); + dirs.push(dir); + for (const [name, body] of Object.entries(files)) { + const path = join(dir, name); + mkdirSync(join(path, '..'), { recursive: true }); + writeFileSync(path, body); + } + + return dir; +} + +/** The request seam, as the templates write it: answer the sentinel before the app handler runs. */ +const ESM_SEAM = ` +import { createServer } from 'node:http'; +import { sentinelAnswer, VERIFY_HEADER } from ${JSON.stringify(SENTINEL)}; + + +createServer(async (req, res) => { + const answered = await sentinelAnswer(req.headers[VERIFY_HEADER]); + if (answered) { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(answered); return; } + res.writeHead(200); res.end('the app'); +}).listen(3000); +`; + +const CJS_SEAM = ` +const { createServer } = require('node:http'); + +createServer((req, res) => { + import(${JSON.stringify(SENTINEL)}).then(async ({ sentinelAnswer, VERIFY_HEADER }) => { + const answered = await sentinelAnswer(req.headers[VERIFY_HEADER]); + if (answered) { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(answered); return; } + res.writeHead(200); res.end('the app'); + }); +}).listen(3000); +`; + +/** + * A throwaway self-signed certificate, generated per test rather than committed: a private key in a + * public repository is a liability even when it only ever protects a loopback port for 30ms. + * + * `openssl` is a maintainer-toolchain requirement for this file. If it is missing the test fails, which + * is the honest outcome — a skip here would quietly stop covering the HTTPS branch. + */ +function certificate(dir: string): { key: string; cert: string } { + execFileSync('openssl', ['req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-keyout', join(dir, 'key.pem'), '-out', join(dir, 'cert.pem'), '-days', '1', '-subj', '/CN=localhost'], { stdio: 'ignore' }); + + return { key: join(dir, 'key.pem'), cert: join(dir, 'cert.pem') }; +} + +/** + * Whether a process — or, for a negative pid, a whole process GROUP — is gone. + * + * Polled rather than sampled once: a signalled process is reaped by whoever is its parent, and how long + * that takes is not this test's business. The group form needs no cooperation from the app, which is + * what makes it usable on a run that ends the moment the app does something: a group exists while any + * member of it does, so `kill(-pid, 0)` failing is the whole claim that nothing was left behind. + */ +async function gone(pid: number, withinMs = 5_000): Promise { + const deadline = Date.now() + withinMs; + for (;;) { + try { + process.kill(pid, 0); + } catch { + return true; // no such process, or no such group + } + if (Date.now() > deadline) return false; + await new Promise((resolve) => setTimeout(resolve, 25)); + } +} + +const run = (cwd: string, entry: string) => + probeRuntimeTraversal({ cwd, entry: join(cwd, entry), timeoutMs: 8_000, settleMs: 300 }); + +describe('the names the seam and the harness share', SLOW, () => { + it('uses the header the seam reads', () => { + expect(VERIFY_HEADER_NAME).toBe(VERIFY_HEADER); + }); + + it('derives the same answer as the seam, from two crypto implementations', async () => { + const challenge = 'c'.repeat(64); + process.env[CHALLENGE_ENV] = challenge; + try { + expect(await sentinelAnswer(challenge)).toBe(expectedAnswerFor(challenge)); + } finally { + delete process.env[CHALLENGE_ENV]; + } + }); + + it('ships the listener reporter alongside the harness', () => { + expect(preloadPath()).not.toBeNull(); + }); +}); + +describe('an app whose seam answers', SLOW, () => { + it('proves traversal for a CommonJS entry', async () => { + const dir = project({ 'server.cjs': CJS_SEAM }); + const result = await run(dir, 'server.cjs'); + expect(result.outcome).toBe('proven'); + expect(result.listeners).toEqual([ + { scheme: 'http', host: '127.0.0.1', port: expect.any(Number), outcome: 'traversed', detail: undefined }, + ]); + }); + + it('proves traversal for a Node ESM entry', async () => { + const dir = project({ 'server.mjs': ESM_SEAM }); + expect((await run(dir, 'server.mjs')).outcome).toBe('proven'); + }); + + it('proves traversal for a bundled entry', async () => { + // A single-file build with no package boundary of its own to speak of. + const dir = project({ 'package.json': '{"type":"module"}', 'dist/bundle.js': ESM_SEAM }); + expect((await run(dir, 'dist/bundle.js')).outcome).toBe('proven'); + }); + + it('proves traversal only when every listener answers', async () => { + const dir = project({ + 'both.mjs': `${ESM_SEAM}\n${ESM_SEAM.split('\n').slice(3).join('\n').replace('listen(3000)', 'listen(3001)')}`, + }); + const result = await run(dir, 'both.mjs'); + expect(result.outcome).toBe('proven'); + expect(result.listeners).toHaveLength(2); + }); + + it('waits for an app that takes its time to boot', async () => { + // The settling interval means "no new listener since the last one". If it started when the process + // did, a framework that spends a second booting would be reported as opening no listener at all. + const dir = project({ 'slow.mjs': `await new Promise((r) => setTimeout(r, 900));\n${ESM_SEAM}` }); + expect((await run(dir, 'slow.mjs')).outcome).toBe('proven'); + }); + + it('leaves nothing behind on a pass', async () => { + const dir = project({ 'server.mjs': ESM_SEAM }); + const result = await run(dir, 'server.mjs'); + expect(result.outcome).toBe('proven'); + expect(await gone(result.pid!)).toBe(true); + }); + +}); + +describe('an HTTPS listener', SLOW, () => { + it('proves traversal, over a certificate nothing in this process trusts', async () => { + const dir = project({}); + const { key, cert } = certificate(dir); + writeFileSync( + join(dir, 'secure.mjs'), + `import { readFileSync } from 'node:fs'; +import { createServer } from 'node:https'; +import { sentinelAnswer, VERIFY_HEADER } from ${JSON.stringify(SENTINEL)}; + +createServer({ key: readFileSync(${JSON.stringify(key)}), cert: readFileSync(${JSON.stringify(cert)}) }, async (req, res) => { + const answered = await sentinelAnswer(req.headers[VERIFY_HEADER]); + if (answered) { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(answered); return; } + res.writeHead(200); res.end('the app'); +}).listen(3000);`, + ); + const result = await run(dir, 'secure.mjs'); + expect(result.listeners).toEqual([ + { scheme: 'https', host: '127.0.0.1', port: expect.any(Number), outcome: 'traversed', detail: undefined }, + ]); + expect(result.outcome).toBe('proven'); + }); + + it('relaxes certificate checking for its own request only, not for the process', async () => { + const dir = project({}); + const { key, cert } = certificate(dir); + writeFileSync( + join(dir, 'secure.mjs'), + `import { readFileSync } from 'node:fs'; +import { createServer } from 'node:https'; +import { sentinelAnswer, VERIFY_HEADER } from ${JSON.stringify(SENTINEL)}; + +createServer({ key: readFileSync(${JSON.stringify(key)}), cert: readFileSync(${JSON.stringify(cert)}) }, async (req, res) => { + res.writeHead(200); res.end((await sentinelAnswer(req.headers[VERIFY_HEADER])) ?? 'the app'); +}).listen(3000);`, + ); + expect((await run(dir, 'secure.mjs')).outcome).toBe('proven'); + // Nothing process-wide was switched off to get there. + expect(process.env.NODE_TLS_REJECT_UNAUTHORIZED).toBeUndefined(); + // And an ordinary request from this process still refuses that certificate. Held open across the + // run above would be circular, so this is a second server from the same throwaway certificate. + const { createServer: createSecure } = await import('node:https'); + const { readFileSync } = await import('node:fs'); + const server = createSecure({ key: readFileSync(key), cert: readFileSync(cert) }, (_req, res) => res.end('x')); + await new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve())); + const port = (server.address() as { port: number }).port; + const { request } = await import('node:https'); + const failure = await new Promise((resolve) => { + const req = request({ host: '127.0.0.1', port, path: '/' }, () => resolve('accepted the certificate')); + req.on('error', (err: NodeJS.ErrnoException) => resolve(err.code ?? err.message)); + req.end(); + }); + server.close(); + expect(failure).toMatch(/SELF_SIGNED|UNABLE_TO_VERIFY|DEPTH_ZERO/); + }); + + it('reports a TLS listener it cannot complete a handshake with as unavailable', async () => { + // `https.createServer()` with no certificate listens happily and fails every handshake. + const dir = project({ 'nocert.mjs': `import { createServer } from 'node:https';\ncreateServer((req, res) => res.end('x')).listen(3000);` }); + const result = await run(dir, 'nocert.mjs'); + expect(result.listeners).toEqual([ + { scheme: 'https', host: '127.0.0.1', port: expect.any(Number), outcome: 'unreachable', detail: expect.any(String) }, + ]); + expect(result.outcome).toBe('unavailable'); + }); +}); + +describe('the port the app asked for', SLOW, () => { + it('does not have to be free, because the listener is moved to an ephemeral loopback port', async () => { + // The blocker takes a port the kernel just handed out, and the app is written to ask for exactly + // that one. Naming a fixed port here would make the test depend on what else is running. + const { createServer } = await import('node:http'); + const blocker = createServer(); + await new Promise((resolve) => blocker.listen(0, '127.0.0.1', () => resolve())); + const taken = (blocker.address() as { port: number }).port; + try { + const dir = project({ 'server.mjs': ESM_SEAM.replace('listen(3000)', `listen(${taken})`) }); + const result = await run(dir, 'server.mjs'); + expect(result.outcome).toBe('proven'); + expect(result.listeners[0].port).not.toBe(taken); + } finally { + blocker.close(); + } + }); +}); + +describe('what must not read as proof', SLOW, () => { + it('reports an app that answers 200 without the sentinel as not traversed', async () => { + const dir = project({ + 'bare.mjs': `import { createServer } from 'node:http'; +createServer((req, res) => { res.writeHead(200); res.end('hello'); }).listen(3000);`, + }); + const result = await run(dir, 'bare.mjs'); + expect(result.outcome).toBe('not-traversed'); + expect(result.listeners[0]).toMatchObject({ outcome: 'answered-without-sentinel' }); + }); + + it('reports an app that echoes the challenge back as not traversed', async () => { + const dir = project({ + 'echo.mjs': `import { createServer } from 'node:http'; +createServer((req, res) => { res.writeHead(200); res.end(req.headers[${JSON.stringify(VERIFY_HEADER)}] ?? ''); }).listen(3000);`, + }); + expect((await run(dir, 'echo.mjs')).outcome).toBe('not-traversed'); + }); + + it('reports a partly guarded app as not traversed, and names both listeners', async () => { + // One server behind the seam, one not — the shape an app takes when a second server was added after + // the guard was wired. A verdict drawn from the guarded one alone would be a pass for half an app. + const dir = project({ + 'partial.mjs': `${ESM_SEAM} +import { createServer as createSecond } from 'node:http'; +createSecond((req, res) => { res.writeHead(200); res.end('the other server'); }).listen(3001);`, + }); + const result = await run(dir, 'partial.mjs'); + expect(result.listeners).toHaveLength(2); + expect(result.listeners.map((l) => l.host)).toEqual(['127.0.0.1', '127.0.0.1']); + expect(result.listeners.map((l) => l.outcome).sort()).toEqual(['answered-without-sentinel', 'traversed']); + expect(result.outcome).toBe('not-traversed'); + }); + + it('reports an app answering a fixed digest as not traversed', async () => { + // The answer is a digest of THIS run's challenge. A body that happens to be a valid-looking digest — + // hard-coded, cached from an earlier run, copied from a doc — is not an answer to the question asked. + const dir = project({ + 'fixed.mjs': `import { createServer } from 'node:http'; +createServer((req, res) => { res.writeHead(200); res.end('${'a'.repeat(64)}'); }).listen(3000);`, + }); + expect((await run(dir, 'fixed.mjs')).outcome).toBe('not-traversed'); + }); + + it('reports an entry that cannot be loaded as unavailable, not as a failure to traverse', async () => { + const dir = project({ 'broken.mjs': 'import { nothing } from "node:this-does-not-exist";' }); + const result = await run(dir, 'broken.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/exited before it listened/); + }); + + it('says the app exited while it was being probed, rather than giving no reason at all', async () => { + const dir = project({ + 'flees.mjs': `import { createServer } from 'node:http'; +const server = createServer((req, res) => res.end('x')); +// A beat, so the listener has certainly been reported before the exit is: the two arrive over +// different channels, and an exit in the same tick as the listen can be delivered first. +server.listen(3000, () => { setTimeout(() => { server.close(); process.exit(0); }, 150); });`, + }); + // A settling window far wider than the app's own delay, so the exit is what ends this run and the + // case under test is the one that runs — not a race between the two. + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'flees.mjs'), timeoutMs: 8_000, settleMs: 2_000 }); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/exited while it was being probed/); + }); + + it('reports a listener that never stops answering as not traversed', async () => { + // Read to a budget and no further: an unverified application must not be able to grow this process, + // and past that budget the answer cannot be the digest whatever else arrives. + const dir = project({ + 'endless.mjs': `import { createServer } from 'node:http'; +createServer((req, res) => { + res.writeHead(200); + const pump = () => { if (res.write('x'.repeat(4096))) setImmediate(pump); else res.once('drain', pump); }; + pump(); +}).listen(3000);`, + }); + expect((await run(dir, 'endless.mjs')).outcome).toBe('not-traversed'); + }); + + it('reports a directory it cannot start the app in as unavailable', async () => { + const dir = project({ 'server.mjs': ESM_SEAM }); + const result = await probeRuntimeTraversal({ + cwd: join(dir, 'not-a-directory'), + entry: join(dir, 'server.mjs'), + timeoutMs: 4_000, + settleMs: 300, + }); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/could not be started/); + }); + + it('reports an app that opens no listener as unavailable', async () => { + const dir = project({ 'quiet.mjs': 'console.log("started, listening to nothing");' }); + const result = await run(dir, 'quiet.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/exited before it listened/); + }); + + it('reports an app that hangs without listening as unavailable', async () => { + const dir = project({ 'hang.mjs': 'setInterval(() => {}, 1000);' }); + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'hang.mjs'), timeoutMs: 1_200, settleMs: 300 }); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/no HTTP listener was observed/); + }); + + it('reports a transport it cannot probe as unavailable', async () => { + const SOCKET_PATH = join(tmpdir(), `ps-probe-${process.pid}.sock`); + const dir = project({ + 'unix.mjs': `import { createServer } from 'node:http'; +const server = createServer((req, res) => res.end('x')); +server.on('listening', () => { console.log('app saw LISTENING'); }); +server.on('error', (err) => { console.log('app saw ' + err.code); }); +server.listen(${JSON.stringify(SOCKET_PATH)}); +// Kept alive deliberately: an app that exits after the refusal would end the run by exiting, which +// would hide whether the refusal itself ended it. +setInterval(() => {}, 1000);`, + }); + // A settling window far longer than this should take: a refused listener ENDS discovery, so the run + // finishes at once rather than waiting to see what else the app opens. + const startedAt = Date.now(); + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'unix.mjs'), timeoutMs: 20_000, settleMs: 5_000 }); + expect(Date.now() - startedAt).toBeLessThan(3_000); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/Unix socket/); + expect(result.listeners).toEqual([]); + // That the refusal HELD — the app never became listening, and no socket was created — is pinned in + // the reporter's own tests, where the child outlives the answer. Here the child is stopped as soon + // as the refusal is heard, which is the contract: discovery ends, and the app is not left starting. + expect(await gone(result.pid!)).toBe(true); + expect(existsSync(SOCKET_PATH)).toBe(false); + }); + + it('distinguishes a reporter that never loaded from an app that never listened', async () => { + // The pair: with the real reporter, a listenerless app is reported as opening no listener (above). + // With a reporter that says nothing, the same app is reported as a reporter that did not load — + // a broken installation, not a finding about the app. No listener is opened either way, so the + // answer does not depend on a port being free on the machine running this. + const dir = project({ 'hang.mjs': 'setInterval(() => {}, 1000);', 'silent.cjs': '// loads, reports nothing' }); + const result = await probeRuntimeTraversal({ + cwd: dir, + entry: join(dir, 'hang.mjs'), + preload: join(dir, 'silent.cjs'), + timeoutMs: 1_200, + settleMs: 300, + }); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/reporter did not load/); + }); + + it('reports and refuses an app that attempts to start another process', async () => { + const dir = project({ + 'spawner.mjs': `import { spawnSync } from 'node:child_process'; +spawnSync('/bin/sh', ['-c', 'exit 0']); +${ESM_SEAM}`, + }); + // Same as a refused listener: discovery ends there, rather than settling out the interval. + const startedAt = Date.now(); + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'spawner.mjs'), timeoutMs: 20_000, settleMs: 5_000 }); + expect(Date.now() - startedAt).toBeLessThan(3_000); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/one-process/); + expect(result.listeners).toEqual([]); + }); + + it('reports and refuses an app that attempts to start another Node process too', async () => { + const dir = project({ + 'worker.mjs': `import { spawn } from 'node:child_process'; +spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { stdio: 'ignore' }); +${ESM_SEAM}`, + }); + const result = await run(dir, 'worker.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/one-process/); + }); + + it('reports an app that starts a worker thread as unavailable', async () => { + // The worker loads the reporter and its listeners are moved to loopback, so containment holds. What + // it has no way to do is report: a worker has no `process.send`. Answering from the main thread's + // listener alone would be a pass for part of an app. + const dir = project({ + 'threaded.mjs': `import { Worker } from 'node:worker_threads'; +new Worker("require('node:http').createServer((q, r) => r.end('unguarded')).listen(3100);", { eval: true }); +${ESM_SEAM}`, + }); + const result = await run(dir, 'threaded.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/worker thread/); + expect(result.listeners).toEqual([]); + }); + + it('reports and refuses an app that goes around the child_process helpers', async () => { + const dir = project({ + 'lowlevel.mjs': `import { ChildProcess } from 'node:child_process'; +const child = new ChildProcess(); +child.spawn({ file: process.execPath, args: [process.execPath, '-e', 'setInterval(() => {}, 1000)'], stdio: 'ignore' }); +${ESM_SEAM}`, + }); + const result = await run(dir, 'lowlevel.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/one-process/); + }); + + it('does not copy a startup command line into the reason it prints', async () => { + // `exec` is handed a whole shell command line, and a startup command commonly carries a token. The + // reason names the launcher and the fact that it was a command line, and nothing that was in it. + const dir = project({ + 'secretive.mjs': `import { execSync } from 'node:child_process'; +execSync('echo ps-secret-71d0e4 >/dev/null'); +${ESM_SEAM}`, + }); + const result = await run(dir, 'secretive.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/shell command line/); + expect(JSON.stringify(result)).not.toContain('ps-secret-71d0e4'); + }); + + it('refuses to answer for a listener that opens after discovery has ended', async () => { + // The first listener is guarded and slow to answer; the second opens while that request is still in + // flight, which is after the set of listeners to probe was fixed. It cannot join that set, and + // ignoring it would let an unguarded listener stand behind a pass on the guarded one. + const dir = project({ + 'late.mjs': `import { createServer } from 'node:http'; +import { sentinelAnswer, VERIFY_HEADER } from ${JSON.stringify(SENTINEL)}; + +createServer(async (req, res) => { + const answered = await sentinelAnswer(req.headers[VERIFY_HEADER]); + await new Promise((r) => setTimeout(r, 1200)); + if (answered) { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(answered); return; } + res.writeHead(200); res.end('the app'); +}).listen(3000); + +setTimeout(() => { createServer((req, res) => res.end('unguarded')).listen(3001); }, 700); +`, + }); + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'late.mjs'), timeoutMs: 15_000, settleMs: 300 }); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/after discovery had ended/); + expect(result.listeners).toEqual([]); + }); + + it('refuses to answer, and leaves nothing behind, when a launch would escape its cleanup', async () => { + // The refusal applies before option details matter. This launch makes both escape routes explicit, + // and must still never reach the child that would open the public listener. + const dir = project({ + 'escaping.mjs': `import { spawn } from 'node:child_process'; +import { writeFileSync } from 'node:fs'; +try { + spawn(process.execPath, ['-e', "require('node:http').createServer().listen(0, '0.0.0.0'); setInterval(() => {}, 1000)"], { + detached: true, + env: { PATH: process.env.PATH }, + stdio: 'ignore', + }).unref(); + writeFileSync(new URL('./escaped', import.meta.url), 'yes'); +} catch { + writeFileSync(new URL('./refused', import.meta.url), 'yes'); +} +${ESM_SEAM}`, + }); + const result = await run(dir, 'escaping.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/one-process/); + expect(existsSync(join(dir, 'escaped')), 'the launch was allowed to happen').toBe(false); + expect(existsSync(join(dir, 'refused'))).toBe(true); + }); + + it('refuses to answer for a worker that would not load the reporter', async () => { + const dir = project({ + 'unscreened.mjs': `import { Worker } from 'node:worker_threads'; +import { writeFileSync } from 'node:fs'; +try { + new Worker("require('node:http').createServer().listen(0, '0.0.0.0');", { eval: true, env: {} }); + writeFileSync(new URL('./escaped', import.meta.url), 'yes'); +} catch { + writeFileSync(new URL('./refused', import.meta.url), 'yes'); +} +${ESM_SEAM}`, + }); + const result = await run(dir, 'unscreened.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/one-process scope/); + expect(existsSync(join(dir, 'escaped')), 'the worker was allowed to start').toBe(false); + expect(existsSync(join(dir, 'refused'))).toBe(true); + }); + + it('refuses to answer for a listener that opens as the last response finishes', async () => { + // The boundary the settled-discovery check alone does not reach. The second listener's report is + // already sent, but its IPC callback can sit behind the HTTP one, so the promise chain here would + // read an empty set of late arrivals, form a pass, and kill the child with the message undelivered. + // Repeated, because a pass here would be a pass with an unguarded listener nobody asked. + const dir = project({ + 'boundary.mjs': `import { createServer } from 'node:http'; +import { sentinelAnswer, VERIFY_HEADER } from ${JSON.stringify(SENTINEL)}; + +createServer(async (req, res) => { + const answered = await sentinelAnswer(req.headers[VERIFY_HEADER]); + const late = createServer((q, r) => r.end('unguarded')); + late.listen(3001, '127.0.0.1', () => { + if (answered) { res.writeHead(200, { 'content-type': 'text/plain' }); res.end(answered); return; } + res.writeHead(200); res.end('the app'); + }); +}).listen(3000); +`, + }); + for (let attempt = 0; attempt < 3; attempt++) { + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'boundary.mjs'), timeoutMs: 15_000, settleMs: 300 }); + expect(result.outcome, `attempt ${attempt}`).toBe('unavailable'); + expect(result.reason, `attempt ${attempt}`).toMatch(/after discovery had ended/); + } + }); + + it('refuses to answer when the app never confirms it has stopped opening listeners', async () => { + // The confirmation is what makes the final read trustworthy, so a reporter that loads and answers + // listener reports but never acknowledges the freeze must not produce a pass. + const dir = project({ + 'server.cjs': CJS_SEAM, + 'mute.cjs': `'use strict'; +const net = require('node:net'); +const http = require('node:http'); +const original = net.Server.prototype.listen; +// Reports listeners like the real one, and deliberately never answers the freeze request. +net.Server.prototype.listen = function (...args) { + const result = original.call(this, 0, '127.0.0.1'); + this.once('listening', () => { + const address = this.address(); + process.send({ patchstackVerify: 'listener', scheme: this instanceof http.Server ? 'http' : 'https', host: address.address, port: address.port }); + }); + + return result; +}; +process.send({ patchstackVerify: 'ready', pid: process.pid }); +`, + }); + const result = await probeRuntimeTraversal({ + cwd: dir, + entry: join(dir, 'server.cjs'), + preload: join(dir, 'mute.cjs'), + timeoutMs: 12_000, + settleMs: 300, + }); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/never confirmed/); + }); + + it('refuses to answer for a listener bound off loopback', async () => { + // Nothing the reporter permits can reach here, so the app is started with the reporter suppressed + // for the listen path: what is under test is the parent's refusal, not the rewrite. + const dir = project({ + 'public.cjs': `'use strict'; +const net = require('node:net'); +const http = require('node:http'); +// A listen the reporter never saw, so it binds exactly as written. +const server = http.createServer((req, res) => res.end('public')); +net.Server.prototype.listen = require('node:net').Server.prototype.listen; +process.send({ patchstackVerify: 'ready', pid: process.pid }); +server.listen(0, '0.0.0.0', () => { + process.send({ patchstackVerify: 'listener', scheme: 'http', host: '0.0.0.0', port: server.address().port }); +}); +`, + 'noop.cjs': '// the reporter, replaced by nothing for this case', + }); + const result = await probeRuntimeTraversal({ + cwd: dir, + entry: join(dir, 'public.cjs'), + preload: join(dir, 'noop.cjs'), + timeoutMs: 6_000, + settleMs: 300, + }); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/must not open/); + expect(result.reason).toMatch(/0\.0\.0\.0/); + expect(result.listeners).toEqual([]); + }); + + it('answers unavailable on Windows without launching anything', async () => { + const dir = project({ 'server.mjs': ESM_SEAM }); + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'server.mjs'), platform: 'win32' }); + expect(result).toEqual({ outcome: 'unavailable', reason: expect.stringMatching(/Windows/), listeners: [], output: '' }); + }); +}); + +describe('the aggregate verdict', SLOW, () => { + const traversed = { scheme: 'http', host: '127.0.0.1', port: 1, outcome: 'traversed' } as const; + + it('treats nothing observed as unavailable rather than as a pass', () => { + expect(verdictOf([])).toBe('unavailable'); + }); + + it('lets one unanswered listener outrank the others', () => { + expect(verdictOf([traversed, { ...traversed, port: 2, outcome: 'answered-without-sentinel' }])).toBe('not-traversed'); + }); + + it('lets an unreachable listener outrank a failure', () => { + expect( + verdictOf([ + { ...traversed, port: 2, outcome: 'answered-without-sentinel' }, + { ...traversed, port: 3, outcome: 'unreachable' }, + ]), + ).toBe('unavailable'); + }); + + it('is proven only when every listener traversed', () => { + expect(verdictOf([traversed, { ...traversed, port: 2 }])).toBe('proven'); + }); +}); + +describe('the deadline', SLOW, () => { + it('covers the probes too, not only discovery', async () => { + // A listener that accepts the connection and answers nothing. With a per-phase timeout the run would + // sit here for the probe's own five seconds after discovery had already finished. + const dir = project({ + 'silent.mjs': `import { createServer } from 'node:http'; +createServer(() => { /* never answers */ }).listen(3000);`, + }); + const startedAt = Date.now(); + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'silent.mjs'), timeoutMs: 2_000, settleMs: 300 }); + const elapsed = Date.now() - startedAt; + expect(result.outcome).toBe('unavailable'); + // Generous, and still far below the 300ms of discovery plus a five-second probe. + expect(elapsed).toBeLessThan(4_000); + }); + + it('is spent, not extended, by a deadline that has already passed', async () => { + const dir = project({ 'server.mjs': ESM_SEAM }); + const startedAt = Date.now(); + const result = await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'server.mjs'), timeoutMs: 100, settleMs: 300 }); + expect(Date.now() - startedAt).toBeLessThan(2_000); + expect(result.outcome).toBe('unavailable'); + }); + + it('does not let settling run past it', async () => { + // A settling interval longer than the whole run must not extend the run. + const dir = project({ 'server.mjs': ESM_SEAM }); + const startedAt = Date.now(); + await probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'server.mjs'), timeoutMs: 1_000, settleMs: 30_000 }); + expect(Date.now() - startedAt).toBeLessThan(3_000); + }); +}); + +describe('the NODE_OPTIONS it carries into the app', SLOW, () => { + const preload = '/opt/patchstack/report-listeners.cjs'; + + it('carries the reporter when nothing was inherited', () => { + expect(propagatedNodeOptions(preload, undefined)).toEqual({ kind: 'options', value: `--require "${preload}"` }); + expect(propagatedNodeOptions(preload, ' ')).toEqual({ kind: 'options', value: `--require "${preload}"` }); + }); + + it('puts the reporter first, ahead of anything it inherited', () => { + // Order in this value is order of execution, and Node parses the whole of it before the command + // line. Anything ahead of the reporter runs before `net.Server.prototype.listen` is patched. + expect(propagatedNodeOptions(preload, '--no-warnings')).toEqual({ + kind: 'options', + value: `--require "${preload}" --no-warnings`, + }); + }); + + it('refuses an inherited value that runs code before the reporter', () => { + for (const existing of ['--require ./boot.cjs', '--import ./boot.mjs', '--experimental-loader ./l.mjs', '--eval "0"', '--inspect']) { + const result = propagatedNodeOptions(preload, existing); + expect(result.kind, existing).toBe('unavailable'); + expect(result.kind === 'unavailable' && result.reason, existing).toMatch(/NODE_OPTIONS/); + } + }); + + it('refuses an inherited value it does not recognise', () => { + const result = propagatedNodeOptions(preload, '--some-future-flag=1'); + expect(result.kind).toBe('unavailable'); + expect(result.kind === 'unavailable' && result.reason).toMatch(/does not recognise/); + }); + + it('refuses an inherited value it cannot split the way Node will', () => { + const result = propagatedNodeOptions(preload, '--require "/opt/boot.cjs'); + expect(result.kind).toBe('unavailable'); + expect(result.kind === 'unavailable' && result.reason).toMatch(/unbalanced double quote/); + }); + + it('refuses a reporter path this value cannot carry', () => { + const result = propagatedNodeOptions('/opt/we"ird/report.cjs', undefined); + expect(result.kind).toBe('unavailable'); + expect(result.kind === 'unavailable' && result.reason).toMatch(/double quote/); + }); + + it('does not launch the app at all when the inherited value is refused', async () => { + // Before the spawn, like the platform check: containment the run cannot guarantee is not something + // to find out about once the application is already running. + const dir = project({ + 'marks.mjs': `import { writeFileSync } from 'node:fs'; +writeFileSync(new URL('./started', import.meta.url), 'yes'); +${ESM_SEAM}`, + }); + const before = process.env.NODE_OPTIONS; + process.env.NODE_OPTIONS = '--require ./boot.cjs'; + try { + const result = await run(dir, 'marks.mjs'); + expect(result.outcome).toBe('unavailable'); + expect(result.reason).toMatch(/NODE_OPTIONS/); + expect(result.pid, 'a process was started anyway').toBeUndefined(); + expect(existsSync(join(dir, 'started')), 'the app was started anyway').toBe(false); + } finally { + if (before === undefined) delete process.env.NODE_OPTIONS; + else process.env.NODE_OPTIONS = before; + } + }); + + it('still proves traversal with a recognised value inherited', async () => { + const dir = project({ 'server.mjs': ESM_SEAM }); + const before = process.env.NODE_OPTIONS; + process.env.NODE_OPTIONS = '--no-warnings'; + try { + expect((await run(dir, 'server.mjs')).outcome).toBe('proven'); + } finally { + if (before === undefined) delete process.env.NODE_OPTIONS; + else process.env.NODE_OPTIONS = before; + } + }); +}); + +describe('the signal handlers it installs while the app is running', SLOW, () => { + const SIGNALS = ['SIGINT', 'SIGTERM', 'SIGHUP', 'SIGQUIT'] as const; + const counts = () => SIGNALS.map((signal) => process.listenerCount(signal)).concat(process.listenerCount('exit')); + + it('are in place while the app is running, not only at the end', async () => { + // The window they exist for is the one where the app is already a detached process group. So they + // have to be observable DURING the run, which is what this waits for. + const before = counts(); + const dir = project({ 'server.mjs': ESM_SEAM }); + const running = probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'server.mjs'), timeoutMs: 20_000, settleMs: 2_000 }); + const deadline = Date.now() + 10_000; + while (counts().join() === before.join() && Date.now() < deadline) await new Promise((r) => setTimeout(r, 10)); + expect(counts(), 'no handler was installed while the app was running').not.toEqual(before); + // Every one of them, so a missing signal cannot hide behind the others. + expect(counts()).toEqual(before.map((n) => n + 1)); + + await running; + expect(counts()).toEqual(before); + }); + + it('do not deliver the signal a second time to a handler that was already there', async () => { + // Removing our own handler restores the default disposition only when ours was the only one. With + // another handler present there is no default to restore, and that handler has already been called + // for this signal — so re-sending it would call it twice, which is not what installing one handler + // asked for. + const seen: string[] = []; + const preexisting = () => seen.push('SIGINT'); + const base = process.listenerCount('SIGINT'); + process.on('SIGINT', preexisting); + try { + const dir = project({ 'server.mjs': ESM_SEAM }); + const running = probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'server.mjs'), timeoutMs: 20_000, settleMs: 2_000 }); + const deadline = Date.now() + 10_000; + while (process.listenerCount('SIGINT') < base + 2 && Date.now() < deadline) await new Promise((r) => setTimeout(r, 10)); + expect(process.listenerCount('SIGINT'), 'the probe never installed its handler').toBeGreaterThanOrEqual(base + 2); + + process.kill(process.pid, 'SIGINT'); + await running; + await new Promise((r) => setTimeout(r, 100)); + expect(seen).toEqual(['SIGINT']); + } finally { + process.removeListener('SIGINT', preexisting); + } + }); + + /** + * Run `body` with SIGINT's listener list emptied, and put it back afterwards. + * + * Both cases below need this process's own signal environment to be exactly what they set up. The + * runner keeps a persistent SIGINT handler of its own, which would sit in every count and every + * ordering assertion and make either case pass whatever the probe did. + */ + const withNoSignalHandlers = async (body: () => Promise): Promise => { + // `listeners()` unwraps a `once` listener. Restoring that function with `on` would silently change + // the test runner's signal semantics, so preserve the raw wrappers instead. + const saved = process.rawListeners('SIGINT'); + process.removeAllListeners('SIGINT'); + try { + await body(); + } finally { + process.removeAllListeners('SIGINT'); + for (const listener of saved) process.on('SIGINT', listener as (...args: unknown[]) => void); + } + }; + + /** Wait until the probe's handler is attached alongside however many are expected beside it. */ + const untilInstalled = async (total: number): Promise => { + const deadline = Date.now() + 10_000; + while (process.listenerCount('SIGINT') < total && Date.now() < deadline) await new Promise((r) => setTimeout(r, 10)); + expect(process.listenerCount('SIGINT'), 'the probe never installed its handler').toBe(total); + }; + + it('run before a handler the caller already registered', async () => { + // Prepending is what makes the count in the handler meaningful, so it is asserted on its own: the + // caller's listener has to still be there, after ours, when ours runs. + await withNoSignalHandlers(async () => { + const sentinel = (): void => {}; + process.on('SIGINT', sentinel); + const dir = project({ 'server.mjs': ESM_SEAM }); + const running = probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'server.mjs'), timeoutMs: 20_000, settleMs: 2_000 }); + await untilInstalled(2); + + const listeners = process.listeners('SIGINT'); + expect(listeners[1], 'the probe did not prepend its handler').toBe(sentinel); + expect(listeners[0]).not.toBe(sentinel); + + await running; + expect(process.listeners('SIGINT')).toEqual([sentinel]); + }); + }); + + it('do not override a pre-existing handler registered with once', async () => { + // A `once` handler removes itself as it runs. Running after one and then counting would read zero — + // the same as nobody listening — and re-sending the signal then kills a process whose only handler + // has already dealt with it. + await withNoSignalHandlers(async () => { + let handled = 0; + process.once('SIGINT', () => { + handled++; + }); + const dir = project({ 'server.mjs': ESM_SEAM }); + const running = probeRuntimeTraversal({ cwd: dir, entry: join(dir, 'server.mjs'), timeoutMs: 20_000, settleMs: 2_000 }); + await untilInstalled(2); + + process.kill(process.pid, 'SIGINT'); + await running; + await new Promise((r) => setTimeout(r, 100)); + // Called once, and this process is still here to say so. + expect(handled).toBe(1); + }); + }); + + it('are removed again, so a session that runs the check twice accumulates none', async () => { + const before = counts(); + const dir = project({ 'server.mjs': ESM_SEAM }); + await run(dir, 'server.mjs'); + await run(dir, 'server.mjs'); + expect(counts()).toEqual(before); + }); +}); + +describe('the child output it retains', SLOW, () => { + it('keeps a bounded excerpt with the challenge and its answer removed', async () => { + const dir = project({ + 'noisy.mjs': `${ESM_SEAM} +console.log('starting with ' + process.env[${JSON.stringify(CHALLENGE_ENV)}]); +console.log('x'.repeat(20000));`, + }); + const result = await run(dir, 'noisy.mjs'); + expect(result.outcome).toBe('proven'); + expect(result.output).toContain('starting with '); + expect(result.output.length).toBeLessThanOrEqual(4_096); + }); + + it('removes a challenge that straddles the bound, rather than keeping half of it', async () => { + // Padded so the challenge begins just under the limit and ends past it. Bounding the text before + // redacting it leaves the first half of the value in output this promises not to carry. + const dir = project({ + 'straddling.mjs': `${ESM_SEAM} +console.log('y'.repeat(4060)); +console.log('the challenge is ' + process.env[${JSON.stringify(CHALLENGE_ENV)}]);`, + }); + const result = await run(dir, 'straddling.mjs'); + expect(result.outcome).toBe('proven'); + expect(result.output).toContain('the challenge is '); + // The challenge is the only hex of that length anything here prints. + expect(result.output).not.toMatch(/[0-9a-f]{16}/); + expect(result.output.length).toBeLessThanOrEqual(4_096); + }); +}); diff --git a/tests/protect/verify-sentinel-seams.test.ts b/tests/protect/verify-sentinel-seams.test.ts new file mode 100644 index 00000000..a01bffb1 --- /dev/null +++ b/tests/protect/verify-sentinel-seams.test.ts @@ -0,0 +1,205 @@ +import { describe, expect, it, afterEach, vi } from 'vitest'; +import { mkdtempSync, writeFileSync, readFileSync, readdirSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { pathToFileURL, fileURLToPath } from 'node:url'; +import { transformSync } from 'esbuild'; +import { VERIFY_HEADER } from '../../src/protect/verify-sentinel.js'; +// The value the harness will compare against, computed its way (node crypto) rather than the seam's +// (web crypto). Asserting the seam against the OTHER implementation is what makes the two agree. +import { expectedAnswerFor } from '../../src/protect/install/runtime/probe.js'; + +/** + * Every seam a scaffolded guard exposes, asked whether a verification request reaches it. + * + * Per (template, SEAM) rather than per file: a template can expose more than one, and the generic guard + * exposes two — a fetch handler and a Node middleware. A file-level case would prove one and imply the + * other. + * + * Two things are asserted for each. The seam answers with the value derived from the challenge, which is + * what a run distinguishes from an app answering by accident. And the app's own handler is not called: a + * seam that answered after passing the request on would report traversal for a request the application + * had already handled. + */ +const TEMPLATE_DIR = new URL('../../src/protect/templates/', import.meta.url); +const CHALLENGE = 'e'.repeat(64); +const RULES = JSON.stringify({ firewall: [], whitelists: [] }); + +type Drive = (api: Record, ranHandler: () => void) => Promise; + +/** The fetch seam: what it answers is a Response body. */ +const fetchSeam: Drive = async (api, ranHandler) => { + const handler = async () => { + ranHandler(); + + return new Response('the app answered', { status: 200 }); + }; + const answer: any = await api.protectFetch(handler)( + new Request('https://app.example.com/', { headers: { [VERIFY_HEADER]: CHALLENGE } }), + ); + + return answer instanceof Response ? (await answer.text()).trim() : null; +}; + +/** The Node/Express seam: what it answers is written to the response, and `next` is the app's turn. */ +const nodeSeam: Drive = async (api, ranHandler) => + new Promise((resolve) => { + const chunks: string[] = []; + const res = { + statusCode: 0, + setHeader: () => {}, + end: (body?: string) => { + if (body) chunks.push(body); + resolve(chunks.join('').trim() || null); + }, + }; + api.patchstackMiddleware({ method: 'GET', url: '/', headers: { [VERIFY_HEADER]: CHALLENGE } }, res, () => { + ranHandler(); + resolve(null); + }); + setTimeout(() => resolve(null), 300); + }); + +/** The Fastify seam: the hook answers through the reply, and reaching the route is the app's turn. */ +const fastifySeam: Drive = async (api, ranHandler) => { + const hooks: Array<(request: unknown, reply: unknown) => unknown> = []; + await api.patchstackFastify({ addHook: (_event: string, hook: never) => hooks.push(hook) }); + let sent: string | null = null; + for (const hook of hooks) { + await hook( + { method: 'GET', url: '/', headers: { host: 'app.example.com', [VERIFY_HEADER]: CHALLENGE } }, + { + code: () => {}, + header: () => {}, + send: (body: unknown) => { + sent = String(body).trim(); + }, + }, + ); + } + if (sent === null) ranHandler(); + + return sent; +}; + +const SEAMS: Record = { + 'generic-guard.ts#protectFetch': fetchSeam, + 'generic-guard.js#protectFetch': fetchSeam, + 'generic-guard.cjs#protectFetch': fetchSeam, + 'generic-guard.ts#patchstackMiddleware': nodeSeam, + 'generic-guard.js#patchstackMiddleware': nodeSeam, + 'generic-guard.cjs#patchstackMiddleware': nodeSeam, + 'express-guard.ts#patchstackMiddleware': nodeSeam, + 'express-guard.js#patchstackMiddleware': nodeSeam, + 'express-guard.cjs#patchstackMiddleware': nodeSeam, + 'fastify-plugin.ts#patchstackFastify': fastifySeam, + 'fastify-plugin.js#patchstackFastify': fastifySeam, + 'fastify-plugin.cjs#patchstackFastify': fastifySeam, +}; + +const dirs: string[] = []; + +afterEach(() => { + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); + vi.unstubAllEnvs(); + vi.restoreAllMocks(); +}); + +/** + * The template as an app receives it, with the package redirected to a stub that carries the REAL + * sentinel — the thing under test — and a protection that would screen if it were ever asked to. + */ +async function seamWith(name: string): Promise> { + const commonjs = name.endsWith('.cjs'); + const dir = mkdtempSync(join(tmpdir(), 'ps-sentinel-')); + dirs.push(dir); + writeFileSync(join(dir, 'package.json'), JSON.stringify({ type: commonjs ? 'commonjs' : 'module' })); + + // `fileURLToPath`, not `.pathname`: a path with a space in it stays percent-encoded there, and the + // module then cannot be found. + const sentinelUrl = new URL('../../src/protect/verify-sentinel.js', import.meta.url).href; + const sentinelPath = fileURLToPath(sentinelUrl); + const stub = commonjs ? 'stub.cjs' : 'stub.mjs'; + // A protection that answers everything, so "the seam answered" cannot be the protection answering. + const protection = + '{ express: () => (_req, _res, next) => next(), node: () => (_req, _res, next) => next(),' + + ' fetchGuard: () => async () => null, screenResponse: async (response) => response }'; + writeFileSync( + join(dir, stub), + commonjs + ? // The sentinel is an ES module, so a CommonJS stub reaches it the way a CommonJS app would have + // to: dynamically. What matters is that the real one answers, not how the stub gets to it. + `let real;\n` + + `const load = async () => (real ??= await import(${JSON.stringify(sentinelUrl)}));\n` + + `module.exports = {\n` + + ` createProtection: async () => (${protection}),\n` + + ` VERIFY_HEADER: "x-patchstack-verify",\n` + + ` sentinelAnswer: async (offered) => (await load()).sentinelAnswer(offered),\n` + + `};\n` + : `export { sentinelAnswer, VERIFY_HEADER } from ${JSON.stringify(sentinelPath)};\n` + + `export const createProtection = async () => (${protection});\n`, + ); + writeFileSync(join(dir, 'rules.json'), RULES); + writeFileSync(join(dir, 'rules-stub.mjs'), `export default ${RULES};\n`); + + let source = readFileSync(new URL(name, TEMPLATE_DIR), 'utf8') + .replace('"@patchstack/connect/protect"', JSON.stringify(`./${stub}`)) + .replace(/from "\.\/(?:patchstack\.)?rules\.json"/, 'from "./rules-stub.mjs"'); + let file = join(dir, name); + if (name.endsWith('.ts')) { + source = transformSync(source, { loader: 'ts', format: 'esm' }).code; + file = join(dir, `${name.slice(0, -3)}.mjs`); + } + writeFileSync(file, source); + const loaded: any = await import(pathToFileURL(file).href); + + return loaded.default && commonjs ? loaded.default : loaded; +} + +describe('a verification request, at every seam a guard exposes', () => { + it('has a case for every seam in every template that carries the sentinel', () => { + const pairs = readdirSync(new URL(TEMPLATE_DIR)) + .filter((name) => /^(?:generic-guard|express-guard|fastify-plugin)\.(?:ts|js|cjs)$/.test(name)) + .flatMap((name) => { + const source = readFileSync(new URL(name, TEMPLATE_DIR), 'utf8'); + // The seams are the exports that answer requests, and the sentinel has to be in each of them. + return ['protectFetch', 'patchstackMiddleware', 'patchstackFastify'] + .filter((seam) => new RegExp(`function ${seam}\\b`).test(source)) + .map((seam) => `${name}#${seam}`); + }); + + expect(pairs.sort()).toEqual(Object.keys(SEAMS).sort()); + expect(pairs).toHaveLength(12); + }); + + for (const [pair, drive] of Object.entries(SEAMS)) { + it(`${pair}: answers it, and the app never sees it`, async () => { + vi.stubEnv('PATCHSTACK_VERIFY_CHALLENGE', CHALLENGE); + let ranHandler = 0; + + const api = await seamWith(pair.split('#')[0]); + const answer = await drive(api, () => { + ranHandler += 1; + }); + + expect(answer, 'the seam answers with the value derived from the challenge').toBe( + expectedAnswerFor(CHALLENGE), + ); + expect(ranHandler, "the app's own handler never ran").toBe(0); + }); + + it(`${pair}: is untouched without a challenge`, async () => { + // The production state: the same request, and the seam does what it always does. + vi.stubEnv('PATCHSTACK_VERIFY_CHALLENGE', ''); + let ranHandler = 0; + + const api = await seamWith(pair.split('#')[0]); + const answer = await drive(api, () => { + ranHandler += 1; + }); + + expect(answer).not.toBe(expectedAnswerFor(CHALLENGE)); + expect(ranHandler, 'the app handled it').toBe(1); + }); + } +}); diff --git a/tests/protect/verify-sentinel.test.ts b/tests/protect/verify-sentinel.test.ts new file mode 100644 index 00000000..ca48e15f --- /dev/null +++ b/tests/protect/verify-sentinel.test.ts @@ -0,0 +1,88 @@ +import { describe, expect, it, afterEach, vi } from 'vitest'; +import { sentinelAnswer, VERIFY_HEADER } from '../../src/protect/verify-sentinel.js'; +// The value the harness will compare against, computed its way (node crypto) rather than the seam's +// (web crypto). Asserting the seam against the OTHER implementation is what makes the two agree. +import { expectedAnswerFor } from '../../src/protect/install/runtime/probe.js'; + +/** + * When a scaffolded guard answers a verification request, and when it does nothing at all. + * + * This code sits on the request path of every app that installs a guard, so every state other than a + * live verification leaves the request untouched. A live verification exists only in a process the + * harness started, holding a challenge it minted for that run. + */ +const CHALLENGE = 'c'.repeat(64); + +afterEach(() => { + vi.unstubAllEnvs(); +}); + +describe('the answer to a verification request', () => { + it('is a digest of the challenge, not the challenge itself', async () => { + vi.stubEnv('PATCHSTACK_VERIFY_CHALLENGE', CHALLENGE); + + const answer = await sentinelAnswer(CHALLENGE); + + // Derived rather than echoed, so the harness can tell this seam's answer from an app that reflects + // request headers — and from a route that happens to return a fixed string. + expect(answer).toBe(expectedAnswerFor(CHALLENGE)); + expect(answer).not.toBe(CHALLENGE); + expect(answer).toMatch(/^[0-9a-f]{64}$/); + }); + + it('differs for a different challenge', async () => { + const other = 'd'.repeat(64); + + expect(expectedAnswerFor(CHALLENGE)).not.toBe(expectedAnswerFor(other)); + }); + + for (const [what, offered] of [ + ['nothing is offered', undefined], + ['the wrong value is offered', 'd'.repeat(64)], + ['a prefix of the challenge is offered', CHALLENGE.slice(0, 32)], + ['the offered value is not a string', 12345], + ] as const) { + it(`is nothing when ${what}`, async () => { + vi.stubEnv('PATCHSTACK_VERIFY_CHALLENGE', CHALLENGE); + + expect(await sentinelAnswer(offered as never)).toBeNull(); + }); + } + + it('is nothing when no challenge was configured, whatever is offered', async () => { + vi.stubEnv('PATCHSTACK_VERIFY_CHALLENGE', ''); + + // The production state: the branch exists and cannot be entered, whoever asks. + expect(await sentinelAnswer(CHALLENGE)).toBeNull(); + expect(await sentinelAnswer('')).toBeNull(); + }); + + it('is nothing when the configured challenge is too short to be one of ours', async () => { + // A guess at the variable rather than a challenge the harness minted. + vi.stubEnv('PATCHSTACK_VERIFY_CHALLENGE', 'short'); + + expect(await sentinelAnswer('short')).toBeNull(); + }); + + it('fails open, rather than throwing, on a runtime with no web crypto', async () => { + vi.stubEnv('PATCHSTACK_VERIFY_CHALLENGE', CHALLENGE); + // Restored from its own descriptor: on Node this is an accessor, and replacing it with a data + // property would leave every later test with a different shape than it had. + const descriptor = Object.getOwnPropertyDescriptor(globalThis, 'crypto'); + // @ts-expect-error - removing it is the point + delete globalThis.crypto; + try { + // What is established is the fail-open: no throw on the request path, and no interception of + // ordinary traffic. Runtime verification needs web crypto and only ever runs where it is present, + // so this state has no verification outcome of its own — a run against it would read as a seam + // the request did not reach. + expect(await sentinelAnswer(CHALLENGE)).toBeNull(); + } finally { + if (descriptor) Object.defineProperty(globalThis, 'crypto', descriptor); + } + }); + + it('names the header the harness sends', () => { + expect(VERIFY_HEADER).toBe('x-patchstack-verify'); + }); +});