feat(cli): machine-output mode — stream discipline, flag propagation, scrubber (#1234)

Three-layer redesign that delivers pipe-safe --json/--dry-run while
scaling to every command and child process socket-cli spawns.

Layer 1 — stream discipline (CLAUDE.md SHARED STANDARDS):
- stdout carries ONLY the data a command was asked to produce.
- stderr carries everything else: progress, spinners, status, warnings,
  errors, dry-run previews, context, banners, prompts.
- Under --json/--markdown, stdout MUST parse as the declared format.
- Targeted audit of output formatters: dry-run previews, scan-report
  status lines ("Writing json report to …"), coana-fix "Copying …"
  message, oops dry-run preview all moved from stdout to stderr.

Layer 2 — per-(tool, subcommand) flag propagation
(utils/spawn/machine-mode.mts):
- coana: --silent + --socket-mode <tempfile>; read file.
- sfw: transparent — forward to inner PM (npm/pnpm/yarn/yarn-berry/zpm/pip).
- npm: --json + --loglevel=error on JSON-aware subcommands.
- pnpm: --reporter=json on JSON-emitting subcommands; --reporter=silent
  otherwise, via a fallbackArgs rule so the two reporter flags never
  collide.
- yarn classic: --json --silent.
- yarn berry: --json where supported + full YARN_ENABLE_* env quieting.
- yarn 6 / zpm: --json on supported cmds; --silent on install+add.
- vltpkg: --view=json uniformly.
- pip/pip3/uv/cargo/gem/go: per-tool quiet/json flags.
- Universal env injection: NO_COLOR, FORCE_COLOR=0, CLICOLOR_FORCE=0.

Layer 3 — output scrubber (utils/output/scrubber.mts):
- Transform stream that catches the residue from tools that don't cooperate
  (synp "Created …" line, zpm ANSI leaks, gem progress dots, unknown
  binaries). NDJSON-aware line classifier:
    1. NUL sentinel spans (socket-cli's own output) → stdout.
    2. JSON.parse succeeds → stdout.
    3. Known noise patterns (log prefixes, status glyphs) → stderr.
    4. Unknown → stderr (safe default).
- Uses ansiRegex() from @socketsecurity/lib/ansi for OSC+CSI stripping.
- Tracing via SOCKET_SCRUB_TRACE=1 for debugging.
- Tool-specific adapters (synp, zpm, gem) plug into the classifier
  registry — small, inspectable, jc-style.
- Soft buffer cap (maxBufferChars) surfaces a one-time "exceeded cap;
  scrubbing continues without fallback" warning. A hard cap with
  passthrough is deliberately deferred — a child tool emitting 100 MB
  without a newline is an upstream bug, and silently flushing a partial
  line as payload would corrupt the stream.

Layer 4 — sentinel wrapping (utils/output/emit-payload.mts):
- emitPayload() wraps the primary payload in NUL-bracketed sentinels
  (\0SOCKET_PAYLOAD_BEGIN\0 / \0SOCKET_PAYLOAD_END\0) under machine mode
  so the scrubber extracts it unambiguously.
- NUL never appears in legitimate JSON/Markdown, so sentinels are
  zero-ambiguity even against a cooperating child tool. Doc comments in
  mode.mts and scrubber.mts explicitly call out the NUL-bracketing so
  the guarantee is grounded in prose, not invisible source bytes.

Ambient mode context (utils/output/ambient-mode.mts):
- meowWithSubcommands / meowOrExit call setMachineOutputMode() once per
  invocation with the parsed flags, resetting prior state so sequential
  vitest cases don't leak mode.
- Spawn wrappers consult getMachineOutputMode() to decide whether to
  apply flag forwarding and scrubbing.

Flag rename:
- --no-log → --quiet. Reframed description: "Route non-essential output
  (status, progress, warnings) to stderr so stdout carries only the
  payload. Implied by --json and --markdown."

Deletions:
- utils/output/no-log.mts + setNoLogMode/isNoLogMode module-level state.
- out() wrapper in utils/dry-run/output.mts — plain logger.error now.

Tests:
- 74 new unit tests across mode, emit-payload, scrubber, pipe-safety,
  machine-mode.
- 134 existing test assertions updated to expect dry-run text on stderr
  (mockLogger.error) rather than stdout (mockLogger.log).
- All 5,202 unit tests pass; type-check and lint clean.

Follow-ups tracked separately:
- Cross-repo: structured NDJSON event contract ("reason" field per
  cargo's --message-format=json).
- Upstream: coana native --machine-output that routes logger to stderr;
  zpm NO_COLOR respect; synp --quiet/--json flags.
- Full socket-cli-wide sweep of the remaining ~500 logger.log call sites
  for stream-discipline conformance (lib-side fix to @socketsecurity/lib
  will automate most of this).

Author: jdalton

CLAUDE.md

@@ -114,6 +114,7 @@ If user repeats instruction 2+ times, ask: "Should I add this to CLAUDE.md?"
 - HTTP Requests: NEVER use `fetch()` — use `httpJson`/`httpText`/`httpRequest` from `@socketsecurity/lib/http-request`
 - `Promise.race` / `Promise.any`: NEVER pass a long-lived promise (interrupt signal, pool member) into a race inside a loop. Each call re-attaches `.then` handlers to every arm; handlers accumulate on surviving promises until they settle. For concurrency limiters, use a single-waiter "slot available" signal (resolved by each task's `.then`) instead of re-racing `executing[]`. See nodejs/node#17469 and `@watchable/unpromise`. Race with two fresh arms (e.g. one-shot `withTimeout`) is safe.
 - File existence: ALWAYS `existsSync` from `node:fs`. NEVER `fs.access`, `fs.stat`-for-existence, or an async `fileExists` wrapper. Import: `import { existsSync, promises as fs } from 'node:fs'`.
+- Stream discipline: stdout carries ONLY the data the command was asked to produce (JSON payload, report, fix output — the thing a script pipes into `jq` or redirects to a file). stderr carries everything else: progress, spinners, status, warnings, errors, dry-run previews, context, banners, prompts. Test: would `command | jq` or `command > file` make sense with only the stdout content? If no, it belongs on stderr. Under `--json` / `--markdown`, stdout MUST parse as the declared format with no prefix/suffix text.
 
 ### Documentation Policy
 

packages/cli/src/commands/fix/coana-fix.mts

@@ -99,10 +99,16 @@ export async function coanaFix(
     spinner,
   } = fixConfig
 
-  // Determine stdio based on output mode:
-  // - 'ignore' when outputKind === 'json': suppress all coana output, return clean JSON response
-  // - 'inherit' otherwise: user sees coana progress in real-time
+  // Under json/markdown mode we route coana's chatter away from our
+  // stdout (its JSON report comes from --output-file, not stdout, so
+  // coana stdout is entirely informational). 'ignore' drops it; that
+  // was the previous behavior and it remains safe. When interactive we
+  // inherit so the user sees coana progress in real-time.
   const coanaStdio = outputKind === 'json' ? 'ignore' : 'inherit'
+  // Ask coana to silence its own Winston logger under json mode. Belt
+  // and braces with stdio:'ignore' and harmless if coana ignores the
+  // flag.
+  const coanaSilenceArgs = outputKind === 'json' ? ['--silent'] : []
 
   const fixEnv = await getFixEnv()
   debugDir({ fixEnv })
@@ -214,6 +220,7 @@ export async function coanaFix(
     try {
       const fixCResult = await spawnCoanaDlx(
         [
+          ...coanaSilenceArgs,
           'compute-fixes-and-upgrade-purls',
           cwd,
           '--manifests-tar-hash',
@@ -254,7 +261,9 @@ export async function coanaFix(
 
       // Copy to outputFile if provided.
       if (outputFile) {
-        logger.info(`Copying fixes result to ${outputFile}`)
+        // Status message — belongs on stderr so stdout stays payload-only
+        // when a consumer is piping `socket fix --json`.
+        logger.error(`Copying fixes result to ${outputFile}`)
         const tmpContent = await fs.readFile(tmpFile, 'utf8')
         await fs.writeFile(outputFile, tmpContent, 'utf8')
       }
@@ -427,6 +436,7 @@ export async function coanaFix(
     // eslint-disable-next-line no-await-in-loop
     const fixCResult = await spawnCoanaDlx(
       [
+        ...coanaSilenceArgs,
         'compute-fixes-and-upgrade-purls',
         cwd,
         '--manifests-tar-hash',

packages/cli/src/commands/oops/cmd-oops.mts

@@ -60,24 +60,26 @@ async function run(
   const dryRun = !!cli.flags['dryRun']
 
   if (dryRun) {
-    logger.log('')
-    logger.log(`${DRY_RUN_LABEL}: Would trigger an intentional error`)
-    logger.log('')
-    logger.log(
+    // Dry-run previews are contextual output; route to stderr per the
+    // stream discipline rule so stdout stays payload-only.
+    logger.error('')
+    logger.error(`${DRY_RUN_LABEL}: Would trigger an intentional error`)
+    logger.error('')
+    logger.error(
       '  This command throws an error for development/testing purposes.',
     )
-    logger.log(`  Error message: "This error was intentionally left blank."`)
-    logger.log('')
+    logger.error(`  Error message: "This error was intentionally left blank."`)
+    logger.error('')
     if (json && !justThrow) {
-      logger.log('  Output format: JSON error response')
+      logger.error('  Output format: JSON error response')
     } else if (markdown && !justThrow) {
-      logger.log('  Output format: Markdown error message')
+      logger.error('  Output format: Markdown error message')
     } else {
-      logger.log('  Output format: Thrown Error exception')
+      logger.error('  Output format: Thrown Error exception')
     }
-    logger.log('')
-    logger.log('  Run without --dry-run to trigger the error.')
-    logger.log('')
+    logger.error('')
+    logger.error('  Run without --dry-run to trigger the error.')
+    logger.error('')
     return
   }
 

packages/cli/src/commands/organization/output-quota.mts

@@ -1,6 +1,7 @@
 import { getDefaultLogger } from '@socketsecurity/lib/logger'
 
 import { failMsgWithBadge } from '../../utils/error/fail-msg-with-badge.mts'
+import { emitPayload } from '../../utils/output/emit-payload.mts'
 import { mdHeader } from '../../utils/output/markdown.mts'
 import { serializeResultJson } from '../../utils/output/result-json.mjs'
 
@@ -59,7 +60,9 @@ export async function outputQuota(
   }
 
   if (outputKind === 'json') {
-    logger.log(serializeResultJson(result))
+    // Sentinel-wrap the JSON so pipe-safety is preserved even if a
+    // downstream spawn in the same process writes to stdout.
+    emitPayload(serializeResultJson(result), { flags: { json: true } })
     return
   }
   if (!result.ok) {
@@ -71,11 +74,10 @@ export async function outputQuota(
   const refreshLine = `Next refresh: ${formatRefresh(result.data.nextWindowRefresh)}`
 
   if (outputKind === 'markdown') {
-    logger.log(mdHeader('Quota'))
-    logger.log('')
-    logger.log(`- ${usageLine}`)
-    logger.log(`- ${refreshLine}`)
-    logger.log('')
+    const md = [mdHeader('Quota'), '', `- ${usageLine}`, `- ${refreshLine}`].join(
+      '\n',
+    )
+    emitPayload(md, { flags: { markdown: true } })
     return
   }
 

packages/cli/src/commands/scan/output-scan-report.mts

@@ -111,7 +111,7 @@ export async function outputScanReport(
       : toJsonReport(scanReport.data as ScanReport, includeLicensePolicy)
 
     if (filepath && filepath !== '-') {
-      logger.log('Writing json report to', filepath)
+      logger.error('Writing json report to', filepath)
       return await fs.writeFile(filepath, json)
     }
 
@@ -129,7 +129,7 @@ export async function outputScanReport(
         )
 
     if (filepath && filepath !== '-') {
-      logger.log('Writing markdown report to', filepath)
+      logger.error('Writing markdown report to', filepath)
       return await fs.writeFile(filepath, md)
     }
 

packages/cli/src/commands/scan/perform-reachability-analysis.mts

@@ -10,6 +10,7 @@ import {
 } from '../../constants/socket.mts'
 import { extractTier1ReachabilityScanId } from '../../utils/coana/extract-scan-id.mjs'
 import { spawnCoanaDlx } from '../../utils/dlx/spawn.mjs'
+import { getMachineOutputMode } from '../../utils/output/ambient-mode.mts'
 import { hasEnterpriseOrgPlan } from '../../utils/organization.mts'
 import { handleApiCall } from '../../utils/socket/api.mjs'
 import { setupSdk } from '../../utils/socket/sdk.mjs'
@@ -173,7 +174,11 @@ export async function performReachabilityAnalysis(
     ? outputPath
     : DOT_SOCKET_DOT_FACTS_JSON
   // Build Coana arguments.
+  // Under machine-output mode, --silent suppresses coana's Winston
+  // logger entirely; the report still lands in --socket-mode's file.
+  const machineMode = getMachineOutputMode()
   const coanaArgs = [
+    ...(machineMode ? ['--silent'] : []),
     'run',
     analysisTarget,
     '--output-dir',
@@ -238,13 +243,15 @@ export async function performReachabilityAnalysis(
     coanaEnv['SOCKET_BRANCH_NAME'] = branchName
   }
 
-  // Run Coana with the manifests tar hash.
+  // Run Coana with the manifests tar hash. Under machine mode we drop
+  // coana stdout; --silent plus 'ignore' ensures our own stdout stays
+  // pipe-safe for --json consumers.
   const coanaResult = await spawnCoanaDlx(coanaArgs, orgSlug, {
     coanaVersion: reachabilityOptions.reachVersion || undefined,
     cwd,
     env: coanaEnv,
     spinner,
-    stdio: 'inherit',
+    stdio: machineMode ? 'ignore' : 'inherit',
   })
 
   if (wasSpinning) {

packages/cli/src/flags.mts

@@ -269,6 +269,12 @@ export const commonFlags: MeowFlags = {
     // Only show in root command in debug mode.
     hidden: true,
   },
+  quiet: {
+    type: 'boolean',
+    default: false,
+    description:
+      'Route non-essential output (status, progress, warnings) to stderr so stdout carries only the payload. Implied by --json and --markdown.',
+  },
   spinner: {
     type: 'boolean',
     default: true,

packages/cli/src/utils/cli/with-subcommands.mts

@@ -41,6 +41,10 @@ import {
 } from '../config.mts'
 import { isDebug } from '../debug.mts'
 import { tildify } from '../fs/home-path.mts'
+import {
+  resetMachineOutputMode,
+  setMachineOutputMode,
+} from '../output/ambient-mode.mts'
 import { getFlagListOutput, getHelpListOutput } from '../output/formatting.mts'
 import { getVisibleTokenPrefix } from '../socket/sdk.mjs'
 import {
@@ -510,15 +514,30 @@ export async function meowWithSubcommands(
   const {
     compactHeader: compactHeaderFlag,
     config: configFlag,
+    json: jsonFlag,
+    markdown: markdownFlag,
     org: orgFlag,
+    quiet: quietFlag,
     spinner: spinnerFlag,
   } = cli1.flags as {
     compactHeader: boolean
     config: string
+    json: boolean | undefined
+    markdown: boolean | undefined
     org: string
+    quiet: boolean | undefined
     spinner: boolean
   }
 
+  // Re-derive from the current argv so ambient mode doesn't leak across
+  // sequential invocations (e.g. multiple vitest cases in one worker).
+  resetMachineOutputMode()
+  setMachineOutputMode({
+    json: jsonFlag,
+    markdown: markdownFlag,
+    quiet: quietFlag,
+  })
+
   const compactMode = !!compactHeaderFlag || !!(getCI() && !VITEST)
   const noSpinner = spinnerFlag === false || isDebug()
 
@@ -937,17 +956,32 @@ export function meowOrExit(
   const {
     compactHeader: compactHeaderFlag,
     help: helpFlag,
+    json: jsonFlag,
+    markdown: markdownFlag,
     org: orgFlag,
+    quiet: quietFlag,
     spinner: spinnerFlag,
     version: versionFlag,
   } = cli.flags as {
     compactHeader: boolean
     help: boolean
+    json: boolean | undefined
+    markdown: boolean | undefined
     org: string
+    quiet: boolean | undefined
     spinner: boolean
     version: boolean | undefined
   }
 
+  // Apply machine-output mode from this command's flags. Reset first
+  // so prior in-worker state doesn't leak across sequential invocations.
+  resetMachineOutputMode()
+  setMachineOutputMode({
+    json: jsonFlag,
+    markdown: markdownFlag,
+    quiet: quietFlag,
+  })
+
   const compactMode = !!compactHeaderFlag || !!(getCI() && !VITEST)
   const noSpinner = spinnerFlag === false || isDebug()
 

packages/cli/src/utils/dlx/spawn.mts

@@ -62,6 +62,10 @@ import { SOCKET_CLI_PYTHON_PATH } from '../../env/socket-cli-python-path.mts'
 import { getSynpVersion } from '../../env/synp-version.mts'
 import { getErrorCause, InputError } from '../error/errors.mts'
 import { isSeaBinary } from '../sea/detect.mts'
+import {
+  applyMachineModeIfActive,
+  inferSubcommand,
+} from '../spawn/apply-machine-mode.mts'
 import { socketHttpRequest } from '../socket/api.mjs'
 import { spawnNode } from '../spawn/spawn-node.mjs'
 import { getDefaultApiToken, getDefaultProxyUrl } from '../socket/sdk.mjs'
@@ -500,6 +504,23 @@ export async function spawnSfwDlx(
   options?: DlxOptions | undefined,
   spawnExtra?: SpawnExtra | undefined,
 ): Promise<DlxSpawnResult> {
+  // sfw is a transparent proxy: args is [innerTool, innerSubcommand?, ...rest].
+  // Machine-mode flags forward to the inner tool so its stdout stays
+  // pipe-safe under --json.
+  const [innerTool, ...innerArgs] = args
+  const innerSubcommand = inferSubcommand(innerArgs)
+  const innerApplied = innerTool
+    ? applyMachineModeIfActive({
+        args: innerArgs,
+        env: undefined,
+        subcommand: innerSubcommand,
+        tool: innerTool,
+      })
+    : { args: [...innerArgs], env: {} }
+  const effectiveArgs = innerTool
+    ? [innerTool, ...innerApplied.args]
+    : [...args]
+
   const resolution = resolveSfw()
 
   // Use local sfw if available.
@@ -511,13 +532,16 @@ export async function spawnSfwDlx(
     } as DlxOptions
 
     const spawnArgs =
-      detection.type === 'binary' ? args : [resolution.path, ...args]
+      detection.type === 'binary'
+        ? effectiveArgs
+        : [resolution.path, ...effectiveArgs]
     const spawnCommand = detection.type === 'binary' ? resolution.path : 'node'
 
     const spawnPromise = spawn(spawnCommand, spawnArgs, {
       ...dlxOptions,
       env: {
         ...process.env,
+        ...innerApplied.env,
         ...spawnEnv,
       },
       stdio: (spawnExtra?.['stdio'] as StdioOptions | undefined) ?? 'inherit',
@@ -534,8 +558,15 @@ export async function spawnSfwDlx(
   }
   return await spawnDlx(
     resolution.details,
-    args,
-    { force: false, ...options },
+    effectiveArgs,
+    {
+      force: false,
+      ...options,
+      env: {
+        ...innerApplied.env,
+        ...options?.env,
+      },
+    },
     spawnExtra,
   )
 }