Replace Web Worker with subprocess-based execution architecture #80

lambdalisue · 2026-01-04T14:12:58Z

Summary

Add subprocess execution infrastructure with NDJSON protocol
Migrate run and list commands to use subprocesses instead of Web Workers
Add template embedding system for distributable CLI
Remove all Web Worker-related code (~1,200 lines deleted)

Why

Web Workers in Deno have critical limitations for our CLI use case:

Problem with Workers:

Cannot resolve bare specifiers dynamically (requires bundling all dependencies)
Cannot pass Deno CLI flags (--unstable-kv, --allow-*, etc.) to worker context
Complex serialization boundaries for error handling
Limited process isolation

Subprocess Benefits:

Each subprocess runs as independent deno run with full module resolution
Natural flag forwarding: users' --unstable-* flags propagate to scenario execution
Simpler error propagation via ErrorObject serialization over NDJSON
Better process isolation and resource cleanup

Technical Implementation:

Template system resolves dependencies at build time and embeds them
NDJSON protocol for structured bidirectional communication
Ready-signal pattern ensures subprocess initialization before sending input
Graceful abort handling via AbortSignal propagation

Test Plan

All existing tests pass (worker_test.ts removed, protocol_test.ts moved to templates)
Run deno task verify to confirm no regressions
Test probitas run with --unstable-kv flag
Test probitas list with selectors
Verify error propagation from scenarios
Test abort signal handling (Ctrl+C during execution)

Copilot

Pull request overview

This PR replaces the Web Worker-based execution architecture with a subprocess-based approach to address critical limitations in Deno's Worker API, particularly around module resolution and CLI flag propagation.

Key Changes:

Implemented subprocess execution infrastructure with NDJSON-based protocol for bidirectional communication
Added template resolution system that embeds dependencies at build time for distributable CLI
Migrated run and list commands to use isolated Deno subprocesses instead of Web Workers

Reviewed changes

Copilot reviewed 17 out of 19 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
`src/cli/subprocess.ts`	Core subprocess spawning utilities with NDJSON stream processing
`src/cli/subprocess_template.ts`	Template resolution system that converts bare specifiers to versioned URLs
`src/cli/_templates/run.ts`	Subprocess entry point for scenario execution with stdin/stdout communication
`src/cli/_templates/list.ts`	Subprocess entry point for scenario listing
`src/cli/_templates/run_protocol.ts`	Protocol definitions and serialization for run command communication
`src/cli/_templates/list_protocol.ts`	Protocol definitions for list command communication
`src/cli/_templates/utils.ts`	Shared subprocess utilities for stdin/stdout and logging configuration
`src/cli/_templates/README.md`	Documentation for subprocess template architecture
`src/cli/commands/run.ts`	Refactored to spawn subprocess instead of Worker for scenario execution
`src/cli/commands/list.ts`	Refactored to use subprocess for scenario loading and filtering
`src/cli/utils.ts`	Added `extractDenoOptions` function to extract `--deno-*` CLI flags
`scripts/build_subprocess_templates.ts`	Build script that generates embedded templates for compiled binaries
`src/cli/_embedded_templates.ts`	Auto-generated file containing resolved template sources
`src/cli/commands/run/worker_test.ts`	Removed legacy Worker tests (~831 lines)
`src/cli/commands/run/worker.ts`	Removed legacy Worker implementation (~231 lines)
`src/cli/_templates/run_protocol_test.ts`	Updated import path from old protocol location
`deno.json`	Added dependencies and compile task for subprocess template embedding
`deno.lock`	Updated lockfile with new dependencies
`.gitignore`	Added patterns for dist/ and embedded templates

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cli/_templates/run.ts

src/cli/_templates/utils.ts

src/cli/utils.ts

src/cli/subprocess.ts

src/cli/subprocess_template.ts

src/cli/_templates/utils.ts

src/cli/_templates/list.ts

src/cli/subprocess.ts

Copilot

Pull request overview

Copilot reviewed 19 out of 21 changed files in this pull request and generated 9 comments.

Comments suppressed due to low confidence (1)

src/cli/_templates/run_protocol.ts:258

The createReporter function accepts a synchronous callback (output: (message: RunOutput) => void), but this is being passed an async function (writeOutput) in run.ts. This type signature mismatch means errors from writeOutput won't propagate correctly, and the Reporter won't know if IPC writes fail. Consider changing the signature to accept an async callback or handling the promise appropriately.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cli/_templates/run.ts

deno.json

src/cli/_templates/list.ts

src/cli/subprocess_template.ts

src/cli/utils.ts

src/cli/commands/list.ts

src/cli/commands/run.ts

src/cli/_templates/README.md

src/cli/_templates/list_protocol.ts

Copilot

Pull request overview

Copilot reviewed 19 out of 21 changed files in this pull request and generated 9 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cli/subprocess_template.ts

src/cli/commands/run.ts

src/cli/subprocess_template.ts

src/cli/utils.ts

src/cli/commands/list.ts

src/cli/commands/run.ts

src/cli/commands/list.ts

src/cli/_templates/utils.ts

src/cli/subprocess.ts

Introduces subprocess execution architecture to replace Web Workers, enabling better process isolation and dependency management. New Components: - subprocess.ts: Core subprocess spawning and NDJSON communication - subprocess_template.ts: Template resolution and embedding system - _templates/: Worker templates for run and list commands - _embedded_templates.ts: Pre-built template data for CLI distribution - build_subprocess_templates.ts: Build script for template embedding Why Subprocess over Worker: Web Workers cannot resolve bare specifiers dynamically, requiring all dependencies to be bundled. Subprocesses allow each worker to use `deno run` with proper module resolution, simplifying dependency management and enabling independent Deno permission control. Technical Details: - NDJSON protocol for structured subprocess communication - Template system resolves import maps before embedding - Ready-signal pattern ensures subprocess initialization - Graceful error propagation via ErrorObject serialization

…ution Refactors run and list commands to use the new subprocess infrastructure, removing all Web Worker-related code. Changes: - run.ts: Replace Worker with subprocess spawning and NDJSON streams - list.ts: Add subprocess-based scenario loading with filtering - utils.ts: Add extractDenoOptions() for --unstable-* flag support - Delete run/worker.ts, run/worker_test.ts (replaced by _templates/run.ts) - Delete run/protocol.ts, run/protocol_test.ts (moved to _templates/) Why This Change: The Worker-based approach required complex serialization boundaries and couldn't pass Deno flags to workers. Subprocess approach enables natural flag forwarding (--unstable-kv, permissions) and simplifies error handling through structured NDJSON communication.

…or subprocess Adds required dependencies and build tooling configuration to support the subprocess-based execution architecture. Changes: - deno.json: Add subprocess dependencies (@deno/graph, @std/streams, etc.) - deno.json: Add "compile" task for template embedding and CLI build - .gitignore: Exclude dist/ and generated _embedded_templates.ts - deno.lock: Update lock file with new dependencies Why These Dependencies: - @deno/graph: Analyze and resolve module dependencies for templates - @std/streams, @std/json: NDJSON communication with subprocesses - @core/streamutil: Stream utilities for subprocess I/O

…P sockets The previous stdin/stdout-based IPC prevented subprocesses from using console.log freely, as it would corrupt NDJSON message streams. This migration enables: - Subprocess console output for debugging (stdout/stderr inherited) - Parent-subprocess communication via dedicated TCP channel - Elimination of "ready" signal (connection itself signals readiness) Technical changes: - Subprocess connects to parent's ephemeral TCP server (--ipc-port arg) - NDJSON protocol unchanged, transport changed from stdio to TCP - IpcConnection abstraction provides typed read/write streams - Exclusive writer usage prevents concurrent write corruption

…ementation - Make createReporter methods async to properly await IPC writes, ensuring message ordering and error propagation - Add explicit process exit to list.ts subprocess to prevent hangs when async operations keep event loop alive - Update documentation comments to reflect TCP-based IPC (not stdin/stdout)

High priority fixes: - Add response.ok check for fetch in subprocess template resolution - Add IPC connection timeout with subprocess exit detection to prevent indefinite hangs when subprocess crashes before connecting Medium priority fixes: - Validate --deno-* options require =value syntax for non-boolean flags - Clarify misleading comment about allScenarios vs filteredScenarios - Improve abort result comment explaining count approximations

Copilot

Pull request overview

Copilot reviewed 19 out of 21 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cli/commands/list.ts

src/cli/subprocess_template.ts

src/cli/commands/run.ts

src/cli/_templates/run.ts

src/cli/_templates/list.ts

- Remove unnecessary type casting in list.ts (use spread instead) - Clarify comment about specifier resolution in subprocess_template.ts

Copilot

Pull request overview

Copilot reviewed 19 out of 21 changed files in this pull request and generated 11 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cli/commands/list.ts

src/cli/utils.ts

src/cli/subprocess.ts

src/cli/_templates/list.ts

src/cli/subprocess_template.ts

src/cli/_templates/run.ts

src/cli/utils.ts

src/cli/subprocess_template.ts

JSON.stringify cannot handle various JavaScript types (BigInt, Symbol, Function, circular references, etc.), which could cause failures in IPC-based step context passing. These scenarios verify the framework handles these edge cases correctly. Covers: BigInt, Function, Symbol, undefined, circular references, Map, Set, RegExp, Date, Error, TypedArray, ArrayBuffer, DataView. Addresses #80

Standard JSON.stringify fails on BigInt, circular references, Symbol, Function, Map, Set, RegExp, TypedArray, and other JavaScript types. This causes step context values to be lost or corrupted when passed between processes via IPC. The custom serializer uses a marker object pattern ($type property) to preserve type information across the IPC boundary, enabling steps to receive BigInt, Map, circular references, and other complex values through ctx.previous and ctx.results. Handles: BigInt, Symbol, Function, undefined, circular references, Map, Set, WeakMap, WeakSet, RegExp, Date, Error, TypedArray, ArrayBuffer, DataView, WeakRef. Addresses #80

JSON.stringify cannot handle various JavaScript types (BigInt, Symbol, Function, circular references, etc.), which could cause failures in IPC-based step context passing. These scenarios verify the framework handles these edge cases correctly. Covers: BigInt, Function, Symbol, undefined, circular references, Map, Set, RegExp, Date, Error, TypedArray, ArrayBuffer, DataView. Addresses #80

Standard JSON.stringify fails on BigInt, circular references, Symbol, Function, Map, Set, RegExp, TypedArray, and other JavaScript types. This causes step context values to be lost or corrupted when passed between processes via IPC. The custom serializer uses a marker object pattern ($type property) to preserve type information across the IPC boundary, enabling steps to receive BigInt, Map, circular references, and other complex values through ctx.previous and ctx.results. Handles: BigInt, Symbol, Function, undefined, circular references, Map, Set, WeakMap, WeakSet, RegExp, Date, Error, TypedArray, ArrayBuffer, DataView, WeakRef. Addresses #80

Copilot AI review requested due to automatic review settings January 4, 2026 14:12

Copilot started reviewing on behalf of lambdalisue January 4, 2026 14:13 View session

Copilot AI reviewed Jan 4, 2026

View reviewed changes

lambdalisue force-pushed the feature/subprocess branch 2 times, most recently from 8a7f2a6 to b8b3b48 Compare January 4, 2026 15:33

lambdalisue requested a review from Copilot January 4, 2026 15:36

Copilot started reviewing on behalf of lambdalisue January 4, 2026 15:37 View session

Copilot AI reviewed Jan 4, 2026

View reviewed changes

lambdalisue requested a review from Copilot January 4, 2026 15:58

Copilot started reviewing on behalf of lambdalisue January 4, 2026 15:58 View session

Copilot AI reviewed Jan 4, 2026

View reviewed changes

lambdalisue added 6 commits January 5, 2026 01:16

lambdalisue force-pushed the feature/subprocess branch from 3e4c196 to 47c8fb5 Compare January 4, 2026 16:16

lambdalisue requested a review from Copilot January 4, 2026 16:20

Copilot started reviewing on behalf of lambdalisue January 4, 2026 16:21 View session

Copilot AI reviewed Jan 4, 2026

View reviewed changes

src/cli/commands/list.ts Outdated Show resolved Hide resolved

src/cli/subprocess_template.ts Outdated Show resolved Hide resolved

src/cli/commands/run.ts Show resolved Hide resolved

src/cli/_templates/run.ts Show resolved Hide resolved

src/cli/_templates/list.ts Show resolved Hide resolved

chore(@probitas/probitas): minor code quality improvements

84504a4

- Remove unnecessary type casting in list.ts (use spread instead) - Clarify comment about specifier resolution in subprocess_template.ts

lambdalisue requested a review from Copilot January 4, 2026 16:34

Copilot started reviewing on behalf of lambdalisue January 4, 2026 16:34 View session

Copilot AI reviewed Jan 4, 2026

View reviewed changes

lambdalisue merged commit ddfd78b into main Jan 4, 2026
10 checks passed

lambdalisue deleted the feature/subprocess branch January 4, 2026 16:47

Uh oh!

Replace Web Worker with subprocess-based execution architecture #80

Replace Web Worker with subprocess-based execution architecture #80

Uh oh!

Conversation

lambdalisue commented Jan 4, 2026

Summary

Why

Test Plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development