perf(core): retain workflow VM across inline steps#2990
Open
NathanColosimo wants to merge 23 commits into
Open
perf(core): retain workflow VM across inline steps#2990NathanColosimo wants to merge 23 commits into
NathanColosimo wants to merge 23 commits into
Conversation
Contributor
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
Contributor
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (9)3498706Fri, 17 Jul 2026 23:54:22 GMT · run logs
e961466Fri, 17 Jul 2026 23:40:11 GMT · run logs
2b73676Fri, 17 Jul 2026 23:10:31 GMT · run logs
43788b8Fri, 17 Jul 2026 22:52:28 GMT · run logs
3bda890Fri, 17 Jul 2026 22:33:42 GMT · run logs
c6d6cb5Fri, 17 Jul 2026 22:02:43 GMT · run logs
bd07e11Fri, 17 Jul 2026 21:34:21 GMT · run logs
93d5afcFri, 17 Jul 2026 21:06:35 GMT · run logs
b32774fFri, 17 Jul 2026 20:51:27 GMT · run logs
Avg deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) 🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
🦋 Changeset detectedLatest commit: 3498706 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
NathanColosimo
force-pushed
the
nathanc/workflow-replay-perf-43e0a8
branch
from
July 17, 2026 21:19
5febf4e to
fd73574
Compare
- A mixed step batch (one unsafe sibling input) now serializes every input through the ordinary VM path: a clone snapshotted before an unsafe sibling's serialization runs its getters could otherwise durably capture stale sibling state. - crypto.subtle.digest rejects SharedArrayBuffer-backed views with TypeError, matching WebCrypto's BufferSource contract.
devalue serializes Map/Set through the realm's iterator protocol and Date/RegExp/typed arrays through prototype getters, all of which workflow code can mutate — so their serialization is not provably passive and their bytes could differ between retained and cold modes. The fast path now accepts only primitives, plain objects, and plain arrays, which devalue traverses exclusively via own-property reads. Slot-bearing exotics decline even with a swapped prototype. The sandbox digest now reads view metadata (buffer/byteOffset/ byteLength) through captured intrinsic getters, so own properties shadowing them cannot change which bytes are hashed or bypass the SharedArrayBuffer rejection.
instanceof dispatch (Symbol.hasInstance via the constructor, Function.prototype, and Object.prototype), the class reducer's value.constructor walk, and devalue's Object/Array traversal all consult intrinsics workflow code could redefine — legally and deterministically — which would make the durable step input depend on WORKFLOW_RETAINED_VM (spoofed values serialize as e.g. Maps on the cold path but as plain clones on the retained path). Freeze Object/Array/Function (constructors and prototypes), the VM collection constructors, and every reducer-referenced global binding (absent ones pinned to undefined) right before the workflow bundle evaluates, so the retained-input equivalence holds by construction. Host-realm constructor escapes (e.g. TextEncoder.constructor) remain out of the determinism contract: code scheduling host timers was never deterministic under ordinary replay either; documented on canRetainWorkflowSession.
Typed-array constructors (and their shared %TypedArray% parent), the Date wrapper, and the session-local AbortController/AbortSignal/ Request/Response bindings were pinned but not frozen, so workflow code could still add Symbol.hasInstance statics that diverge reducer dispatch between the retained clone (host constructors) and ordinary VM serialization. Freeze every binding value that is not the shared host intrinsic; shared host objects are dispatched identically by both paths, so mutations there cannot cause mode divergence.
Replace structuredClone with an explicit deep copy into an SDK-private realm: clones previously inherited host prototypes, which workflow code can reach (e.g. via structuredClone's return values) and vandalize with Symbol.toStringTag or constructor overrides, shifting devalue's classification of the clone relative to the ordinary VM path. The pristine realm is unreachable by any user code, and the explicit copy serializes exactly what devalue traverses (own indices, own enumerable string props). Arrays also now decline own constructor properties, which the class reducer reads even when non-enumerable.
Host intrinsics are shared with the whole process and cannot be frozen, but workflow code can reach them (structuredClone results, exposed host classes) and install Symbol.hasInstance predicates that distinguish the original from its clone — or WORKFLOW_SERIALIZE statics on host Object/Array that the class reducer reads for host-prototype originals (hydrated step results). prepareRetainedStepInput now verifies, via own-descriptor reads only, that every host dispatch point is pristine and declines retention before any clone exists — so a spoofed predicate can never observe or capture a pristine-realm object.
Reducers dispatch on symbol tags (e.g. the workflow abort-signal markers) that are non-enumerable and dropped by the pristine-realm copy, so a tagged object would serialize as an abort descriptor on the cold path but as plain data on the retained path.
Hidden own keys of any kind — non-enumerable properties, accessors, symbols — can be observed by serialization dispatch (reducer probes like .signal, thenable checks, the class reducer) while the pristine clone drops them. With no hidden own keys, every probe on an accepted object resolves deterministically through validated data or pristine prototypes.
Symbol.hasInstance dispatch walks the constructor's prototype chain, so the frozen Date wrapper still exposed the unfrozen original VM Date it delegates statics to. Freeze each non-shared binding's full chain (stopping at host Function/Object prototypes) and verify host Object.prototype carries no added hasInstance on the detection side.
Member
|
I'm re-running benchmarks one more time to see if the TTFS downsides hold |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
crypto.subtle.digest, noAtomics.waitAsync/async-WebAssembly/WeakRef/FinalizationRegistryWORKFLOW_RETAINED_VM=0kill switch (default on) restores replay-from-scratch on every iterationworkflow.runspans withworkflow.execution.mode=replay|retainedfor direct production decompositionrunWorkflowas the one-shot compatibility APICombines closes #2984 (retained-session architecture) with the best tactics from closes #2966 (env kill switch, loop-level single-VM A/B test), plus review-driven sandbox hardening.
Design
executeWorkflowowns the invocation-local VM behind two discriminated unions: a request (replay|resume) and a result (completed|suspended|replay). Overloads guarantee at the type level that a fresh replay can never request another replay — only resuming a stale or diverged session can.The session itself is a small state machine:
running,suspended,failed,replay, orcompleted. The runtime retains a session only for a pure step boundary —stepCount > 0, zero hooks/waits/attributes/hook-disposals/aborts, and no open hook or wait anywhere in the event history — since hooks and waits can wake concurrent invocations and attributes require replay.Unconditional VM quiescence
Retention is only sound if a suspended VM cannot make progress the event log cannot replay (found by review: a workflow racing host-timed async against a step could advance while the inline step ran and commit behavior no replay reconstructs). Rather than tracking such APIs, the sandbox now has none:
crypto.subtle.digestcomputes synchronously vianode:crypto(createHash— byte-identical values, verified against WebCrypto for SHA-1/256/384/512; stable and undeprecated through Node 26), so its promise settles on a deterministic microtask. Digest-using workflows stay retainable, and digest race timing becomes deterministic and identical across live, retained, and replay execution.Atomics.waitAsync(a wall-clock timer viaSharedArrayBuffer), the asyncWebAssemblycompilation entry points,WeakRef, andFinalizationRegistryare deleted from the sandbox — wall-clock timing and GC observation are unreplayable with or without retention. Syncnew WebAssembly.Module()/Instance()remain. Dynamicimport()settles within a microtask.A suspended VM is therefore parked exclusively on log-driven promises: its state is a pure function of the consumed event prefix — exactly what a replay rebuilds.
Serialization isolation (found by review, several rounds)
handleSuspensiondehydrates step arguments with the live VM, and ordinary serialization can execute workflow code (getters,WORKFLOW_SERIALIZEhooks, spoofed dispatch) — side effects a later cold replay would not repeat. For retention candidates the inputs are therefore prepared without touching workflow code, all-or-nothing per batch:retained-step-input.ts) accepts only primitives, plain objects, and plain arrays — the types devalue traverses purely via own-property reads — inspecting descriptors only (getters never fire, proxies/symbol tags/constructor/thendecline);Object/Array/Function+ prototypes,%TypedArray%, all non-shared serialization bindings; absent bindings pinned toundefined);Symbol.hasInstance, noWORKFLOW_SERIALIZE/classIdstatics, unmodifiedconstructor/toStringTag) before any clone exists — dirt declines retention for the boundary.Any input that fails these checks serializes through the ordinary VM path and the session falls back to replay — behavior identical to
WORKFLOW_RETAINED_VM=0.Known residual (deliberate): workflow code that escapes to the host realm via exposed host-class constructor chains (e.g.
TextEncoder.constructor) can schedule real timers and was never deterministic under ordinary replay either; that root cause needs realm-local sandbox intrinsics and is tracked as a follow-up.On resume, the session verifies that every previously observed event is still an exact
eventIdprefix of the authoritative log, appends only the new events to the existing consumer, and wakes the suspended execution. A prefix mismatch or a second suspension boundary from an unobserved/discarded session moves it permanently toreplay. Background workflow code can only enqueue in-memory work; the active observer remains the sole owner of durable writes and terminal drains.This is intentionally invocation-local. Re-invocation, retry, rollover, queued execution, and process loss discard the session and use normal durable replay. The event log remains the source of truth.
Why the kill switch
Retained execution is expected to be observationally identical to a fresh replay, but that equivalence rests on VM-internal state (deterministic RNG stream position, ULID factory state, VM clock) the event-prefix guard cannot check.
WORKFLOW_RETAINED_VM=0gives targeted mitigation and incident bisection without a dependency rollback. The "off" path is not a second implementation — it forces the always-alive fallback that hooks, waits, and divergence already exercise in production. CI runs the full core suite in both modes, andretained-vm-loop.test.tsasserts the dehydrated run output is byte-identical between modes while the retained mode builds exactly one VM per run.Validation
pnpm --filter @workflow/core typecheckpnpm --filter @workflow/core buildWORKFLOW_RETAINED_VM=0): 71 files, 1,485 passed, 3 expected failures eachBenchmark
STSO results land in the sticky benchmark comment below (the benchmark workflow runs against this PR's preview deployment).
For reference, #2984 measured the same architecture (on top of #2980, now merged) at:
Docs Preview
Behind deployment protection (Vercel team access required).
WORKFLOW_RETAINED_VMWORKFLOW_SERIALIZE— hook purity constraints