feat(core): experimental in-process VM continuation for inline replay#2966
feat(core): experimental in-process VM continuation for inline replay#2966VaguelySerious wants to merge 2 commits into
Conversation
🦋 Changeset detectedLatest commit: 8467bdf The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
|
Deployment failed with the following error: |
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (2)ff3959bFri, 17 Jul 2026 17:06:35 GMT · run logs
ff3959bFri, 17 Jul 2026 16:25:18 GMT · run logs
Avg deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) 🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
Combines the retained-session architecture from #2984 with the env kill switch and loop-level single-VM test from #2966. - executeWorkflow with discriminated request/result types and a WorkflowSession state machine (running/suspended/failed/replay/completed) - EventsConsumer.append: only newly durable events feed the live VM - WORKFLOW_RETAINED_VM=0 kill switch (default on) - retained-vm-loop.test.ts: proves one VM per run and byte-identical output vs the from-scratch replay path
Add an opt-in path (WORKFLOW_VM_CONTINUATION=1, off by default) that keeps a suspended workflow VM alive across inline-loop iterations instead of rebuilding the vm.Context and replaying the event log from scratch each pass. On a step-only suspension, runWorkflow attaches a `continuation` handle to the WorkflowSuspension. The inline loop, after appending the step's terminal events, feeds them into the SAME live VM via EventsConsumer.resume(), resolving the pending step promise so the workflow body advances to its next checkpoint. Scope and safety: - Only within a single invocation's inline loop (same process owns the writes, so the durable event log stays authoritative). - Only for step-only suspensions; hooks/waits/attribute writes fall back to a full replay (they involve out-of-band invocations and delivery-barrier ordering the replay path handles specially). - resume() prefix-checks the consumed events against the authoritative log by eventId and throws ReplayDivergenceError on any mismatch; the loop swallows it and falls back to a fresh replay. Worst case == today's behavior. Verified: full core suite green with the flag both off and on (1488 passed); a loop-level test proves the VM is constructed once (resumed per step) with the flag on vs rebuilt per replay off, producing byte-identical output. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Flip the flag to default-on so CI and benchmarks exercise and measure the continuation path. `WORKFLOW_VM_CONTINUATION=0` reverts to rebuilding the VM and replaying from scratch. Updates the loop test to drive the baseline via the kill switch and the ON case via the (now default) unset env. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ff3959b to
8467bdf
Compare
1 participant
Summary
Adds in-process VM continuation — on by default, kill switch
WORKFLOW_VM_CONTINUATION=0. When active, the inline replay loop keeps a suspended workflow VM alive across iterations and feeds newly-appended events into it, instead of rebuilding thevm.Contextand replaying the event log from event 0 on every pass.Default-on so CI lanes and benchmarks actually exercise and measure the new path. The kill switch restores the previous rebuild-and-replay behavior exactly.
This is the "cache the VM" idea, scoped to the only place it's sound: within a single invocation's inline loop, where the same process owns the appended writes so the durable event log stays authoritative. It does not cache a VM across process/invocation boundaries (sleeps, hooks, redelivery) — there is no live VM to resume there and a snapshot would drift from the log.
How it works
runWorkflowattaches acontinuationhandle to the thrownWorkflowSuspension.runWorkflow, it callscontinuation.resume(events):EventsConsumer.resume()adopts the authoritative event array and re-drives the still-registered step consumers, resolving the pending step promise so the same live VM advances to its next checkpoint.The step consumer already stays registered after hitting end-of-events (it only removes itself on terminal events), so resuming is just feeding it the events a fresh replay would have consumed next — the same deterministic call sequence, not restarted. RNG/clock/correlation-id state advance naturally in the live VM, exactly matching what a from-scratch replay reconstructs.
Safety (why this can't corrupt a run)
continuationunset → the loop falls back to a full replay. Those paths involve out-of-band resume/parallel invocations and delivery-barrier ordering that the replay path handles specially and that aren't validated for continuation here.resume()checks that the events already consumed by the live VM are a byte-stable prefix (byeventId) of the authoritative log. Any mismatch throwsReplayDivergenceError; the loop swallows it and falls back to a fresh replay.WORKFLOW_VM_CONTINUATION=0disables the continuation attach entirely; the suspend/complete/error control flow is byte-identical to before (verified by the full suite passing under the kill switch).Scope / limitations (intentionally not in this PR)
Verification
packages/corefull suite green with the default (on) and under the kill switch (=0): 1488 passed, 3 pre-existing expected-fails.vm-continuation.test.ts: provesresume()advances a live two-step workflow across appended events in one VM, plus the divergence guard.vm-continuation-loop.test.ts: drives a 2-step sequential workflow through the realworkflowEntrypointloop and assertscreateContext(VM construction) is called once by default (resumed per step) vs >1 under the kill switch (rebuilt per replay), and that the dehydratedrun_completedoutput is byte-identical between the two modes.pnpm --filter @workflow/core build+typecheckclean.Notes
WORKFLOW_VM_CONTINUATION=0is the per-deployment kill switch.world-localpaths.🤖 Generated with Claude Code