perf(core): retain workflow VM across inline steps#2984
Conversation
🦋 Changeset detectedLatest commit: 60ac8fb The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests▲ Vercel Production (32 failed)nextjs-turbopack (29 failed):
nuxt (3 failed):
vercel-multi-region (3 failed)nextjs-turbopack (3 failed):
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
❌ vercel-multi-region
❌ Some E2E test jobs failed:
Check the workflow run for details. |
Retained VM benchmark and trace decompositionBenchmark metrics
Does VM execution still scale with event count?No, once the invocation retains the session:
There are only two fresh replays in this trace, so those are exact observations rather than a statistically useful replay distribution. The retained sample is approximately 1,020 calls. The result is still decisive for the scaling question: the 1,000th hot continuation does not replay 1,000 promises; it resumes the existing VM in about 2.4 ms. Exact STSO decompositionEach benchmark gap is approximately:
Late-window outliers are visible directly in the spans:
The retained VM is not responsible for either tail sample. One representative normal late stepTrace span
ConclusionRetaining the VM removes the event-count-dependent replay cost for hot serial steps. It improves average STSO by 42%, 28%, and 59% in the three windows, and turns the late-window typical path from about 572 ms p75 into 227 ms p75. It cannot reach 20 ms by itself. The remaining path contains two sequential durable transitions, each paying client/network, workflow-server, and DynamoDB materialization/write latency. The next performance change must collapse, overlap, or eliminate one or both round trips, for example by atomically completing the current step and claiming/starting the next one in one server transition. VM retention and that protocol optimization are complementary. Alternatives and extension path
The chosen state machine is deliberately invocation-local so it can later sit inside an actor/owner without changing its session API. The most valuable immediate follow-up is protocol fusion; cross-invocation ownership is useful only after deciding that retaining locality is worth its operational cost. Review and verification
|
📊 Workflow Benchmarkscommit Backend:
Avg deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader) Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges 🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request. |
1 participant
Summary
runWorkflowas the one-shot compatibility APIworkflow.runspans withworkflow.execution.mode=replay|retainedfor direct production decompositionDesign
runWorkflowSessionowns the invocation-local VM and exposes a small discriminated state machine:running,suspended,failed,replay, orcompleted. The runtime retains a session only for a completed inline step when no attributes, hooks, waits, or replay divergence require the normal durable path.On resume, the session verifies that every previously observed event is still an exact prefix, appends only the new events to the existing consumer, and wakes the suspended execution. A prefix mismatch or a second suspension boundary from an unobserved/discarded session moves it permanently to
replay. Background workflow code can only enqueue in-memory work; the active observer remains the sole owner of durable writes and terminal drains.This is intentionally invocation-local. Re-invocation, retry, rollover, queued execution, and process loss discard the session and use normal durable replay. The event log remains the source of truth.
Stacked on #2980, which prepares and caches replay payloads.
Validation
pnpm --filter @workflow/core typecheckpnpm --filter @workflow/core buildBenchmark
Run against workflow-server #632 at
063557ca5565d9dc3b182f602372c0222a7c9379. The benchmark-only SDK commitbed8acd4bc0042119c21dac2607f9c81f6e3b609is the clean PR headd1907dc8fe22ddf331c7f047f6394e93224e9357plus only the workflow-server preview URL override; the override is not present in this PR.In the exact 1,020-step trace, retained
workflow.runcalls are approximately 2.4 ms even at 3,000+ events. The remaining typical STSO is almost entirely the previousstep_completeddurable write plus the nextstep_starteddurable write. Two DynamoDB/write-path stalls dominate the retained late-window p90/p99; the VM remains constant during those samples.The detailed trace decomposition and non-STSO metrics are in the benchmark comment below.