Stream event delivery to clients lags 2-3 minutes behind durable writes (steps finish in seconds)

[Colin-Buffamin]

Summary

Events written to a workflow run's stream become visible to readers only minutes later. The workflow itself completes its steps in seconds, but a client holding GET .../stream receives the events in ~120s reconnect cycles, and even a fresh stream read (replay from startIndex=0) does not show the recorded events until ~90s after the step finished.

Environment

• Vercel-managed Workflow (production, Fluid Compute), workflow spec v5
• Reached through the eve framework, which bundles @workflow/core — reproduced identically on eve 0.11.10 and 0.18.1, so this does not appear to be an eve regression
• Local dev runtime delivers the same streams in real time (<20s end-to-end), so this is specific to the hosted workflow backend

Measurements

Single structured-output turn; the model step completes in seconds — confirmed via /flow invocation logs.

Read strategy	Time until result.completed reached the client
Hold one stream connection (default client)	124s / 179s / 199s (3 runs)
Fresh short-lived stream GETs every 3s, startIndex=0 replay	90s (17 polls before the event appeared)
Same code against local dev runtime	~15–20s

Request-log pattern for the held-connection case: POST /session 202 in ~1s → /.well-known/workflow/v1/flow invocations complete within seconds → stream GET reconnects at exactly +120s intervals, and the result arrives only on a later cycle. Setting WORKFLOW_V2_TIMEOUT_MS=30000 had no effect on delivery timing.

The replay experiment is the interesting datapoint: since a brand-new streams.get() read at +60s still did not include events recorded at ~+10s, the lag appears to be on the write-visibility side of the streams backend, not in client reconnect behavior.

Impact

Interactive use (chat-style refinement against a durable session) is effectively blocked: every turn costs 2–3 minutes of waiting for a result that was ready in seconds. Possibly related to #943 (functions hanging after steps complete) if the underlying cause is stream finalization.

Happy to provide session ids / timestamps privately for backend-side tracing.

[KaBankz]

We hit what looks like the same failure mode on workflow@4.5.0 (managed Vercel backend, Next.js app in iad1, run stream consumed via run.readable in the POST route / getRun().getReadable({ startIndex }) on reconnect).

Forensics from one occurrence today (run wrun_01KWMM7NDYVFDNFXC4QKR7V7K0, team august-law, project august, preview env):

• run_created 18:35:39.392Z → step_started 18:35:40.160Z (dispatch healthy, 768ms)
• the step generated and wrote 113,270 bytes of stream data (streamStorageBytes)
• the connected client received zero chunks for 37 seconds and the user cancelled at 18:36:16Z
• local dev (world-local) delivers the identical workload's streams in real time

Separately measured on healthy runs in the same app: live-tail delivery throttles to ~13 chunk-batches/s (a 500-word answer whose model call finished in 18s kept draining to the client for ~50s; step_completed confirms the step stayed alive writing the whole time — so at least part of the lag appears write-side, in the sequential per-flush PUTs). The total-blackout case above is the one that reads like the delivery/visibility bug in this issue.

Happy to provide more run IDs or timings if useful.

[Colin-Buffamin]

Adding a fresh production occurrence from today — it matches the ~120s reconnect-quantization variant, which may be worth tracking separately from the write-side throttle @KaBankz measured above:

• 2026-07-06 04:43:59Z, managed Vercel backend, production: session POST accepted (202), run wrun_01KWTVV0T2SHQ00N4Q47K85GJE
• flow steps (/.well-known/workflow/v1/flow) kept POSTing in between, but the stream reader GETs reconnected at almost exactly 120s boundaries (04:44:00 → 04:46:00 → …) with nothing delivered in between
• downstream: a synchronous serverless caller waiting on this run hit its 300s ceiling → a user-facing 504. On the managed backend the effective delivery floor seems to be the reconnect window, regardless of actual step duration
• consistent with the table in the original report: WORKFLOW_V2_TIMEOUT_MS override had no effect, SDK upgrade didn't change it

Happy to share more run IDs/timings.

[tuliren]

I am also hitting this on the managed Vercel backend. Same setup as @KaBankz: getRun(runId).getReadable({ startIndex: 0 }) piped as NDJSON from a POST route (Next.js 16). A streamText / getWritable() step delivers zero chunks to the client for the whole generation — skeleton UI, content only shows after completion + reload.

Key points:

• SDK-version-independent: identical on workflow 4.2.4 and 4.6.0 (@workflow/world-vercel 4.5.0).
• Backend, not app code: it worked before, regressed, and rolling deployments back weeks doesn't restore it.
• Same window, /v2/runs/* broadly slow: 84× GET /v2/runs/<wrun>?remoteRefBehavior=resolve timed out (~88s–24min) Jun 30–Jul 7, e.g. wrun_01KWXGVZNRBC06N3QZGM1657YE, wrun_01KWG5GW3SZKZ5KJ14NY8PFCP5 (likely overlaps Bug: Workflow steps hang until 300s timeout despite completing in seconds #943).