fix(core): fail runs delivered to the wrong deployment#2960
Conversation
🦋 Changeset detectedLatest commit: a359b5e The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (2)3ef6bccThu, 16 Jul 2026 18:47:46 GMT · run logs
1da67faThu, 16 Jul 2026 18:23:02 GMT · run logs
Avg deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader) Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges 🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request. |
1da67fa to
3ef6bcc
Compare
3ef6bcc to
589da5d
Compare
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
589da5d to
a359b5e
Compare
|
Deployment failed with the following error: |
1 participant
Summary
A run created on one deployment whose queue callbacks route to another (common on preview branches, where a superseded deployment is deactivated) fails cryptically: the step handler resolves the per-run encryption key from the bare
runId, derives it from the wrong deployment's master key, and throws aRuntimeDecryptionErrorbefore user code runs. The queue retry callback swallows it, so the run dies as a blank "exceeded max retries."A run may only execute on the deployment that created it (continuing elsewhere risks code skew). This detects delivery to the wrong deployment and fails the run with the new
DEPLOYMENT_MISMATCHerror code.errorCodeis what observability and the UI key off (describeRunErrormaps it to an actionable hint).RUN_ERROR_CODES.DEPLOYMENT_MISMATCH;WorkflowDeploymentMismatchErrorclassifies to it.Closes #2816. The issue's second half — the queue retry callback swallowing errors — was fixed separately.
Docs Preview
errors/deployment-mismatchFollow-up (separate repo)
DEPLOYMENT_MISMATCHcode to the new docs page.Testing
pnpm --filter @workflow/core... buildpnpm --filter @workflow/core exec vitest run src --exclude src/vm/uint8array-base64.test.ts(1,430 tests)pnpm --filter @workflow/errors test(38 tests)The excluded VM test has existing cross-realm assertion failures on local Node 26; the repository supports through Node 24.