Skip to content

Stale hardcoded .swc loader cache silently ships new "use step" functions as inline code → zero journaling + CORRUPTED_EVENT_LOG on resume #2793

Description

@mani-blink

Summary

@workflow/next hardcodes the SWC plugin/loader cache to <projectRoot>/.swc with no invalidation key and no opt-out:

  • @workflow/next/dist/swc-plugin-cache.js:17cacheRoot: join(projectRoot, '.swc')
  • @workflow/next/dist/loader.js:187 — same path in the loader

In CI/Docker environments where .swc persists between builds (cached layers, persistent build contexts), a newly added "use step" function can compile inline (untransformed) into the workflow bundle while the manifest still discovers and counts it. Nothing fails at build time. At runtime the function executes inline in the workflow body — producing zero workflow_steps journal rows — and every subsequent replay/resume of an affected run dies with CORRUPTED_EVENT_LOG (the inline await diverges the event log).

Verified on workflow@5.0.0-beta.25; the cache code is unchanged in 5.0.0-beta.27.

What makes this nasty

  • Build succeeds; manifest counts the new step (workflows build complete (N steps…) includes it).
  • The turn that runs the inline function works live — the failure surfaces only later, on wake/resume of parked runs, as CORRUPTED_EVENT_LOG.
  • Discovery is not the problem (we confirmed fastDiscoverEntries reads fresh; the step count incremented correctly). The stale layer is specifically the SWC loader cache on the compile path that turns "use step" source into registered/proxied steps.

Repro shape (from our production incident, Next.js 16 + Turbopack, Docker on Railway)

  1. Deploy an app with an existing workflow (steps journal normally).
  2. Add a new "use step" function (new file OR inside the existing "use workflow" entry — both reproduce) that the workflow body awaits.
  3. Build in an environment where .swc survives from a previous build.
  4. Observe: manifest counts the new step; at runtime the function executes but journals nothing; wake/resume of a parked run fails with CORRUPTED_EVENT_LOG.
  5. Add rm -rf .swc before next build → same code journals correctly and resumes work (this was our fix, verified by journal assertion + resume test).

We reproduced the failure twice (two independent deploys, once with the step in a new module and once defined inside the workflow entry file) and confirmed the rm -rf .swc fix on the third deploy with the identical source.

Ask

Either (a) include a build-id / source-hash / plugin-version invalidation key in the loader cache, or (b) expose a cacheRoot / no-cache option on withWorkflow() so CI builds can opt out. Today the only remedy is deleting .swc out-of-band, and the failure mode when you don't is silent data-plane corruption on resume.

Happy to provide build logs (step counts, timings) and the journal evidence from our incident if useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions