Skip to content

feat(ipc): async server handlers + server-side per-fork ordering - #24210

Closed
charlielye wants to merge 0 commit into
cl/ipc-wsdb-concurrent-serverfrom
cl/ipc-wsdb-server-ordering
Closed

charlielye wants to merge 0 commit into
cl/ipc-wsdb-concurrent-serverfrom
cl/ipc-wsdb-server-ordering

Conversation

@charlielye

@charlielye charlielye commented Jun 20, 2026 •

Copy link
Copy Markdown
Contributor

What

Moves request ordering from the client into the database. Previously each TS client serialized its own reads/writes per fork (WorldStateOpsQueue); a second async client could corrupt state by skipping that contract. This makes the wsdb server own per-fork ordering, built on an async server-handler codegen pattern adopted symmetrically across all four languages.

Stacked on #24198 (concurrent reactor).

Codegen — async server handlers (C++, Rust, Zig; TS already async)

The generated server dispatch hands each handler the decoded command plus a respond callback, instead of returning a value. A handler may run inline or defer to a thread pool and call respond when ready. The wire protocol is unchanged, so the cross-language interop matrix and golden fixtures are untouched.

  • C++: void handle_x(Ctx&, Cmd&&, Responder<Resp>); make_*_handler returns void(span, RawRespond); serve() drives it via run_reactor.
  • Rust: Handler trait methods take Responder<Resp>; handle_request collects synchronously so the runtime/echo glue is unchanged.
  • Zig: dispatch hands handlers a Responder (ok/err); handleRequest unchanged.
  • TS: already Promise-returning — no change.

Each language's echo server updated to the new shape (~6 handlers each).

wsdb — server-side per-fork ordering

Each of the 40 handlers declares its own ordering by calling schedule_read (reads run concurrently; committed reads bypass ordering) or schedule_write (exclusive on its fork), extracting its fork from the typed command. WsdbScheduler implements the read-batch / write-barrier model per fork, with an inline fast path when idle and a dispatch pool distinct from WorldState's intra-op pool. The hand-rolled request classifier is gone — read/write + fork come from the typed handler.

Testing

  • Cross-language echo matrix (build + interop + goldens 22/22) green across C++/Rust/Zig/TS.
  • aztec-wsdb builds; ipc-runtime reactor/echo gtests green.
  • native_world_state.test.ts (50 tests incl. Concurrent requests, which asserts read-your-writes ordering) green against the async server.
  • parallel_read.bench SHM: ~3.7x read scaling (8.1k → 30.4k reads/s, c=1 → c=16).
  • Full root bootstrap: wsdb + yarn-project + bb build clean.

Client queue removed

With the server owning per-fork ordering, the TS client's per-fork WorldStateOpsQueue is now redundant and has been deleted: IpcWorldState sends every op immediately and the server serializes writes / parallelizes reads. With the client queue gone, native_world_state.test.ts's Concurrent requests test now genuinely exercises server-side read/write ordering (and passes), and the TS-consumer no longer pays the queue's per-op overhead.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XfznXp9wzGWLU6LC8EHtpi

@charlielye

Copy link
Copy Markdown
Contributor Author

Superseded by the restack. The reactor (#24198) and ordering work have been combined into a single machinery commit on #24198, which now targets next and sits below the wsdb cutover (#23036). The WorldStateOpsQueue removal moved into the cutover PR. Closing as the commits here are now redistributed across #24198 and #23036.

@charlielye charlielye closed this Jun 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant