Server 3.2.0: connection context, targeted events, one ordered outbound queue per connection

[dmitrat]

What

OutWit.Communication.Server 3.1.1 → 3.2.0: what a shared WitServer needs before it can serve many clients that must not see each other's events (the WitCloud /worker/v2 case: one endpoint for every node instead of one server per node). Planned in WitCloud/@Docs/Roadmap/plan-witrpc-connection-context-and-targeted-callbacks.md; additive on the server package only.

Connection context for a service

• ConnectionContext.Current — an AsyncLocal the server sets inside ProcessMessage (visible to the validator, the processor, the service method and whatever it awaits or starts; gone when the request completes) and around the authorization handshake. Carries ConnectionId, ServerId / ServerName / Transport, Principal, AuthorizedAtUtc.
• The principal lives on ConnectionInfo, established once at authorization through the opt-in IConnectionAuthenticator (TryAuthenticate(token, out ClaimsPrincipal?)); IAccessTokenValidator is untouched and a validator that does not opt in leaves it null.
• IConnectionContextAccessor / ConnectionContextAccessor for injection, ConnectionContext.BeginScope(...) for unit tests of services.

Targeted events

• CallbackScope.Target(connectionId) / Target(connectionIds) / TargetCaller() around an ordinary C# event raise. WitServer.OnCallback reads the target when the callback reaches it — the raise chain event → HandleEvent → Callback → OnCallback is synchronous, so neither processor nor IRequestProcessor changes.
• The scope reports per server and connection (Report now, Completion once every accepted send finished): Sent, UnknownConnection, NotAuthorized, QueueFull, SendFailed, SendTimedOut, Refused. A service registered in two servers is raised in both; the one owning the connection delivers, the other reports the id unknown.
• WithTargetedCallbacksOnly() refuses an untargeted raise (logged, Refused). Without a scope the server broadcasts exactly as before.

One ordered outbound queue per connection

• ConnectionOutbox: one queue and one writer task per connection for responses, handshake replies and callbacks alike, replacing the fire-and-forget send task per callback per connection. Wire order is a stated guarantee: a callback raised inside a handler precedes the handler's response; the AEAD counter advances in wire order by construction; a stuck transport holds only its own queue.
• WithCallbackDelivery(...) / CallbackDeliveryOptions: MaxPendingCallbacks, MaxPendingCallbackBytes (0 = unbounded), OverflowPolicy = Log (default: queue anyway, warn, keep the connection — the 3.1 behaviour) / CloseConnection (refuse, close; a callback send timeout closes too) / DropNewest. Responses are never counted and never refused. This is the outbound half of the 2026-08-30 audit's P0.2; the inbound bound stays for its Stage 1.
• Diagnostics: WitServer.GetPendingCallbacks(connectionId), AuthorizedConnectionCount, ConnectionInfo.PendingCallbacks / PendingCallbackBytes.

Compatibility

Nothing on the wire, nothing in the core, the client packages, Blazor, REST, InterProcess or the DI package (its floor stays Server >= 3.1.1; a consumer takes 3.2.0 with an explicit pin and behaves as before until it opens a scope, reads the context or sets delivery options). The old WitServer constructor stays; a new overload takes CallbackDeliveryOptions.

Tests

Suite	Result
OutWit.Communication.Tests net10.0-windows (CI filter, 46 new)	912/912
OutWit.Communication.Tests net8.0-windows	912/912
OutWit.InterProcess.Tests (real agent processes)	13/13
CI solution filter, Release, warnings as errors, all TFMs	builds

New fixtures: ConnectionContextTests (Pipes / WebSocket / TCP: id per connection, flows across awaits, no leak across 6 clients × 20 concurrent calls, principal from the authenticator, null for a plain validator, refusal closes), TargetedCallbackTests (only the target / a set / broadcast unchanged / unknown id / targeted-only / 200 in order / shared service in two servers), ConnectionOutboxTests on a stub transport that can block a write (callback-before-response, 1000 in order, responses and callbacks in one order, slow neighbour, disconnect mid-send, queue-full close only the offender, Log keeps queuing, DropNewest, byte budget, responses never counted, send timeout close vs warn, dispose with pending sends, targeted-only refusal, per-connection report).

Load harness

Communication/OutWit.Communication.LoadTests (console, in OutWit.slnx, not in the CI gate): one shared server with targeted callbacks vs N per-client servers vs the pre-3.2 broadcast, 100 / 300 / 1000 nodes over real WebSocket on loopback, a throttled or stalled socket for the slow-neighbour case. Numbers below and in WitCloud/@Docs/Roadmap/baseline-2026-09-07-witrpc-load.md.

Ryzen 9 5950X, one process (server + N WitClients), WebSocket over loopback, JSON, AES-GCM, 256 B payload, 20 s steady dispatch, dispatch → ack = callback down + request up:

Model	Nodes	Rate	Acked	p50 / p95 / p99 / max	CPU (whole process)
shared, targeted (3.2)	100	1 000/s	100 %	0.40 / 0.66 / 2.98 / 24.7 ms	18.6 s
per-client servers (1.7.x)	100	1 000/s	100 %	0.40 / 0.95 / 4.97 / 31.2 ms	24.7 s
shared, broadcast (pre-3.2)	100	1 000/s	100 %	11.6 / 23.4 / 47.6 / 193 ms	457 s
shared, targeted	300	2 000/s	100 %	0.64 / 1.39 / 6.85 / 27.4 ms	48 s
per-client servers	300	2 000/s	100 %	0.60 / 2.79 / 18.8 / 59.3 ms	48 s
shared, targeted	1 000	3 000/s	100 %	0.86 / 1.50 / 13.4 / 63.1 ms	118 s
shared, targeted, 8 KB	1 000	3 000/s	100 %	0.85 / 3.19 / 4.98 / 45.1 ms	121 s
shared, broadcast	300	1 000/s (10 s)	11 %	3 421 / 12 285 / 13 380 ms	633 s, 2.3 GB

Slow neighbour (node 7's socket 200 ms per write, or never draining), the other 99 nodes: shared Log 0.43 / 0.62 / 2.90 ms, shared CloseConnection (bound 50, send timeout 2 s) 0.42 / 0.67 / 3.06 ms, per-client 0.41 / 0.73 / 3.76 ms — all at the no-slow-node baseline (0.40 / 0.66 / 2.98 ms); the stuck frames accumulate only in the slow node's own queue (105–199). Under CloseConnection the stalled node is closed 2 s after its first stuck frame and every later task to it reports UnknownConnection at once (the directory's cue to requeue); under Log (= 3.1) the frames sit in that queue and the connection stays. The queued-callbacks maximum over healthy nodes was 1 in every targeted run.

Docs

Server README ("Connection context and targeted events (3.2)"), CHANGELOG (Server 3.2.0), ROADMAP-v3 ("3.2.0 follow-up"), MIGRATION-3 (Step 4: no consumer action).

🤖 Generated with Claude Code