Phase C6: NetworkStack and NetworkNode (completes the full-node API)

[ptesavol]

Summary

Ports the NetworkStack / NetworkNode layer from the pinned TypeScript (af966cf03, v103.8.0-rc.3) — the public full-node API that milestone C8 (full-node e2e + TS interop) will build on. Also lands the two proxy end-to-end test suites deferred from C5, which uncovered and drove the fixing of three real concurrency defects.

New modules

• NetworkStack — layer-0 DhtNode + ContentDeliveryManager composition with lifecycle: entry-point self-join, ensureConnectedToControlLayer (the TS setImmediate joinDht as a bounded GuardedAsyncScope task), the node-info RPC service, createNodeInfo/fetchNodeInfo over the NodeInfoResponse proto. It supplies the manager's injected discovery-layer factory with createStreamPartDiscoveryLayerNode (the C5 seam). The TS module-level instances list and process/window exit handlers are a JS-runtime concern and are not ported.
• NetworkNode + createNetworkNode — the client-facing API: start/stop, join (with neighbor requirement), broadcast, setProxies, message listeners (add returns a HandlerToken; remove takes it — C++ emitter idiom), stream-part entry points, getters, external RPC registration.
• ExternalNetworkRpc — the application RPC service; the TS class-object parameters become template parameters.
• NodeInfoRpcLocal (response factory injected instead of holding the stack — avoids a module cycle), NodeInfoRpcRemote, NodeInfoClient.

DhtNode / proxy plumbing

• DhtNode::getTransport() / getConnectionLocker() accessors; allowIncomingPrivateConnections option passed through to the owned ConnectionManager (TS sets it from acceptProxyConnections).
• Optional proxy direction end to end (absent = bidirectional, TS parity): the wire field is only set when present; server connections and the client definition store std::optional<ProxyDirection>.
• ProxyClient::updateConnections: signed connection-count diff — the unsigned subtraction could never take the shrink branch ("close one of two proxies" was broken).

Concurrency defects found by the new proxy tests (all runtime-evidence diagnosed)

1. RecursiveOperationManager::execute zero-connections early return skipped session->stop() — the session communicator died with its transport Message handler still registered; any later routed message walked into freed memory (three distinct crash signatures). TS has the same early return but its GC makes it a leak, not a use-after-free. Completed sessions are additionally retired into a bounded ring (mirroring the existing retired-communicators pattern) so no in-flight handler invocation can outlive its session.
2. RecursiveOperationSession::stop lock order — rpcCommunicator.destroy() moved outside the session mutex (the message handler takes the same mutex).
3. ProxyClient data race — TS is single-threaded; in C++, setProxies (API thread) raced onNodeDisconnected (websocket/rtc event threads) over definition/connections (crash: dead map keys under attemptConnection on the RTC poll thread). The state is now mutex-guarded with a strict snapshot-under-lock / act-outside-lock discipline — the mutex is never held across a blocking RPC wait, because a transport event thread parked on it can be the very thread an in-flight send needs. onNodeDisconnected no longer fires the TS automatic reconnection attempt (it always fails against a gone proxy, and blocking RPCs from transport event threads deadlock); reconnection is an explicit setProxies re-issue.
4. Teardown latency — ContentDeliveryManager::destroy now aborts every part's split-avoidance/reconnect loops before joinScope.close(); the join otherwise waits out the full avoidNetworkSplit exponential backoff (~60 s per part). Proxy test times dropped 68 s → 5.5 s.

Tests (8 new files, in test/unit/ per package convention)

NetworkNodeTest (stubbed-stack message listener), NetworkRpcTest, NetworkNodeIntegrationTest (simulator: broadcast/subscribe + fetchNodeInfo), NetworkStackTest (real websockets), NodeInfoRpcTest, ExternalNetworkRpcTest, and the C5-deferred ProxyConnectionsTest (11 tests incl. bidirectional proxies and reconnect) and ProxyAndFullNodeTest. Adaptations: the reconnect test's first until() upstream passes only through a JS event-loop timing artifact (the connection map is polled before the queued setImmediate removal runs) — the C++ version asserts the portable intent, driving reconnection through setProxies. Fixture TearDowns are null-guarded (SetUp can throw when a fixed websocket port is still in TIME_WAIT).

Validation

• dht regression: unit 181/181, integration 43/43, end-to-end 7/7 (all green post-fixes).
• trackerless: the full 111-test suite green end-to-end; proxy suite 4/4 consecutive full passes in isolation.
• Package lints (trackerless, dht, eventemitter) and root lint.sh green. Three new test files join the documented clangd-tidy std-type false-positive exclusion list.

Known residual + follow-up

Under repeated back-to-back full-suite runs on a loaded machine, a rare crash remains in the pre-existing destroy-vs-in-flight-handler race family (an emitter handler can still be mid-invocation on another thread when its owner is freed right after off()). A full off()-waits-for-in-flight-invocation hardening was implemented and deliberately reverted: it deadlocks the 64-node scale tests through cyclic waits between handler chains that call off() transitively. A follow-up task is filed with the reverted patch and the investigation notes.

Remaining phases

C8 (full-node e2e + mixed C++/TS interop via granular npm packages) is the last milestone-C phase; C4/C7 merged earlier.

🤖 Generated with Claude Code