Skip to content

Phase C8: full-node end-to-end and TS interop (completes Milestone C)#89

Merged
ptesavol merged 2 commits into
mainfrom
claude/phase-c8
Jul 13, 2026
Merged

Phase C8: full-node end-to-end and TS interop (completes Milestone C)#89
ptesavol merged 2 commits into
mainfrom
claude/phase-c8

Conversation

@ptesavol

Copy link
Copy Markdown
Collaborator

Summary

The final milestone-C phase: the three remaining end-to-end tests from the TS pin (af966cf03, v103.8.0-rc.3) plus the new mixed C++/TS stream interop test — the milestone exit criterion.

Milestone C exit criterion: a C++ NetworkNode joins a stream-part overlay directly, publishes and subscribes, and interoperates with TS 103.8 nodes.Met — see TsInteropStreamTest below.

Ported end-to-end tests

  • WebsocketFullNodeNetworkTest — entry point + N NetworkStacks over real websockets form one stream-part overlay; a broadcast reaches every node. Nodes start concurrently (TS Promise.all parity).
  • WebrtcFullNodeNetworkTest — same shape, nodes without websocket servers so their connections form over WebRTC.
  • ContentDeliveryLayerNodeRealConnectionsTest — five content-delivery nodes over layer-0 DhtNodes used directly as per-stream discovery layers, real websockets; fully-connected-topology and propagation tests.

Scaling deviation: the two network tests run with 8 nodes instead of the TS 12/22. Measured cause: the C++ entry point's websocket accept path degrades under a concurrent connectivity-check storm (~9 accepts in the first second, then ~1/s), so late joiners' checks exceed the TS-parity 1-second connect timeout. A follow-up task is filed to offload accepted-connection processing from the libdatachannel callback thread and restore the TS sizes; the test files carry a pointer to it.

Mixed C++/TS stream interop (test/ts-interop/)

Mirrors the B3 harness pattern: package.json pins @streamr/trackerless-network 103.8.0-rc.3 from npm (granular packages — the TS monorepo is never built), fullNode.js runs a TS NetworkStack as the websocket + stream-part entry point speaking a DESCRIPTOR/RECEIVED/SENT stdout protocol with a PUBLISH stdin command. The C++ test starts two NetworkNodes that join through the TS node, then asserts both directions: a C++ publish reaches the other C++ node and the TS node; a TS publish reaches both C++ nodes. Skips only when no node ≥ 20 runtime exists; all other failures are hard.

Cleanups

  • Removes the ListeningRpcCommunicator lifecycle diagnostics that slipped into Phase C6: NetworkStack and NetworkNode (completes the full-node API) #88 (INSTR logs plus their accessor).
  • lint.sh: the test sweep now excludes node_modules (the npm harness ships C++ sources); the four new test files join the documented clangd-tidy std-type false-positive exclusion list; two genuine readability-use-anyofallof findings fixed with std::ranges::all_of.

Validation

  • All four new tests green, interop included.
  • dht regression: unit 181/181, integration 43/43, end-to-end 7/7.
  • Package lints and root lint.sh green.
  • The known pre-existing destroy-vs-in-flight-handler residual (documented in Phase C6: NetworkStack and NetworkNode (completes the full-node API) #88, follow-up chip filed) surfaced once during a full-suite run under load, in a C6-era proxy test — unrelated to this phase's changes.

Milestone C complete

All phases merged: C1 (#79), C2 (#85), C3 (#86), C4 (#82), C5 (#87), C6 (#88), C7 (#84), C8 (this PR). Next per the plan: Milestone D — the shared-library C API for the full node (streamrNode* handle API alongside the existing proxyClient* surface).

🤖 Generated with Claude Code

Ports the three remaining end-to-end tests from the TS pin (af966cf03,
v103.8.0-rc.3) and adds the new mixed C++/TS stream interop test — the
milestone C exit criterion: a C++ NetworkNode joins a stream-part
overlay directly, publishes and subscribes, and interoperates with TS
103.8 nodes.

- WebsocketFullNodeNetworkTest / WebrtcFullNodeNetworkTest: an entry
  point + N NetworkStacks over real websockets (respectively WebRTC
  connections with the entry point's websocket for signalling) form one
  stream-part overlay; a broadcast reaches every node. Nodes start
  CONCURRENTLY (TS Promise.all parity). Scaled to 8 nodes from the TS
  12/22: the C++ entry point's websocket accept path degrades under a
  concurrent connectivity-check storm (measured ~9 accepts in the first
  second, then ~1/s; the late checks then exceed the TS-parity 1 s
  connect timeout) — follow-up task filed to offload the accept path
  from the rtc callback thread and restore the TS sizes.
- ContentDeliveryLayerNodeRealConnectionsTest: five content-delivery
  nodes over layer-0 DhtNodes as their per-stream discovery layers
  (DhtNodeDiscoveryLayer), real websockets; fully-connected topology
  and propagation.
- test/ts-interop/: the mixed-network interop harness (B3 pattern) —
  package.json pinning @streamr/trackerless-network 103.8.0-rc.3 from
  npm, fullNode.js driving a TS NetworkStack entry point with a
  DESCRIPTOR/RECEIVED/SENT + PUBLISH stdin/stdout line protocol, and
  TsInteropStreamTest: two C++ NetworkNodes join the stream part
  through the TS node; a C++ publish reaches the other C++ node AND the
  TS node, and a TS publish reaches both C++ nodes. Skips only when no
  node runtime >= 20 exists.
- Removes the ListeningRpcCommunicator lifecycle diagnostics that
  slipped into #88 (INSTR logs + the accessor they used).
- lint.sh: the test-file sweep excludes node_modules (the npm harness
  ships C++ sources); the three e2e tests and the interop test join the
  documented clangd-tidy std-type false-positive exclusion list.

Validation: dht 181 unit + 43 integration + 7 e2e green; all four new
tests green (interop included); package + root lints green. The known
pre-existing destroy-vs-in-flight-handler residual (documented in #88,
follow-up filed) surfaced once in a full-suite run under load, in a C6
test, unrelated to this phase's changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nners

The linux-arm64 runner failed WebsocketFullNodeNetworkTest twice over:
the 8 stacks did not all reach 4 neighbors within 30 s (3-core runner),
and after that wait threw, the SEQUENTIAL stack stops in TearDown
stacked their bounded per-stop timeouts into a ~19-minute teardown that
tripped the 1200 s ctest kill.

- neighborTimeout 30 s -> 90 s (the TS test tolerates up to its 220 s
  jest budget for convergence).
- TearDown stops all stacks CONCURRENTLY (TS afterEach Promise.all
  parity), bounding an unconverged teardown by the slowest stack.

Both changes applied to the WebRTC variant too (same fixture shape,
same hazard). Verified locally: both tests green twice in a row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ptesavol
ptesavol merged commit 9045807 into main Jul 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant