Skip to content

fix(swarm): let Node.js clients connect to a firewalled peeroxide server - #31

Merged
eshork merged 8 commits into
Rightbracket:mainfrom
alaa-eddine:fix/node-client-to-rust-server
Nov 2, 2026
Merged

eshork merged 8 commits into
Rightbracket:mainfrom
alaa-eddine:fix/node-client-to-rust-server

Conversation

@alaa-eddine

Copy link
Copy Markdown
Contributor

Summary

Node.js Hyperswarm clients could not connect to a firewalled peeroxide server; every attempt timed
out. This PR makes the server side of the handshake match Node hyperdht lib/server.js on the three
points described below.

Problem

A Node.js Hyperswarm client cannot connect to a peeroxide 1.7.3 swarm that announces a topic from
behind a NAT. The opposite direction, Rust client to Node server, works; that is what
hyperswarm_cross_language_connect covers.

Every attempt from Node fails:

  • peeroxide logs: passive holepunch: replacing stale in-flight entry, then
    passive holepunch timed out.
  • Node: dht.connect() fails with HOLEPUNCH_ABORTED.

Seen against hyperswarm 4.17.0 / hyperdht 6.30.0 (the versions tests/node pins) and 4.17.2 /
6.34.1, both on a local testnet and on the public DHT from behind a home NAT.

How to reproduce

  • Public DHT: run cargo run --example swarm_announce -p peeroxide behind a NAT, then
    node tests/node/hyperswarm-join.js <topic>. They never connect.
  • Local: the test hyperswarm_node_client_holepunches_to_rust_server, added in this PR,
    fails on main the same way.

Causes

There are three, and any one of them is enough to break the connection. Line numbers are for
hyperdht 6.30.0.

  1. Duplicate PEER_HANDSHAKE.
    • The Node client builds its Noise message once (lib/connect.js:416).
    • It sends that message through up to two relays in parallel (lib/connect.js:355,
      Semaphore(2)) and acts on the first reply.
    • Node's server deduplicates by the Noise bytes and answers every copy with the same reply
      (lib/server.js:466-475). It keeps the entry until 10 s after the handshake settles
      (HANDSHAKE_CLEAR_WAIT).
    • peeroxide's handle_server_handshake runs a fresh NoiseWrap responder for every copy, with
      a new ephemeral key, stream id and holepunch id. commit_passive_holepunch then replaces the
      first session with the second.
    • So the client continues with a reply whose session no longer exists.
  2. Wrong peer_address in holepunch.relays.
    • In Node, an announcer relay is { relayAddress: msg.from, peerAddress: msg.to }
      (lib/announcer.js:265): the relay, and our address as that relay saw it.
    • The client sends PEER_HOLEPUNCH to relayAddress, and the relay forwards it to peerAddress
      (lib/router.js:213-218).
    • peeroxide fills peer_address with the client's address (swarm.rs,
      peer_address: client_address.clone()).
    • So the relay sends the holepunch back to the client, which drops it, and the server never sees
      a round.
  3. Open or direct clients are given a holepunch.
    • When the client says it is FIREWALL_OPEN, or reached the server without a relay, Node's
      server dials it straight away (lib/server.js:392-396). An open Node client waits for that
      dial (lib/connect.js:236-239).
    • peeroxide decides whether to holepunch from its own firewall state only. So it stages a
      passive holepunch that the client will never perform.
    • This affects any Node peer with a public address, for example on a VPS, that connects to a
      NATed Rust peer.

Changes

peeroxide (src/swarm.rs)

  • HandshakeReplyCache answers every copy of a PEER_HANDSHAKE with the same Noise reply, keyed by
    the client's Noise message. Entries last 20 s: the 10 s passive-holepunch window plus Node's 10 s
    HANDSHAKE_CLEAR_WAIT.
  • The reply envelope moves into encode_server_handshake_reply, shared by the first answer and the
    duplicates.
  • holepunch.relays now comes from HyperDhtHandle::current_relays(), so each relay is paired with
    our own address as that relay saw it.
  • A client that is FIREWALL_OPEN, or that reached us directly, is dialled through the existing
    direct path instead of being given a holepunch.

peeroxide-dht (additive API only)

  • QueryReply::to (new pub field on a #[non_exhaustive] Result-role type): our address as the
    responder saw it. It is filled from the to field already decoded off the wire.
  • HyperDhtHandle::current_relays() -> Vec<RelayInfo> (new method on a Handle; RelayInfo is the
    existing wire envelope): the self-announce relays, each with that to.
  • The handle now stores RelayInfo instead of bare relay addresses. current_relay_addresses()
    keeps its signature and behaviour; it maps over the same list.

API and semver

  • No existing public signature changes. The new items follow VISIBILITY_POLICY.md: the field is
    on a #[non_exhaustive] Result type, the method is on a Handle, and RelayInfo is a wire
    envelope.
  • peeroxide-dht changes are additive (minor); peeroxide changes are internal (patch). release-plz
    bumps the version group.

Tests

Each new test fails on main and passes with the fix. Each fix was also switched off on its own to
check that the matching test catches it.

Test Covers Fails without
peeroxide/tests/hyperswarm_interop.rs::hyperswarm_node_client_holepunches_to_rust_server Firewalled Node client → firewalled Rust server: two parallel handshakes, then holepunch rounds through the advertised relays the dedup fix, or the relay fix
peeroxide/tests/hyperswarm_interop.rs::hyperswarm_open_node_client_connects_to_rust_server Reachable Node client → firewalled Rust server: the server dials the dial fix
peeroxide/tests/server_handshake_dedup.rs Rust-only: the same PEER_HANDSHAKE sent twice gets byte-identical replies, and the reply completes the handshake the dedup fix
swarm::tests::handshake_reply_cache_returns_the_first_reply_until_it_expires Cache hit, miss, and 20 s expiry (paused tokio clock) –
swarm::tests::server_handshake_reply_follows_the_inbound_mode FROM_RELAY/FROM_SECOND_RELAY → FROM_SERVER with the client's address; FROM_CLIENT → REPLY –
hyperdht::tests::current_relays_pair_each_relay_with_our_observed_address After an announce, each relay's peer_address is our own socket as the relay saw it the to plumbing

Test helper (tests/node/hyperswarm-server.js): two opt-in flags. Without them, the helper
behaves exactly as before.

  • --remote-client: the client skips the same-host LAN shortcut, as a client on another network
    would. It also starts one more DHT node, because a holepunching Node client needs 3 NAT samples
    (lib/nat.js:103).
  • --open-client: the client's DHT node is reachable.

Also checked, on the public DHT from behind a home NAT: a real Node.js app (hyperswarm 4.17.2 /
hyperdht 6.34.1) and a Rust peer built on this branch.

  • The Node peer dialled a server-only Rust peer; the holepunch landed and the app protocol ran
    end to end.
  • The Rust peer dialled the Node peer.

Not fixed here (possible follow-ups)

  • Same-host / LAN shortcut.
    • When both peers share a public IP, a Node client pings an address from the server's
      addresses4 and connects its stream there (lib/connect.js:242-262).
    • peeroxide's passive-holepunch stream listens only on the puncher socket, and that socket does
      not answer DHT pings.
    • On one host (loopback): the first entry is the DHT socket. The ping succeeds, the stream
      connects to the wrong socket, and the client gets ETIMEDOUT.
    • On a LAN: the ping goes to the puncher port and fails, so the holepunch wins the race. It
      works, just slower.
    • Possible fixes: let the passive stream accept its first packet on the DHT sockets too (Node
      UDX streams are not tied to one socket), or advertise only addresses where the stream listens.
  • peeroxide_dht::hyperdht::run_server (not the swarm) also runs a fresh responder per copy.
    It is untouched here.
  • Duplicate-connection tie-break. hyperswarm 4.17 keeps the new connection once the existing
    one has both sent and received bytes (index.js, existingIsOutdated). peeroxide compares keys
    only on outgoing connections, and drops incoming duplicates.

Checklist

  • cargo test --workspace --no-fail-fast on Windows 11: 1065 passed, 9 failed. All 9 fail the
    same way on an untouched main checkout on that machine, so none come from this PR:

    • libudx: stream_lifecycle::stream_write_read_10mb and throughput::throughput_sustained
      (debug-build timeouts; libudx is unchanged);
    • peeroxide-cli: chat_integration (2) and local_commands (5 dd tests).

    Not yet run on Linux, which is what CI uses.

  • cargo test -p peeroxide-cli --test live_commands -- --ignored: not run.

  • cargo clippy --workspace --all-targets -- -D warnings

  • cargo doc --workspace --no-deps

  • MSRV 1.85 cargo check: not run locally (no 1.85 toolchain here; nothing newer is used).

  • No public API breaking changes.

  • CHANGELOG updated (peeroxide, peeroxide-dht); TESTING.md updated.

  • No task-artifact files in the branch: the only added file is
    peeroxide/tests/server_handshake_dedup.rs.

🤖 Generated with Claude Code

alaa-eddine and others added 2 commits October 15, 2026 14:42
`QueryReply` gains `to`: our own address as the responding node saw it,
already decoded off the wire. `HyperDhtHandle` gains `current_relays()`:
the self-announce relays, each paired with that address, mirroring Node
hyperdht's `Announcer.relays` (`{ relayAddress: msg.from, peerAddress:
msg.to }`).

A server advertises these pairs in the `holepunch.relays` of its
handshake reply, and a relay forwards a client's PEER_HOLEPUNCH to
`peer_address`, so it has to be how that relay reaches us.
`current_relay_addresses()` keeps its signature and behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Node.js Hyperswarm client could not connect to a firewalled peeroxide
server: every attempt timed out ("passive holepunch timed out" here,
HOLEPUNCH_ABORTED on the Node side). The server side of the handshake
now matches Node hyperdht `lib/server.js` on three points:

- Duplicate PEER_HANDSHAKE. A Node client sends one Noise message
  through up to two relays in parallel and acts on the first reply. Every
  copy now gets the same reply, cached by the Noise message for 20 s.
  Before, each copy got a fresh responder, and the second replaced the
  holepunch session of the first.
- Holepunch relays. `holepunch.relays[].peer_address` is now our own
  address as each relay saw it (`HyperDhtHandle::current_relays()`).
  Before, it held the client's address, so the relay sent the client's
  PEER_HOLEPUNCH back to the client.
- Open or direct clients. A client that says it is FIREWALL_OPEN, or
  that reached us without a relay, is now dialled directly, as Node does.
  Before, it was given a holepunch, while it waited for that dial.

Tests: two Node-client -> Rust-server interop tests (a firewalled client
that holepunches through the relays, and a reachable client that waits
to be dialled), a Rust-only test that sends one PEER_HANDSHAKE twice,
and unit tests for the reply cache and envelope. The Node helper gains
two opt-in flags, `--remote-client` and `--open-client`; without them it
behaves as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alaa-eddine
alaa-eddine marked this pull request as ready for review October 15, 2026 13:23

@eshork eshork left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The diagnosis is right: I reproduced it locally. The three new tests fail against main's source and pass on this branch (and stayed green over 10 reruns). The dedup fix and the relay peer_address fix both look correct. Requesting changes for two things, plus some test gaps.

1. Open/direct clients: the server still never dials (blocking)

Node handles a FIREWALL_OPEN (or direct) client in two parts:

  • the reply still includes holepunch: { id, relays } whenever the server itself is firewalled (lib/server.js:360, which only looks at ourRemoteAddr);
  • then the server dials the client: hs.onsocket(sock, clientAddress.port, clientAddress.host) (lib/server.js:392-396).

This PR does neither. It drops holepunch from the reply, and create_server_connection still ignores _client_address; it only installs a passive firewall hook. With no holepunch info, the Node client takes the "relayed && !remoteHolepunchable" branch of connect.js holepunch() and dials the firewalled server itself. That works on loopback and behind full-cone NATs. Behind the more common address- or port-restricted NATs, the client's unsolicited packets get dropped. So I'd expect open clients (e.g. on a VPS) to still fail in practice, now with a stream timeout instead of HOLEPUNCH_ABORTED.

Suggested fix: for open/direct clients, connect the server stream to client_address so the server sends first (like Node's onsocket), and keep advertising { id, relays } when the server is firewalled.

2. HandshakeReplyCache has no size limit

  • get and insert both run prune(), which retains over the whole map, so each handshake costs two full scans.
  • Nothing limits the number of entries.
  • Insertion happens before the already-connected and max_peers checks.

Anyone who knows the announced public key can send distinct valid IK first messages, each one a new key. That makes the cost grow quadratically, and all of it runs inside the serial swarm actor, so a flood also stalls joins, connections and holepunches. Expired entries are also only removed when another handshake arrives. Suggest insertion-ordered expiry (e.g. a VecDeque<(Instant, key)> popped from the front) plus a hard cap on entries.

Test gaps

  • hyperswarm_open_node_client_connects_to_rust_server passes because of loopback reachability, not because the server dials. It wouldn't catch (1).
  • hyperswarm_node_client_holepunches_to_rust_server doesn't assert the Rust side is actually FIREWALL_CONSISTENT/RANDOM when the handshake arrives. If NAT classification is still UNKNOWN, it takes the direct path and passes anyway.
  • server_handshake_dedup.rs sends the handshake straight to the server, so no passive holepunch is ever staged. It checks the replies are byte-identical, but not the bug that actually broke things: commit_passive_holepunch replacing the first session. Worth adding a duplicate that arrives while a holepunch is staged, and asserting a PEER_HOLEPUNCH for the first reply's id still works.
  • Also missing: duplicates through two real relays (FROM_RELAY + FROM_SECOND_RELAY with different peer_address), a check that the server sends the first packet to an open client, and the cache's size cap / idle expiry.

Non-blocking: run_server still runs a fresh responder per copy (you already noted it). Worth a follow-up issue so it doesn't get lost.

Comment thread peeroxide/src/swarm.rs Resolved
Comment thread peeroxide/src/swarm.rs Resolved
Comment thread peeroxide/src/swarm.rs Resolved
Comment thread peeroxide/tests/hyperswarm_interop.rs Resolved
Comment thread peeroxide/tests/hyperswarm_interop.rs Resolved
@eshork

eshork commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

A few corrections and clarifications to my review above:

  • Scope of (1): this only affects FIREWALL_OPEN clients that come through a relay. Direct (non-relayed) clients are fine: they reached us once, so they can reach us again when they dial. The restricted-NAT failure is also my inference from the code paths (connect.js:216-237); I haven't reproduced it behind a real NAT.
  • The fix needs both halves together: advertise { id, relays } and have the server dial the client, without staging a passive holepunch. That's what Node does. Advertising the holepunch info without the dial brings back main's 10 s passiveConnectTimeout abort. Staging a passive holepunch as well as dialing risks two connection paths.
  • If you'd rather not do the dial in this PR, that's fine with me. The dedup and relay peer_address fixes are solid on their own, and this PR is still better than main for open clients. In that case, please reword the PR description, the CHANGELOG, and the hyperswarm_open_node_client_connects_to_rust_server doc comment so they don't say the server dials, and we'll open a follow-up issue.
  • CI: no checks have run on this branch yet, so it hasn't been tested on Linux. Let's get CI green before merge.

eshork and others added 4 commits November 1, 2026 21:00
Follow-up corrections to the Node-client-to-firewalled-server fix:

- Open or direct clients: the server now dials the client (its stream
  connects to the client's address, so it sends the first packet and opens
  its own NAT mapping), as Node's `hs.onsocket(...)` does. A firewalled
  server keeps advertising `holepunch { id, relays }` to such a client, as
  Node does, so an open Node client waits for that dial instead of dialling
  the firewalled server itself.
- Handshake reply cache: keyed by a hash of the client's Noise message,
  expired in insertion order (amortised O(1)) and by a periodic sweep,
  capped at 1024 entries (oldest evicted first), and skips handshakes the
  server rejects.
- Tests: the interop tests wait until the Rust server classifies itself as
  firewalled and check Node's punch counters to prove which side dialled;
  the open client is now ephemeral so it really goes through relays; a new
  direct-client test covers the direct path; the remote client reports
  FIREWALL_UNKNOWN deterministically. A new Rust-only test sends one
  handshake through two real relays to a firewalled server and checks that
  the staged holepunch session for the first reply survives the duplicate.
  Cache unit tests cover first-reply-wins, the capacity bound and idle expiry.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…en tests

Review follow-ups on the previous commit:

- A relayed handshake the server would reject (already connected, at
  max_peers, no UDX info) no longer keeps its staged passive holepunch or a
  reply-cache entry; the session would only have been dropped when its punch
  landed. `HolepunchSetup::release()` also frees the puncher on the Noise
  error paths, not just the abort timer.
- `--direct-client` now reports FIREWALL_UNKNOWN, so the direct-client
  interop test can only pass through the direct (non-relayed) path.
- New tests: a server at max_peers = 0 neither caches replies nor keeps a
  staged holepunch; a Rust client connecting straight to a firewalled swarm
  server still works now that the server dials it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A relayed --direct-client attempt would otherwise take Node's same-host LAN
shortcut, which leaves the punch counters at zero, so the counters could
not tell a direct handshake from a relayed one.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
`select_server_path` replaces the `server_firewalled` / `client_dialable` /
`attempt_holepunch` flags with an explicit `ServerPath` (Relay, Dial,
Holepunch, Listen), mirroring Node `lib/server.js _addHandshake`, and is
unit-tested as a decision table. `Dial { advertise_holepunch }` carries the
"firewalled server still offers a holepunch to a client it dials" rule.

The free-slot and UDX checks no longer need the client's key, so they now
run before a passive holepunch is staged: a full swarm no longer opens a
puncher socket only to release it. Only the already-connected check still
releases a staged session.

Tests: the max_peers test now asserts no holepunch is staged; a new test
covers the already-connected release path.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@eshork

eshork commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

@alaa-eddine thanks for this. Your diagnosis and the relay / current_relays() fix are kept as-is. I've pushed four commits on top that address my review:

  • bbefa8b: the server now dials open/direct clients (stream connects to the client's address, so we send first), and keeps advertising holepunch { id, relays } when firewalled, as Node does. HandshakeReplyCache is bounded (1024 entries, oldest first), expires in insertion order plus a periodic sweep, and is keyed by a hash of the Noise message.
  • dca6b64: handshakes we'd reject (already connected, at max_peers, no UDX info) aren't cached, and their staged holepunch is released.
  • f814e36: test-only. The --direct-client helper skips the LAN shortcut, so the punch counters prove the handshake went direct.
  • 97492be: select_server_path → ServerPath { Relay, Dial, Holepunch, Listen } replaces the boolean flags and is unit-tested against Node's _addHandshake. A full swarm no longer builds a puncher just to release it.

Tests

  • The interop tests now wait until the Rust server is classified as firewalled, and use Node's punch counters to check which side dialled.
  • --open-client is now ephemeral, so it really goes through relays. Before this, it found the server's announce itself and connected directly.
  • --remote-client pins FIREWALL_UNKNOWN. Without that, about 40% of runs had dht-rpc classify the Node client as open on loopback, so it was dialled and never holepunched.
  • New tests:
    • a duplicate handshake through two real relays, checking the staged session survives
    • rejection with no free slot (max_peers = 0)
    • rejection from an already-connected peer
    • a Rust client connecting directly to a firewalled server

Validation (macOS)

  • cargo test --workspace: 1082 passed, 0 failed
  • cargo test -p peeroxide-cli --test live_commands -- --ignored: 6/6
  • New tests: 10 repeated runs, all passed
  • cargo clippy --workspace --all-targets -- -D warnings (1.95): clean
  • cargo +1.85 check (MSRV): clean

No public API changes beyond yours.

eshork and others added 2 commits November 2, 2026 10:47
clippy 1.98 denies `drain(..).collect()` into the same collection type
(clippy::drain_collect). Use `std::mem::take` instead; behaviour is
unchanged. Three sites: libudx stream timeout path, protomux cork batch
and pending remote-channel messages.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Rust 1.99 deprecates `AtomicUsize::fetch_update` in favour of
`try_update`, which does not exist at our MSRV (1.85), so CI clippy
(-D warnings) fails on stable. Move the three identical saturating
decrements into one `saturating_dec` helper with a scoped
`#[allow(deprecated)]`; behaviour is unchanged.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@eshork
eshork merged commit d3e335a into Rightbracket:main Nov 2, 2026
5 checks passed
@github-actions github-actions Bot mentioned this pull request Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants