Initial-peers WAN topology with a cycle (N>=3 routers) enters a self-sustaining SEDP subscription-announcement storm; does DDS-Pipe v1.5.3 (#196) cover the multi-hop case?

[middlestvrain]

Summary

Three DDS Router instances (v3.5.1, also reproduced on v3.5.2) in an initial-peers unicast full mesh enter a self-sustaining discovery storm carrying no user data: ~1 CPU core per router and ~17,000 pkt/s on the WAN link, sustained indefinitely, with every application publisher stopped.

We root-caused it with an RTPS capture and believe it is closely related to the two-pipe discovery saturation fixed in DDS-Pipe #196 (v1.5.3) — but our case is the N>=3 cyclic WAN generalization, which #196's N=2 single-domain reproduction does not cover. We are testing v3.5.3 now and will report results here; filing so the topology-level behavior is on record either way.

Environment

• DDS Router v3.5.1 (also v3.5.2), built from source via ddsrouter.repos (Fast DDS v3.6.1, DDS-Pipe v1.5.1/v1.5.2)
• Ubuntu 24.04, containers (one router per network namespace), FASTDDS_BUILTIN_TRANSPORTS=UDPv4, --reload-time 0
• Per router: one local participant (its own domain, ~8 participants with a handful of endpoints) + one initial-peers WAN participant; ~6-topic allowlist with explicit RELIABLE/VOLATILE/KEEP_LAST QoS pinned
• Topology: every router's WAN participant dials every other (full mesh) — at N=3 the graph contains a cycle

Observed

With all application publishers stopped:

topology	per-router CPU
1 router	0.1 %
2 routers	0.0 % (stable >2 min)
3 routers	~106–111 % each
back to 2 (stop the 3rd)	still ~120 % — does not recover
2 routers, both restarted	0.0 % — cleared

RTPS capture on the WAN link (20,000 frames in 1.15 s ≈ 17,400 pkt/s):

• Essentially every DATA submessage is from ENTITYID_SEDP_BUILTIN_SUBSCRIPTIONS_WRITER (0x000004c2). Only 3 SPDP announcements in the whole capture; zero user data.
• The SEDP subscriptions writers' sequence numbers advance ~4,400/s per router — new samples, not retransmits — while only 4 distinct guidPrefixes appear. It is re-announcement, not endpoint proliferation.
• The same five allowlisted topics are re-announced ~3,000x/s each. The one allowlisted topic with no reader never appears — pinning the loop to subscription mirroring specifically.

Behavioral fingerprints

1. Publisher-independent — full intensity with every user publisher stopped.
2. Needs real seed endpoints — 3 routers plus trivial talkers on otherwise-empty domains stay at 0.0 %; the storm needs a moderately rich endpoint graph on the local domains.
3. Needs the third router — two routers with the identical config are silent.
4. It latches — removing the third router does not clear it; only restarting the routers does. Once triggered it appears to be a state of the SEDP writers, not a function of the live topology.

Our understanding of the mechanism

Each router mirrors subscriptions discovered on its WAN participant onto its local participant and re-announces them to every WAN peer. With three mutually-visible routers the announcement echo always has a fresh destination, and (pre-v1.5.3) the unconditional re-announcement in Track::update_reader_partitions → CommonReader::update_partitions → update_reader(...) lets it free-run — the same amplifier #196 fixed for two pipes on one domain.

Endpoint::discoverer_participant_id names the local participant that saw the endpoint, so there is nothing in the current data model that could suppress a cycle spanning more than one hop: no origin marker that survives a hop, no hop count, no TTL.

Questions

1. Is DDS-Pipe v1.5.3 / Build Fast DDS for Windows on build_fastdds workflow #196 expected to make cyclic N>=3 initial-peers topologies converge, or is announce-on-change damping only guaranteed for the two-pipe case? (We will post our v3.5.3 measurements here.)
2. Is a cyclic router topology a supported configuration at all? The repeater use-case page documents hub topologies only, and we found no documented topology restrictions. If cycles are out of contract, a note in the docs (and ideally a startup warning) would save others this diagnosis.
3. Would you be open to a discovery-plane analogue of RepeaterDataFilter — e.g. stamping an origin-router property on mirrored endpoints (the app_id/app_metadata PropertyPolicy hook already exists in CommonParticipant) plus a hop limit, so redundant router graphs are safe by construction? We may be able to contribute a reproduction and/or a patch.

We can share pcaps and the full reproduction recipe.

[middlestvrain]

Follow-up with measurements, as promised: v3.5.3 (DDS-Pipe v1.5.3 / #196) resolves our N>=3 cyclic-topology storm.

Same rig, same config, only DDS_ROUTER_REF changed:

metric	v3.5.1	v3.5.3
N=3 full mesh, per-router CPU	~106 %	0.1–0.2 %
N=3, loopback pkt/s (shared netns)	~37,000	~560
N=6 full mesh, 6 routers (cgroup sum)	432.7 %	1.8 %
latch behavior	storm persists after removing the 3rd router; only restart clears	stop/start of a router twice: returns to noise each time
endpoint churn	sustains the free-run	short burst, converges

An N=3 mesh also stayed quiet over a ~2.7 h soak. So announce-on-change damping does cover our multi-hop cyclic case in practice (question 1 answered empirically, N<=6, one deployment). Questions 2 and 3 stand: we'd still value a documented statement on whether cyclic topologies are supported going forward, since nothing structural (origin/hop tracking) prevents a future regression — this failure mode was invisible until N=3.

One small adoption blocker worth flagging: the native YAML validator (since v3.5.2) rejects connection-addresses: [] ("array has too few items"), which listen-only repeater/hub configs and single-node bring-ups naturally emit. Omitting the key entirely passes. If an empty connect list is intended to be invalid, a clearer error (or accepting the empty list as "listen-only") would help.