Skip to content

fix(relay): backport the mesh gossip fixes and relay counters to release - #7108

Merged
kixelated merged 6 commits into
releasefrom
backport/mesh-gossip
Oct 28, 2026
Merged

kixelated merged 6 commits into
releasefrom
backport/mesh-gossip

Conversation

@kixelated

@kixelated kixelated commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

On release, a relay mesh under announce churn pays for it several times over:

  • Every announce cursor rescans its whole pending map on each poll, so a withdrawal wave costs each cursor O(K²) (moq-dev/moq#6422).
  • A draining relay's empty-URI GOAWAY makes every cluster peer redial it immediately, and each new session replays both announce tables for the whole drain window. The cluster redial backoff also doubles to 300s without jitter (moq-dev/moq#6423).
  • Hop paths are accepted up to the wire maximum of 32, so small meshes can hunt long detours during withdrawals (moq-dev/moq#6427).

Operators also cannot see it: /metrics omits the announce counters the relay already keeps (moq-dev/moq#5696) and has no control-message byte attribution (moq-dev/moq#6235).

Approach

Backport those five main PRs, one commit each, plus one review fix. release's moq-net has diverged too far to cherry-pick, so each was ported by hand to release's code, keeping main's semantics, names, and public items. Each commit message notes its adaptations.

Commit From Adaptations for release
c44953eb3 #5696 announce metrics release already counts announces; this adds only the /metrics rendering, its test, and the doc line.
bb42e2711 #6235 control byte counters Control half only. release's codec has no Encoder/Decoder context, so classes come from crate-private Encode/Decode hooks, and raw IETF frame reads count through a private Reader::read_message. Messages release doesn't implement stay uncounted (class="auth" stays 0).
061110a33 #6422 linear announce cursors Due/parked queues, read-only readiness check, allocation-free no-op updates. release delivers in prefix order rather than insertion order, so due is keyed by prefix. Keeps release's optional clock. Ports the K log K step test, the allocation test, and both bench groups.
f1e8cb6fc #6423 draining-peer redial moq_tokio::Restart::{Redial, Wait} and Client::with_restart; cluster dials use Wait; jittered redial capped at 30s. release has no moq-clock, so the drain test runs on Tokio's paused clock.
b279140c3 #6427 cluster.max_hops Same name, flag, env var, range (1..=32), and default (32) as main. The egress cap joins release's cursor Horizon, so selection falls back to an in-cap route; requests stay uncapped. main's random-mesh harness doesn't exist here, so a capped 8-relay withdrawal test replaces it.
ff6b11554 #6235 review fix Draft 14's SUBSCRIBE_ERROR (0x05) counts as subscribe; from draft 15 on, 0x05 is the shared REQUEST_ERROR and stays other.

Impact

  • Public API: additive only. cargo semver-checks reports no semver update required for moq-net, moq-relay, and moq-tokio. Every new item matches main:
    • moq-net: stats::Control, stats::Bytes, Snapshot::control(), origin::HopLimit (new, get), origin::Config.max_hops
    • moq-tokio: Restart, Client::with_restart
    • moq-relay: cluster::Config.max_hops
  • Wire: unchanged.
  • Behavior: cluster peers stay on a draining relay until it closes instead of redialing it, and the cluster redial backoff is jittered and capped at 30s. New /metrics series: moq_relay_announces_{started,ended}_total and moq_relay_control_{tx,rx}_bytes_total{tier,class}.
  • max_hops defaults to 32, the old limit. release still stamps a random hop ahead of an unnamed publisher's path, so such a route counts one more hop against the cap than on main.
  • No version bump or publication.

Validation

  • just check origin/release passes on b279140c3, including 5162 Rust tests. On ff6b11554, cargo test -p moq-net -p moq-relay, cargo fmt --check, and cargo clippy -D warnings pass.
  • cargo semver-checks -p moq-net -p moq-relay -p moq-tokio --baseline-rev origin/release: no semver update required.
  • Each new test fails without its fix: the withdrawal wave took 3848 steps at K=64 against a bound of 1792; the drain test opened 31 sessions instead of 1; the fallback test fails when the cap filters only the winner; removing either the receive or the send cap fails a test.
  • The drain test passed 50 of 50 stress runs.

Alternatives

Follow-ups

(Written by Claude Opus 5.5)

kixelated and others added 5 commits October 28, 2026 01:03
Release already counts announces; render them on /metrics. Drop main's
refusal and request metric tests, which cover code release lacks.

(cherry picked from commit ec0b3d0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Port the control-message byte counters: moq_net::stats::{Control, Bytes},
Snapshot::control(), and moq_relay_control_{tx,rx}_bytes_total{tier,class},
counted at the codec for lite 01-07 and IETF drafts 14-22. Release's codec has
no Encoder/Decoder context, so classes come from defaulted crate-private
Encode/Decode hooks, and raw IETF frame reads count through Reader::read_message.
The wire_{tx,rx} counters are not ported: they need #5773's wire registry.

(cherry picked from commit 8ee3c80)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ports the due/parked pending queues, the per-entry update hold, the read-only
readiness check in poll_next/try_next, the allocation-free no-op
TableCursor::update, RouteEntry::reach, lazy RouteTable::covering, and
TableCursor::best. Release keeps its lexicographic delivery order (due is a
BTreeSet of prefixes) and its Option clock; it has no epochs or restarts.

(cherry picked from commit 45007f6)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ackport #6423)

Ports moq_tokio::Restart / Client::with_restart, the relay's Restart::Wait for
cluster dials, the jittered 30s-capped cluster redial, and the cluster docs.
The cluster_drain test runs on Tokio's paused clock (release has no moq-clock)
in a flat tests/ file; the old empty-URI redial test in goaway_cluster is removed.

(cherry picked from commit 4e5c68b)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ort #6427)

Adds cluster.max_hops (--cluster-max-hops, MOQ_CLUSTER_MAX_HOPS; 1..=32, default 32)
via origin::Config.max_hops: origin::HopLimit. Receive drops longer paths on lite and
IETF subscribers, counting release's anonymous-publisher stamp; the egress cap joins
release's cursor Horizon so selection falls back to an in-cap route.

(cherry picked from commit 1582fdc)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T15:57:51.313549Z ff6b115 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b279140c37

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rs/moq-net/src/ietf/message.rs Resolved

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: b279140

Finding already reported: I agree with the existing P2 draft-14 SUBSCRIBE_ERROR classification finding in rs/moq-net/src/ietf/message.rs:20–23. The subscription-specific refusal is counted as Other; add the draft-14-only classification and a refusal-counter regression test while preserving later drafts' shared error classification. Referencing the existing thread rather than duplicating its inline comment. No additional actionable bugs found.

Direction: The release-specific adaptation is reasonable: selecting an in-cap route before advertising preserves fallback, and separating due/parked announcements avoids repeated full pending scans. Keeping these changes on the existing codec and reconnect abstractions is simpler than importing main's surrounding refactors; the added bookkeeping is justified by those constraints.

Verification limits: Static review of the GitHub diff and relevant surrounding code, including hop-limit, cursor/hold, reconnect, control-accounting paths and added tests. I did not execute tests, benchmarks, or semver checks; the PR's reported results were not independently verified.

Draft 14 sends 0x05 only to refuse a subscription; later drafts reuse it for
the shared REQUEST_ERROR, which stays in the other class.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: ff6b115

Compared with my previous review at b279140 (base unchanged). The previously referenced P2 classification finding is fixed: rs/moq-net/src/ietf/message.rs:24–27 classifies SUBSCRIBE_ERROR as Subscribe only for draft 14, preserving later drafts' shared-error classification. The added test covers draft 14 and draft 15.

No new actionable bugs found in this update. The narrowly version-gated fix is the simplest appropriate approach and does not add unnecessary abstraction.

Verification: static inspection of the one-file, one-commit update and classifier context; I did not execute the tests.

@kixelated
kixelated merged commit b7d6970 into release Oct 28, 2026
22 checks passed
@kixelated
kixelated deleted the backport/mesh-gossip branch October 28, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant