Skip to content

test(net): check refusal state cheaply and benchmark its scaling - #7549

Merged
kixelated merged 5 commits into
mainfrom
quest/m0/ci/test-cost-followups
Nov 2, 2026
Merged

kixelated merged 5 commits into
mainfrom
quest/m0/ci/test-cost-followups

Conversation

@kixelated

@kixelated kixelated commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Finishes quest/m0/ci/test-cost-followups.md by making the last cut. The remaining refused-fill test spent about two CPU seconds repeating 100,000 refusals to expose quadratic work, and it never asserted the retained state directly.

The other three cuts already landed: recovery encode/reference-decode caching in moq-dev/moq#7074, tracing isolation in moq-dev/moq#7071, and the stats test budget in moq-dev/moq#7072. Current source and measurements confirm those fixes remain. This PR does not repeat them.

Approach

  • Check that every refusal advances the fill, releases the consumed answer, and leaves one coalesced resolved run. The fast test uses 32 refusals with these stronger assertions.
  • Keep the 100,000-refusal workload in track_fill_refusal, alongside a 1/8/64-reader by 16/256/4096-refusal sweep. The existing nightly moq-net --bench '*' lane runs it. This keeps the scaling coverage, including a cheap copy per refusal, without charging every merge-gate run for the long workload.
  • The quest's goal is met, so this PR removes the quest and its link from quest/m0/ci/README.md. moq-dev/moq#7542 owns other fill/range implementation costs; none are duplicated here. Relay-auth trust, export-idle polling, and mesh-session performance stay out of scope.

Measurements

Per-test-process user+system CPU, using the unchanged wait4 wrapper from moq-dev/moq#6914, under just rs test, zero retries. One run before and after on the same shared host, with warm recovery caches:

Selected tests Before (s) After (s)
Refused fill (1) 1.9217 0.0049
Recovery (20) 5.2912 5.2398
Stats oversize (2) 0.0195 0.0209
Tokio integration (159) 6.6855 6.7981

Only the refused-fill saving comes from this PR. All 182 selected tests passed before and after. The PRs above have the before/after measurements for the cuts that already landed.

Release benchmark: cargo bench -p moq-net --bench track -- track_fill_refusal --warm-up-time 0.1 --measurement-time 0.2 --sample-size 10. One reader costs 19.36 us / 300.69 us / 4.773 ms / 115.76 ms at 16 / 256 / 4096 / 100,000 refusals. At 64 readers, 16 / 256 / 4096 refusals cost 0.835 / 13.315 / 212.31 ms. The cost per reader per refusal stays flat; production code is unchanged.

Impact

  • Public API: none. The new resolved-run accessor is crate-private and #[cfg(all(test, feature = "async"))].
  • Wire: none.
  • Production behavior: none.

Alternatives

Simply shrinking the old loop would weaken its only scaling guard. Direct state assertions plus a nightly benchmark keep both the behavior coverage and the performance coverage. Reimplementing the three landed cuts would duplicate existing work.

Checks

  • Full quest gate env BASH_ENV= just ci origin/main: passed, 9,300 Rust tests plus selected lint and repository checks, on source head 500466f8c0c30531c8b470086b7bf93d9fe164bd.
  • The 182-test measurement selection, release benchmark, benchmark clippy, formatting, and quest check --base origin/main: passed.
  • After merging origin/main (rs/moq-net/src/model/track.rs auto-merged) and removing the quest, on head 8cf1ccbdf3: cargo test -p moq-net --features async --lib a_fill_refused_group_by_group_stays_linear, cargo clippy -p moq-net --all-features --benches --tests -- -D warnings, and quest check --base origin/main pass.
  • No interop run is needed: no wire, FFI, or gateway code changed.

Review

Local review skipped under the simple-review rule (moq-dev/quest#364): the change touches only tests, a benchmark, and quests. The only src/ changes are inside mod test in rs/moq-net/src/model/track.rs and a #[cfg(all(test, feature = "async"))] accessor in rs/moq-net/src/model/track/fill.rs. No online-reviewer threads were open on head c1550fa126. The skip applies to head 8cf1ccbdf3, which adds only a conflict-free merge of origin/main and the quest removal. Online reviewers (CodeRabbit, Pullfrog) run once the PR is ready.

Follow-ups

None. The Codex shell startup hook made mbx/Cargo fail with Argument list too long inside the gate, which is an environment issue outside this repository; running with BASH_ENV= works around it.

(Written by Claude Opus 5.5)

kixelated and others added 3 commits November 1, 2026 03:57
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title The next test-CPU cuts test(net): check refusal state cheaply and benchmark its scaling Nov 1, 2026
kixelated and others added 2 commits November 1, 2026 10:16
All four cuts from #6987 have landed or land with this PR, so remove the
quest and its link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 1, 2026 18:18
@coderabbitai

coderabbitai Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 7 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 704e2401-b1e9-4497-84df-6680557b1e4a
Commits

Reviewing files that changed from the base of the PR and between 951cca6 and 8cf1ccb.

Files selected for processing: 5
  • quest/m0/ci/README.md
  • quest/m0/ci/test-cost-followups.md
  • rs/moq-net/benches/track.rs
  • rs/moq-net/src/model/track.rs
  • rs/moq-net/src/model/track/fill.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed commit: 8cf1ccbdf34c3cc858be0244b42c11bd75e5bd8a.

No actionable bugs found. The direction is sound: the short test now checks progress, consumed-answer release, and resolved-range coalescing directly, while the benchmark retains the long refusal history and sweeps reader count. This is stronger than merely shortening the old loop, with no production API or wire change. The accessor stays test-only, and the existing nightly Rust bench selection includes the new workload.

Verification limits: static GitHub review of the diff, relevant fill/subscriber code and benchmark wiring; I did not run tests or reproduce the reported timings.

(Written by OpenAI)

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Refused-fill test made cheap and stronger: a_fill_refused_group_by_group_stays_linear now walks 32 refusals and asserts, per iteration, that wanted() advances by one, answers() is 0 after the reader consumes the refusal, and resolved_runs() stays one coalesced run.
  • Test-only accessor: Fill::resolved_runs() in fill.rs is pub(super) and #[cfg(all(test, feature = "async"))], mirroring the existing answers(); no production or public API surface changes.
  • Nightly scaling benchmark: track_fill_refusal builds one shared descending fill, sweeps readers [1, 8, 64] against refusals [16, 256, 4096], and keeps the old 100,000-refusal single-reader point. It is registered in criterion_group!, and the existing nightly moq-net --bench '*' lane picks it up.
  • Quest completion: the finished quest/m0/ci/test-cost-followups.md and its README link are removed, with no dangling references left in the tree.

Verified locally: the focused test passes, cargo test -p moq-net --features async --benches --no-run compiles, and the 8-reader/256 and 64-reader/16 benchmark points run without panicking. The release criterion baseline in target/criterion shows roughly flat per-refusal and per-reader cost across the sweep.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated enabled auto-merge (squash) November 1, 2026 18:54
@kixelated
kixelated added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 305f90a Nov 2, 2026
7 checks passed
@kixelated
kixelated deleted the quest/m0/ci/test-cost-followups branch November 2, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant