Skip to content

fix(bench): expect stable MSF catalogs during estimate churn - #7264

Merged
kixelated merged 1 commit into
mainfrom
fix-codspeed-ci
Oct 30, 2026
Merged

kixelated merged 1 commit into
mainfrom
fix-codspeed-ci

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

Codspeed run 40902359739 panics in the MSF catalog fanout benchmark. moq-dev/moq#7161 intentionally fixed MSF estimates at declaration, but the benchmark still expects two estimate-driven updates.

Approach

  • Expect exactly two hang updates and zero MSF updates across the existing track/subscriber sweep.
  • Assert initial catalog delivery and rendition counts before measuring, so an inactive subscriber cannot satisfy the zero-update expectation.
  • Update the benchmark comments to describe both formats.

Impact

  • Public API: none.
  • Wire: none.
  • Benchmark assertions only; production behavior and Codspeed coverage are unchanged.

Validation

  • Reproduced the original assertion locally with Criterion: actual (0, 0), expected (2, 2).
  • cargo bench --locked -p moq-mux --bench catalog -- --test: all 24 cases passed.
  • cargo codspeed build --locked -m simulation -p moq-mux --bench catalog and cargo codspeed run -m simulation -p moq-mux --bench catalog: all 24 cases checked locally. Hosted instruction measurements were not run.
  • just ci and git diff --check: passed.
  • Local self-review: expectations match the producer's fixed MSF declarations; initial delivery still exercises namespace conversion.

Alternatives

Skipping the MSF cases would lose coverage; changing production would undo the intended suppression.

Follow-ups

None.

(Written by GPT-6.1 Sol)

Co-Authored-By: Codex <noreply@openai.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T17:51:53.810252Z 5d2cf76 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 5d2cf767f7eccdd9474d6374c9cd7827b97e89b5 against base 14537ef416a15c2140ae5137993bbbb66425268a.

No actionable findings. The format-specific counts match the producer: hang continues publishing throttled estimate changes, while MSF retains each track's declared estimates and suppresses identical catalog JSON. The new setup assertion checks initial catalog delivery and the aggregate rendition count before accepting zero MSF churn, and the hang cases retain the two-update regression check.

Keeping the MSF sweep rather than skipping it is the right direction. Namespace conversion remains exercised during setup; it is not newly included in the timed estimate-churn closure. Production behavior is unchanged.

Verification: GitHub-only static review of the complete benchmark and relevant catalog producer/freezing logic and tests. No Criterion/Codspeed runs, compilation, CI, or performance measurements were executed; the reported 24-case results were not independently reproduced.

@coderabbitai

coderabbitai Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 65ee095c-1639-4674-a955-f580cf110c59
Commits

Reviewing files that changed from the base of the PR and between 14537ef and 5d2cf76.

Files selected for processing: 1
  • rs/moq-mux/benches/catalog.rs

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

Walkthrough

The catalog benchmark now checks the initial catalog and per-track subscription updates. Its run method accepts an expected publish count and checks the measured publishes and updates. The estimate-fanout benchmark expects zero publishes for MSF references and two for hang. The benchmark documentation describes these cases.

Priority: Low

Merge Risk: Minimal · up to 5d2cf

No actionable merge-blocking risk was identified in this benchmark update.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes the benchmark fix for stable MSF catalogs during estimate churn.
Description check Passed The description directly explains the benchmark failure, the expected MSF and hang updates, the assertions, and validation results.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 3 functions across 1 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary: CI passed (10/10) on head 5d2cf767f7eccdd9474d6374c9cd7827b97e89b5. Reviews on this head: Codex 👍 (no findings), the automated maintainer-account review (no actionable findings), CodeRabbit (no actionable comments; its docstring-coverage warning is not a repo requirement). No decisions outstanding; quest check against origin/main passed. Landing through the merge queue.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 5a5c146 Oct 30, 2026
10 checks passed
@kixelated
kixelated deleted the fix-codspeed-ci branch October 30, 2026 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant