From 01e22890593f6145470283f95102a960abca48c4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 16 Oct 2026 17:39:06 -0700 Subject: [PATCH] quest: pin benchmark cores and fold moq-quic into bench CI Follow-ups from the moq-quic benchmarks (#5812). Co-Authored-By: Claude Opus 5.5 --- quest/m1/quic/coverage/README.md | 1 + quest/m1/quic/coverage/bench-pin.md | 20 ++++++++++++++++++++ quest/m2/bench-ci.md | 6 ++++++ 3 files changed, 27 insertions(+) create mode 100644 quest/m1/quic/coverage/bench-pin.md diff --git a/quest/m1/quic/coverage/README.md b/quest/m1/quic/coverage/README.md index 5fc0a32b00f..2e6d0fbe3f8 100644 --- a/quest/m1/quic/coverage/README.md +++ b/quest/m1/quic/coverage/README.md @@ -45,6 +45,7 @@ and the harness defaults to Cubic while production runs BBR3. - [Benchmarks](/quest/m1/quic/coverage/bench.md) - Criterion sweeps of stream scheduling, receive and reset, endpoint routing, and the wire codec, on the baseline - [Characterization tests](/quest/m1/quic/coverage/characterize.md) - golden scheduling order, and every uncovered frame, reset, limit, and endpoint path, on today's harness - [Seeded simulation](/quest/m1/quic/coverage/sim.md) - a deterministic, impaired network in `Pair`, BBR3 as the harness default, and property tests over random workloads +- [Pin benchmark cores](/quest/m1/quic/coverage/bench-pin.md) - local base/head comparisons run on one core type on hybrid CPUs ## Related diff --git a/quest/m1/quic/coverage/bench-pin.md b/quest/m1/quic/coverage/bench-pin.md new file mode 100644 index 00000000000..e3a481044a8 --- /dev/null +++ b/quest/m1/quic/coverage/bench-pin.md @@ -0,0 +1,20 @@ +# [XS] Pin local benchmark comparisons to one core type + +## Goal + +On a hybrid CPU (performance and efficiency cores), `bench/run.sh` runs +both sides of a comparison on the same core type, so a base/head delta no +longer depends on where the scheduler put each run. On a CPU with one core +type nothing changes. + +## Plan + +Decided 2026-10-16 with the maintainer. Seen while landing the moq-quic +benchmarks: on an 8P/16E host the same unpinned case measured 420 µs or +690 µs depending on the core it ran on. Detect the performance cores where +the kernel reports them (for example `/sys/devices/cpu_core/cpus` on Intel +hybrid parts) and pin each benchmark process to them with `taskset`. Print +which cores were used beside the comparison, so a reader knows the run was +pinned. + +Public API: none. Wire: none. diff --git a/quest/m2/bench-ci.md b/quest/m2/bench-ci.md index 959affee2c6..a6e6c8cca29 100644 --- a/quest/m2/bench-ci.md +++ b/quest/m2/bench-ci.md @@ -39,6 +39,12 @@ a GitHub App: own alert disabled. The `moq-uring` socket benches run only here, and skip loudly when the runner's kernel is too old. Add the workflow's name to `alert.yml` if it isn't `Nightly`. +- Decided 2026-10-16: moq-quic's Criterion targets (#5812) join the + selection. They time thread CPU rather than wall time, which already cut + most host noise; on a shared hybrid-core host, identical builds still + swing about 5% on small cases and 20-130% on large-working-set ones, so + the noise floor must come from A/A runs on the CI runner itself, never + from a laptop. - Validate with an A/A run (the same commit twice) and a deliberately slowed bench before trusting either signal. - Document the comment, the trend, and local reproduction in