Repository navigation
Conversation
Generate bench/large.md and bench/huge.md in the bench job and run hyperfine parameterized over the three docs. bench-compare pairs baseline/PR results by the doc parameter and renders one row per doc; any failing row fails the step.
denolfe
changed the title
Uncapped --render lays out at most RENDER_MAX_HEIGHT rows, so huge.md measured the same 2000-row layout as large.md plus parse time inside the noise band.
denolfe
changed the title
Startup benchmark (
|
| doc | baseline (main@3b6d9f2) | PR | ratio | verdict |
|---|---|---|---|---|
| test/exhaustive.md | 360.3ms ± 4.4ms | 358.5ms ± 5.1ms | 0.99× | ✅ ok |
| bench/large.md | 7379.9ms ± 217.3ms | 6985.5ms ± 209.0ms | 0.95× | ✅ ok |
Thresholds: warn ≥ 1.1×, fail ≥ 1.25×. Baseline built from main.
Owner
Author
|
Closing: the extra --render row cannot observe the interactive-path regressions it was meant to guard, and the 2000-row render cap makes it redundant with the existing row. A first-frame/scroll CI bench is the useful follow-up. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the PR startup benchmark from one doc to two.
test/exhaustive.mdstays as the small-doc row; a generated 5k-line fixture adds a row where layout runs to the render cap. Each row gets its own ratio and verdict in the sticky comment.Key Changes
bench/large.mdis gitignored, so the workflow runsbench/gen-fixture.tsbefore hyperfine. The baseline and PR binaries read the same file.-L docover both fixtures, baseline command first. hyperfine emits results doc-outer, command-inner, and tags each withparameters.doc.bench-compare.tspairs results byparameters.docinstead of assuming exactly two results. It renders one table row per doc.Why no bigger fixture: uncapped
--renderlays out at mostRENDER_MAX_HEIGHT(2000) rows. A 26k-line fixture measured the same 2000-row layout as the 5k-line one plus about 40ms of parse, inside that row's ±40ms noise band. Parse scaling is covered bybench/stages.tsfrom source.