Skip to content

📝 docs: summarize perf gains and refresh the benchmark tables#686

Merged
gaborbernat merged 4 commits into
tox-dev:mainfrom
gaborbernat:docs/perf-changelog-1.5
Jul 19, 2026
Merged

📝 docs: summarize perf gains and refresh the benchmark tables#686
gaborbernat merged 4 commits into
tox-dev:mainfrom
gaborbernat:docs/perf-changelog-1.5

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Jul 18, 2026

Copy link
Copy Markdown
Member

A single changelog entry for the 1.5.0 cycle's performance work, summarizing the C-core optimizations landed since 1.4.0 (#648 through #680) rather than leaving them without a user-facing note. The numbers come from a same-machine A/B of the release branch against the 1.4.0 tag on a fixed workload: an id-selective XPath lookup about 260x faster, overlapping text search over 100x, Unicode normalization about 6x, and a spread of validation, extraction, and minification operations 2x to 5x, alongside the serialize scan's ~24% gain. The entry groups the changes by technique so a reader sees the shape of the work, not a flat list of issue numbers.

Publishing those gains surfaced a defect in the bench tooling. report.py names each emitted feed after its operation, but the performance guide names its tables for the section they sit in (querying-2.json, building-3.json), and nothing mapped the two. So --table-json docs/development/bench could never refresh the committed feeds; it dropped operation-named files beside them, and every refresh was a hand copy. That is how the guide's tables went stale and quietly lost competitor columns, while report.py's own docstring claimed a straight copy was all it took. bench.docs_feeds now holds that mapping and writes the committed feeds, mirroring bench.migration, which already did the job for the per-library feeds, and a test pins the map against the guide's own bench-table directives so the two cannot drift apart again.

Every feed is regenerated from a full profile-guided run through that generator, so the committed files are reproducible output rather than a hand copy, and the refresh procedure is written into the guide instead of being folklore. The read-path tables pick up the wider competitor field the suites gained, and the prose across the performance and migration guides is rewritten so each ratio, competitor list, and comparative claim tracks a measured number. Claims that had flipped are corrected: turbohtml now leads libxml2 on XML parsing and libxslt on XSLT, lightningcss on every stylesheet it accepts, and parsel no longer sits out the text collectors. The competitor timeout also moves from 900s to 5400s, so a library that reparses the page on every mutation iteration, or walks multi-megabyte input, gets measured instead of dropped for being slow.

@gaborbernat gaborbernat added the documentation Improvements or additions to documentation label Jul 18, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 18, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 106 untouched benchmarks
⏩ 20 skipped benchmarks1


Comparing gaborbernat:docs/perf-changelog-1.5 (c956f53) with main (e8af0a7)

Open in CodSpeed

Footnotes

  1. 20 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

The harness writes one feed per operation, but the performance guide names
its tables for the section they sit in, and nothing mapped the two. So
`--table-json` could never refresh the committed feeds: it dropped op-named
files beside them, and every refresh was a hand copy. That is how the guide's
tables went stale and quietly lost competitor columns, while report.py's own
docstring claimed a straight copy was all it took.

bench.docs_feeds holds the mapping and writes the committed feeds, mirroring
bench.migration, which already did this for the per-library feeds. Three
tables predate a split of their operation and still show the combined view
the guide describes, so they name the operation behind each row and column. A
test pins the map against the guide's own directives so the two cannot drift
apart again.

The competitor timeout moves from 900s to 5400s. A library that reparses the
page every mutation iteration, or walks multi-megabyte input, was being
dropped for being slow rather than measured. XPath case labels double as the
evaluated expression, so the emitted feed carried a bare star that broke the
RST build; the display label is now inline-literal while the expression stays
raw.
The comparison tables in the performance and migration guides had drifted
from the bench. Operation titles, competitor sets, and case names moved as
the suites grew, and the committed feeds carried stale numbers and
competitors that had dropped out, so the published ratios no longer matched
what the harness measures.

Every feed is regenerated from a full profile-guided run through the new
generator, so the committed files are reproducible output rather than a hand
copy. The read-path tables pick up the wider competitor field the suites
gained, and the prose across both guides is rewritten so each ratio,
competitor list, and comparative claim tracks a measured number. Claims that
had flipped are corrected: turbohtml now leads libxml2 on XML parsing and
libxslt on XSLT, lightningcss on every stylesheet it accepts, and parsel no
longer sits out the text collectors.
@gaborbernat gaborbernat changed the title 📝 doc: summarize perf gains since 1.4.0 📝 docs: summarize perf gains and refresh the benchmark tables Jul 19, 2026
@gaborbernat
gaborbernat enabled auto-merge (squash) July 19, 2026 16:28
@gaborbernat
gaborbernat disabled auto-merge July 19, 2026 16:34
@gaborbernat
gaborbernat marked this pull request as draft July 19, 2026 16:34
Raising the competitor timeout let four operations finish with libraries the
old 900s bound had cut: BeautifulSoup on the three mutation sweeps, dominate
on unescape. Their tables and prose carried an absence that no longer holds,
and the claim that those libraries "cannot finish within the harness's time
bound" contradicted the timeout change shipping alongside it.

Regenerated the four through bench.docs_feeds and rewrote the prose to the
measured numbers. BeautifulSoup trails by 5 to 11 times on the rel-tagging
sweep and 12 to 44 on the content setters, reparsing the page each iteration
where the others splice into a live tree. dominate sits with the rest on short
strings but takes whole seconds on multi-megabyte input, trailing by 327 times
on the 4 MiB book and past 2,000 on the escaped copy.
@gaborbernat
gaborbernat marked this pull request as ready for review July 19, 2026 22:18
@gaborbernat
gaborbernat merged commit 46d22e5 into tox-dev:main Jul 19, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant