docs(performance): disclose that the tables are a 3.3.0 snapshot - #447
Merged
Conversation
The Results provenance block named the measurement date and version but was silent on the fact that a release never regenerates the tables, so a reader took them as current. Adds a disclosure adjacent to the tables and points a reader on a newer version at `just bench-report`. No published figure changes. Closes #437 Claude-Session: https://claude.ai/code/session_01FBic7HFGR6sdRfSW3cGEv2
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: 05999d2 | Previous: 6f67540 | Ratio |
|---|---|---|---|
benchmarks/test_guard_by_type.py::test_g16_resolve_by_type |
5787196.1179252835 iter/sec (stddev: 4.947042644519418e-9) |
3205455.6342188003 iter/sec (stddev: 1.0528672764767563e-8) |
0.55 |
benchmarks/test_guard_by_type.py::test_g17_resolve_by_type_large_registry |
5828812.554886123 iter/sec (stddev: 3.739837913258584e-9) |
3187185.4745330936 iter/sec (stddev: 1.0296778470929436e-8) |
0.55 |
benchmarks/test_guard_cold.py::test_g8_cold_first_resolve |
50123.60961175268 iter/sec (stddev: 0.000002028819450429028) |
23547.814601633792 iter/sec (stddev: 0.000005075884786377669) |
0.47 |
benchmarks/test_guard_cold.py::test_g8b_cold_first_resolve_cached |
39991.792160809164 iter/sec (stddev: 0.0000015285391219845316) |
18176.964019491872 iter/sec (stddev: 0.0000049408642721595806) |
0.45 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[1] |
854.5255350409437 iter/sec (stddev: 0.00005816365271536236) |
453.680798029442 iter/sec (stddev: 0.00003598789074031733) |
0.53 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[2] |
803.5870031741475 iter/sec (stddev: 0.000045940314571031134) |
435.14603740694537 iter/sec (stddev: 0.00003065167807239478) |
0.54 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[4] |
737.2866828885494 iter/sec (stddev: 0.00003222339236009992) |
395.8255470762083 iter/sec (stddev: 0.00004845839813536915) |
0.54 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[1] |
4641.916740265547 iter/sec (stddev: 0.000015980733469463767) |
2237.462991437241 iter/sec (stddev: 0.00003066750230688572) |
0.48 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[2] |
3280.1809172863173 iter/sec (stddev: 0.0001860787181486862) |
1628.71107077806 iter/sec (stddev: 0.00016989398870783106) |
0.50 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[4] |
2281.9762781748636 iter/sec (stddev: 0.000023665050169198007) |
1154.9869255497606 iter/sec (stddev: 0.000031339169738511176) |
0.51 |
benchmarks/test_guard_lifecycle.py::test_g6_build_child_container |
1385167.4057945262 iter/sec (stddev: 2.4273953674301242e-8) |
752448.3068982477 iter/sec (stddev: 5.630491921340849e-8) |
0.54 |
benchmarks/test_guard_lifecycle.py::test_g6b_build_child_container_auto_scope |
1245209.368255738 iter/sec (stddev: 1.2735549319542264e-8) |
684103.7647189967 iter/sec (stddev: 4.718667038338458e-8) |
0.55 |
benchmarks/test_guard_lifecycle.py::test_g7_request_lifecycle_batch |
4612.150813290217 iter/sec (stddev: 0.00000470849839308595) |
2346.0451804816475 iter/sec (stddev: 0.000013521817664666) |
0.51 |
benchmarks/test_guard_lifecycle.py::test_g7c_event_loop_floor_control |
120533.258819918 iter/sec (stddev: 7.305800999249184e-7) |
60942.5184357266 iter/sec (stddev: 0.0000025882477312840462) |
0.51 |
benchmarks/test_guard_lifecycle.py::test_g13_teardown_at_scale |
84978.36233417281 iter/sec (stddev: 7.591588756505079e-7) |
46163.40710244865 iter/sec (stddev: 0.0000029018998137983025) |
0.54 |
benchmarks/test_guard_resolve.py::test_g1_transient_resolve |
3948176.2386848517 iter/sec (stddev: 1.668169782909242e-8) |
1948716.219888309 iter/sec (stddev: 2.8579463781773687e-8) |
0.49 |
benchmarks/test_guard_resolve.py::test_g2_cached_resolve |
6426382.097888438 iter/sec (stddev: 6.315201088047079e-9) |
3579207.2112673293 iter/sec (stddev: 1.2320663285516314e-8) |
0.56 |
benchmarks/test_guard_resolve.py::test_g3_deep_chain |
1493794.386002668 iter/sec (stddev: 9.195117558908416e-8) |
740067.7587531799 iter/sec (stddev: 1.2139170651420763e-7) |
0.50 |
benchmarks/test_guard_resolve.py::test_g4_wide_resolve |
750809.2315744263 iter/sec (stddev: 5.246775179481212e-8) |
388182.87481581495 iter/sec (stddev: 4.3603175743809157e-7) |
0.52 |
benchmarks/test_guard_resolve.py::test_g5_cross_scope |
3286038.7468258287 iter/sec (stddev: 1.161459330145565e-8) |
1384818.1965701173 iter/sec (stddev: 1.9440519258309444e-7) |
0.42 |
benchmarks/test_guard_resolve.py::test_g9_context_resolve |
1617305.099876306 iter/sec (stddev: 2.0102115702840954e-8) |
770855.7628303291 iter/sec (stddev: 2.8777347044699594e-7) |
0.48 |
benchmarks/test_guard_resolve.py::test_g12_override_active_resolve |
981943.4831628556 iter/sec (stddev: 2.5064126830522014e-8) |
495208.9524314781 iter/sec (stddev: 7.140706897468433e-8) |
0.50 |
benchmarks/test_guard_resolve.py::test_g18_alias_hop |
4375848.066708667 iter/sec (stddev: 6.075812656517837e-9) |
2412589.703683903 iter/sec (stddev: 1.3780975826847452e-8) |
0.55 |
benchmarks/test_guard_validate.py::test_g10_validate_deep_chain |
58189.10500942102 iter/sec (stddev: 0.0000014601016703224283) |
27569.099605101164 iter/sec (stddev: 0.00000434712551107999) |
0.47 |
benchmarks/test_guard_validate.py::test_g11_validate_wide |
34679.505821185296 iter/sec (stddev: 0.00000118024600925649) |
17122.1374973838 iter/sec (stddev: 0.000005124850992113708) |
0.49 |
This comment was automatically generated by workflow using github-action-benchmark.
Two bolded lead-ins in "What the numbers show" read "modern-di is now faster than dishka" and "the by-type surcharge is now small enough", where "now" anchors to the reader's present rather than to the measured version. Names the version instead. Also drops the republication rationale from the snapshot blockquote, which restates benchmarks/README.md, and points it at the version line above rather than repeating the number. No published figure changes. Claude-Session: https://claude.ai/code/session_01FBic7HFGR6sdRfSW3cGEv2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
docs/introduction/performance.mdis the most public artifact in the repo: it makes namedcomparative claims against four other frameworks. Its
## Resultsblock states the measurementdate, the modern-di version, the machine, the CPython build and the rival versions, and it is
accurate about every one of them. It is silent about one thing: the tables do not update when a
new modern-di ships. A reader therefore takes them as current, and they are only current until
the next release.
The page has been behind the shipping version before, twice on 2026-08-03, once by a margin that
materially understated the library. It is behind again today: the page reads 3.3.0 and 3.4.0 has
shipped. Regenerating the tables needs the isolated comparative environment (four rival packages)
and a quiet controlled machine to reproduce the page's stated methodology, so the republish
necessarily lags the release rather than being part of it.
Closes #437.
Design
Prose only. A blockquote goes directly under the
## Resultsprovenance paragraph, adjacent tothe tables rather than only in the page intro, so a reader who skims straight to the numbers still
meets it. It states three things: the figures are a snapshot of 3.3.0, a release does not update
them, and a reader on a later version should run
just bench-reportthemselves. It links to thepage's existing
Reproduce it yourselfsection rather than restating the command's setup.The blockquote matches the idiom the page already uses for its other caveat (the machine-variance
note at the top), which covers a different axis and is left alone.
Two bolded lead-ins under
## What the numbers showworked against that: "modern-di is nowfaster than
dishka" and "the by-type surcharge is now small enough to stop mattering" bothanchor "now" to the reader's present rather than to the measured version, which is the implication
the disclosure exists to remove. Both now name 3.3.0 instead. No figure moves.
The effect is that a newer release now makes this page older, which it discloses, rather than
wrong, which it previously implied.
Non-goals
lint-cior_checks.yml(bothgate every PR, and the window between a tag and the republish is a state that legitimately
exists for days), not in
scheduled.yml, not inrelease.yml, not as a line inplanning/_templates/release.md. The page going stale between a release and its republish isan accepted cost; the fix is to stop the page claiming otherwise.
just bench-reportwas not run; theMeasured 2026-08-03 with modern-di 3.3.0line names the same date and version it named before. No ratio, absolute timing, IQR or±figure changes anywhere on the page.committed benchmark result files (
benchmarks/README.mdkeeps that policy).Verification
No code changed, so no test was added; the page is prose and the gates that cover it are the
doc ones.
just docs-build(mkdocs--strict, this page is in the nav) — passes.just check-links—links: OK; the new#reproduce-it-yourselfanchor resolves.just lint-ci— passes.just test-ci— 512 passed, 100% line coverage.