Add method_churn: type inference as a GC workload - #104
Open
gbaraldi wants to merge 1 commit into
Open
Conversation
Repeatedly define call trees of fresh gensym-named methods and run code_typed over them, so every iteration re-runs inference and optimization from scratch with no cache reuse. This exercises the GC under the compiler's own allocation profile (Exprs, CodeInfos, IR, type objects) - the profile of package loading and REPL work - rather than the uniform data structures of the other benchmarks. Complements inference_benchmarks.jl, which measures the compiler's speed rather than the GC underneath it. Runs ~6s measured with ~2.8GB allocated and ~10% GC time on Julia 1.12. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
|
This seems pretty synthetic. Is it actually useful? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repeatedly defines call trees of fresh
gensym-named methods (6 leaves of varying body shapes, 2 mids, 1 top per unit; 40 units × 5 iterations) and runscode_typedover each top, so every iteration re-runs inference and optimization from scratch.Why this earns a slot next to
inference_benchmarks.jl: both exercise the GC under the compiler's allocation profile, and they are about equally GC-bound — measured on Julia 1.12.6, this runs ~6 s with ~2.8 GB allocated at ~10% GC time, whileinference_benchmarks.jlruns ~18 s with ~8.9 GB at ~10–11% GC time (near-identical allocation rates, ~0.47 vs ~0.49 GB/s, mark/sweep split ~60/40 in both). The differences are in benchmark quality rather than GC fraction:inference_benchmarks.jlruns a fixed target set against caches that fill as it goes (several of its targets specifically measure caching, e.g.CachedMethodTable, GlobalRef caching). Fresh gensym'd methods make every iteration identical work, so the GC load is steady and repeatable.UNITS_PER_ITER/NITERscale the workload linearly; the fixed suite does not.Uses
invokelatestfor the post-evalbinding lookups, so it is clean under 1.12's strict binding world-age semantics.Disclosure: this PR was written with the assistance of generative AI (Claude).
🤖 Generated with Claude Code