Skip to content

perf: plain counts on structs only one task can reach - #1230

Merged
kacy merged 10 commits into
mainfrom
perf/task-local-counts
Oct 28, 2026
Merged

kacy merged 10 commits into
mainfrom
perf/task-local-counts

Conversation

@kacy

@kacy kacy commented Oct 28, 2026

Copy link
Copy Markdown
Owner

A struct's strong and weak counts are now plain loads and stores while only the task that built it can reach it, and atomic once it is marked shared. On struct graphs the locked read-modify-write was most of the count cost (an unsafe all-plain prototype took cyclic_graph from 384 to 232 ms); everywhere else the counts were not the bottleneck.

How it works (cranelift/runtime/src/share.rs):

  • A struct's mark is a bit in its header's flag word. Lists, maps and closures carry the same mark in their flag byte; their own counts stay atomic, and the mark only records that the walk has been through them.
  • A value is marked, with everything it reaches, where it can cross to another task: what a spawn captures (pith_spawn), a task's result (the spawn wrapper), a channel send and a select send arm, a store into a module global and the global's initializer, and a store into a struct, list or map that is already shared (an inline test of the holder's mark at field stores, a check of the container's mark in the list and map store paths). The barrier runs before the store.
  • The walk uses the cycle collector's tracers, which now also report weak fields, under a kind the collector skips: a task that can read a weak reference can upgrade it.
  • Every module's tracers are registered before any global is initialized, since a global is marked as it is stored.
  • A finished task's threadlocal values are marked before the worker releases them outside the task.
  • With the cycle collector on, every count stays atomic. PITH_LOCAL_COUNTS=0 keeps every count atomic for comparison.

PITH_RC_CHECK=1 records the task that built each struct and aborts when another task makes a plain count update on it. make rc-check-only runs the green corpus and the green-thread tests that way, and CI runs it. It found three gaps while this was being built: the field barrier ran after the store, tracers were registered after global init, and function-typed globals were not marked.

Interleaved against main 6a0132b (5 rounds, 9 for the noisy rows): cyclic_graph 401 to 282 ms, cyclic_graph_strong 780 to 724 ms, http_head_read 711 to 683 ms, lock_sweep/labels 61 to 57 ms. task_churn reads 4 to 6% slower; it does not change with PITH_LOCAL_COUNTS, so it is the added checks on the alloc and free paths or placement. Other rows within noise.

New case test_shared_marking crosses tasks through every path above. It is not in the memcheck list: its channel-receive loop shows two invalid reads under valgrind on main too, a separate bug.

@kacy
kacy merged commit 7830a9f into main Oct 28, 2026
1 of 2 checks passed
@kacy
kacy deleted the perf/task-local-counts branch October 28, 2026 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant