Skip to content

perf: keep each green task's join, closure and threadlocals in its sync block - #1235

Merged
kacy merged 4 commits into
mainfrom
perf/spawn-path
Oct 29, 2026
Merged

kacy merged 4 commits into
mainfrom
perf/spawn-path

Conversation

@kacy

@kacy kacy commented Oct 28, 2026

Copy link
Copy Markdown
Owner

Stacked on #1233.

A spawn plus an await took about a dozen lock/unlock pairs and three allocations per task:

  • a join Arc<(Mutex, Condvar)> and a boxed threadlocal map;
  • three handle-registry shard locks;
  • the slab shard lock, taken up to five times;
  • the per-worker stack mutex, taken twice;
  • the join lock.

green_fanout was 3.6x Go.

Task state moves into the task's SyncBlock. These blocks sit in an arena whose chunks never move or free. The join (generation, done, result, waiters, OS-thread waiter count, behind one small mutex and a condvar), the threadlocal map and the closure now live there inline.

  • A handle is still (index, generation). Await, detach and is_done check the generation (and a live flag) under the join lock, so a stale or garbage handle resolves to nothing. That makes the registry lookups redundant for green tasks.
  • The first await retires the task: it bumps the generation and frees the id. A second await of the same handle now returns 0. Before, both awaits took the result as owned.
  • The slab's per-task entries are gone. Ids come from a free pool each worker thread owns, then a shared pool, then a fresh arena chunk.

Thread-owned worker cache. Each worker's stack and id cache sits behind an owner-only cell. A separate thread-local, which only worker_loop sets, keys it, so a test or probe posing as a worker uses the locked shared pools instead.

Thieves skip held queues. A thief or injector popper that finds a queue locked moves on instead of blocking. 15 idle workers used to queue up in the kernel on one mutex. The queue's atomic length still reads non-zero, so no worker parks while work remains. steal_half also no longer allocates a Vec per steal.

green_fanout (2M spawn+await in batches of 64), ms:

case before after
as written (main awaits from its OS thread), 16 workers 860 620
as written, 1 worker 700 500
loop inside a task, 16 workers 965 556
loop inside a task, 1 worker 596 326

Tried and dropped: a short spin before an OS-thread await blocks. It was 4% faster at 4 workers and 6% slower at 16.

Local corpus, leak, memcheck, green, result-box, cargo test, seed, TLS-barrier and panic gates pass. The slab unit tests are replaced by lifecycle tests: retiring on await, stale generations, and detaching running and finished tasks.

@kacy
kacy changed the base branch from perf/pinned-steal to main October 28, 2026 23:52
# Conflicts:
#	cranelift/runtime/src/concurrency/green.rs
@kacy
kacy merged commit a8788c5 into main Oct 29, 2026
4 checks passed
@kacy
kacy deleted the perf/spawn-path branch October 29, 2026 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant