You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/explanation/scheduler.md
+32-32Lines changed: 32 additions & 32 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,9 +9,9 @@ For the user-facing view of pools, priorities, groups, and idle tasks, see [Thre
9
9
Every reaction execution is a **task** (`ReactionTask`) submitted to the scheduler. The `PowerPlant` owns a single `Scheduler` instance and forwards all work to it:
10
10
11
11
1. A trigger (message emit, timer, IO event, etc.) creates a `ReactionTask`.
1. The scheduler resolves the target **pool**, acquires any required **group** tokens, and enqueues the task.
14
+
1. A pool worker dequeues the task, runs the callback, and releases group locks when the callback returns.
15
15
16
16
`PowerPlant::start()` calls `Scheduler::start()`, which starts worker pools and then blocks the calling thread in the **MainThread** pool until shutdown. `PowerPlant::shutdown()` emits the shutdown event and calls `Scheduler::stop()`.
17
17
@@ -129,11 +129,11 @@ If a reaction is bound with `Inline` and belongs to a single group, the schedule
129
129
130
130
Each pool holds an array of five `Queue<Task>` instances — one per priority bucket. At construction time the pool chooses the concrete queue type:
131
131
132
-
| Pool kind | Queue type | Why |
133
-
| --------- | ---------- | --- |
134
-
| Default pool (`Pool<>`) |`TaskQueue` (MPMC) | Concurrency may differ from the descriptor's nominal value; multiple workers dequeue concurrently. |
135
-
|`MainThread`, Trace pool, any pool with `concurrency == 1`|`MPSCQueue` (MPSC) | Exactly one consumer; simpler and cheaper than MPMC. |
136
-
| Custom pools with `concurrency > 1`|`TaskQueue` (MPMC) | Multiple workers compete for tasks. |
| Default pool (`Pool<>`) |`TaskQueue` (MPMC) | Concurrency may differ from the descriptor's nominal value; multiple workers dequeue concurrently. |
135
+
|`MainThread`, Trace pool, any pool with `concurrency == 1`|`MPSCQueue` (MPSC) | Exactly one consumer; simpler and cheaper than MPMC. |
136
+
| Custom pools with `concurrency > 1`|`TaskQueue` (MPMC) | Multiple workers compete for tasks.|
137
137
138
138
The virtual `Queue` interface lets `Pool` store both implementations in one `std::array` without templating the entire pool. The virtual call cost is negligible compared to the atomic operations inside enqueue and dequeue.
139
139
@@ -143,13 +143,13 @@ Workers identify themselves via a thread-local `Pool::current_pool` pointer, set
143
143
144
144
Tasks are not kept in one monolithic priority queue. Instead, each pool has **five fixed buckets** scanned from highest to lowest priority:
`Pool::try_dequeue_task()` walks buckets 0→4 and returns the first available task. Within a bucket, ordering is **FIFO** (per-producer FIFO in the MPMC queue; strict FIFO in MPSC). Priority therefore dominates bucket order; tie-breaking within a bucket follows enqueue order, not reaction ID.
155
155
@@ -198,8 +198,8 @@ The hot-path slot claim via `fetch_add` is wait-free within a non-full block. Se
198
198
Most reactions belong to at most one group (including `Sync<T>`). For these, `Group::try_submit()`:
199
199
200
200
1. Tries to decrement `tokens` with a compare-exchange.
201
-
2. On success, submits to the pool immediately with a `RunningLock` that calls `release_token()` on destruction.
202
-
3. On failure, **parks** the task in priority-ordered waiter buckets via `park_publish()` / `park_reconcile()`.
201
+
1. On success, submits to the pool immediately with a `RunningLock` that calls `release_token()` on destruction.
202
+
1. On failure, **parks** the task in priority-ordered waiter buckets via `park_publish()` / `park_reconcile()`.
203
203
204
204
The token counter can go **negative** when waiters reserve slots they have not yet consumed. This signed counter, combined with per-waiter **arbiter slots** (`atomic<bool>`), ensures no lost wakeups and exact accounting when multiple waiters race with draining threads.
205
205
@@ -230,36 +230,36 @@ Idle reactions (`on<Idle<>>`, `on<Idle<Pool<T>>>`) are registered via `PowerPlan
230
230
When a pool worker finds no runnable task:
231
231
232
232
1. It tries `get_idle_task()` — acquiring counting locks that track per-thread and per-pool idle state.
233
-
2. When all threads in a pool are idle and the pool holds the global idle lock, global idle reactions are collected.
234
-
3. A synthetic `ReactionTask` runs that re-submits each idle reaction's task via `scheduler.submit()`.
233
+
1. When all threads in a pool are idle and the pool holds the global idle lock, global idle reactions are collected.
234
+
1. A synthetic `ReactionTask` runs that re-submits each idle reaction's task via `scheduler.submit()`.
235
235
236
236
`global_idle_count` is an atomic so pools can cheaply check whether global idle exists without locking the scheduler on every external-waiter registration.
237
237
238
238
### Shutdown sequence
239
239
240
240
`Scheduler::stop(force)` sets `running = false` and stops all pools.
241
241
242
-
| Stop type | Behaviour |
243
-
| --------- | --------- |
244
-
|`NORMAL`| Pools stop accepting new work (except **persistent** pools, which keep accepting during shutdown). Workers drain queued tasks. |
245
-
|`FINAL`| Used after the main thread exits `start()`; even persistent pools stop once their queues empty. |
246
-
|`FORCE`| Clears queues and wakes all threads; used for forced test timeouts. MPSC pools require the consumer thread to perform the drain. |
|`NORMAL`| Pools stop accepting new work (except **persistent** pools, which keep accepting during shutdown). Workers drain queued tasks.|
245
+
|`FINAL`| Used after the main thread exits `start()`; even persistent pools stop once their queues empty.|
246
+
|`FORCE`| Clears queues and wakes all threads; used for forced test timeouts. MPSC pools require the consumer thread to perform the drain. |
247
247
248
248
`Scheduler::start()` starts worker pools first, then blocks in `MainThread::start()`. When the main thread pool exits (after shutdown), pools are stopped in order — non-persistent pools before persistent ones — then joined.
249
249
250
250
Persistent pools (`ThreadPoolDescriptor::persistent`) continue accepting tasks during a normal shutdown so networking or logging reactors can finish in-flight work.
251
251
252
252
## Design tradeoffs
253
253
254
-
| Choice | Rationale |
255
-
| ------ | --------- |
256
-
| Virtual `Queue` interface | One bucket array in `Pool` without templating the entire pool; indirection cost is dwarfed by atomics. |
257
-
| Separate `MPSCQueue`| Single-consumer pools avoid MPMC CAS on dequeue; meaningful win for `MainThread` and concurrency-1 pools. |
| Virtual `Queue` interface | One bucket array in `Pool` without templating the entire pool; indirection cost is dwarfed by atomics.|
257
+
| Separate `MPSCQueue`| Single-consumer pools avoid MPMC CAS on dequeue; meaningful win for `MainThread` and concurrency-1 pools.|
258
258
| Priority buckets vs one sorted queue | Fixed five buckets give O(1) bucket selection and lock-free queues per level; fine-grained priority within a bucket is FIFO, not strict global ordering by task ID. |
259
-
| Lock-free group fast path | Single-group `Sync` is the common case; parking in lock-free buckets avoids mutex contention on submission. |
260
-
| Mutex for pool/group maps | Pools and groups are created once per descriptor; mutex cost is paid on first use, not every submit. |
261
-
| Condition variable for workers | Lock-free queues hold tasks, but workers must sleep when idle; CV + `live` flag avoids busy-waiting. |
262
-
| Non-preemptive execution | Simpler reasoning, no priority inversion from preemption; long tasks hold a thread until completion. |
259
+
| Lock-free group fast path | Single-group `Sync` is the common case; parking in lock-free buckets avoids mutex contention on submission.|
260
+
| Mutex for pool/group maps | Pools and groups are created once per descriptor; mutex cost is paid on first use, not every submit.|
261
+
| Condition variable for workers | Lock-free queues hold tasks, but workers must sleep when idle; CV + `live` flag avoids busy-waiting.|
262
+
| Non-preemptive execution | Simpler reasoning, no priority inversion from preemption; long tasks hold a thread until completion.|
0 commit comments