Skip to content

Commit 2503827

Browse files
committed
feat(stream): per-task reclaim, prefetch backpressure, NOGROUP self-heal
RedisStreamBroker reclaim was driven by a fixed idle_timeout and a broker-side Redis lock, which both double-ran long tasks and recovered short crashed tasks slowly. Replace it with per-task-deadline reclaim: - Resolve reclaim deadline from the message's `timeout` label (via formatter.loads) plus reclaim_timeout_grace, falling back to idle_timeout for messages without a timeout label. - Drop the autoclaim Redis lock; rely on XCLAIM min-idle-time for server-side atomic dedup (unacknowledged_lock_timeout is deprecated and ignored). - Protect only messages held by the current listener instance from reclaim (tracked via a local delivered set), so a worker sharing a consumer_name with a dead predecessor still recovers its pending. - Gate reclaim sweeps with reclaim_interval (default 30s) to avoid scanning pending on every listen iteration. - Enforce prefetch backpressure: do not XREADGROUP while xread_count delivered-but-unacked messages are outstanding. - Recreate a missing consumer group on NOGROUP during xpending/xreadgroup. Tests cover timeout-label reclaim, idle_timeout reclaim, shared consumer_name reclaim, prefetch backpressure, and NOGROUP self-heal. README documents the new reclaim/backpressure behavior.
1 parent 39c6b7c commit 2503827

3 files changed

Lines changed: 707 additions & 70 deletions

File tree

README.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,23 @@ Stream brokers use redis [stream type](https://redis.io/docs/latest/develop/data
9898
> This broker **supports** acknowledgements and therefore is fine to use in cases when data durability is
9999
> required.
100100
101+
`RedisStreamBroker` is the recommended stream broker for single-node redis. It uses a consumer group so every message is delivered to exactly one worker, and pending entries are tracked by redis until they are acknowledged.
102+
103+
Recovery of messages orphaned by a crashed worker is driven by `XCLAIM`:
104+
105+
* Messages with a `timeout` label are reclaimed once they have been pending for longer than the task's own timeout plus `reclaim_timeout_grace`. A long-running task is never stolen while it is still within its declared timeout.
106+
* Messages without a `timeout` label fall back to `idle_timeout` (10 minutes by default).
107+
* Reclaim sweeps run at most every `reclaim_interval` (30 seconds by default) to avoid hammering redis on every listen loop. Set it to `0` to scan on every iteration.
108+
* A worker that shares its `consumer_name` with a previous, dead worker can still reclaim that worker's pending messages; only messages actually held by the current listener instance are protected from re-delivery.
109+
110+
To avoid one worker hoarding the backlog, the listener does not fetch new messages while it already has `xread_count` delivered but unacknowledged messages. Set `xread_count=None` to disable this limit.
111+
112+
When a listener is closed with messages already fetched from redis but not yet yielded to taskiq, those buffered entries are claimed to an internal `abandoned` consumer and stamped as very idle. The next reclaim sweep can recover them immediately instead of waiting for `idle_timeout`.
113+
114+
If the consumer group is removed out of band, the broker recreates it on the next `XREADGROUP` so an in-use queue heals instead of spinning on errors.
115+
116+
The legacy `unacknowledged_lock_timeout` parameter is deprecated and ignored; reclaim correctness now relies on the `XCLAIM` min-idle-time check instead of a broker-side redis lock.
117+
101118
## RedisAsyncResultBackend configuration
102119

103120
RedisAsyncResultBackend parameters:

0 commit comments

Comments
 (0)