Skip to content

celery: shard subscriber functions per derived config - #316

Open
nsidnev wants to merge 1 commit into
nsidnev/dramatiq-max-durationfrom
nsidnev/celery-max-duration
Open

celery: shard subscriber functions per derived config#316
nsidnev wants to merge 1 commit into
nsidnev/dramatiq-max-durationfrom
nsidnev/celery-max-duration

Conversation

@nsidnev

@nsidnev nsidnev commented Aug 20, 2026

Copy link
Copy Markdown
Member

Makes the Celery adapter respect max_duration for subscriber functions and shard queue topics, so that tasks with different hard time_limit values are deployed as separate functions with the appropriate maxDuration. Tasks without an explicit limit stay on the queue's base topic, whose limit comes from the new max_duration entry in broker_transport_options or the app's task_time_limit setting.

Registration now finalizes the app first, because task declarations on an unfinalized app are still pending decorators, so reading them earlier would silently register no shards and Celery's finalize hooks run in nondeterministic order, so registration is deferred to on_after_finalize, when all tasks are known.

Like the Dramatiq adapter and the SDK, time_limit values above 30 minutes are capped with a warning instead of an error so user code remains deployable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant