Skip to content

Global scheduler pause override (preserve per-DAG is_paused) #69699

Open

Description

@pedro-cf

Description

Add a deployment-wide global pause toggle (UI, REST API, and CLI) that makes the scheduler treat every DAG as paused without changing any DAG's stored is_paused value.

While global pause is enabled:

  • The scheduler schedules no new DAG runs.
  • The scheduler schedules no new tasks.
  • Per-DAG is_paused in the metadata DB stays untouched.

When global pause is disabled:

  • Each DAG immediately returns to its previous behavior based on its own is_paused state.
  • No restore step and no tracking of which DAGs were paused before.

This is a scheduler-level override, not bulk pause/unpause of individual DAGs.

The UI should show when global pause is active. Per-DAG pause toggles should still reflect each DAG's real is_paused value (not the effective scheduling state), so operators can see which DAGs were already paused before the freeze.

Mockup

Image

Use case/motivation

We need a way to freeze an entire Airflow deployment for operational work (upgrades, migrations, incident response) without losing per-DAG pause state.

Pausing every DAG individually overwrites is_paused and makes it hard to restore the previous mix of paused and active DAGs on large deployments. Suspend/resume style approaches that mutate is_paused and restore later have the same problem.

Typical scenario: a deployment with many DAGs, some intentionally paused and some active. An admin needs to stop all scheduling quickly, do maintenance, then release the freeze and have every DAG behave exactly as it did before. No custom restore script, no risk of unpausing DAGs that were paused on purpose.

Related issues

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. shivaam commented on Jul 14, 2026

    @shivaam
    Contributor

    @Moemenmohamed24 we dont assign issues any more. You can directly start working on it and create a PR if you want to.

    https://github.com/apache/airflow/blob/main/contributing-docs/04_how_to_contribute.rst#contribute-code-changes

  2. bbovenzi commented on Oct 1, 2026

    @bbovenzi
    Contributor

    This is partially enabled in the UI in 3.4.0 with filtering dags by not paused dags and then pausing all or draining all at once. It has some pagination limits, but it will be a lot less bad than before.

    Image
  3. pedro-cf commented on Oct 1, 2026

    @pedro-cf
    Author

    Thanks @bbovenzi , I think the existing bulk pause/drain functionality addresses a related use case, but not quite the one proposed here.

    The key requirement is that a global pause must not modify the individual DAGs' is_paused state.

    For example, if A and B are active and C is intentionally paused, a global pause should leave the stored state as:

    A: is_paused = false
    B: is_paused = false
    C: is_paused = true
    

    The scheduler would simply treat all three as paused while the global override is enabled. Disabling the override would then immediately restore the previous behavior without needing to track or restore any DAG state.

    With the current bulk pause approach, A and B would be changed to is_paused = true, and bulk unpausing them later could also unpause C unless the previous state is tracked separately.

    So I see the 3.4 UI functionality as helpful for bulk operations, but the proposal here is specifically for a scheduler level override, rather than a bulk mutation of per DAG pause state.

  4. pedro-cf commented on Oct 2, 2026

    @pedro-cf
    Author

    @bbovenzi

    Mockup:

    Image
  5. removed
    needs-triagelabel for new issues that we didn't triage yet
    on Oct 6, 2026
  6. bbovenzi commented on Oct 6, 2026

    @bbovenzi
    Contributor

    Yes, that's why I didn't close the issue. Just providing additional context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions