Add ChainRetryPolicy to common.compat for Airflow 3.3

[kaxil]

Follow-up to #73508, which added ChainRetryPolicy to the Task SDK in Airflow 3.4. Retry policies themselves arrived in 3.3, so a Dag on 3.3 can run a chain as long as the class comes from somewhere. This makes airflow.providers.common.compat.sdk that place: where the SDK has the class (3.4 and later) the compat module returns the SDK class, on 3.3 a copy in this provider stands in for it, and on 3.2 and below the name is absent, as retry policies are.

from airflow.providers.common.compat.sdk import ChainRetryPolicy, ExceptionRetryPolicy, RetryAction, RetryRule

The other retry policy names (RetryPolicy, RetryDecision, RetryAction, RetryRule, ExceptionRetryPolicy) are exported from the same module, so a chain is one import line. Those map straight to the SDK on every version that has them.

The tests ran on main (Airflow 3.4.0), where compat resolves to the SDK class, and against a released Airflow 3.3.2 with Task SDK 1.3.2 and this provider installed from a wheel, where compat resolves to the copy. The behaviour tests import the copy directly and one drives it through the worker's own _evaluate_retry_policy, so both run on both versions.

Design rationale

A copy in compat, rather than raising the Common AI floor to 3.4. Common AI floors apache-airflow>=3.0.0; requiring 3.4 for one class would drop 3.0 to 3.3 for the whole provider, and it would still leave every other Dag author on 3.3 without the chain. Vendoring the class privately inside Common AI would serve one consumer. The compat provider exists for this gap and already carries class copies (the lineage entities) and polyfills, so a copy here serves any Dag on 3.3 through one import line. The cost is that the copy stays in the package until 3.3 leaves compat's support window, which is why the tests compare the source of both classes and fail on drift.

Resolved through the existing lazy import map, not a version gate at import time. The compat sdk module resolves names on first access. Adding map entries means importing airflow.providers.common.compat.sdk gains no new eager import, and the SDK path is listed first, so wherever the SDK has the class the copy is never imported. The copy imports RetryPolicy, RetryDecision and RetryAction from the SDK rather than duplicating them, so it produces SDK decisions and is a subclass of the SDK base; the worker only calls evaluate and only a has_retry_policy flag is serialized, so nothing can tell the two classes apart.

Gated on 3.3 like the other 3-only names. On Airflow 2 and on 3.0 to 3.2 the import raises ImportError, the way DownstreamTasksSkipped does there. A provider that supports those versions and imports any of the six names at module level has to guard the import, as Common AI already does for its SDK retry imports.

The Common AI retry guide will point 3.3 users at this import once it is in a compat release.

Live run on Airflow 3.3.2

A released Airflow 3.3.2 (Task SDK 1.3.2) under Breeze with this provider installed from a wheel, and a Dag whose only retry-policy imports come from airflow.providers.common.compat.sdk. One chain for every task: a rule set that fails a PermissionError, a deliberately broken policy that raises, then a floor that retries a ConnectionError after 15 seconds. Each task logs the class it is running, which on 3.3.2 is the copy in this provider.

Task	Failure raised	Decided by	Decision and reason
rules_first__403	PermissionError	first rule set	FAIL, ExceptionRetryPolicy: never retry a 403
floor__connection_refused	ConnectionError	the floor, after the broken policy is skipped	RETRY 15s, ExceptionRetryPolicy: floor (after ExceptionRetryPolicy: no decision; BrokenPolicy: raised RuntimeError)
nothing_decides__value_error	ValueError	nobody; the task's own retries apply	DEFAULT, no policy decided (ExceptionRetryPolicy: no decision; BrokenPolicy: raised RuntimeError; ExceptionRetryPolicy: no decision)

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.