Build docs and registry CI images separately so docs stay on default Python - #70959
Merged
Conversation
…Python The docs build and the provider registry shared a single CI image, so they also shared its Python. That image is built at 3.12 because the registry has always extracted on 3.12, which silently moved the docs build there too and broke publishing on 2026-07-28: Sphinx mocks third-party modules while building provider docs, and from 3.12 functools copies __type_params__, for which a mock hands back another mock rather than a tuple. Any provider decorating methods with functools.wraps over a mocked callable then fails to import, and the build also began reporting cross-reference ambiguities it had not reported before. The image job already matrixes over the Python versions it is given, and every cache it touches - registry buildx tag, image stash, commit marker and mount cache - is keyed by Python version. Asking it for both versions therefore builds them in parallel, each seeding and storing its own cache, and needs no second job. Docs go back to the default Python and the registry keeps 3.12. The stashes are also named for what they are rather than for the ref alone, so release images stop sharing a key with anything else that stashes an image for the same ref.
Docs and the registry now want different Pythons, and a job that builds an image for another job has to hand it over as a multi-gigabyte tarball - exported, stashed, restored and imported - for an image whose only consumer is a single downstream job. Building in place skips that round trip entirely, and the two builds still run in parallel because both jobs now wait only on build-info. What is worth keeping between runs is the BuildKit mount cache rather than the image, so the docs job restores and stashes it under this ref and its own Python. The registry side already knew how to build its own image when the caller had not stashed one; it just could not be told which ref to build, so it would have documented main rather than the tag being published.
potiuk
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496 and
jscheffl
as code owners
August 3, 2026 02:25
boring-cyborg
Bot
added
area:dev-tools
area:registry
backport-to-v3-3-test
`breeze build-docs` gained a --python option, and because that option reads PYTHON_MAJOR_MINOR_VERSION, which every job of the docs publishing workflow sets, the docs quietly started building on whatever the workflow happened to export rather than on the Python they are meant for. That is how they ended up on 3.12, where Sphinx's mocking of third-party modules makes functools.wraps raise and providers fail to import. Which interpreter documents Airflow is not a per-invocation choice, so the option is gone and the build pins itself to the default again. The workflow reads the same constant instead of naming a version of its own, so the image it prepares cannot drift away from the one the docs build asks for.
potiuk
requested review from
choo121600,
ephraimbuddy,
jedcunningham and
vatsrahul1001
as code owners
August 3, 2026 02:34
The prefix exists to keep these images from sharing a key with anything else stashing an image for the same ref, so it should say which workflow put them there rather than describe them as release artifacts.
Each image now has a single consumer, so a job that only builds one can do no more than export it and have the consumer import it straight back. Both docs and the registry build in place instead, which leaves the registry workflow with one job as well. What that job hands to the next run is unchanged: the image, the commit it was built from and the mount cache are stashed under the ref, so a later run for the same ref seeds its build from them and skips the build entirely when the sources have not moved. That is the whole reason the stash exists - publishing an RC and then the final docs is the same ref twice - so it moves into the jobs with the build rather than being dropped along with the job that used to do it. The logic lives in a composite action so both jobs share one copy; ci-image-build.yml stays for callers that do want an image job of their own.
Removing the build-ci-image guard left the parenthesis that opened its group, so GitHub refused to parse the workflow and the dispatch failed before any job ran.
The registry still runs alongside the docs build - nothing it does depends on them - but the publish now waits for it, so a green run means both halves of a publish are actually out rather than only the S3 upload. Its result is checked explicitly because the registry is skipped for non-provider distributions, and a skipped dependency would otherwise skip the publish with it.
A local action is resolved against the workspace, not against the ref the workflow definition came from, so publishing a tag cut before the action existed found nothing to run and both image builds failed immediately. The workflow's own version is now checked out alongside the sources being published and the action is taken from there, which is what lets a workflow branch be tested against an older tag at all. The image itself is still built from the checked out sources.
gopidesupavan
approved these changes
Aug 3, 2026
Contributor
Backport failed to create: v3-3-test. View the failure log Run detailsNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
You can attempt to backport this manually by running: cherry_picker 115eb43 v3-3-testThis should apply the commit to the v3-3-test branch and leave the commit in conflict state marking After you have resolved the conflicts, you can continue the backport process by running: cherry_picker --continueIf you don't have cherry-picker installed, see the installation guide. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Publishing the provider docs has failed since 2026-07-28.
build-ci-imagebuilds one image and bothbuild-docsandupdate-registryconsume it, so they share its Python. It is built at 3.12 because that is what the registry has always extracted on — which silently moved the docs build onto 3.12 too.The last green run built with no
--pythonflag; every run since passes--python 3.12. Nothing in the repo caused it: only four commits sit between the last green and the first red run (INTHEWILD.md, a secrets-backend change, a UI toggle, Helm docs).Two things break there, and both go away on the default Python:
autodoc_mock_imports, and from 3.12functoolscopies__type_params__, for which a mock returns another mock rather than a tuple.SambaHookdecorates all 24 of its methods withfunctools.wraps(smbclient.…), so the module dies at class-definition time and the provider cannot be documented at all. Reproduces identically on Sphinx 8.2.3, 9.0.4 and 9.1.0, so it is not a Sphinx regression.amazon,googleandopenlineagethat were not reported before.The change
ci-image-build.ymlalready matrixes over thepython-versionsit is handed, and every cache it touches is keyed by Python version — the registry buildx tag (…/ci/python{version}:cache-linux-amd64), the image stash, the commit marker and the BuildKit mount cache. Passing it both versions therefore builds the two images in parallel, each seeding from and storing to its own cache, with no eviction between them and no second job.Docs return to the default Python (3.10); the registry stays on 3.12.
Stashes are also now keyed
release-build-<ref>rather than the bare ref, so these images — built from a release tag and retained for days so the next publish of the same tag reuses them — stop sharing a key with anything else stashing an image for that ref.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines