Build the cryptography wheel once for all downstream jobs#14976
Closed
alex wants to merge 3 commits into
Closed
Conversation
Every linux-downstream job was independently compiling the same cryptography wheel (~25-60s of Rust compilation each, 11 times per CI run). Instead, build the wheel once in a dedicated job and have the downstream jobs install it from an artifact. There is deliberately no needs: edge between the jobs: the downstream jobs' own setup (checkouts, caches, installing the downstream project's dependencies) overlaps with the wheel build, and they only wait at the point where they previously would have built the wheel themselves.
alex
force-pushed
the
claude/optimistic-rubin-enmq4
branch
from
9668eed to
1ff3f56
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every
linux-downstreamjob was independently compiling the same cryptography wheel (~25–60s of Rust compilation each, 11 times per CI run — the verbose logs from #14974 showed theuv pip install .time is ~entirely cargo). Instead, build the wheel once in a dedicatedlinux-downstream-wheeljob and have the downstream jobs install it from an artifact.There is deliberately no
needs:edge between the jobs: the downstream jobs' own setup (checkouts, caches, installing the downstream project's dependencies) overlaps with the wheel build, and they only wait — by polling the run's artifact list — at the point where they previously would have built the wheel themselves. Typical expected wait is a few seconds; the poll has a 5-minute cap so a failed wheel build doesn't hang them for long, andall-greennow also needslinux-downstream-wheelso such a failure is surfaced directly.The upload uses
overwrite: trueso "re-run all jobs" doesn't conflict with the artifact from the previous attempt (same commit, same wheel). Note the new job gets its own rust-cache key, so the first run on this PR pays one cold build; subsequent runs warm up.https://claude.ai/code/session_01YUH4HCt55jz7EM16P8rc2K
Generated by Claude Code