ci: use native arm64 runners and GHA layer cache#35
Open
marc0olo wants to merge 6 commits into
Open
Conversation
Splits each image build into parallel amd64 (ubuntu-latest) and arm64 (ubuntu-24.04-arm) jobs, then merges them into a multi-arch manifest via `docker buildx imagetools create`. This eliminates QEMU emulation for arm64, which was responsible for the ~50 min build time on rust/all. Adds GHA layer caching (type=gha, mode=max, scoped per image+platform) so that unchanged layers (e.g. candid-extractor when only icp-cli bumps) are served from cache on subsequent releases. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR refactors the container-image publishing workflow to build linux/amd64 and linux/arm64 images on native runners (eliminating QEMU emulation) and introduces GitHub Actions BuildKit layer caching to significantly reduce CI build times—especially for Rust-heavy images.
Changes:
- Split each image build into parallel per-architecture jobs (amd64 on
ubuntu-latest, arm64 onubuntu-24.04-arm) and add a merge job to publish a combined multi-arch manifest. - Add
cache-from/cache-toBuildKit GHA cache configuration, scoped per image + platform. - Remove QEMU setup from the workflow since cross-arch emulation is no longer needed.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Addresses Copilot review on #35: - Switch platform builds to push-by-digest (outputs: push-by-digest=true) so no SHA-suffixed intermediate tags accumulate in GHCR; digests are passed between jobs via outputs and referenced directly in imagetools create - Add type=ref,event=tag to all merge jobs so workflow_dispatch on a tag ref always produces at least one --tag destination Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace echo with printf + grep -v '^$' in the TAGS construction of all three merge jobs. echo always emits a trailing newline so an empty tags output would produce '--tag ' (with no value), causing imagetools create to fail. The grep filter drops blank lines so TAGS is truly empty when no tags are generated. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Required for cache-from/cache-to: type=gha — without it the GHA cache backend denies both reads and writes when permissions are explicitly set. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This reverts commit 943e827.
Member
Author
|
Reverting the The The suggestion cited a real failure mode (cache access being denied in restricted-permission contexts) but the wrong fix. The cache will work correctly with just |
grep -v '^$' exits 1 when all input lines are empty, which would fail the merge step under bash -e before imagetools create runs. awk NF filters blank lines identically but always exits 0. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
All three images were built using QEMU to emulate
linux/arm64on an amd64 runner. Rust compilation under QEMU is ~10–20× slower than native, which caused therustandallimage builds to take ~50 minutes.Changes
Native arm64 runners — each image build is split into two parallel jobs:
build-{name}-amd64onubuntu-latest(native amd64)build-{name}-arm64onubuntu-24.04-arm(native arm64, free for public repos)A third
merge-{name}job waits for both, then assembles the final multi-arch manifest viadocker buildx imagetools create. The published result is identical to before: a single multi-arch image tag covering both platforms.Push by digest — platform builds use
push-by-digest=trueand pass their digest as a job output. The merge job references images by digest ({image}@sha256:...) rather than SHA-suffixed tags, so no intermediate tags accumulate in GHCR.GHA layer cache —
cache-from/cache-to: type=gha,mode=maxis added to every platform build, scoped per image and platform (e.g.scope=rust-arm64). When onlyICP_CLI_VERSIONchanges, the slowcargo binstall candid-extractorlayer is served from cache and skipped entirely. Noactions: writepermission is needed — thetype=ghacache backend usesACTIONS_RUNTIME_TOKEN, notGITHUB_TOKEN.Tag ref support — added
type=ref,event=tagto all merge jobs soworkflow_dispatchtriggered on a tag ref always produces at least one--tagdestination forimagetools create.Safe TAGS construction — the merge steps use
printf+grep -v '^$'to filter blank lines before building the--tagargument list, preventing a bare--tagwith no value if the tags output were ever empty.QEMU setup is removed from all jobs — it's no longer needed.
Expected impact
Test plan
workflow_dispatchon this branch and verify all 9 jobs succeedlinux/amd64andlinux/arm64in the manifest for each imagerustandallarm64 jobs🤖 Generated with Claude Code