From ee4061591afe9985293a04737b2d1dbd52a15eaf Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Sun, 5 Jul 2026 14:11:53 +0000 Subject: [PATCH] perf(build): cache Go build + module dirs in Dockerfile builder MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The builder stage had no BuildKit cache mount, so every `docker build` started with an empty Go build cache and recompiled the entire module — client-go, controller-runtime, all deps — from scratch (~100s), even for a one-line source change. This is the dominant cost of the e2e image-refresh leg, which deliberately changes a source file and rebuilds the controller ~5 times per run to prove that a change triggers a rebuild + rollout. On the sharded CI matrix that leg is the critical path at ~15 min (run 28681013051), almost all of it cold recompiles. Mount the Go build cache (/root/.cache/go-build) and module cache (/go/pkg/mod) as BuildKit cache mounts so they persist across builds. A source change still recompiles (only the changed packages + dependents), so the image-refresh assertions still hold, but a rebuild no longer recompiles the world. Verified locally: after appending a comment to cmd/main.go, the go build step dropped from a cold full compile to ~2s incremental. Also speeds up local `task test-e2e` rebuilds and the CI build job on warm cache. The cache is content-addressed and keyed by GOOS/GOARCH/flags, so cross-arch and coverage (GOCOVER) builds stay isolated. Requires BuildKit, the default on modern Docker. See docs/design/e2e-ci-runner-sharding-plan.md for the sharding analysis that surfaced this. Co-Authored-By: Claude Opus 4.8 (1M context) --- Dockerfile | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/Dockerfile b/Dockerfile index 05d64757..8d2489f4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -22,8 +22,9 @@ WORKDIR /workspaces # Copy the Go Modules manifests COPY go.mod go.sum ./ # cache deps before building and copying source so that we don't need to re-download as much -# and so that source changes don't invalidate our downloaded layer -RUN go mod download +# and so that source changes don't invalidate our downloaded layer. The module +# cache is a BuildKit cache mount so it also survives across builds. +RUN --mount=type=cache,target=/go/pkg/mod go mod download # Copy the go source COPY cmd/ cmd/ @@ -32,7 +33,18 @@ COPY internal/ internal/ # Build for the target platform. ${GOCOVER:+...} expands to the coverage flags # only when GOCOVER is non-empty, instrumenting every package in the module. -RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build \ +# +# The Go build and module caches are BuildKit cache mounts, so they persist +# across builds instead of starting empty every time. A source change still +# recompiles (only the changed packages + dependents), but a rebuild no longer +# recompiles the whole module + all deps from scratch — the dominant cost of the +# e2e image-refresh chain, which rebuilds the controller several times per run +# (see docs/design/e2e-ci-runner-sharding-plan.md). Also speeds up local +# `task test-e2e` rebuilds. The cache is content-addressed and keyed by +# GOOS/GOARCH/flags, so cross-arch and coverage builds stay isolated. +RUN --mount=type=cache,target=/root/.cache/go-build \ + --mount=type=cache,target=/go/pkg/mod \ + CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build \ ${GOCOVER:+-cover -covermode=atomic -coverpkg=github.com/ConfigButler/gitops-reverser/...} \ -ldflags "-X main.version=${VERSION} -X main.gitCommit=${GIT_COMMIT} -X main.gitDirty=${GIT_DIRTY} -X main.buildDate=${BUILD_DATE}" \ -o manager ./cmd