Cache misses using a remote build cache when sharing docker targets

[tg-freigmbh]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

We are affected by an interesting remote cache problem. We get cache hits for each individual target, but not when multiple targets are build at the same time.
In the linked repo:

1. set variable registry to your cache repo
2. docker buildx bake
3. docker builder prune -f
4. docker buildx bake --progress=plain alpha beta

This will usually result in something like:

#14 [beta build 3/3] RUN echo "This should been optimized away beta"
#14 0.266 This should been optimized away beta
#14 DONE 0.3s

#11 [alpha build 3/3] RUN echo "This should been optimized away alpha"
#11 CACHED

So a cache hit for one build but not for the other.
However executing
2. docker buildx bake
3. docker builder prune -f
4. docker buildx bake --progress=plain alpha
5. docker builder prune -f
6. docker buildx bake --progress=plain beta
Will result in a full cache hit and never print the statement

Expected behaviour

docker buildx bake alpha beta
should have the same cache hits as

docker buildx bake alpha
docker buildx bake beta

Actual behaviour

1 Target gets a cache hit, the other targets dont.

Buildx version

github.com/docker/buildx v0.33.0-desktop.1 7f91f038ac14cbf5c4b2a6b76470860814424da1

Docker info

Client:
 Version:    29.4.3
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  agent: Docker AI Agent Runner (Docker Inc.)
    Version:  v1.57.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-agent.exe
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v1.20.2
    Path:     C:\Program Files\Docker\cli-plugins\docker-ai.exe
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.33.0-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v5.1.3
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.47
    Path:     C:\Program Files\Docker\cli-plugins\docker-debug.exe
  desktop: Docker Desktop commands (Docker Inc.)
    Version:  v0.3.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-desktop.exe
  dhi: CLI for managing Docker Hardened Images (Docker Inc.)
    Version:  v0.0.3
    Path:     C:\Program Files\Docker\cli-plugins\docker-dhi.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.31
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  mcp: Docker MCP Plugin (Docker Inc.)
    Version:  v0.42.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-mcp.exe
  model: Docker Model Runner (Docker Inc.)
    Version:  v1.1.37
    Path:     C:\Program Files\Docker\cli-plugins\docker-model.exe
  offload: Docker Offload (Docker Inc.)
    Version:  v0.5.89
    Path:     C:\Program Files\Docker\cli-plugins\docker-offload.exe
  pass: Docker Pass Secrets Manager Plugin (beta) (Docker Inc.)
    Version:  v0.0.27
    Path:     C:\Program Files\Docker\cli-plugins\docker-pass.exe
  sandbox: Docker Sandbox (Docker Inc.)
    Version:  v0.12.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sandbox.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scout: Docker Scout (Docker Inc.)
    Version:  v1.20.4
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 34
  Running: 34
  Paused: 0
  Stopped: 0
 Images: 55
 Server Version: 29.4.3
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Discovered Devices:
  cdi: docker.com/gpu=webgpu
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 77c84241c7cbdd9b4eca2591793e3d4f4317c590
 runc version: v1.3.5-0-g488fc13e
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.114.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 31.28GiB
 Name: docker-desktop
 ID: 28518f49-cc95-4c07-bc7c-7a5805d4d3e3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=npipe://\\.\pipe\docker_cli
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Registry Mirrors:
  https://docker.freigmbh.de/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 192.168.128.0/17, Size: 26
 Firewall Backend: iptables

Builders list

NAME/NODE           DRIVER/ENDPOINT     STATUS    BUILDKIT   PLATFORMS
default             docker                                   
 \_ default          \_ default         running   v0.29.0    linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (2 more)
desktop-linux*      docker                                   
 \_ desktop-linux    \_ desktop-linux   running   v0.29.0    linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (2 more)

Configuration

I made a reproducer here:
https://github.com/tg-freigmbh/repro-bake/tree/main

Build logs

Additional info

This seems to be very much tight to the docker buildtarget "build".
Something along the lines that due to the ARG SVC it could result in 2 different images, but in fact it doesnt.

For our productive problem we were able to solve this by inlining the target "build" into "runtime"-> Since docker lazy resolves ARG and shares everything anyway this seems currently as a good workaround without downsides.