Skip to content

Release v3.0.1: fix the multi-architecture Docker build - #8

Merged
ecrum19 merged 1 commit into
mainfrom
dev
Sep 12, 2026
Merged

ecrum19 merged 1 commit into
mainfrom
dev

Conversation

@ecrum19

@ecrum19 ecrum19 commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Why

v3.0.0 is on PyPI but has no container image: the linux/arm64 half of the
multi-architecture build failed, so publish-docker never produced one. This
releases 3.0.1 with that build fixed.

PyPI reserves a version permanently once uploaded, so 3.0.0 could not be
rebuilt and re-released under the same number. 3.0.0 on PyPI is itself correct
and is left untouched.

What was wrong

Two defects, one hidden behind the other.

1. Hardcoded architecture path. The runtime stage copied QLever's shared
libraries out of /lib/x86_64-linux-gnu by soname. Upstream publishes an arm64
image whose multiarch directory is /lib/aarch64-linux-gnu, so the COPY could
not resolve its sources and the build failed before doing any work.

2. Non-root staging. The first fix introduced a staging step that upstream's
image could not run: it executes as a non-root user, and /opt is not writable
by it. This failed identically on both platforms.

What changed

  • Libraries are resolved with ldd inside the QLever stage and staged to a
    fixed path, so the runtime stage copies from architecture-independent
    locations. This is also soname-independent: an upstream rebuild that bumps
    Boost or ICU no longer requires an edit here.
  • The QLever stage runs as root and stages into /tmp, which is writable in
    every base image.
  • QLever acquisition is best-effort throughout, which is what the surrounding
    comments always claimed: an image without it is still fully functional, and the
    build-time marker records why it is unavailable. Previously a missing COPY
    source failed the build before that marker could be written, and the marker
    could not detect a missing binary at all — ldd on a nonexistent file does not
    print not found, so the status would have wrongly read ok.
  • The QLever image is pinned by digest rather than :latest, which is a
    rolling build of upstream's main branch rebuilt within a day of most commits.
    The pin is the multi-architecture index digest, so it resolves on both
    platforms — a per-architecture manifest digest would break the other one. The
    pinned build is bfd5741, the one this repository's QLever assertions were
    actually measured against (the qlever-index flags and the
    xsd:integer→xsd:int canonicalisation that preflight_position_datatype
    accommodates).
  • publish-python skips already-published files. Re-pushing a release tag to
    correct anything outside the Python artifact used to fail that job on a
    duplicate upload. Tag re-pushes are now idempotent.

Verification

  • docker buildx build --platform linux/amd64,linux/arm64 --target qlever —
    passes on both architectures, staging both binaries and the same nine
    libraries the original hardcoded list named.
  • Full multi-architecture image build.
  • python3 -m unittest discover -s test -p "test_*_unit.py" — 641 tests, OK.
  • python3 scripts/release.py --check-tag v3.0.1 — metadata agrees.

Audited for the same class of failure elsewhere: no other hardcoded architecture
paths remain, both base images (eclipse-temurin:11-jre, rust:1.93-slim)
publish amd64 and arm64, and all six pinned pip packages ship aarch64 wheels or
are pure Python — so nothing in the image needs to compile from source on arm64.

After merge

The existing v3.0.1 tag points at a commit that predates these fixes and has no
version bump behind it. It must be deleted and recreated on the merge commit:

git push origin :refs/tags/v3.0.1 && git tag -d v3.0.1
git tag -a v3.0.1 -m "VCF-RDFizer v3.0.1" && git push origin v3.0.1

That tag push publishes the image as 3.0.1, v3.0.1 and latest, and uploads
3.0.1 to PyPI. Afterwards, record the conda checksum:

python3 scripts/release.py 3.0.1 --fetch-conda-sha256

🤖 Generated with Claude Code

The v3.0.0 image never published: the linux/arm64 build failed outright, so
3.0.0 exists on PyPI with no matching container image.

Two defects, one behind the other:

- The runtime stage copied QLever's shared libraries from
  /lib/x86_64-linux-gnu by soname. Upstream publishes an arm64 image whose
  multiarch directory is /lib/aarch64-linux-gnu, so the COPY could not resolve
  its sources. The libraries are now resolved with ldd inside the QLever stage
  and staged into a fixed path, which is both architecture- and
  soname-independent: an upstream rebuild that bumps Boost or ICU no longer
  needs an edit here. Verified to stage the same nine libraries on both
  architectures.

- That staging step then failed on both platforms, because upstream's image
  runs as a non-root user and /opt is not writable by it. The stage switches to
  root and stages into /tmp.

QLever acquisition is now best-effort throughout, which is what the surrounding
comments always claimed: an image without it is still fully functional and the
build-time marker says why. Previously a missing COPY source failed the build
before that marker could be written, and the marker could not detect a missing
binary at all, since ldd on a nonexistent file does not print "not found".

Also pins the QLever image by its multi-architecture index digest rather than
:latest, which is a rolling build of upstream's main branch. The pinned digest
is build bfd5741 -- the one this repository's QLever assertions were actually
measured against.

Finally, publish-python skips already-published PyPI files. PyPI reserves a
version permanently once uploaded, so re-pushing a release tag to correct
anything outside the Python artifact used to fail that job on the duplicate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@ecrum19
ecrum19 merged commit 9b6d85e into main Sep 12, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants