perf(box): speed cold docker builds and fix update --check estimates

CHANGELOG.md

@@ -54,6 +54,38 @@ All notable changes to the Lager platform are documented here. For detailed rele
   and false of the file that grants `apt-get` one line above them. Treat
   anyone holding the box login account's SSH key as holding root on that box.
 
+- **Cold box image builds spend less time installing things nothing uses.**
+  Three changes to `box.Dockerfile`, all of them about build time rather than
+  behavior:
+
+  Node and npm now come from the official upstream tarball, verified against
+  its `SHASUMS256.txt` before extraction, instead of Debian's `nodejs npm`
+  meta-packages — which pull in roughly 400 `node-*` packages the box never
+  touches. `start_box.sh` needs npm only to install the packages named in
+  `box_config.npm_packages`, which the tarball provides.
+
+  `cryptography` moves from 38.0.4 to 43.0.3. The old pin has no cp312 wheel,
+  so every cold pip layer compiled it from Rust source; 43.0.3 ships a
+  manylinux wheel for the image's Python. The BluFi cipher, its only consumer
+  in this tree, switches from `algorithms.AES128` to `algorithms.AES` — stable
+  across both versions and byte-identical for BluFi's fixed 16-byte key.
+
+  `flex` and `bison` move into the uldaq layer, the only stage whose
+  `autoreconf` needs them. `ccache` and `ninja-build` are dropped outright:
+  nothing in this repo invokes either, and no build here was wired to use
+  them.
+
+  **A box carrying globally-installed npm packages should be updated once with
+  `--force`.** Node's major version moves from 18 to 20, and the
+  `lager-npm-global` volume holding those packages survives an ordinary image
+  rebuild — only `--force` wipes it. Any package with a compiled native module
+  needs reinstalling under the new ABI.
+
+- **The update progress bar names what the container build is currently
+  doing** — `Building container... [pip install ...]` — instead of holding one
+  unchanging label for the several minutes a cold build takes. Parsed from
+  BuildKit's own step output; `--verbose` is unchanged.
+
 ### Fixed
 
 - **Every successful `lager update` that rebuilt the container warned that its
@@ -117,6 +149,30 @@ All notable changes to the Lager platform are documented here. For detailed rele
   gate would rebuild. `--check`'s exit code now accounts for a pending
   flatten too, so a box needing one no longer reports `Nothing to do`.
 
+- **`lager update --check` still promised a cached build when the ref it was
+  about to check out changed the image recipe.** The companion to the case
+  above, and the more common one. The probe measured the Dockerfile,
+  requirements and box source in the box's *current* working tree — which on a
+  box a long way behind its target still matched `/etc/lager/build-hash`
+  exactly. The preview printed `Estimated: ~90s (cached build)`; the pull then
+  landed a different Dockerfile and the update took the full six minutes.
+
+  `--check` now reads those same build inputs at the target ref, straight out
+  of the box's git object database via `git cat-file` / `git show` — no
+  checkout, no mutation, and one extra SSH round-trip on the `--check` path
+  only. The snippet is composed to emit a byte-identical digest to the
+  working-tree hasher for an identical tree, so the two are comparable by
+  construction rather than by coincidence; tests execute both under `sh`
+  against a fixture repo and assert the digests agree.
+
+  When the target ref can be measured it replaces the working tree as the
+  basis for the whole preview, which also turns the old
+  `unknown until pull (older ref may differ)` guess on rollbacks and branch
+  switches into a measured answer. When it cannot be measured — a sparse
+  checkout, an odd ref — the preview says so rather than falling back to the
+  pre-pull tree and calling the cache valid. A pending flatten still forces a
+  rebuild whatever the target digest says.
+
 - **The sudoers bootstrap snippet printed by `lager box-config mount` wrote a
   strict subset of the file it was overwriting.** It teed a single
   `NOPASSWD: /bin/mkdir, /bin/chown` line over `/etc/sudoers.d/lager-box-config`,

box/lager/blufi/security/aes.py

@@ -8,9 +8,9 @@
     # path with removal upstream calls imminent but has not scheduled.
     from cryptography.hazmat.decrepit.ciphers.modes import CFB
 except ImportError:
-    # Older installs predate the decrepit package entirely -- the box
-    # runtime pins cryptography==38.0.4 (box.Dockerfile) and the unit
-    # floor is >=42. Same class either way, byte-identical output.
+    # Older installs predate the decrepit package entirely. Box runtime
+    # pins cryptography==43.0.3 (box.Dockerfile); unit floor is >=42.
+    # Same class either way, byte-identical output.
     from cryptography.hazmat.primitives.ciphers.modes import CFB
 
 # https://cryptography.io/en/latest/hazmat/primitives/symmetric-encryption/#cryptography.hazmat.primitives.ciphers.algorithms.AES
@@ -19,7 +19,10 @@ class BlufiAES(object):
     def __init__(self, key, iv):
         self.key = key
         self.iv = iv
-        self.cipher = Cipher(algorithms.AES128(self.key), CFB(self.iv))
+        # AES128 was a fixed-size alias; AES(key) is the stable API across
+        # cryptography versions (and what manylinux wheels for 3.12 expect).
+        # BluFi always uses a 16-byte key.
+        self.cipher = Cipher(algorithms.AES(self.key), CFB(self.iv))
         self.encryptor = self.cipher.encryptor()
         self.decryptor = self.cipher.decryptor()
 

box/lager/docker/box.Dockerfile

@@ -5,16 +5,21 @@ ENV PYTHONDONTWRITEBYTECODE=1
 ENV PYTHONUNBUFFERED=1
 ENV PYTHONPATH=/app/lager
 
+# flex/bison are not here: they moved to the uldaq stage below, the only
+# place that needs them (autoreconf). ccache/ninja-build are dropped
+# outright -- nothing in this repo invokes either and no build here was
+# wired to use them, so they were pure image weight; a container script
+# that wants them has to install them itself. Node comes from the official
+# tarball below rather than Debian's `nodejs npm` meta-packages, which pull
+# ~400 unused `node-*` packages and dominate cold-build time.
 RUN apt-get update && apt-get install -y ca-certificates libusb-1.0-0-dev libudev-dev \
 	libhidapi-dev git gcc python-dev-is-python3 python3-pip python3-venv xz-utils build-essential bluetooth ssh openssh-client \
 	cups-client lpr zlib1g-dev wget libjpeg-dev libpng-dev libfreetype6-dev \
 	fswebcam automake g++ libtool libleptonica-dev make pkg-config libpango1.0-dev gdb-multiarch tesseract-ocr libtesseract-dev \
-	flex bison ccache ninja-build \
 	libturbojpeg0-dev v4l-utils \
 	wireless-tools \
 	tini \
 	gnupg \
-	nodejs npm \
 	openocd \
 	&& wget -qO /usr/share/keyrings/phidgets.gpg https://www.phidgets.com/gpgkey/pubring.gpg \
 	&& echo deb [signed-by=/usr/share/keyrings/phidgets.gpg] http://www.phidgets.com/debian bookworm main > /etc/apt/sources.list.d/phidgets.list \
@@ -27,14 +32,33 @@ RUN apt-get update && apt-get install -y ca-certificates libusb-1.0-0-dev libude
       virtualenv \
     && :
 
+# Node.js + npm from the official binary tarball (~50MB) instead of apt's
+# `nodejs` meta-package (~500MB of unused node-* tooling). Needed at runtime
+# so start_box.sh can `npm install -g` box_config.npm_packages. Multi-arch:
+# dpkg arch → node dist arch.
+# Pin + verify: Debian-signed apt packages are replaced by an upstream tarball,
+# so HTTPS alone is not enough — check SHASUMS256.txt before extracting.
+ENV NODE_VERSION=20.18.1
+RUN arch="$(dpkg --print-architecture)" \
+	&& case "$arch" in \
+		amd64) node_arch=x64 ;; \
+		arm64) node_arch=arm64 ;; \
+		armhf) node_arch=armv7l ;; \
+		*) echo "unsupported arch for Node.js: $arch" >&2; exit 1 ;; \
+	esac \
+	&& cd /tmp \
+	&& tarball="node-v${NODE_VERSION}-linux-${node_arch}.tar.xz" \
+	&& wget -q "https://nodejs.org/dist/v${NODE_VERSION}/${tarball}" \
+	&& wget -q "https://nodejs.org/dist/v${NODE_VERSION}/SHASUMS256.txt" \
+	&& grep " ${tarball}$" SHASUMS256.txt | sha256sum -c - \
+	&& tar -xJf "${tarball}" --strip-components=1 -C /usr/local \
+	&& rm -f "${tarball}" SHASUMS256.txt \
+	&& node --version && npm --version
+
 # PicoScope SDK is mounted from host at runtime via start_box.sh
 # The SDK library is at /opt/picoscope/lib/libps2000.so on the host
 # This avoids the systemd reload error that occurs when installing in Docker
 
-
-
-
-
 # MCC uldaq C library -- required by uldaq Python bindings for USB-202 DAQ devices
 # See: https://github.com/mccdaq/uldaq
 #
@@ -56,7 +80,10 @@ RUN apt-get update && apt-get install -y ca-certificates libusb-1.0-0-dev libude
 # a memory-safety bug rather than close it, so the warnings stay: noisy once
 # per image build, and honest. Fixing it means patching upstream source at
 # build time or dropping the library -- upstream has not released since 2022.
-RUN apt-get update && apt-get install -y autoconf automake libtool libusb-1.0-0-dev && rm -rf /var/lib/apt/lists/* \
+#
+# flex/bison stay in this layer only (autoreconf); not needed at runtime.
+RUN apt-get update && apt-get install -y autoconf automake libtool libusb-1.0-0-dev flex bison \
+	&& rm -rf /var/lib/apt/lists/* \
 	&& git clone --depth 1 --branch v1.2.1 https://github.com/mccdaq/uldaq.git /tmp/uldaq \
 	&& cd /tmp/uldaq \
 	&& autoreconf -i \
@@ -111,7 +138,9 @@ RUN --mount=type=cache,target=/root/.cache/pip,sharing=locked \
 	'simplejson==3.18.0' \
 	'labjack-ljm==1.23.0' \
 	'pygdbmi==0.11.0.0' \
-	'cryptography==38.0.4' \
+	# 38.0.4 has no cp312 wheel — every cold pip layer compiled Rust from
+	# source (~2-3 min). >=42 ships manylinux wheels for python 3.12.
+	'cryptography==43.0.3' \
 	'psycopg2-binary==2.9.9' \
 	'pyvisa-py==0.5.2' \
 	'PyVISA==1.11.3' \