Publish verified notebook environment images to GHCR; requirements.in submission contract

[bendichter]

This implements the "two-pronged" strategy we discussed at the DANDI meeting: keep the one-click Colab path exactly as it is, and add a stamped path where each notebook environment is published as a persistent, self-contained container image. The Colab flow works today, but it depends on Colab still shipping Python 3.12 and on PyPI still resolving the pins. A published image freezes the whole working environment, so the notebook stays runnable after both of those have moved on. As @yarikoptic pointed out, CI already effectively builds this environment on every run; this PR makes it a downloadable artifact.

What This Adds

A container-publishing workflow (build-notebook-images.yml, manual dispatch only for now). Notebooks are grouped by directory and pin set (33 groups across the 51 Colab-ready notebooks), and each group gets one image at ghcr.io/dandi/example-notebooks/<group>, for example:

docker run --rm -p 8888:8888 ghcr.io/dandi/example-notebooks/001550-paganlab:latest

That opens JupyterLab on the notebook with all pinned dependencies preinstalled. The key property is that an image is only pushed after every notebook in the group has been executed successfully inside the candidate image, using the unmodified run_notebook.py harness. latest is therefore always a verified snapshot, and each push also gets sha-<commit>, <YYYY-MM-DD>, and hash-<inputs> tags. The hash tag doubles as the rebuild-skip check: if an image built from identical inputs already exists, the job skips it, which keeps a future full-fleet cron cheap.

Inside the image the kernel environment is the system Python with the pins installed via uv pip install --system, matching the Colab bootstrap cell and the CI harness exactly. JupyterLab runs from an isolated uv tool environment so its own dependency tree can never perturb the pinned set. The base is a digest-pinned python:3.12-slim; the BASE_IMAGE arg is the knob for a possible future variant on Google's official Colab runtime image (deferred, since that image is around 24 GB against roughly 14 GB of free runner disk). Images are built for linux/amd64, the platform the pins were resolved for. Details are in .github/docker/README.md.

A simpler submission contract (lock_notebook.py). Contributors now commit a requirements.in next to their notebook listing only its direct dependencies, and run:

python .github/scripts/lock_notebook.py <notebook>.ipynb

The script compiles the full pinned set with uv pip compile, constrained to Colab's preinstalled versions, and writes the four bootstrap cells into the notebook, prepending them when absent or refreshing the pin block in place (helper !curl lines are preserved, and nbformat_minor is bumped when needed). Committing the .in file also gives re-locking a source of truth: today the pins exist only inside the notebook JSON, so re-resolving after a Colab runtime bump means reverse-engineering the direct dependencies from a 160-line pin list. The README and docs/adding-notebooks.md are updated accordingly; the old conda env export instructions are retired for new submissions.

Verification

I ran the pipeline end to end locally on the 001550/PaganLab pilot group: the image builds at 380 MB, both notebooks execute green inside it (stage: done, ok: true), JupyterLab serves with the pinned kernel correctly resolved, and a second run of the pin-set grouping and build hash reproduces identical values. The lock script was tested on scratch copies for both the refresh and prepend paths, including the cell-source newline and nbformat_minor gotchas from the original #149 rollout. Nothing in the existing test, index, or preview workflows is touched, apart from the missing-install-cell error message in run_notebook.py, which now points contributors at the new flow.

Remaining Steps Before This Is Useful

1. Confirm the dandi org allows GITHUB_TOKEN package creation for this repo (org settings, Packages).
2. Merge, then dispatch the Build notebook images workflow with the default 001550/PaganLab filter.
3. Flip the resulting 001550-paganlab package to public and pull-and-run it on a fresh machine.
4. Roll out to the remaining groups on this PR's thread, starting with a helper-using group (001636/TurnerLab/motor_cortex) to exercise helper baking.

Index-page integration (a copyable docker run command next to the Colab badge), push-to-master and monthly cron triggers, and converging the weekly test sweep onto these images are follow-ups once the pilot is validated end to end. Backfilling requirements.in for the 51 existing notebooks is another open follow-up; it can happen gradually as notebooks need re-locking.

🤖 Generated with Claude Code

[github-actions]

Preview for this PR has been removed (PR closed).

[bendichter]

Fleet Rollout Scorecard

The full fleet has now been built: 28 of 33 image groups are published as multi-arch (amd64 + arm64) manifests, with every notebook in every published group executed successfully inside its image on both architectures, on native runners with no emulation. Runs: 32320269117 (initial fleet) and 32323493938 (re-run after #183; completed builds skipped via hash tags as designed, so the re-run only performed the missing merges and one push retry).

Two fixes landed during the rollout. #182 switched to multi-arch images after validation on Apple Silicon showed that Rosetta emulation can deadlock kernel subprocess spawns; it also added a non-root user and prebaked the matplotlib font cache. #183 fixed the merge job so one group's failure does not block every other group's manifest.

Groups that did not publish

These five groups failed in-image verification, so nothing was pushed for them (by design; only verified images publish). The causes are notebook or upstream issues, not image issues, and the first three presumably affect Colab and the weekly sweep as well.

Group	Cause
000409-ibl	brainglobe's GIN atlas server refused connections from the runners (twice); related work is open in #177
001712-...-anatomical-localization-widefield	DANDI asset pattern matches zero files (sub-CSK-im-011/..._desc-*.nwb); content or dandiset-side change
001712-...-processed-widefield	same asset-pattern failure
dandi-dandi-user-guide-part-i	uses the h5py ros3 driver, which PyPI h5py wheels do not include; should stream via remfile per docs/adding-notebooks.md
demos (NWBWidget-demo)	same ros3 issue

Action needed (org admin)

GHCR has no API for visibility, so each new package needs a one-time manual flip to public at https://github.com/orgs/dandi/packages (select the package, then Package settings, then Change visibility). 001550-paganlab is already public; the other 27 published packages are currently private.

Once flipped, any of them runs with, for example:

docker run --rm -p 127.0.0.1:8888:8888 ghcr.io/dandi/example-notebooks/tutorials-cosyne-2023:latest

Follow-ups from here: fix or exclude the five failing groups, add the push-to-master and monthly cron triggers, and then index-page integration.

🤖 Generated with Claude Code