Convert ros3 streaming to remfile in User Guide Part I and NWBWidget demo - #185
Conversation
…demo Both notebooks opened DANDI assets with h5py's ros3 driver, which PyPI h5py wheels do not include, so they failed in CI and in the container image verification (and would fail for any pip-installed user). They now stream through remfile per docs/adding-notebooks.md, with remfile==0.1.14 added to their install cells (its dependencies were already pinned). Fixing ros3 unmasked a second blocker in the NWBWidget demo: its local- file section printed a download URL, asked the reader to click it, and then opened ~/Downloads/<file>, which can never run headlessly. That section now downloads a 44 MB asset from the same dandiset programmatically to a temp path and opens that, preserving the point of the demonstration. Both notebooks verified green end to end inside freshly built container images. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
Preview for this PR has been removed (PR closed). |
The remfile wrapping now sits on its own lines with a disk cache, per the streaming guidance in docs/adding-notebooks.md. Both notebooks gain a committed requirements file per the new submission contract (a shared requirements.in for demos/, and a per-notebook file for Part I since Part II shares the dandi/ directory), with install cells re-locked from those files via lock_notebook.py. Re-locking surfaced a real conflict: nwbwidgets 0.9.1 only imports on the pre-2024 stack (pynwb<3, hdmf<4, zarr<3; newer zarr removed zarr.core.Array and newer hdmf cannot resolve its bundled ndx-spectrum spec), while a DANDI-server-compatible client needs dandi>=0.76.5, which requires pynwb>=3.1 under Colab's current click pin. The demos notebook, whose subject is NWB Widgets and which does not use the dandi client, keeps the bounded old stack. The User Guide, whose subject is the dandi client, drops nwbwidgets and shows files via pynwb's rich Jupyter repr instead, on the current stack (dandi 0.77, pynwb 4.1). Both notebooks verified green end to end inside freshly built container images. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Updated per review: the remfile wrapping is split onto its own lines with Re-locking from those files surfaced a genuine conflict worth knowing about: nwbwidgets 0.9.1 only imports on the pre-2024 stack (newer zarr removed Both notebooks verified green end to end inside freshly built container images. |
These are two of the five groups that failed verification in the container-image fleet rollout (see the scorecard on #180). Both opened DANDI assets with h5py's
ros3driver, which PyPI h5py wheels do not ship, so they failed in the image verification and in headless CI, and would fail for any user in a pip-installed environment. They now stream through remfile, following the pattern in docs/adding-notebooks.md, withremfile==0.1.14added to their install cells (numpy, h5py, and requests were already pinned).Fixing ros3 in the NWBWidget demo unmasked a second, older problem: the "Running NWBWidgets locally" section printed a download URL, instructed the reader to click it, and then opened
~/Downloads/<file>. That can never run unattended, so the ros3 failure had been hiding it. The section now downloads a 44 MB asset from the same dandiset programmatically to a temporary path and opens that, which keeps the demonstration of local-path reading intact while making the notebook runnable end to end.Verified by building both container image groups locally and running each notebook inside its image with the CI harness:
DANDI User Guide, Part Ifinished in 17s andNWBWidget-demoin 17s, both green. After merge, dispatching the image workflow will publish these two groups, bringing the fleet to 30 of 33.🤖 Generated with Claude Code