Skip to content

Convert ros3 streaming to remfile in User Guide Part I and NWBWidget demo - #185

Merged
bendichter merged 2 commits into
masterfrom
ros3-to-remfile
Aug 20, 2026
Merged

Convert ros3 streaming to remfile in User Guide Part I and NWBWidget demo#185
bendichter merged 2 commits into
masterfrom
ros3-to-remfile

Conversation

@bendichter

Copy link
Copy Markdown
Member

These are two of the five groups that failed verification in the container-image fleet rollout (see the scorecard on #180). Both opened DANDI assets with h5py's ros3 driver, which PyPI h5py wheels do not ship, so they failed in the image verification and in headless CI, and would fail for any user in a pip-installed environment. They now stream through remfile, following the pattern in docs/adding-notebooks.md, with remfile==0.1.14 added to their install cells (numpy, h5py, and requests were already pinned).

Fixing ros3 in the NWBWidget demo unmasked a second, older problem: the "Running NWBWidgets locally" section printed a download URL, instructed the reader to click it, and then opened ~/Downloads/<file>. That can never run unattended, so the ros3 failure had been hiding it. The section now downloads a 44 MB asset from the same dandiset programmatically to a temporary path and opens that, which keeps the demonstration of local-path reading intact while making the notebook runnable end to end.

Verified by building both container image groups locally and running each notebook inside its image with the CI harness: DANDI User Guide, Part I finished in 17s and NWBWidget-demo in 17s, both green. After merge, dispatching the image workflow will publish these two groups, bringing the fleet to 30 of 33.

🤖 Generated with Claude Code

…demo

Both notebooks opened DANDI assets with h5py's ros3 driver, which PyPI
h5py wheels do not include, so they failed in CI and in the container
image verification (and would fail for any pip-installed user). They now
stream through remfile per docs/adding-notebooks.md, with remfile==0.1.14
added to their install cells (its dependencies were already pinned).

Fixing ros3 unmasked a second blocker in the NWBWidget demo: its local-
file section printed a download URL, asked the reader to click it, and
then opened ~/Downloads/<file>, which can never run headlessly. That
section now downloads a 44 MB asset from the same dandiset
programmatically to a temp path and opens that, preserving the point of
the demonstration.

Both notebooks verified green end to end inside freshly built container
images.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions

github-actions Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Preview for this PR has been removed (PR closed).

github-actions Bot added a commit that referenced this pull request Aug 20, 2026
The remfile wrapping now sits on its own lines with a disk cache, per
the streaming guidance in docs/adding-notebooks.md. Both notebooks gain
a committed requirements file per the new submission contract (a shared
requirements.in for demos/, and a per-notebook file for Part I since
Part II shares the dandi/ directory), with install cells re-locked from
those files via lock_notebook.py.

Re-locking surfaced a real conflict: nwbwidgets 0.9.1 only imports on
the pre-2024 stack (pynwb<3, hdmf<4, zarr<3; newer zarr removed
zarr.core.Array and newer hdmf cannot resolve its bundled ndx-spectrum
spec), while a DANDI-server-compatible client needs dandi>=0.76.5,
which requires pynwb>=3.1 under Colab's current click pin. The demos
notebook, whose subject is NWB Widgets and which does not use the dandi
client, keeps the bounded old stack. The User Guide, whose subject is
the dandi client, drops nwbwidgets and shows files via pynwb's rich
Jupyter repr instead, on the current stack (dandi 0.77, pynwb 4.1).

Both notebooks verified green end to end inside freshly built container
images.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bendichter

Copy link
Copy Markdown
Member Author

Updated per review: the remfile wrapping is split onto its own lines with remfile.DiskCache("nwb-cache") in both notebooks, and both directories now carry committed requirements files per the new contract (demos/requirements.in shared; a per-notebook DANDI User Guide, Part I.requirements.in since Part II shares the dandi/ directory with a different dependency set).

Re-locking from those files surfaced a genuine conflict worth knowing about: nwbwidgets 0.9.1 only imports on the pre-2024 stack (newer zarr removed zarr.core.Array, and newer hdmf cannot resolve its bundled ndx-spectrum spec), while a DANDI-server-compatible client needs dandi>=0.76.5, which under Colab's current click pin requires pynwb>=3.1. The two cannot coexist. Resolution: the NWBWidget demo, whose subject is the widgets and which does not use the dandi client, keeps the bounded old stack (pynwb<3, hdmf<4, zarr<3); the User Guide, whose subject is the dandi client, drops nwbwidgets and shows files via pynwb's rich Jupyter repr on the current stack (dandi 0.77.0, pynwb 4.1.0).

Both notebooks verified green end to end inside freshly built container images.

github-actions Bot added a commit that referenced this pull request Aug 20, 2026
@bendichter
bendichter merged commit 7f1a83b into master Aug 20, 2026
6 checks passed
github-actions Bot added a commit that referenced this pull request Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant