Run the harness headlessly for plotly and matplotlib; un-exclude 001170 - #209
Merged
Conversation
The ReimerLab notebook was excluded from CI with the note that it calls webbrowser.open. It does not: it builds a plotly express figure and calls fig.show(), and outside a Jupyter kernel plotly resolves its default renderer to "browser", which is what reaches webbrowser.open and raises on a headless runner. Matplotlib has the same shape of problem in the other direction: on a machine with a display it picks a GUI backend, so running the harness locally opens plot windows that CI never sees. Both are properties of the environment rather than of any notebook, so the harness now sets PLOTLY_RENDERER and MPLBACKEND for the ipython path it controls. Colab and JupyterLab inside the notebook image run a real kernel, which selects inline rendering for both by itself, and the kernel path used for PR previews is deliberately left alone so its figures still appear in the executed copy. With that, the notebook runs headlessly, so its exclusion is removed and it gains a container image. It also gets the requirements.in the submission contract expects, which re-locks it onto the 3.13 pin set, and streams through a remfile.DiskCache like the other notebooks that read one file repeatedly. Verified locally on Python 3.13: it now runs in 43 s, where before the cache it took 826 s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011NuStoQykizhQCx2KBrMCN
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Contributor
|
Preview for this PR has been removed (PR closed). |
Contributor
Executed notebooksCI executes the notebooks changed in this PR through a Jupyter kernel and publishes each one, with its outputs, as soon as it finishes. Links appear below as notebooks complete; an hourglass means the notebook is still running. 1 of 1 finished. Commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
001170/ReimerLab/public_demo/001170_demo.ipynbhas been excluded from CI with this note:It never calls
webbrowser.open. What it does is build a plotly express figure and callfig.show(). Outside a Jupyter kernel, plotly resolves its default renderer tobrowser, and that renderer is what callswebbrowser.open:Matplotlib has the same shape of problem pointing the other way. On a machine with a display it selects a GUI backend, so running the harness locally opens plot windows; on a Linux runner it falls back to
Aggby itself, which is why CI never showed it.Neither is a property of a notebook, so both are fixed in the harness rather than in notebook content.
run_notebook.pynow setsPLOTLY_RENDERER=jsonandMPLBACKEND=Aggfor the ipython subprocess it launches, with both respecting an existing value from the environment. The kernel path added in #205 is deliberately untouched, since Colab and JupyterLab inside the notebook image select inline rendering on their own and the executed PR previews need their figures. Any future notebook that uses plotly gets this for free.What that unblocks
With the harness fixed, the notebook runs headlessly, so its exclusion is removed. It is the only notebook in the repo that carried a Colab badge but no image, and it now gets both. It also picks up two things it was missing:
requirements.in, which the submission contract expects and which the directory did not have (it still carries anenvironment.ymlfrom before the contract). Re-locking against it moves the notebook onto the 3.13 pin set; its install cell header still said Python 3.12, since the Re-lock every notebook for Colab's Python 3.13 runtime #206 sweep only covered notebooks that were not excluded.remfile.DiskCache, matching the other notebooks that read one file repeatedly.Verification
Run through the CI harness on Python 3.13, in the same fresh-environment mode CI uses: green, and 43 s against 826 s before the disk cache, since the notebook reads overlapping regions of the same file for each of its plots.
🤖 Generated with Claude Code
https://claude.ai/code/session_011NuStoQykizhQCx2KBrMCN