Convert to MySTMD - #315
lundybernard wants to merge 15 commits into
Conversation
✅ Deploy Preview for learn-scientific-python-org ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
I've put together a plan for the migration, including REQUIREMENTS.md, and Architectural Decision Records (ADRs), which need review, and a detailed PLAN.md document. Recommended Review order:
Ai Tools Disclaimer: |
0f29f3f to
4f6f3c4
Compare
|
From what I see, the plan looks fine. While I was excited about the Jekyll -> Hugo migration (fast, secure single binary, static easily themeable HTML), I'm not quite as excited about mystmd transition (Frankenstein of nodejs+Python, JS-powered pages). I get several source map loading errors when opening https://mystmd.org/guide. But I do really like it for course material, and some of our material basically is that. I'd probably not worry about trying to replicate the theme at all at first (which I think matches the decisions made, like the deferred footer). I'm curious if you've done the basic conversation table for the other subrepos? I'm curious to see what it says about the special stuff in cookie, like our tabs. I know exactly how to do that in Hugo (I maintain a Hugo theme ;) ), I'm curious to know how those would map to mystmd. I'm very slightly tempted to do Jekyll -> Hugo with cookie, and then look at Hugo -> mystmd. But yes, plan looks good. What do you feed into the next step, just the PLAN.md? |
|
I've only done a cursory look at converting the other sub-repos, a quick "is it possible, any major blockers?" check. Now's probably a good time to build a report and add it to the parent ticket. I take a very hand-on approach with Ai tools, so I wont feed a big "one-shot" prompt into an agent and lets it run for hours. I've been using claude code, with broad read-only permissions, and it pulls in a lot of context about the repo when I start it up. So, the input is a heuristic search of the learn repository, and access to external docs as-needed. |
a85c0ad to
03f979b
Compare
|
For generic landing pages, we've started working on https://github.com/scientific-python/scientific-python-myst-theme/ (early stages of a cookie cutter). Agreed that porting content first is a good idea: we can tweak the theme later. myst is not supposed to be a "Frankenstein of nodejs+Python, JS-powered pages" — it is a typescript application (runtime Node or Bun). But, I get the concern about the complexity, and of course the source map errors should be addressed. Hugo's big strength is in how easily extendible it is, without needing to tinker with the code, whereas myst themes are all full-fledged web servers & logic. |
|
Surfaced a bug: Confirmed on the live site, these are dead-links on https://learn.scientific-python.org/contributors/getting-started/ |
fd8312a to
58ef866
Compare
8d756c1 to
6c54a4c
Compare
Known Regressions/ChangesDiff between the live site and the myst preview that I've found so far 🤖 AI text below 🤖 R1: Footer removedProduction footer contains:
Preview footer: only "Made with MyST" attribution. The Hugo theme provided this footer. R2: On-page TOC relabeled (minor)Production: right-side panel is labeled "On this page." Preview: same panel, same links, labeled "CONTENTS." MyST default label. No content difference. R3: Breadcrumb nav replaced by full left sidebar (structural change)Production: a minimal breadcrumb path below the top bar, e.g. Preview: a full left-side column showing the entire site index, expanded to the This is MyST's default sidebar behavior. R4: Page title format reversed (minor)
Consistent across all preview pages. Affects R5: Prev/next buttons show "Setup" instead of page title (new feature, label bug)Prev/next navigation buttons are new; Hugo did not have them. They work Root cause: the toc group in - title: Setup
children:
- file: contributors/setup/install.md
- file: contributors/setup/ecosystem.md
- file: contributors/setup/next-steps.mdThe Fix options:
R6: Maintainers section now visible in sidebar (change)On the Hugo site, On the MyST preview, Maintainers appears as a top-level entry in the left R7: Favicon missing in preview (fix pending)The production Hugo site serves a favicon via the Hugo theme. The MyST preview Fix options:
R8: Dead links on /contributors/getting-started/ fixed (change)
These are fixed in the PR. Pre-existing non-issues (not regressions)
|
e91c5ad to
9e3fb56
Compare
31ea4b8 to
b016851
Compare
|
Updated to account for the Cookie migration. Ready for review |
871f6c9 to
9363c6f
Compare
|
|
If this is an accurate preview, https://deploy-preview-315--learn-scientific-python-org.netlify.app/, the home page looks pretty bad, way more blank space, footer is particularly bad. 
|
|
docs/decisions feels like a ton of extra file that are not needed, they are historical. Once this is done, no one cares about the old site. Maybe these could be summarized into a single historical file, or attached to the PR text? |
@henryiii The footer is being fixed in scientific-python/scientific-python-myst-theme#7 which will be incorporated here once it's merged |
|
@henryiii on the ADRs, I can compress those docs significantly. Most of the ADR docs are already fairly short, but could use some formatting cleanup (those tables are messy). I'll add a fixup commit with some changes when I have a few minutes. |
The records served the planning phase of PR scientific-python#315 and stay in this branch's history. README.md and content/myst.yml drop their pointers. Refs scientific-python#315 Assisted-by: claude-code:claude-fable-5-1 [3M1LY]
0b739fe to
3a8ffdd
Compare
The records served the planning phase of PR scientific-python#315 and stay in this branch's history. README.md and content/myst.yml drop their pointers. Refs scientific-python#315 Assisted-by: claude-code:claude-fable-5-1 [3M1LY]
3a8ffdd to
3152ac3
Compare
The header should now show "Learn Scientific Python" (waiting for build to confirm), The font change will require a change to the theme template. Consistent footers will require applying the copier template to the other sites, I can open a PR against the cookie site for starters. |
The records served the planning phase of PR scientific-python#315 and stay in this branch's history. README.md and content/myst.yml drop their pointers. Refs scientific-python#315 Assisted-by: claude-code:claude-fable-5-1 [3M1LY]
3152ac3 to
cae9672
Compare
|
Requires: scientific-python/scientific-python-myst-theme#10 to prevent CI's prettier from reformatting after each update, I manually reformatted the files in this PR already. |
|
Thanks, Lundy, this looks great! There's one last issue with the width of the footer columns: 
I am going to approve this; once Dan has taken a final look and the footer is fixed, I think we'll go ahead and merge. |
|
One other thing: I don't see the social icon static files in the scientific-python-myst- theme template. As much as possible, this site should just be a content specialization of the copier version. Anything we can upstream to that theme, please do so. |
they are under template/assets (https://github.com/scientific-python/scientific-python-myst-theme/tree/main/template/assets/icons) Copier vendors them in the target project. |
Adds the full planning record for the Hugo → MyST-MD migration to docs/decisions/0001-myst-migration/: - REQUIREMENTS.md — what the migration must achieve and why - PLAN.md — commit-by-commit 8-phase implementation sequence - 0001–0007 ADR files — one decision per file, all Status: Proposed - README.md — index of the above ADR format is used so the decision rationale survives as a reviewable archive. All ADRs are Proposed pending maintainer review; the plan and requirements are the working basis for the migration work to follow. Assisted-by: claude-code:claude-opus-4-8
- myst.yml: new; project.id, inline toc for content/ tree, excludes submodules - fixed project.id UUID, exclude submodule dirs (external-content/, themes/) - .gitignore: add _build/ the MyST output directory - ADR 0001: Accepted Assisted-by: claude-code:claude-opus-4-8
_index.md is a Hugo idiom; MyST excludes _-prefixed files from auto-discovery. Renames done before content changes so all later commits reference the final filenames. Assisted-by: claude-code:claude-opus-4-8
Assisted-by: claude-code:claude-opus-4-8
Mapping (per ADR 0002):
{{< admonition warning >}}…{{< /admonition >}} → :::{warning}…:::
{{< admonition note >}}…{{< /admonition >}} → :::{note}…:::
Assisted-by: claude-code:claude-opus-4-8
shortcutDepth was a Hugo theme option controlling sidebar depth. MyST has no equivalent and warns on unknown frontmatter keys. Assisted-by: claude-code:claude-opus-4-8
Hugo silently rendered [text]() as href=""; MyST raises build errors. Added URLs for astropy, sunpy, numpy, scipy, matplotlib, and a Wikipedia link for "integrated development environment". Assisted-by: claude-code:claude-opus-4-8
- html: hugo → myst build --html (output now in _build/html/) - html-all: copy _build/html/ to public/ then build external content - serve: hugo server → myst start - clean: also removes _build/ Assisted-by: claude-code:claude-opus-4-8
Remove Hugo/Dart Sass toolchain (HUGO_VERSION, DART_SASS_VERSION, DART_SASS_URL env vars and the curl/tar/PATH install block). MyST ships pre-built CSS; no Sass compilation step required. Add `pip install mystmd` before `make html-all`. Keep PYTHON_VERSION, publish = "public", and netlify-plugin-checklinks unchanged. Assisted-by: claude-code:claude-opus-4-8
Port description, nav, domain, and logo from config.yaml. Rename static/ → assets/ (Hugo directory convention; MyST does not serve static/ automatically — assets/ is the expected path). No dark logo or favicon: neither existed in the Hugo static/ dir; MyST supplies a default favicon. Footer/quicklinks deferred per ADR 0005. Assisted-by: claude-code:claude-opus-4-8
- Delete config.yaml (metadata ported to myst.yml in previous commit) - Remove scientific-python-hugo-theme submodule (only consumed by Hugo; other SP repos vendor their own copy — ADR 0003) - Drop .hugo* and resources/ from .gitignore; both are Hugo-specific. _build/ was already present; public/ kept for html-all output. Assisted-by: claude-code:claude-opus-4-8
Vendor scientific-python/scientific-python-myst-theme at 9002ff6 with copier. The footer grid reads its links and social icons from a JSON file through the footer-items.mjs plugin. The template ships the scientific-python.org link set, whose relative paths do not exist on this site. footer_items.json carries the links and icons from the Hugo footer on main instead, plus the Bluesky account the main site links: About and Roadmap stay local, the other five links point at scientific-python.org. Assisted-by: claude-code:claude-opus-4-8 Assisted-by: claude-code:claude-fable-5-1 [3M1LY]
- Document build/preview process in README - Document ADRs in README - add small threaded server to scripts/serve.py Assisted-by: claude-code:claude-opus-4-8
The records served the planning phase of PR scientific-python#315 and stay in this branch's history. README.md and content/myst.yml drop their pointers. Refs scientific-python#315 Assisted-by: claude-code:claude-fable-5-1 [3M1LY]
cae9672 to
7703c47
Compare
I added a fixup commit to the end of the branch to address this, the preview looks good to me now. up-streamed the change to the theme in this PR: scientific-python/scientific-python-myst-theme#12 If it looks good to y'all let me know and I'll fixup the history. |
4 participants
Migrate from Hugo to MyST-MD
Fixes: #279