fix(changelog): render release notes as markdown instead of raw text

[NitinKumar004]

The release notes on gofr.dev/changelog are hard to read for most releases. Tables show up as raw | a | b | lines, links as [text](url), #### headings keep their hashes, nested lists are flattened and code has no highlighting. Some content is missing entirely.

Cause

src/app/changelog/page.jsx rendered the GitHub release bodies with a small hand-written line parser. It only understood code fences, ### headings, - bullets, **bold** and `code`. Across the 117 published releases, that means:

• links render as raw text in 39 releases;
• nested bullets are flattened in 32;
• #### headings show the hashes in 26;
• bullets that wrap onto a second line break apart in 13;
• tables render as raw pipes in 6, including v1.61.0;
• text before the first ## section is dropped in 11;
• pasted screenshots show as raw <img …> text;
• about 15 releases use # or ### for their Features/Fixes headings and lose their section cards. In v1.34.0 the Features heading disappears.

Fix

• Markdoc for the release notes. This is the same renderer the docs pages use, with the site's own components:

  • Prose for typography and tables;
  • Fence for syntax highlighting and the copy button;
  • AutoLink, which drops unsafe schemes such as javascript: and opens external links in a new tab.

• GitHub-style pre-processing. It never touches code blocks or inline code:

  • bare URLs are auto-linked;
  • pasted <img> screenshots become images, https only;
  • other raw HTML stays escaped text, so <METHOD> or <script> show as text.

• Code fences are marked process=false. A {% %} inside a code sample (Jinja, Helm, CI templates) can no longer make Markdoc lose the fence's end, which would swallow the rest of the release and fail the build.

• Better section splitting:

  • sections are found at #, ##, and at ### when the heading is a section name such as "Bug Fixes";
  • text before the first section is kept as an Overview;
  • version-title headings (## Release - v1.38.0) are skipped;
  • a heading-only notice, like v1.14.0's breaking-change warning, is kept as a Note;
  • card labels are plain text, taken from the real heading.

• Rendered at build time. page.jsx is now a server component that pre-renders the notes. The client component (ChangelogClient.jsx) keeps the pagination, deep links, version rail and expand/collapse. The browser no longer downloads Markdoc or the raw release bodies:

  	changelog page JS	first load JS
  before	71.3 kB	156 kB
  after	8.24 kB	118 kB

  Gzipped HTML plus page JS grows by about 17 kB, and it now carries fully rendered notes for every release.

• Always dark. The notes stay on dark prose styles even when the site theme is light, since the changelog background is always dark.

The first commit only moves the old page.jsx into ChangelogClient.jsx, byte for byte. The second commit holds all the changes, so its diff shows exactly what changed in the page.

Before / after

Screenshots were taken with headless Chrome at 2x: live gofr.dev on the left, this branch on the right.

Release	Before (gofr.dev)	After (this PR)
v1.61.0 — table
v1.61.0 — tables
v1.39.0 — table, numbered heading
v1.53.0 — nested list, #### heading, code
v1.54.5 — links
v1.56.1 — wrapped bullets
v1.16.0 — pasted screenshots
v1.14.0 — breaking-change notice
v1.34.0 — sections with # headings

Testing

• Rendered all 117 releases from the GitHub API through the new code. No raw markdown is left in the output, no text is lost compared with the old parser, there are no Markdoc validation errors, and the React server render throws nothing.
• Ran about 25 synthetic edge cases:
  • tables containing | in code, and a table right after a paragraph;
  • ~~~ fences, 4-backtick fences and an unclosed fence;
  • {% %} and {{ }} in prose and in code;
  • fences indented inside lists;
  • javascript: and data: links and images, <script>, <iframe>;
  • CRLF line endings, emoji shortcodes, URLs in parentheses;
  • long tokens.
• Checked in the browser:
  • expand and collapse, the "Show older" pagination, and /changelog#<tag> deep links on old releases;
  • no console errors or hydration warnings;
  • still readable with the light theme.
• next build succeeds with the static export, and the new files pass Prettier.

src/app/changelog/releases.json is not changed here, because yarn refresh-data regenerates it at deploy time. The RSS feed (changelog.xml) still embeds the raw markdown; that is a separate follow-up.

[aryanmehrotra]

Thanks for this. The old parser was losing a lot, and the before/after table makes the case well. I checked it rather than just reading it.

Commit 1 is a byte-for-byte move (diff of the base page.jsx against c44f2c05d:ChangelogClient.jsx is empty), so commit 2's diff really is the whole change. I refreshed releases.json to all 117 current releases and ran every section through splitReleaseSections → normalizeReleaseMarkdown → Markdoc. The build is clean. javascript:, data:, <script> and <img src=javascript:> all render as inert text. On size, measured against the base with the same 117 releases: HTML gzip goes 18.2 → 99.8 kB and page JS 71.4 → 8.38 kB, so a net gain of about 18 kB, which matches your figure.

Requesting changes on one thing.

The fence scanner doesn't close a fence when its list item ends

mapOutsideCode (releaseMarkdown.mjs:100) and splitReleaseSections (:188) both keep a fence open until they see a matching closing fence. In CommonMark, a fence inside a list item also ends when that item ends.

v1.27.0 hits this today. Its body has ```go inside item 1 (line 9) that is never closed, so ## 🛠️ Fixes (line 41) is treated as code and never split out:

splitReleaseSections(v1.27.0.body)  ->  [features]        // old parser: features, fixes

The Fixes heading ends up as an <h3> inside the Features card, and the release loses its Fixes badge and card. It's the only one of the 117 releases that loses a card.

The bigger problem is that the same scanner decides where the {% escape and the prose rewrites run. Minimal input:

## ✨ Features

1. Adds a thing
    ```go
    x := 1

2. Templates now accept {% raw %} blocks in values, see https://gofr.dev

## 🛠️ Fixes

- fixed y

Result: one section (features), Markdoc.validate reports 8 critical errors, the rendered text reads "accept blocks in values" ({% raw %} is silently dropped), and the URL stays unlinked. The build doesn't notice, because ReleaseNotes never calls Markdoc.validate.

I'll leave the fix to you. Markdoc's own tokenizer already knows where each fence starts and ends. Whatever you choose, the scanners need to agree with the parser. It would also help to run Markdoc.validate during the build and fail on anything critical, so the next case like this breaks the build instead of silently dropping text.

Smaller notes, no action needed

• Bare URL inside link text: BARE_URL (:28) breaks the link. [see https://gofr.dev/docs here](https://gofr.dev) renders with the outer link as raw text. None of the current releases do this.
• Stale comment: releaseMarkdown.mjs:4 refers to "the changelog checks", but they aren't in the repo. Please either commit a check over the real releases (it's the only thing that would catch a regression here) or drop the reference.
• Broken internal link: v1.42.0's /advanced-guide/dealing-with-sql/ is now clickable and returns 404. That needs fixing in the release body, not in this PR.

Gates: the build is clean against all 117 releases. The 3 new files pass Prettier; ChangelogClient.jsx is flagged, but the base page.jsx already was. There's no ESLint config, so next lint isn't a real gate here.

[NitinKumar004]

Thanks for checking this so thoroughly. The list-item fence case was a real hole, and your minimal input made it easy to pin down.

Fence scanner vs the parser. I removed the hand-written scanners. mapOutsideCode and splitReleaseSections now both take their positions from Markdoc's own tokenizer (new Markdoc.Tokenizer().tokenize()):

• code ranges come from each fence token's map, so a fence left open inside a list item ends with the item, as in CommonMark;
• section boundaries come from top-level ATX heading_open tokens, so headings inside code, list items or quotes never split a section.

The rewrites ({% escape, bare URLs, process=false on fences) and the section split therefore always agree with how Markdoc parses the body.

• v1.27.0 is back to features, fixes.
• Your minimal input gives two sections (Features, Fixes) and 0 critical errors. It renders "Templates now accept {% raw %} blocks in values", with the URL linked and the go fence ending with item 1.
• Across all 117 current releases, the rendered output is identical to the previous commit except for v1.27.0.

Validate at build time. ReleaseNotes now runs Markdoc.validate and throws on anything critical, naming the release and line. For example: changelog: release notes of v1.x.y don't parse: line 3: Node 'fence' is missing closing. The next case like this fails the build instead of dropping text.

A committed check. utils/check-changelog.mjs renders every release in releases.json exactly the way the page does. It fails if:

• a release has a critical validate error;
• a release loses all its content;
• raw markdown (a table row, [text](url) or a # heading) survives outside code.

It runs in prebuild, so CI's build job runs it, and it's also available as yarn check:changelog. The comment in releaseMarkdown.mjs now points to it.

To make sure the check isn't vacuous, I switched process=false off and added a release with {% if %} inside a fence. The check failed with the fence's missing-closing error and exited 1. With the real code the same release renders cleanly.

Bare URL inside link text. [see https://gofr.dev/docs here](https://gofr.dev) now keeps the outer link. Existing [text](url) and <url> links are skipped when bare URLs get linked.

v1.42.0's /advanced-guide/dealing-with-sql/ link. Agreed, that needs fixing in the release body on GitHub. I'll leave it out of this PR.

Gates: yarn build, which runs prebuild including the new check, is clean against both the committed releases.json (108) and a fresh fetch (117). The changelog page JS stays at 8.24 kB. The new and changed files pass Prettier.

[aryanmehrotra]

The R1 case is fixed, and I checked it rather than reading it:

R1 item	At 6e26b1129
v1.27.0 Fixes card	back to features, fixes
minimal input ({% raw %} in a list item after an indented fence)	2 sections, 0 critical, {% raw %} rendered, URL linked
output across all 117 current releases vs 9234747a1	only v1.27.0 changed; 0 critical at head
bare URL inside link text	outer link kept
validate at build time	ReleaseNotes throws on critical, names the tag

next build is clean (/changelog 8.38 kB) and the 5 changed files pass Prettier. Moving onto the tokenizer was the right call.

Requesting changes on two things.

1. The tokenizer reads the raw text, but rendering parses the escaped text

fenceRanges and topLevelHeadings tokenize the body before transformProse escapes {%. A {% … %} alone on a line therefore opens a Markdoc tag at tokenize time. Every heading after it is no longer at level 0, so splitReleaseSections never splits on it. Rendering then parses the escaped text, where it's plain prose:

## Features
- Supports {{ .Values.x }} and
{% include "a" %}          <- tokenizer: open tag; headings below become level 1
## Fixes                   <- not a section; renders as <h3>Fixes</h3> in the Features card

input	sections
{% raw %} alone on a line, then a list	features
continuation line {% include "a" %}	features
{% raw %} inline in a sentence	features, fixes
{% if x %} alone on a line	features, fixes

Both failing cases give 0 critical validation errors, so the new build gate stays green. No current release hits this, but it's the same disagreement between scanner and parser as R1. The positions need to come from the same text Markdoc finally parses. How you get there is up to you.

2. check-changelog.mjs doesn't catch a lost section

I pointed the committed check at the old scanner from 9234747a1, the one that dropped v1.27.0's Fixes card, with all 117 releases:

[check-changelog] all 117 release(s) render cleanly.

A swallowed section renders as a real <h3>, not a raw #, so none of the three RAW_MARKDOWN patterns match. The same is true for the case in (1): I added it as a 118th release and the check passed. Your non-vacuity test broke process=false, which produces a parse error, so it proved the check catches parse errors, not lost sections.

As committed, the check guards against parse errors and raw markdown. It doesn't guard against the regression R1 was about. Could it also fail on the R1 case itself, i.e. v1.27.0 through the old scanner? Then it protects the thing this PR fixed.

No action needed

• Agreed on v1.42.0's /advanced-guide/dealing-with-sql/ — that's a release-body fix.
• A heads-up rather than an ask: with the check in prebuild and the throw in ReleaseNotes, a GitHub release body that doesn't parse now blocks the site deploy. That's what I asked for and I think it's right, but it's worth whoever publishes releases knowing about it.

[NitinKumar004]

Thanks. Both are real, and the second one was the more useful catch: the check wasn't guarding the thing this PR is about.

1. Tokenizer vs parser

The positions now come from text with no active tags, which is what Markdoc finally parses. The tokenizer that finds fences and headings is still Markdoc's own, but with its tag rules turned off (tokenizer.parser.disable(['annotations', 'containers'])). The text that is rendered has every prose {% escaped and every fence marked process=false, so no tag is active in it, and reading the raw text with tags off gives the block structure the final parse sees. The same tokenizer drives the fence ranges for the rewrites and the section split. annotations/containers are Markdoc's internal rule names. If an upgrade renames them, markdown-it throws Failed to disable unknown rule(s) when the module loads, so the build fails loudly rather than quietly tokenizing with tags on. I noted that next to the call.

Your table at db3769c:

input	sections
{% raw %} alone on a line, then a list	features, fixes
continuation line {% include "a" %}	features, fixes
{% raw %} inline in a sentence	features, fixes
{% if x %} alone on a line	features, fixes

All four have 0 critical errors, and the {% … %} renders as literal text. A {% include %} line followed by a fence containing {% if %} also keeps the fence verbatim. Across the 117 current releases, the output is identical to 6e26b11.

2. The check now catches a lost section

check-changelog.mjs also reads each section's own Markdoc parse, the text that is actually rendered, and fails if a top-level heading there should have been a section of its own (#/##, or a ### section name) or a version title. Because it reads the parse and not the splitter's output, a splitter that swallows a section can't hide it.

Your two runs, repeated:

# committed check, splitter from 9234747, all 117 releases
[check-changelog] v1.27.0:
  - Features: "## Fixes" should be its own section but is inside this one
[check-changelog] 1 of 117 release(s) don't render correctly.        (exit 1)

# committed check, splitter from 6e26b11, 117 releases + your case as the 118th
[check-changelog] v9.9.9-case:
  - Features: "## Fixes" should be its own section but is inside this one
[check-changelog] 1 of 118 release(s) don't render correctly.        (exit 1)

# this commit, same 118
[check-changelog] all 118 release(s) render cleanly.                  (exit 0)

On the deploy note: agreed, that's intended. A release body that doesn't render now fails the check and the build, with the release and line named. I'll make sure whoever publishes releases knows.

Gates: yarn build (prebuild with the check, then next build) is clean with the committed releases.json (108) and a fresh fetch (117). /changelog page JS is 8.24 kB, and both changed files pass Prettier.

[aryanmehrotra]

Both are fixed, and I checked by mutation.

1. The tag rules are off in the positions tokenizer, and the rule names match what the pinned @markdoc/markdoc@0.3.0 registers (annotations on block and core, containers on inline). My four cases from last round all split into features, fixes with 0 critical errors. So do six more: {% /if %}, {% comment %}, {% if %} inside a fence, an include line followed by a fence, {% raw %} in a code span, and my R1 input. Across 117 releases the output is identical to 6e26b1129.

2. The check now fails for every broken splitter I gave it:

splitter from 9234747a1, 117 releases                -> v1.27.0: "## Fixes" should be its own section   (exit 1)
splitter from 6e26b1129, 117 + {% include %} case    -> case flagged                               (exit 1)
head with the tag rules switched back on             -> case flagged                               (exit 1)
head as committed, 117 + case / committed 108        -> clean                                      (exit 0)

Reading the headings from the rendered parse rather than from the splitter is what makes it hold.

next build is clean and Prettier passes on both changed files. Approving. Thanks for turning both rounds around quickly.