[cnbrkdmrci]

Requested by Can · project thread

Intent: make the Worker's main module small enough for hosts that cap each uploaded part (open-compute: 8 MiB), so Diffinery can deploy there (#24), without changing anything on Cloudflare or celld beyond a lighter cold start.

Opened by Claude Code on behalf of Can (GitHub cnbrkdmrci).

What this changes

The main module bundled to 14.4 MB. Two generated things were most of it: the docs content (src/docs/content.generated.ts, 4.4 MB of pages, images and downloads as JS literals) and Shiki's grammars (3.2 MB of JS modules that the bundler inlined, though they were meant to load on first use). Both now upload as text modules, the mechanism the Kumo islands already use (*.txt, wrangler's default Text rule, also supported by celld):

• Docs: scripts/build-docs.mjs writes the bulky values to build/docs/content.txt (JSON). content.generated.ts imports it and JSON.parses it once, keeping every export's name and type, so no caller changes. Parsing the JSON takes about half the time of parsing the same JS literal (14 ms against 27 ms here).
• Grammars: the new scripts/build-grammars.mjs writes each Shiki grammar the Worker uses to build/grammars/<name>.txt (73 grammars for the 62 languages, each once, though JavaScript, CSS and HTML are embedded by many), and src/web/grammars.generated.ts imports them. highlight.ts takes its language list from that file and parses a grammar on first use. The language list moved from highlight.ts to the build script.

Results from wrangler deploy --dry-run:

	Before	After
index.js (main module)	14.4 MB	6.8 MB
Largest other module	0.48 MB	4.4 MB (content.txt)
Source map	20.5 MB	11.7 MB
Total upload (gzip)	4.63 MB	4.62 MB

The source map is still over 8 MiB, so the docs tell open-compute users to set upload_source_maps to false. On Cloudflare nothing changes in what is uploaded or how it behaves.

How it was tested

• pnpm -r typecheck passes. apps/diffinery unit tests all pass (1251), including the highlighting tests that compare Diffinery's tokens with Shiki's own modules (highlight.test.ts) and the celld guard. The new main-module-size.test.ts fails if the generated content goes back into JavaScript. In the full pnpm -r test run, containers/runner's PTY sandbox test failed once under load. This change doesn't touch that package, and the test passes when run again.
• Ran the Worker with wrangler dev -e dev (workerd): docs pages render with Shiki highlighting, and docs images, the site pages, /install, the CLI bundle, the VS Code extension and the Claude plugin zip are served.

Checklist

• pnpm -r typecheck and pnpm -r test pass
• User- or admin-visible changes update the docs in apps/diffinery/docs (self-hosting/architecture.md, self-hosting/configuration.md), plus a CHANGELOG entry
• New D1 migrations use the next free number in apps/diffinery/migrations (none)

Refs #24

---

Generated by Claude Code