Add static site generator with client-side search by razodin137 · Pull Request #445 · basecamp/writebook

razodin137 · 2026-07-06T16:00:40Z

Add a static site generator with client-side search

This adds a self-contained static site generator to Writebook: it renders the
public library — the menu of books, every book's table of contents, and every
leaf — to a hostable static HTML directory with all CSS/JS assets and image
blobs copied in, and no Rails process, login, or editing machinery. A
hosted export behaves like a read-only visitor: the same bytes, no server.

On top of that, it adds client-side search for the static export, so a static
copy can be searched with no Rails process at all — mirroring Writebook's own
server-side SQLite FTS5 search.

The live app is untouched. The exporter renders the read views logged-out
through an Integration::Session and post-processes the result, so no
application or template code is modified for the export path.

Summary

Export the whole library to a static site. Renders the read views exactly
as a visitor sees them — the library menu, every book TOC, every leaf, the
bookmark overlay frames, front-matter .md alternates, the PWA manifest, and
every referenced asset and image blob (ActiveStorage covers/pictures + /u/
uploads) — and writes it all to one self-contained directory you can host
anywhere. Driven by Writebook::StaticExporter in
lib/writebook/static_exporter.rb.
Make the export dramatically smaller. The sidebar was previously inlined
into every leaf — O(n²) bytes, so a 1255-leaf book was ~390 MB. It's now one
fetched fragment per book (O(n); that book drops to ~39 MB). The fetch
resolves against location.pathname with trailing-slash normalization, so the
same export works at a domain root and under any subpath (the in-app
/static-site/ preview, a GitHub Pages project site, any /repo/ prefix).
Report to the operator in a browser, not to a developer over a shell. The
old result page showed a server filesystem path, a shell command, and a server
log — useless on a Dockerized deploy where tmp/static-site lives in a
container the admin has no shell for. The result page now streams the generated
site as a single .zip (#download), serves it in-browser under
/static-site/ admin-gated (#preview) so it can be seen with no separate web
server, and renders a distinct "Nothing to export yet" page when there are no
published books instead of zeroed counts.
Don't 500 the result page during export. The export renders every page
through a nested request, and that nested request was wiping the live
request's session state — so the result page crashed in Puma (it passed in the
test suite because tests scope that state differently than Puma does). Run the
export in a separate thread wrapped in Rails.application.executor so the
nested request's state stays isolated; the thread is joined, so the request is
still synchronous. The landing page's "Include unpublished drafts" checkbox
runs the same rolled-back Book.transaction the STATIC_ALL=1 rake task uses,
so the live DB is never mutated.
Export a single book, not just the whole library. The landing page gets a
scope selector: all published books, or one book picked from a list. The
exporter itself is unchanged (it always renders whatever Book.published
returns); the controller scopes that set inside a rolled-back transaction by
temporarily making the chosen book the only published one. Handy for sharing
or hosting a single title.
Make search work on a static host. Native Writebook search is server-side
FTS5 (POST /books/:id/search → Books::SearchesController#create → the
leaf_search_index virtual table), which can't run without Rails. The exporter
previously "neutralized" the search form — stripped its action and short-circuited
submit — leaving a dialog that opened but did nothing. This replaces that with
a real client-side search that mirrors the server's behavior, with no view-template
changes: only post-processing on the rendered HTML plus one new Stimulus
controller (app/javascript/controllers/static_search_controller.js).

How it stays faithful: a per-book _search.json index (title + plain-text
searchable_content — the same fields the FTS index holds) is written
alongside _sidebar.html at export time. The existing search dialog is reused;
its empty <form id="search_form"> is handed to a static-search controller
with the relative path to the book's _search.json. The controller fetches the
index lazily on first interaction, builds an in-memory inverted index, and on
submit renders up to 50 results into the existing <turbo-frame id="search">
using the server's _results/_result markup: title with <mark>, a ~20-token
content snippet with <mark>, a reading-mode link carrying ?search=, and a
"No matches." empty state. Title hits weighted 2×, mirroring
bm25(leaf_search_index, 2.0). No vendored dependency — a hand-rolled index
matches the fork's no-bundler, inline-script convention.

A small inline highlight script is injected into leaf pages so arriving from a
result link (which carries ?search=) wraps whole-word matches in <mark>
and scrolls the first into view, mirroring the server's
highlight_searched_content + scroll_to_highlight_controller. Exported leaf
HTML has no baked-in marks; the script only runs when the live URL carries the
query.

What's left out (and why)

A static export is read-only by design, so anything that needs a Rails process
or a logged-in session is intentionally absent from the export — and because the
live app is unchanged, none of it is lost:

Login / editing / admin. The export renders the read views logged-out.
Drafts can be included via the checkbox, which uses a rolled-back transaction
so the live DB is never mutated.
Server-side FTS5 search. Replaced, not removed: the client-side index
carries the same title + searchable_content the FTS index holds, ranked the
same way (title 2×). The live search UI and server code are untouched; the
static-search controller is eager-loaded in the live app too but only
activates where data-controller="static-search" is present, which the
exporter injects solely into exported pages.
Turbo Drive on the export. Exported pages are plain HTML served from a
static host; the search form uses data-turbo="false" so no request fires.

Test plan

bin/rails test — 186 runs, 709 assertions, 0 failures, 0 errors, 0 skips
on the branch tip, run in the ghcr.io/razodin137/writebook:static-search
container. New coverage lives in
test/controllers/static_exports_controller_test.rb and
test/lib/writebook/static_exporter_test.rb (exporter tests run in-container
against the real rendered output), including: drafts export (asserts the draft
is exported and still unpublished in the DB after rollback), the
no-books-at-all empty case, the single-book scope, and the PRG redirect that
fixes the Generate form's Turbo error.
Manually: exported the full library and a single book from the admin "Export
to static site" button; downloaded the .zip; previewed under /static-site/;
opened the search dialog on an exported page and ran a query (title + snippet
results with <mark>, "No matches." empty state); followed a result link and
confirmed destination highlighting + scroll; confirmed the same export renders
correctly at a domain root and under a subpath.

Screenshots

The export landing page: all-books / single-book scope selector, "Include unpublished drafts" checkbox, Generate button.

The admin "Export to static site" page.

Result page after Generate: counts, hosting next steps, and the Download .zip / Preview buttons.

Client-side search on a static export — title + snippet results with <mark> highlights, no Rails process.

Files

lib/writebook/static_exporter.rb — the exporter (506 lines).
app/controllers/static_exports_controller.rb — admin-gated export / download
/ preview / result, PRG redirect, single-book scope, drafts checkbox.
app/javascript/controllers/static_search_controller.js — client-side search
Stimulus controller (inverted index, ≤50 results, server's result markup).
app/views/static_exports/{create,empty,show}.html.erb — landing / result /
empty pages.
app/views/books/{index,show}.html.erb — the download icon in the library
header; one stray-quote fix.
config/routes.rb, lib/tasks/static.rake — routes; bin/rails static:generate
with STATIC_HOST / STATIC_ALL env support.
README.md, STATIC_SITE_GENERATOR.md — docs.
test/controllers/static_exports_controller_test.rb,
test/lib/writebook/static_exporter_test.rb — coverage.

14 files changed, +1784 / −1.

Drafted with assistance from Claude Code.

…ML site Renders the published library — the menu of books, every book's table of contents, and every leaf — to a self-contained static HTML directory with all CSS/JS assets and image blobs copied in, no Rails process, login, or editing machinery. - Writebook::StaticExporter drives an Integration::Session to render the read views logged-out, exactly as a visitor sees them — no application/template code is modified. - Renders the library menu, every book, every leaf, the bookmark overlay frames, the front-matter markdown (.md) alternate routes, the PWA manifest, and every referenced asset and image blob (ActiveStorage covers/pictures + /u/ uploads). - Externalizes the per-book sidebar into one shared fetched fragment (O(n^2) -> O(n); a 1255-leaf book goes ~390 MB -> ~39 MB). - bin/rails static:generate with STATIC_HOST / STATIC_ALL env support (STATIC_ALL includes unpublished books inside a rolled-back transaction so the live DB is untouched). - Admin "Export to static site" button in the library header runs the export synchronously and shows a result page with counts + next steps (StaticExportsController + views; admin-gated via ensure_can_administer). Tests: 171 runs, 597 assertions, 0 failures. Docs: README.md + STATIC_SITE_GENERATOR.md.

The export result page reported the outcome to a developer (a server filesystem path, a shell command, a server log) rather than to the operator looking at it in a browser -- which is especially opaque on a Dockerized deploy where tmp/static-site lives inside a container the admin has no shell for. Three additive changes, none of which bake any operator's host into the code: * Download .zip: a #download action streams the generated site as a single zip the operator can save from the browser. A downloaded file is the success signal, and its location is wherever the browser put it. * Preview site: a #preview action serves tmp/static-site from inside the running app under /static-site/, admin-gated, so the export can be seen in a browser tab with no server shell or separate web server. Root-relative asset/upload URLs resolve against the live app (the same bytes the export copied); the relative sidebar fetch and per-book files resolve within the mounted path. Header CSP is cleared so the static HTML's own meta policy governs, like a real static host. * Empty-case: when there are no published books the result page said "Your static site is ready" with zeros. Now it renders a distinct "Nothing to export yet" page without invoking the exporter (the library root redirects when there are no published books, so the exporter can't start anyway). Image-copy failures are also surfaced as their own warning rather than buried in the counts sentence. rubyzip (already a transitive dependency) backs the zip; no Gemfile change.

Generate was 500ing in the live Puma request: the exporter renders every page through a nested ActionDispatch::Integration::Session, which re-enters the middleware stack and resets ActiveSupport::CurrentAttributes. Running that inside the request's own thread wiped the request's Current, so the layout's signed_in? -> Current.user raised NoMethodError ([] on nil) the moment the result page rendered. It worked in the test suite and rails runner only because those scope CurrentAttributes differently than Puma does. Run the exporter in a separate thread wrapped in Rails.application.executor instead, so the nested session's CurrentAttributes resets stay scoped to that thread and the live request's Current is left intact. The thread is joined, so the request is still synchronous and returns the result. Validated in the live container: export completes, outer Current.user intact. Make drafts a button, not a shell command: the landing page gets an "Include unpublished drafts" checkbox that runs the same rolled-back Book.transaction the STATIC_ALL=1 rake task uses, so the live DB is untouched. Drop the shell-first framing from the UI (kept only as the escape hatch for libraries large enough to time out a browser request). Add visible "Generating..." feedback via data-turbo-submits-with on the Generate / Generate-again buttons, so the click isn't silent while the export runs. Tests: drafts export (asserts the draft is exported AND still unpublished in the DB after rollback), and no-books-at-all-with-drafts renders the empty page. 177 runs, 629 assertions, 0 failures.

Sidebar 404 injected into every leaf page (static_exporter.rb): - fetch("../../_sidebar.html") resolved one level too shallow when the browser URL lacked a trailing slash (Turbo nav links render slash-less), 404ing; and the .then(r=>r.text()) never checked r.ok, so the server's 404 body was outerHTML'd into the placeholder <aside>. fetch does not reject on 404 -- the .catch only guards network errors. - Make the fetch path absolute and root-relative (/<book_rel>/_sidebar.html) so it resolves against the host root regardless of baseURI/trailing slash, and guard with if(!r.ok)return null so a non-OK response can't be injected. Search form 404 on every page: <form id="search_form"> posted to /books/:id/search, a route that only exists in the live app, 404ing into Turbo's "content missing" fallback. Strip its action and add data-turbo="false" + onsubmit="return false" so the dialog still opens (visual parity) but no request fires. Malformed HTML in books/show.html.erb:62: stray " in the class attribute (txt-tight-lines"") -- replicated to every exported book TOC page. Removed.

The admin "Export to static site" form is Turbo-driven (turbo_submits_with: "Generating…"), posting to static_exports#create, which rendered the result page with 200 OK. Turbo Drive requires form submissions to redirect; a 200 HTML response made it throw "Form responses must redirect to another location" -- so Generate appeared to do nothing in the browser. This was missed by earlier server-side verification (curl / Integration::Session return 200 fine) because the failure is client-side JS only. Switch to post-redirect-get: #create runs the export, stashes the Result in the session, and redirects 303 to a new GET #result, which renders :create (or :empty). Turbo follows the redirect (so "Generating…" stays active during the POST), the result URL is now refresh-safe, and the "Generate again" button on the empty page works the same way. A direct GET to #result with no stashed result falls back to the landing page.

…ubpath The exported sidebar was fetched via a root-relative "/<book_rel>/_sidebar.html", which 404s whenever the site is hosted under a subpath (the in-app /static-site/ preview, a GitHub Pages project site, any /repo/ prefix). Resolve the fetch against location.pathname instead, with a trailing-slash normalization, so the same export works at a domain root and under any subpath -- the same relative resolution the per-book search index fetch already uses.

The export landing page now has a scope selector: "All published books" (the existing behavior, still with the "Include unpublished drafts" checkbox) or a single book picked from a list. Exporting one book is handy for sharing or hosting a single title at a time. The exporter itself is unchanged -- it always renders whatever Book.published returns. The controller scopes that set inside a rolled-back transaction (the same trick the drafts checkbox already uses): for a single book, the chosen book is temporarily made the only published one, so the library menu and the book render alone, regardless of the book's real published state. The live database is never touched. The result page names the book when one was exported, and the download link carries book_id so the .zip is named writebook-<slug>.zip (vs writebook-static-site.zip for the whole library) and a bookmarked download URL regenerates just that book. "Generate again" repeats the same scope.

Native Writebook search is server-side SQLite FTS5 (POST /books/:id/search -> Books::SearchesController#create -> the leaf_search_index virtual table), which cannot run on a static host. The exporter previously "neutralized" the search form (stripped its action, short-circuited submit), leaving a dialog that opened but did nothing. Replace that with a real client-side search that mirrors the server's behavior, with no Rails process required and no view template changes -- only post-processing on the rendered HTML, plus one new Stimulus controller. * A per-book _search.json index is written alongside _sidebar.html at export time, from the live Leaf model -- the same title + plain-text searchable_content the FTS index holds. (build_search_index) * The existing search dialog is kept and its empty <form id="search_form"> is handed to a new static-search Stimulus controller with the relative path to the book's _search.json. The path is computed per page (../../_search.json on leaf pages, _search.json on the book TOC) and resolved against location.pathname with the same trailing-slash normalization as the sidebar fetch, so it works at a domain root and under any subpath. (wire_search_form) * static_search_controller.js fetches the index lazily on first interaction, builds an in-memory inverted index, and on submit (or debounced input) renders up to 50 results into the existing <turbo-frame id="search"> using the server's _results/_result markup: title with <mark>, a ~20-token content snippet with <mark>, a reading-mode link carrying ?search=, and a "No matches." empty state. Title hits weighted 2x, mirroring bm25(leaf_search_index, 2.0). No vendored dependency -- a hand-rolled index matches the fork's no-bundler, inline-script convention. * A small inline highlight script is injected into leaf pages so arriving via a result link (which carries ?search=) wraps whole-word matches in <mark> and scrolls the first into view, mirroring the server's highlight_searched_content + scroll_to_highlight_controller. Exported leaf HTML has no baked-in marks (fetched without ?search=), so the script only runs when the live URL carries the query. (inject_destination_highlight) The live search UI and server code are untouched; the controller is eager-loaded in the live app too but only activates where data-controller="static-search" is present, which the exporter injects solely into exported pages. 10 exporter tests pass in-container.

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Copilot

Pull request overview

This PR adds a static site export feature to Writebook, generating a fully hostable read-only HTML copy of the public library (including assets/blobs) and introducing client-side search for exported pages, while keeping the live app’s read path unchanged.

Changes:

Introduces Writebook::StaticExporter to render library/book/leaf pages to a static directory and copy referenced assets/resources.
Adds an admin-only UI flow (generate → result → download zip / in-app preview) plus routes and a static:generate rake task.
Implements client-side search for exports via a new Stimulus controller and a per-book _search.json index.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`lib/writebook/static_exporter.rb`	Core static exporter: render read views, rewrite HTML, externalize sidebar, build search index, copy assets/resources.
`app/controllers/static_exports_controller.rb`	Admin controller for generate/result/download/preview, with session-stashed result and rollback scoping.
`app/javascript/controllers/static_search_controller.js`	Client-side search over exported per-book JSON index; renders results into existing dialog/frame.
`app/views/static_exports/show.html.erb`	Export landing page (scope selector + drafts checkbox).
`app/views/static_exports/create.html.erb`	Result page (counts, download zip, preview, instructions).
`app/views/static_exports/empty.html.erb`	“Nothing to export” result path when there are no (published) books.
`config/routes.rb`	Adds static export routes + in-app preview path mapping.
`lib/tasks/static.rake`	Adds `bin/rails static:generate` with `STATIC_HOST` / `STATIC_ALL` support.
`app/views/books/index.html.erb`	Adds admin-only “Export to static site” button in library header.
`app/views/books/show.html.erb`	Fixes an extra stray quote in a class attribute.
`test/lib/writebook/static_exporter_test.rb`	Exporter integration tests for rendered output, resource copying, sidebar/search wiring, markdown alternates.
`test/controllers/static_exports_controller_test.rb`	Controller tests for auth/admin gating, scoping, rollback behavior, download/preview/result flows.
`README.md`	Adds fork-specific static exporter overview and pointers to full docs.
`STATIC_SITE_GENERATOR.md`	Adds full documentation for running exports, included content, design principles, and file map.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+          terms=terms.filter(function(t,i){return terms.indexOf(t)===i;});
+          if(!terms.length)return;
+          terms.sort(function(a,b){return b.length-a.length;});
+          var esc=function(s){return s.replace(/[&<>"']/g,function(c){return {"&":"&amp;","<":"&lt;",">":"&gt;",'"':"&quot;"}[c];});};


+      def dir_size
+        `du -sh #{@output_dir}`.strip.split.first
+      end


+            result = exporter.call
+          end
+        end
+      end.join


+    def ensure_static_site_generated(book: nil)
+      generate(book: book) unless static_dir.join("index.html").exist?
+    end


+    def zip_static_site
+      require "zip"
+
+      zip_path = Rails.root.join("tmp/writebook-static-site.zip")
+      FileUtils.rm_f(zip_path)
+      root = static_dir
+


+    <p class="margin-block-none txt-subtle">
+      Only published books are exported here. To include unpublished drafts, or to
+      export without holding a web request open, run
+      <code>bin/rails static:generate STATIC_ALL=1</code> from the server shell —


+    results.innerHTML = hits.map(({ entry }) => {
+      const href = entry.url + "?search=" + encodeURIComponent(q)
+      const title = this.highlight(entry.title, terms)
+      const snippet = this.snippet(entry.content, terms)
+      return `<a class="search__result hide_from_edit_mode txt-ink" data-turbo-frame="_top" href="${this.escapeAttr(href)}"><strong>${title}:</strong> ${snippet}</a>`


razodin137 added 8 commits July 6, 2026 05:17

Copilot AI review requested due to automatic review settings July 6, 2026 16:00

claude Bot reviewed Jul 6, 2026

View reviewed changes

Copilot started reviewing on behalf of razodin137 July 6, 2026 16:01 View session

Copilot AI reviewed Jul 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add static site generator with client-side search#445

Add static site generator with client-side search#445
razodin137 wants to merge 8 commits into
basecamp:mainfrom
razodin137:static-search

razodin137 commented Jul 6, 2026

Uh oh!

claude Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

razodin137 commented Jul 6, 2026

Add a static site generator with client-side search

Summary

What's left out (and why)

Test plan

Screenshots

Files

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants