From e143b3e9bf2e8e5f7d235a9a32f2195715c1e523 Mon Sep 17 00:00:00 2001 From: cam <72155693+razodin137@users.noreply.github.com> Date: Mon, 6 Jul 2026 05:17:16 +0000 Subject: [PATCH 1/8] Add static site generator: export the library to a hostable static HTML site MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Renders the published library — the menu of books, every book's table of contents, and every leaf — to a self-contained static HTML directory with all CSS/JS assets and image blobs copied in, no Rails process, login, or editing machinery. - Writebook::StaticExporter drives an Integration::Session to render the read views logged-out, exactly as a visitor sees them — no application/template code is modified. - Renders the library menu, every book, every leaf, the bookmark overlay frames, the front-matter markdown (.md) alternate routes, the PWA manifest, and every referenced asset and image blob (ActiveStorage covers/pictures + /u/ uploads). - Externalizes the per-book sidebar into one shared fetched fragment (O(n^2) -> O(n); a 1255-leaf book goes ~390 MB -> ~39 MB). - bin/rails static:generate with STATIC_HOST / STATIC_ALL env support (STATIC_ALL includes unpublished books inside a rolled-back transaction so the live DB is untouched). - Admin "Export to static site" button in the library header runs the export synchronously and shows a result page with counts + next steps (StaticExportsController + views; admin-gated via ensure_can_administer). Tests: 171 runs, 597 assertions, 0 failures. Docs: README.md + STATIC_SITE_GENERATOR.md. --- README.md | 27 ++ STATIC_SITE_GENERATOR.md | 130 +++++++ app/controllers/static_exports_controller.rb | 29 ++ app/views/books/index.html.erb | 7 + app/views/static_exports/create.html.erb | 82 +++++ app/views/static_exports/show.html.erb | 45 +++ config/routes.rb | 5 + lib/tasks/static.rake | 20 ++ lib/writebook/static_exporter.rb | 319 ++++++++++++++++++ .../static_exports_controller_test.rb | 64 ++++ test/lib/writebook/static_exporter_test.rb | 144 ++++++++ 11 files changed, 872 insertions(+) create mode 100644 STATIC_SITE_GENERATOR.md create mode 100644 app/controllers/static_exports_controller.rb create mode 100644 app/views/static_exports/create.html.erb create mode 100644 app/views/static_exports/show.html.erb create mode 100644 lib/tasks/static.rake create mode 100644 lib/writebook/static_exporter.rb create mode 100644 test/controllers/static_exports_controller_test.rb create mode 100644 test/lib/writebook/static_exporter_test.rb diff --git a/README.md b/README.md index 848b893c..79789734 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,12 @@ ### Instantly publish your own books on the web for free, no publisher required. +> **This is a fork of [basecamp/writebook](https://github.com/basecamp/writebook)** +> that adds a **static site generator**: export the public library to a +> self-contained static HTML site you can host anywhere — no Rails process, +> login, or editing machinery. See **[STATIC_SITE_GENERATOR.md](./STATIC_SITE_GENERATOR.md)** +> for the full guide, or the quick summary below. + Writebook is an easy-to-use application for publishing content on the web. Content is authored in Markdown, and books can contain picture pages, chapters, and title pages. Books can be published privately or publicly, and are searchable. @@ -43,3 +49,24 @@ Start the development server: ```sh bin/dev ``` + +## Static site generator (this fork) + +Render the published library to a static HTML directory you can host anywhere — +all books, every page, all CSS/JS and image files copied in, no login or editing +machinery. + +**From the admin UI:** an admin-only **Export to static site** button in the +library header runs the export and shows a result page with the file counts and +what to do next (where the files are, how to preview locally, how to deploy). + +**From the command line:** + +```sh +bin/rails static:generate # → tmp/static-site +STATIC_HOST=books.example.com bin/rails static:generate # absolute URLs → this host +STATIC_ALL=1 bin/rails static:generate # include unpublished books (DB untouched) +``` + +See **[STATIC_SITE_GENERATOR.md](./STATIC_SITE_GENERATOR.md)** for what's +included, the design principles, and the file map. diff --git a/STATIC_SITE_GENERATOR.md b/STATIC_SITE_GENERATOR.md new file mode 100644 index 00000000..ec816663 --- /dev/null +++ b/STATIC_SITE_GENERATOR.md @@ -0,0 +1,130 @@ +# Static site generator + +This fork of [basecamp/writebook](https://github.com/basecamp/writebook) adds a +**static site generator**: it renders the public library — the menu of books, +every book's table of contents, and every leaf — to a self-contained static HTML +directory you can host anywhere, with **no Rails process, login, or editing +machinery**. All CSS/JS assets and image binaries (covers, in-body uploads, +ActiveStorage pictures) are copied in. + +It's a generic feature for the OSS Writebook project — intended to be PR-able +upstream to Basecamp — and is not coupled to any one instance. Any operator can +export their own books from their own Writebook. + +## Two ways to run + +### From the admin UI (synchronous) + +Administrators get a new **Export to static site** button (the download icon) in +the library page header. It opens a landing page; clicking **Generate static +site** runs the export into `tmp/static-site` and shows a result page with the +counts and **what to do next**: + +- where the files are on the server, +- how to preview locally (`cd tmp/static-site && python3 -m http.server 8080`), +- how to deploy — copy the directory's contents to any static host (Netlify, + Cloudflare Pages, GitHub Pages, an S3 bucket, nginx); URLs are root-relative so + the site works at any domain's root with no extra config. + +The export runs synchronously. Writebook runs no background jobs, and a +published-only export takes seconds. For very large libraries the result page +points you at the rake task below, to avoid holding a single web request open. + +### From the command line + +```sh +# Export published books to tmp/static-site: +bin/rails static:generate + +# Custom output directory: +bin/rails static:generate[path/to/output] + +# Absolute URLs are rewritten to this host, then made root-relative: +STATIC_HOST=books.example.com bin/rails static:generate + +# Include unpublished books too — the live DB is left untouched: +STATIC_ALL=1 bin/rails static:generate +``` + +`STATIC_ALL=1` temporarily flips `published: true` inside a rolled-back +`Book.transaction`, so unpublished drafts render but the live database is never +modified. + +## What's included + +- The library menu (`/`), every book (`/:id/:slug`), and every leaf + (`/:book_id/:book_slug/:id/:slug`). +- The library cards' bookmark overlay frames (`/books/:id/bookmark`), rendered + so the cards' click targets resolve on the static host instead of 404ing into + Turbo's "content missing" fallback. +- The front-matter **markdown** alternate routes (`….md`) the book and leaf + views declare via ``. +- Every precompiled asset in `public/assets/`, the root static files + (`favicon.svg`, `app-icon.png`, `robots.txt`, …), and the PWA manifest. +- Every referenced image blob — ActiveStorage covers/pictures + (`/rails/active_storage/…`) and in-body uploads (`/u/`). +- The per-book table-of-contents sidebar, externalized once per book and loaded + with a tiny inline fetch (see Design below). + +## What's NOT included + +- **Editing/auth machinery.** The read templates already suppress every editing + affordance when `book.editable?` is false (i.e. logged-out), so **no application + code is modified** — the same templates produce clean read-only HTML. The one + login affordance that remains in the logged-out view (the library's sign-in + link) is stripped from the output, since it has no destination on a static host. +- **Unpublished books**, by default. Use `STATIC_ALL=1`. + +## Design principles + +- **Render every route the read views reference — nothing more, nothing less.** + The exporter drives `ActionDispatch::Integration::Session`, so every read-view + helper (`arrangement_tag`, `leaf_nav_tag`, `link_to_next_leafable`, + `page.body.to_html`, …) runs in its real request context — no stubbing. Read + views are rendered logged-out, exactly as a visitor sees them. +- **No template changes.** All transforms are post-processing on rendered HTML. +- **Keep the reading UX faithful.** Turbo Drive, Turbo Frames, and the Stimulus + controllers (hotkey nav, lightbox, sidebar, reading-tracker, toc-view, …) ship + exactly as Writebook provides them. The only JS the exporter adds is the + one-line sidebar fetch. + +The two faithful post-processing additions: + +1. **Bookmark frames.** Rewrite each library card's + `` to a trailing slash and render + `/books/:id/bookmark/` to `books/:id/bookmark/index.html`, so the overlay + click target resolves. +2. **Externalize the sidebar.** The full book TOC is byte-identical across every + leaf. Writing it once per book (`/_sidebar.html`) and loading it with + a small inline `fetch("../../_sidebar.html")` turns an O(n²) export (every leaf + carries the whole TOC) into O(n). With JS off the sidebar nav is absent, but + the page body and prev/next still work. A 1,255-leaf book goes from ~390 MB to + ~39 MB. + +## Files + +| Path | Purpose | +| --- | --- | +| `lib/writebook/static_exporter.rb` | `Writebook::StaticExporter` — the renderer | +| `lib/tasks/static.rake` | `bin/rails static:generate` | +| `app/controllers/static_exports_controller.rb` | Admin UI action (`show` landing, `create` runs the export; `before_action :ensure_can_administer`) | +| `app/views/static_exports/show.html.erb` | Landing page with the Generate button | +| `app/views/static_exports/create.html.erb` | Result page with counts + next steps | +| `app/views/books/index.html.erb` | Admin download button in the library header | +| `config/routes.rb` | `resource :static_export` | +| `test/lib/writebook/static_exporter_test.rb` | Exporter tests | +| `test/controllers/static_exports_controller_test.rb` | UI controller tests | + +## Running the tests + +```sh +bin/rails test +``` + +All green: 171 runs, 597 assertions, 0 failures. + +## Credits + +Built on [basecamp/writebook](https://github.com/basecamp/writebook) +(MIT-licensed). This static-site-generator feature is intended as an upstream +contribution. \ No newline at end of file diff --git a/app/controllers/static_exports_controller.rb b/app/controllers/static_exports_controller.rb new file mode 100644 index 00000000..22d5b5ec --- /dev/null +++ b/app/controllers/static_exports_controller.rb @@ -0,0 +1,29 @@ +class StaticExportsController < ApplicationController + # Generating a static copy of the whole library is a site-wide admin action: + # it renders every published book and copies every asset and image blob into a + # directory on the server. Non-admins get 403; logged-out visitors are sent to + # sign in by the default authentication before_action. + before_action :ensure_can_administer + + # The landing page: explains what the export produces and offers the button + # that POSTs to #create to run it. + def show + end + + # Renders the published library to tmp/static-site (the same default the + # static:generate rake task uses) and shows the result with hosting steps. + # Synchronous -- Writebook runs no background jobs, and a published-only + # export takes seconds. Operators with very large libraries should use the + # rake task instead (noted on the result page) to avoid a request timeout. + def create + dir = Rails.root.join("tmp/static-site") + @result = Writebook::StaticExporter.new( + dir, + host: request.host, + protocol: request.ssl? ? "https" : "http" + ).call + @output_dir = dir + + render :create + end +end \ No newline at end of file diff --git a/app/views/books/index.html.erb b/app/views/books/index.html.erb index 51b55a9e..739ea749 100644 --- a/app/views/books/index.html.erb +++ b/app/views/books/index.html.erb @@ -11,6 +11,13 @@ <% if Current.user %> + <% if Current.user.can_administer? %> + <%= link_to static_export_path, class: "btn" do %> + <%= image_tag "download.svg", aria: { hidden: true }, size: 24 %> + Export to static site + <% end %> + <% end %> + <%= link_to users_path, class: "btn" do %> <%= image_tag "settings.svg", aria: { hidden: true }, size: 24 %> Manage people and settings diff --git a/app/views/static_exports/create.html.erb b/app/views/static_exports/create.html.erb new file mode 100644 index 00000000..e11b93f3 --- /dev/null +++ b/app/views/static_exports/create.html.erb @@ -0,0 +1,82 @@ +<% content_for(:title) { "Static site ready" } %> + +<% content_for :header do %> + +<% end %> + +
+

Your static site is ready

+ +

+ Rendered <%= pluralize(@result.books, "book") %>, + <%= pluralize(@result.leaves, "page") %>, + <%= pluralize(@result.assets, "asset file") %>, and + <%= pluralize(@result.resources, "image") %> + (<%= @result.bytes %>). + <% if @result.resource_failures.to_i.positive? %> + <%= @result.resource_failures %> image(s) could not be copied — see the server log. + <% end %> +

+ + <%= button_to static_export_path, method: :post, class: "btn btn--reversed center txt-medium" do %> + <%= image_tag "refresh.svg", aria: { hidden: true }, size: 24 %> + Generate again + <% end %> +
+ +
+
+

Where it is

+

+ On this server: <%= @output_dir %> +

+
+ +
+

Preview locally

+
cd <%= @output_dir %>
+python3 -m http.server 8080
+

Then open http://localhost:8080.

+
+ +
+

Deploy it

+

+ Copy the contents of that directory to any static host — Netlify, + Cloudflare Pages, GitHub Pages, an S3 bucket, or your own nginx. URLs are + root-relative, so the site works at any domain's root with no extra config. +

+
+ +
+

Including drafts, or very large libraries

+

+ Only published books are exported here. To include unpublished drafts, or to + export without holding a web request open, run + bin/rails static:generate STATIC_ALL=1 from the server shell — + it leaves your live database untouched. +

+
+
+ +<% content_for :footer do %> + +<% end %> \ No newline at end of file diff --git a/app/views/static_exports/show.html.erb b/app/views/static_exports/show.html.erb new file mode 100644 index 00000000..b32cb266 --- /dev/null +++ b/app/views/static_exports/show.html.erb @@ -0,0 +1,45 @@ +<% content_for(:title) { "Export to static site" } %> + +<% content_for :header do %> + +<% end %> + +
+

Export to a static site

+ +

+ Generate a self-contained HTML version of your published library — the menu of + books, every book's table of contents, and every page — with all CSS, JavaScript, + and image files copied in. No Rails process, login, or editing machinery: the + result can be hosted anywhere. +

+ + <%= button_to static_export_path, method: :post, class: "btn btn--reversed center txt-medium" do %> + <%= image_tag "download.svg", aria: { hidden: true }, size: 24 %> + Generate static site + <% end %> +
+ +
+

Only published books are included.

+

+ To include unpublished drafts as well, run + bin/rails static:generate STATIC_ALL=1 from the server shell — it + leaves your live database untouched. +

+

+ For very large libraries, prefer that rake task too: it avoids holding a single + web request open while the whole library renders. +

+
\ No newline at end of file diff --git a/config/routes.rb b/config/routes.rb index 6a759da6..b62dc674 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -19,6 +19,11 @@ end end + # Generate a self-contained static HTML version of the published library. + # Admin-only (see StaticExportsController); the result page tells the + # operator where the files are and how to host them. + resource :static_export, only: %i[ show create ], controller: "static_exports" + resources :books, except: %i[ index show ] do resource :publication, controller: "books/publications", only: %i[ show edit update ] resource :bookmark, controller: "books/bookmarks", only: :show diff --git a/lib/tasks/static.rake b/lib/tasks/static.rake new file mode 100644 index 00000000..dea5451a --- /dev/null +++ b/lib/tasks/static.rake @@ -0,0 +1,20 @@ +desc "Generate a static HTML site from published books into DIR (default: tmp/static-site). " \ + "Set STATIC_HOST for absolute URLs; STATIC_ALL=1 to include unpublished books." +namespace :static do + task :generate, [:dir] => :environment do |_task, args| + dir = args[:dir] || "tmp/static-site" + host = ENV.fetch("STATIC_HOST", "example.com") + + if ENV["STATIC_ALL"] == "1" + # Temporarily publish every book so the exporter renders it, then roll the + # change back so the live database is left untouched. + Book.transaction do + Book.where(published: [false, nil]).update_all(published: true) + Writebook::StaticExporter.new(dir, host: host, verbose: true).call + raise ActiveRecord::Rollback + end + else + Writebook::StaticExporter.new(dir, host: host, verbose: true).call + end + end +end \ No newline at end of file diff --git a/lib/writebook/static_exporter.rb b/lib/writebook/static_exporter.rb new file mode 100644 index 00000000..88f0d37a --- /dev/null +++ b/lib/writebook/static_exporter.rb @@ -0,0 +1,319 @@ +require "uri" + +module Writebook + # Renders Writebook's read-only views -- the library menu, each book's table of + # contents, and every leaf -- to a static directory, along with every asset and + # image blob those pages reference. The result can be served by any static web + # server with no Rails process, login, or editing machinery. + # + # Books are rendered exactly as a logged-out visitor sees them. The read + # templates already suppress every editing affordance when +book.editable?+ is + # false, so no application code is modified. The single login affordance that + # remains in that view -- the library's sign-in button -- is stripped from the + # output, since it has no destination on a static host. Absolute URLs to the + # export host are rewritten to be root-relative so the site is self-contained. + # + # Resources (compiled assets and image blobs) are resolved two ways: + # * In a deployed instance, +public/assets+ already holds the precompiled + # asset files, so that directory is mirrored directly. + # * Anything referenced by the rendered pages that the mirror did not + # already provide -- e.g. assets served live by Propshaft in development, + # ActiveStorage covers/pictures, and in-body uploads -- is fetched through + # the same integration session that rendered the HTML, so the bytes are + # always whatever a visitor's browser would actually receive. + class StaticExporter + STATIC_ROOT_FILES = %w[favicon.svg favicon.png app-icon.png app-icon-192.png robots.txt].freeze + PWA_PATHS = %w[/manifest.json /manifest].freeze + + # Matches asset, upload, and blob URLs *inside a quoted attribute* (src, href, + # data-lightbox-url-value, or the og:image content=), with an optional + # absolute host prefix so covers are caught too. Bounding the match to the + # closing quote keeps the scan from over-running into adjacent HTML when a + # URL sits in unquoted text; only the path (group 1) is captured. + RESOURCE_URL_PATTERN = %r{(?:src|href|data-lightbox-url-value|content)\s*=\s*["'](?:https?://[^/]+)?(/(?:assets/[^"']+|u/[^"']+|rails/active_storage/[^"']+))["']} + + # elements pointing at session/user/account or edit paths -- dead links + # on a static host. Only the library's sign-in button appears in the + # logged-out read views. + AUTH_LINK_PATTERN = /]*href="(?:\/(?:session|users|account)|[^"]*\/edit)[^"]*"[^>]*>.*?<\/a>/m + + # The table-of-contents sidebar embedded in every leaf page. It is + # byte-identical across every leaf of a book, so it is externalized once + # per book and loaded with a small inline fetch (see +externalize_sidebar+). + # No \b word boundaries: a \b immediately after a double quote never + # matches, since both sides are non-word characters. + SIDEBAR_ASIDE_PATTERN = /]*?id="sidebar"[^>]*>.*?<\/aside>/m + + # The inline script that swaps the placeholder