Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/explorations/2026-07-05-privacy-analytics-cloudflare.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,10 @@ beacon script 即可,最不破坏隐私定位。

## 验证(chrome-devtools + 构建产物)

| 构建 | 预期 | 实测 |
| --- | --- | --- |
| 无 token(默认) | 不注入 | ✅ bundle 里零 `cloudflareinsights` 引用(死代码消除) |
| 有 token · 顶层 | 注入 beacon | ✅ `data-cf-beacon={"token":...}` 正确 |
| 有 token · `?embed=1` | 不注入 | ✅ beacon 缺席,`embed-mode` class 存在 |
| 构建 | 预期 | 实测 |
| --------------------- | ----------- | ------------------------------------------------------ |
| 无 token(默认) | 不注入 | ✅ bundle 里零 `cloudflareinsights` 引用(死代码消除) |
| 有 token · 顶层 | 注入 beacon | ✅ `data-cf-beacon={"token":...}` 正确 |
| 有 token · `?embed=1` | 不注入 | ✅ beacon 缺席,`embed-mode` class 存在 |

`lint:ts` ✅ / prettier ✅ / `build` ✅。
10 changes: 5 additions & 5 deletions docs/explorations/2026-07-05-sw-stale-build-auto-reload.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ const hadController = !!navigator.serviceWorker.controller;
let reloadingForUpdate = false;
navigator.serviceWorker.addEventListener('controllerchange', () => {
if (!hadController || reloadingForUpdate) return; // 1. 仅更新(非首装);2. 只 reload 一次
if (getDocmentObj().fileName) return; // 3. 有文档打开时不 reload,避免丢失编辑
if (getDocmentObj().fileName) return; // 3. 有文档打开时不 reload,避免丢失编辑
reloadingForUpdate = true;
window.location.reload();
});
Expand All @@ -63,11 +63,11 @@ navigator.serviceWorker.addEventListener('controllerchange', () => {

在 `vite preview`(dist)上:

| 场景 | 预期 | 实测 |
| --- | --- | --- |
| 首次安装(无旧控制器) | 不 reload | ✅ 载入计数停在 1 |
| 场景 | 预期 | 实测 |
| ------------------------------------------------------------ | ---------------- | ----------------------------------------------------- |
| 首次安装(无旧控制器) | 不 reload | ✅ 载入计数停在 1 |
| 有旧控制器 + 未开文档(模拟新部署,合成 `controllerchange`) | 自动 reload 一次 | ✅ window marker 丢失、sessionStorage 保留 = 同页刷新 |
| 文档已打开时 `controllerchange` | 不 reload | ✅ marker 仍在、编辑器 iframe 完好 |
| 文档已打开时 `controllerchange` | 不 reload | ✅ marker 仍在、编辑器 iframe 完好 |

`pnpm run lint:ts` 通过;`sw-routing.test.ts` 31/31 通过;`pnpm run build` 成功。

Expand Down
55 changes: 55 additions & 0 deletions docs/explorations/2026-07-05-x2t-wasm-gzip-cf-pages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# x2t WASM 走 gzip 客户端解压 —— 解除 Cloudflare Pages 迁移的唯一硬阻塞

> 2026-07-05

## 背景

document 项目计划从 `ranuts.github.io/document` 迁到 `edit.chaxus.com`(Cloudflare Pages)。
唯一的托管硬阻塞:**CF Pages 单文件上限 25 MiB**,而 `public/wasm/x2t/x2t.wasm` = **55 M**,部署时会被直接拒收(dist 文件数 336,远低于 2 万上限,无其它阻塞)。

目标:让浏览器仍拿到 55M 解压后的 wasm,但**部署产物里不再有超限的裸文件**。

## 方案

`x2t.js`(Emscripten 胶水)默认 `fetch('x2t.wasm')` + `instantiateStreaming`。但它:

1. **复用已存在的全局 `Module`**(`if (!Module) Module = ...`,并 `moduleOverrides = Object.assign({}, Module)` 回填)。
2. 第 284 行 `if (Module['wasmBinary']) wasmBinary = Module['wasmBinary']` —— 一旦 `Module.wasmBinary` 有值,`getBinarySync` 直接返回它、`instantiateAsync` 跳过 streaming fetch,**根本不请求 `x2t.wasm`**。

于是在 `@ranuts/converter` 的 `loadScript()` 里、注入 x2t.js **之前**新增 `prepareWasmBinary()`:

- `fetch(x2t.wasm.gz)`(11M,under 25MiB)
- 用浏览器原生 **`DecompressionStream('gzip')`** 解压(无新依赖)
- `window.Module = { ...window.Module, wasmBinary }`

部署产物只保留 `x2t.wasm.gz`(11M) + `x2t.js`,删掉 `x2t.wasm`(55M)与 `x2t.wasm.br`(8M,brotli 需额外解码器,不用)。

## 关键坑:服务器对 `.gz` 的处理不一致

一开始直接 `DecompressionStream('gzip')`,在 `vite preview` 下报 `Z_DATA_ERROR: incorrect header check`。

原因:**vite 的 dev/preview 服务器识别 `.gz` 后缀,给响应加 `Content-Encoding: gzip`**(实测 `Content-Type: application/wasm` + `Content-Encoding: gzip`),浏览器/fetch 层已透明解压了一次,我再手动解压就是对已解压数据二次解压。

而静态托管(CF Pages / GitHub Pages)通常发**原始 gzip 字节**(不加 Content-Encoding)。两种行为都要兼容。

**解法:按 magic bytes 探测**——收到的首字节是 `1f 8b`(gzip)才手动解压;是 `00 61 73 6d`(`\0asm`,已是裸 wasm)就直接用。对任何服务器都正确。

## 验证(端到端)

| 环节 | 方法 | 结果 |
| ----------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| isGzip=false 分支(preview 自动解压) | Node fetch preview 的 .gz → 探测 → compile | 54.7M,magic OK,`WebAssembly.compile` OK(28 exports) |
| isGzip=true 分支(静态托管原始 gzip) | Node 直接读磁盘 .gz 原始字节 → 手动解压 → compile | 54.7M,compile OK |
| 浏览器集成 | Playwright:注入 binary → 加载真实 x2t.js → 等 `onRuntimeInitialized` + 监听网络 | **运行时初始化 OK;FS/ccall 可用;对 `x2t.wasm` 请求 0 次(只请求 `x2t.wasm.gz`)** |
| 构建产物 | `find dist -size +25M` | **空**(最大文件 = x2t.wasm.gz 11M) |
| 回归 | `pnpm run lint:ts` / `pnpm run test` | 通过 / 19 files · 240 tests 全过 |

## 结论 & 影响

- **迁 CF Pages 的唯一硬阻塞已解除**:dist 无 >25MiB 文件。
- 改动集中在 `@ranuts/converter`(`prepareWasmBinary`)+ 删两个大文件,主 app 代码零改动。
- 对现有 GitHub Pages 部署也安全(magic 探测 + 现存 .gz)。

## 遗留(不阻塞 CF 迁移)

- `build:single`(单 HTML,`bin/bundle_single_html.js`)靠内联裸 wasm,删裸文件后该 target 的 wasm 需改为内联 `.gz` + 同款解压逻辑。非部署链,单列跟进。
53 changes: 52 additions & 1 deletion packages/converter/src/document-converter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,22 @@ export class X2TConverter {

private readonly WORKING_DIRS = ['/working', '/working/media', '/working/fonts', '/working/themes'];
private readonly SCRIPT_PATH = `${BASE_PATH}wasm/x2t/x2t.js`;
private readonly WASM_GZ_PATH = `${BASE_PATH}wasm/x2t/x2t.wasm.gz`;
private readonly INIT_TIMEOUT = 300000;

/**
* Load X2T script file (using ranuts scriptOnLoad utility)
* Load X2T script file (using ranuts scriptOnLoad utility).
*
* We first decompress the gzipped WASM and hand the bytes to Emscripten via
* `Module.wasmBinary` (see prepareWasmBinary), so x2t.js never fetches the raw
* 55 MB `x2t.wasm` itself. This lets us ship only the ~11 MB `x2t.wasm.gz`,
* staying under Cloudflare Pages' 25 MiB-per-file deploy limit.
*/
async loadScript(): Promise<void> {
if (this.hasScriptLoaded) return;

try {
await this.prepareWasmBinary();
// scriptOnLoad accepts an array of URLs
await scriptOnLoad([this.SCRIPT_PATH]);
this.hasScriptLoaded = true;
Expand All @@ -56,6 +63,50 @@ export class X2TConverter {
}
}

/**
* Fetch the gzipped x2t WASM, decompress it in the browser, and stash the raw
* bytes on `window.Module.wasmBinary` *before* x2t.js runs. Emscripten checks
* `if (Module['wasmBinary']) wasmBinary = Module['wasmBinary']` and then skips
* its own fetch/instantiateStreaming of `x2t.wasm` entirely.
*
* Uses the native `DecompressionStream('gzip')` — no extra dependency.
*
* Servers disagree on how they serve a `.gz` file: some (e.g. Vite's dev /
* preview server) send it with `Content-Encoding: gzip`, so the browser has
* already transparently decompressed it by the time we read the body; others
* (static hosts like Cloudflare Pages / GitHub Pages) serve the raw gzip
* bytes. We detect which by the leading magic bytes and only decompress when
* the payload is still gzip (`1f 8b`) rather than an already-raw wasm module
* (`00 61 73 6d`). This keeps it correct on every host.
*/
private async prepareWasmBinary(): Promise<void> {
const globalScope = window as unknown as {
Module?: Record<string, unknown> & { wasmBinary?: ArrayBuffer };
};
if (globalScope.Module?.wasmBinary) return; // already prepared

const response = await fetch(this.WASM_GZ_PATH);
if (!response.ok) {
throw new Error(`Failed to fetch x2t WASM at '${this.WASM_GZ_PATH}' (${response.status})`);
}
const raw = await response.arrayBuffer();
const head = new Uint8Array(raw, 0, Math.min(2, raw.byteLength));
const isGzip = head[0] === 0x1f && head[1] === 0x8b;

let wasmBinary: ArrayBuffer;
if (isGzip) {
const stream = new Response(raw).body!.pipeThrough(new DecompressionStream('gzip'));
wasmBinary = await new Response(stream).arrayBuffer();
} else {
// Already decompressed by the browser via Content-Encoding.
wasmBinary = raw;
}

// Pre-seed the global Module so x2t.js (which reuses an existing global
// Module) picks up the binary. Preserve any properties already set.
globalScope.Module = { ...globalScope.Module, wasmBinary };
}

/**
* Initialize X2T module
*/
Expand Down
Binary file removed public/wasm/x2t/x2t.wasm
Binary file not shown.
Binary file removed public/wasm/x2t/x2t.wasm.br
Binary file not shown.