diff --git a/.claude/rules/platform-and-vfs.md b/.claude/rules/platform-and-vfs.md index 03945b3d..fb20408e 100644 --- a/.claude/rules/platform-and-vfs.md +++ b/.claude/rules/platform-and-vfs.md @@ -24,6 +24,7 @@ paths: - Bytecode generation requires the target architecture binary. - Use `--no-bytecode` for cross-arch builds. - Linux: QEMU for emulation. macOS: Rosetta 2 for arm64 building x64. +- Windows: Wine for `win-x64` bytecode from Linux — opt-in via `--cross-bytecode` (same CPU arch only; needs a `binfmt_misc` MZ→Wine handler). ## ESM diff --git a/README.md b/README.md index 19b456de..5d32c2d6 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,8 @@ `pkg` takes your Node.js project and ships it as a single binary that runs on devices without Node.js installed. Cross-compile for Linux, macOS, and Windows from any host. +> Building a `win-x64` binary on a non-Windows host **with bytecode** (source protection) runs the Windows Node under Wine — install Wine, register a `binfmt_misc` `MZ` handler, and pass `--cross-bytecode`. See [Building Windows binaries on Linux (Wine)](https://yao-pkg.github.io/pkg/guide/targets#building-windows-binaries-on-linux-wine). + ## Install ```sh diff --git a/docs-site/guide/bytecode.md b/docs-site/guide/bytecode.md index e0fcfc1e..439a3559 100644 --- a/docs-site/guide/bytecode.md +++ b/docs-site/guide/bytecode.md @@ -34,6 +34,8 @@ Bytecode generation requires running the **target** architecture's Node.js to co pkg --no-bytecode --public-packages "*" --public -t node22-linux-arm64 index.js ``` +For a **Windows** target on Linux the analogous setup runs the target Node under Wine. To **keep** bytecode (and source protection) for `win-x64` instead of disabling it, build with `--cross-bytecode` — see [Targets → Building Windows binaries on Linux (Wine)](/guide/targets#building-windows-binaries-on-linux-wine). + See [Targets → Cross-compilation support](/guide/targets#cross-compilation-support). ## Licenses and `--public-packages` diff --git a/docs-site/guide/recipes.md b/docs-site/guide/recipes.md index b183030f..769a0756 100644 --- a/docs-site/guide/recipes.md +++ b/docs-site/guide/recipes.md @@ -169,6 +169,14 @@ pkg --no-bytecode --public-packages '*' --public -t node22-linux-arm64 . Skips the bytecode step, so there's no need to run an arm64 interpreter on your x64 host. Trade-off: source is plaintext in the binary. See [Targets → Cross-compilation support](/guide/targets#cross-compilation-support). +## Cross-compile to Windows x64 on Linux with Wine + +```sh +pkg --cross-bytecode -t node22-win-x64 . +``` + +Runs the Windows target Node under Wine to fabricate Windows-native bytecode, so the `.exe` keeps source protection (no plaintext sources) — unlike `--no-bytecode`. Requires Wine and a `binfmt_misc` `MZ` handler registered from a privileged container or the host kernel. See [Targets → Building Windows binaries on Linux (Wine)](/guide/targets#building-windows-binaries-on-linux-wine). + ## Exclude test and doc directories from dependencies ```json diff --git a/docs-site/guide/sea-vs-standard.md b/docs-site/guide/sea-vs-standard.md index 0c1eb2fa..f92ecb01 100644 --- a/docs-site/guide/sea-vs-standard.md +++ b/docs-site/guide/sea-vs-standard.md @@ -25,19 +25,19 @@ Everything else — bytecode, worker threads, native addons, bundling strategy ## Feature matrix -| Feature | **Standard** | **Enhanced SEA** | -| ---------------------------------- | ---------------------------------------------------------------------- | ------------------------ | -| **Node.js binary** | Custom patched (`pkg-fetch`) | **Stock Node.js** ✨ | -| Source protection (V8 bytecode) | ✅ | ❌ plaintext | -| Compression (Brotli / GZip / Zstd) | ✅ | ✅ | -| Build speed | Slower | **Faster** | -| Cross-compile | ⚠️ broken on Node 22 ([see](/guide/targets#cross-compilation-support)) | ✅ | -| Worker threads | ✅ | ✅ | -| Native addons | ✅ | ✅ | -| ESM + top-level await | Partial | ✅ every target | -| Maintenance burden | High — patch each Node release | **Low — stock binaries** | -| Security updates | Wait for `pkg-fetch` rebuild | **Immediate** | -| Future path | Tied to `pkg-fetch` | Migrates to `node:vfs` | +| Feature | **Standard** | **Enhanced SEA** | +| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------ | +| **Node.js binary** | Custom patched (`pkg-fetch`) | **Stock Node.js** ✨ | +| Source protection (V8 bytecode) | ✅ | ❌ plaintext | +| Compression (Brotli / GZip / Zstd) | ✅ | ✅ | +| Build speed | Slower | **Faster** | +| Cross-compile | ⚠️ Node 22 regression; `win-x64` keeps bytecode via Wine + `--cross-bytecode` ([see](/guide/targets#cross-compilation-support)) | ✅ | +| Worker threads | ✅ | ✅ | +| Native addons | ✅ | ✅ | +| ESM + top-level await | Partial | ✅ every target | +| Maintenance burden | High — patch each Node release | **Low — stock binaries** | +| Security updates | Wait for `pkg-fetch` rebuild | **Immediate** | +| Future path | Tied to `pkg-fetch` | Migrates to `node:vfs` | ## When to pick which diff --git a/docs-site/guide/targets.md b/docs-site/guide/targets.md index ee3cce10..a3411277 100644 --- a/docs-site/guide/targets.md +++ b/docs-site/guide/targets.md @@ -79,12 +79,13 @@ On Node 22, Standard cross-compile **builds cleanly but produces a broken execut - `linux-arm64` crashes with `Error: UNEXPECTED-20` in `readFileFromSnapshot` - `win-x64` exits silently with no stdout (EXIT=4) -Tracked in [#87](https://github.com/yao-pkg/pkg/issues/87) and [#181](https://github.com/yao-pkg/pkg/issues/181). Three workarounds: +Tracked in [#87](https://github.com/yao-pkg/pkg/issues/87) and [#181](https://github.com/yao-pkg/pkg/issues/181). Workarounds: 1. **Switch to SEA** — `pkg . --sea`. Avoids the V8 bytecode step entirely. 2. **Disable bytecode** — `pkg . --no-bytecode --public-packages "*" --public`. Keeps Standard mode, stores source as plaintext. 3. **Fallback to source** — `pkg . --fallback-to-source`. Keeps bytecode for files that compile successfully and ships the rest as plain source. See [Bytecode → Fallback to source](/guide/bytecode#fallback-to-source-on-failure). 4. **Target Node 24** — the regression is gone on `node24-*` targets. +5. **Keep bytecode for `win-x64`** — build on Linux with Wine and pass `--cross-bytecode`. The only workaround that preserves source protection for Windows targets. See [Building Windows binaries on Linux (Wine)](#building-windows-binaries-on-linux-wine) below. ::: @@ -100,6 +101,35 @@ Regardless of the bug above, the V8 bytecode fabricator in Standard mode needs t Enhanced SEA doesn't have this limitation when the host and target share the same Node major: pkg uses `process.execPath` to generate the SEA blob, so no target-arch interpreter is needed. Cross-major SEA builds (e.g. building `node22-*` targets on a Node 24 host) still require an interpreter for the downloaded target binary. +### Building Windows binaries on Linux (Wine) + +When a `win-x64` target's bytecode is fabricated with the **host** (Linux) Node, the target's Windows V8 can reject it at runtime — producing an executable that fails to start (this is the failure mode behind the Node 22 regression above). Passing `--cross-bytecode` runs the **Windows** target Node under [Wine](https://www.winehq.org/) to generate Windows-native bytecode — the OS analogue of using QEMU for a foreign arch. + +Wine is an **OS-ABI translation layer**, not a CPU emulator: because `win-x64` is the same CPU arch as an x64 Linux host, there is **no CPU emulation** and fabrication runs at near-native speed. This path is **`x64` host → `win-x64` only**; `win-arm64` from an x64 host would also need CPU emulation and is not supported. + +Setup (Debian/Ubuntu shown; adapt for your distro): + +```sh +# 1. Install Wine (same CPU arch as the host) +apt-get update && apt-get install -y --no-install-recommends wine wine64 + +# 2. Register a binfmt_misc handler so the kernel runs .exe files through Wine. +# pkg forwards the Wine environment (WINEPREFIX, HOME, PATH, …) to the +# fabricator, so a plain handler with no wrapper script is enough: +echo ':winePE:M::MZ::/usr/bin/wine:' > /proc/sys/fs/binfmt_misc/register + +# 3. Build, opting in with --cross-bytecode +pkg --cross-bytecode -t node22-win-x64 index.js +``` + +::: warning Privileged context required to register binfmt +Registering a `binfmt_misc` handler writes to `/proc/sys/fs/binfmt_misc` and needs a **privileged** context — a rootful `docker run --privileged` / `docker:dind`, or the host kernel. **Rootless** containers cannot mount or write `binfmt_misc`; register the handler on the host instead. It is a kernel-wide setting, so it only needs registering once per host/boot. +::: + +`--cross-bytecode` is **off by default** — without it, `win-x64` builds on Linux behave exactly as before. It can also be set as `crossBytecode` in the pkg config. If Wine or the binfmt handler is missing, the build **fails with an error** pointing back here rather than silently producing a broken binary. + +To verify end to end, run the produced `.exe` on Windows: it should start with no `V8 rejected the bytecode cache` error, and the app code stays shipped as bytecode (no plaintext sources). + ## macOS arm64 `macos-arm64` is experimental. Be careful about the [mandatory code signing requirement](https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-universal-apps-release-notes). The final executable has to be signed (an ad-hoc signature is sufficient) with the `codesign` utility on macOS (or the `ldid` utility on Linux). Otherwise the executable will be killed by the kernel and the end user has no way to permit it to run. `pkg` tries to ad-hoc sign the final executable. If necessary, replace this signature with your own trusted Apple Developer ID. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 7f697731..ef5ff0a1 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -132,6 +132,14 @@ CLI (lib/index.ts) └─ Write final executable ``` +### Fabricator (cross-arch and cross-OS bytecode) + +V8 cached-data bytecode must be produced by a Node/V8 that matches the **target**, so `lib/fabricator.ts` runs a _fabricator_ — a Node binary chosen by `fabricatorForTarget()` in `lib/index.ts`: + +- **Native** — same platform and arch as the host. A persistent child is spawned once per `[cmd, bakes]` and reused across every file; the snap/body and the resulting cached data are exchanged over inherited **stdin/stdout pipes**. +- **Cross-arch (QEMU)** — a different CPU arch on a Linux/Alpine host selects the `linuxstatic` fabricator, executed under QEMU via `binfmt_misc`. QEMU passes stdio through, so it keeps the pipe path. +- **Cross-OS (Wine)** — opt-in via `--cross-bytecode`, a same-arch `win` target on a Linux/Alpine host selects the **`win`** fabricator, executed under Wine via a `binfmt_misc` `MZ` handler. A Windows Node under Wine cannot expose inherited Unix pipes as Windows stdio handles, so this case alone uses a **file-based IPC** transport (`fabricateViaFile`): the snap/body and the cached data travel through temp files (paths translated to Wine's `Z:` drive) and the Wine environment is forwarded into the spawn. The `vm.Script(…, { produceCachedData: true, sourceless: true })` call is byte-for-byte identical to the pipe path, so the produced bytecode is the same. Wine is an OS-ABI layer, not a CPU emulator — `x64` host → `win-x64` only. + ### Binary Format The traditional executable has this layout: @@ -522,14 +530,14 @@ SIZE_LIMIT_PKG=1048576 DEBUG_PKG=1 ./my-packaged-app # flag files > 1MB ## Performance Comparison -| Aspect | Traditional `pkg` | Enhanced SEA | -| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Startup time** | V8 bytecode loads faster than parsing source — bytecode is pre-compiled. `vm.Script` with `cachedData` skips the parsing phase | `useCodeCache: true` provides similar optimization. Without it, every launch re-parses source from scratch | -| **Memory footprint** | Payload accessed via file descriptor reads on demand at computed offsets. Files loaded only when accessed | `sea.getRawAsset('__pkg_archive__')` loads the entire archive as a zero-copy `ArrayBuffer`. Individual files are extracted via `Buffer.subarray()` and cached in a `Map` on first access | -| **Executable size** | Brotli/GZip/Zstd compression reduces payload by 60-80%. Dictionary path compression adds 5-15% reduction | Per-file Brotli/GZip/Zstd compression (opt-in via `--compress`). Decompression is lazy — only files actually read at runtime pay the cost. Uncompressed by default | -| **Build time** | V8 bytecode compilation spawns a Node.js process per file via fabricator. Cross-arch bytecode needs QEMU/Rosetta. Expensive for large projects | No bytecode step. Pipeline: walk deps, write assets, generate blob, inject. Significantly faster | -| **Module loading** | Custom `require` implementation in bootstrap. Each module loaded from VFS via binary offset reads. Synchronous only | VFS polyfill patches `require`/`import` at module resolution level. 164+ fs functions intercepted. ESM module hooks supported natively | -| **Native addons** | Extracted to `~/.cache/pkg//` on first load, SHA256-verified, persisted across runs | Same extraction strategy via shared `patchDlopen()`. Uses `fs.cpSync` for package folder copying | +| Aspect | Traditional `pkg` | Enhanced SEA | +| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Startup time** | V8 bytecode loads faster than parsing source — bytecode is pre-compiled. `vm.Script` with `cachedData` skips the parsing phase | `useCodeCache: true` provides similar optimization. Without it, every launch re-parses source from scratch | +| **Memory footprint** | Payload accessed via file descriptor reads on demand at computed offsets. Files loaded only when accessed | `sea.getRawAsset('__pkg_archive__')` loads the entire archive as a zero-copy `ArrayBuffer`. Individual files are extracted via `Buffer.subarray()` and cached in a `Map` on first access | +| **Executable size** | Brotli/GZip/Zstd compression reduces payload by 60-80%. Dictionary path compression adds 5-15% reduction | Per-file Brotli/GZip/Zstd compression (opt-in via `--compress`). Decompression is lazy — only files actually read at runtime pay the cost. Uncompressed by default | +| **Build time** | V8 bytecode compilation spawns a Node.js process per file via fabricator. Cross-arch bytecode needs QEMU/Rosetta; cross-OS `win-x64` bytecode on Linux needs Wine (`--cross-bytecode`). Expensive for large projects | No bytecode step. Pipeline: walk deps, write assets, generate blob, inject. Significantly faster | +| **Module loading** | Custom `require` implementation in bootstrap. Each module loaded from VFS via binary offset reads. Synchronous only | VFS polyfill patches `require`/`import` at module resolution level. 164+ fs functions intercepted. ESM module hooks supported natively | +| **Native addons** | Extracted to `~/.cache/pkg//` on first load, SHA256-verified, persisted across runs | Same extraction strategy via shared `patchDlopen()`. Uses `fs.cpSync` for package folder copying | ### Note on `--no-bytecode` diff --git a/lib/config.ts b/lib/config.ts index 716febe0..982e5732 100644 --- a/lib/config.ts +++ b/lib/config.ts @@ -165,6 +165,13 @@ const FLAG_SPECS: readonly FlagSpec[] = [ resolved: 'noDictionary', kind: 'list', }, + { + cli: 'cross-bytecode', + cfg: 'crossBytecode', + resolved: 'crossBytecode', + kind: 'bool', + default: false, + }, ]; /** Programmatic option key for a flag (defaults to the config key). */ @@ -488,6 +495,7 @@ export interface ResolvedFlags { publicPackages: string[] | undefined; noDictionary: string[] | undefined; bakeOptions: string[] | undefined; + crossBytecode: boolean; } /** Narrow an arbitrary value to `string | string[] | undefined` or `undefined`. */ diff --git a/lib/fabricator.ts b/lib/fabricator.ts index 4549ff80..46982e6f 100644 --- a/lib/fabricator.ts +++ b/lib/fabricator.ts @@ -1,8 +1,15 @@ -import { spawn, ChildProcessByStdio } from 'child_process'; +import { spawn, spawnSync, ChildProcessByStdio } from 'child_process'; +import { readFileSync, writeFileSync, unlinkSync } from 'fs'; +import { tmpdir } from 'os'; +import { join } from 'path'; +import { randomBytes } from 'crypto'; import { Readable, Writable } from 'stream'; +import { system } from '@yao-pkg/pkg-fetch'; import { log } from './log'; import { Target } from './types'; +const { hostPlatform } = system; + const script = ` var vm = require('vm'); var module = require('module'); @@ -41,26 +48,218 @@ const script = ` process.stdin.resume(); `; +// Same compile as `script` above, but the IPC is done through files instead of +// inherited stdin/stdout pipes. This is required for a cross-OS fabricator run +// under an ABI layer (a Windows Node under Wine) which cannot expose inherited +// Unix pipes as Windows stdio handles. Paths arrive via env vars so the child +// never touches process.std* — under Wine those throw `EBADF`. The vm.Script +// call must stay byte-for-byte identical to `script` so the produced bytecode +// is the same. +const fileScript = ` + var vm = require('vm'); + var fs = require('fs'); + var module = require('module'); + var inPath = process.env.PKG_FAB_IN; + var outPath = process.env.PKG_FAB_OUT; + var errPath = process.env.PKG_FAB_ERR; + try { + var stdin = fs.readFileSync(inPath); + var sizeOfSnap = stdin.readInt32LE(0); + var snap = stdin.toString('utf8', 4, 4 + sizeOfSnap); + var sizeOfBody = stdin.readInt32LE(4 + sizeOfSnap); + var startOfBody = 4 + sizeOfSnap + 4; + var body = Buffer.alloc(sizeOfBody); + stdin.copy(body, 0, startOfBody, startOfBody + sizeOfBody); + var code = module.wrap(body); + var s = new vm.Script(code, { + filename: snap, + produceCachedData: true, + sourceless: true + }); + if (!s.cachedDataProduced) { + fs.writeFileSync(errPath, 'Pkg: Cached data not produced.'); + process.exit(2); + } + var b = s.cachedData; + var h = Buffer.alloc(4); + h.writeInt32LE(b.length, 0); + fs.writeFileSync(outPath, Buffer.concat([ h, b ])); + } catch (err) { + try { fs.writeFileSync(errPath, String((err && err.stack) || err)); } catch (e) {} + process.exit(1); + } +`; + const children: Record< string, ChildProcessByStdio > = {}; -export function fabricate( +// Bakes that don't influence the produced bytecode and so are dropped before +// the fabricator is spawned (keeps the persistent-child key stable too). +function bytecodeBakes(bakes: string[]) { + return bakes.filter((bake) => { + const bake2 = bake.replace(/_/g, '-'); + + return !['--prof', '--v8-options', '--trace-opt', '--trace-deopt'].includes( + bake2, + ); + }); +} + +// True when the fabricator is a Windows binary executed on a non-Windows host, +// i.e. run under Wine via a binfmt_misc MZ handler. The cross-arch (QEMU) +// `linuxstatic` fabricator is NOT included: QEMU user emulation passes stdio +// through fine, so it keeps the faster persistent-pipe path below. +function runsUnderWine(fabricator: Target) { + return fabricator.platform === 'win' && hostPlatform !== 'win'; +} + +let fileIpcCounter = 0; + +function crossFabricatorError( + fabricator: Target, + snap: string, + wine: boolean, + detail: string, +): Error { + const base = `Failed to make bytecode ${fabricator.nodeRange}-${fabricator.arch} for file ${snap}`; + const hint = wine + ? ' — building a Windows target on this host runs the target Node under Wine; ' + + 'ensure Wine and a binfmt_misc MZ handler are configured (see the cross-compile guide)' + : ''; + return new Error(`${base} (${detail})${hint}`); +} + +// File-based fabrication. Spawns the fabricator once per file (no persistent +// child) and exchanges the snap/body and the resulting cached data through +// temp files. Exported so the file protocol can be exercised in tests with the +// host Node as the fabricator, independently of Wine. +export function fabricateViaFile( bakes: string[], fabricator: Target, snap: string, body: Buffer, cb: (error?: Error, buffer?: Buffer) => void, ) { - const activeBakes = bakes.filter((bake) => { - // list of bakes that don't influence the bytecode - const bake2 = bake.replace(/_/g, '-'); + const activeBakes = bytecodeBakes(bakes); + const wine = runsUnderWine(fabricator); - return !['--prof', '--v8-options', '--trace-opt', '--trace-deopt'].includes( - bake2, + const uniq = `${process.pid}-${(fileIpcCounter += 1)}-${randomBytes( + 6, + ).toString('hex')}`; + const dir = tmpdir(); + const inPath = join(dir, `pkg-fab-${uniq}.in`); + const outPath = join(dir, `pkg-fab-${uniq}.out`); + const errPath = join(dir, `pkg-fab-${uniq}.err`); + + // Wine maps the unix filesystem root onto its `Z:` drive, so a win fabricator + // sees Windows-style paths while we read/write the same files via unix paths. + const childPath = (p: string) => (wine ? `Z:${p.replace(/\//g, '\\')}` : p); + + const cleanup = () => { + for (const p of [inPath, outPath, errPath]) { + try { + unlinkSync(p); + } catch { + /* best-effort */ + } + } + }; + + try { + // [int32 snapLen][snap][int32 bodyLen][body] + const snapBuf = Buffer.from(snap); + const head1 = Buffer.alloc(4); + head1.writeInt32LE(snapBuf.length, 0); + const head2 = Buffer.alloc(4); + head2.writeInt32LE(body.length, 0); + writeFileSync(inPath, Buffer.concat([head1, snapBuf, head2, body])); + + const env: NodeJS.ProcessEnv = { + PKG_EXECPATH: 'PKG_INVOKE_NODEJS', + PKG_FAB_IN: childPath(inPath), + PKG_FAB_OUT: childPath(outPath), + PKG_FAB_ERR: childPath(errPath), + }; + + if (wine) { + // The native spawn path strips the env entirely; for Wine that drops + // HOME/WINEPREFIX and breaks. Forward the Wine-relevant vars so a plain + // `:MZ:…:/usr/bin/wine:` handler works with no wrapper script. + for (const key of [ + 'WINEPREFIX', + 'WINEARCH', + 'WINEDEBUG', + 'WINEDLLOVERRIDES', + 'HOME', + 'XDG_RUNTIME_DIR', + 'PATH', + ]) { + const value = process.env[key]; + if (value !== undefined) env[key] = value; + } + } + + const stderr = log.debugMode ? 'inherit' : 'ignore'; + const result = spawnSync( + fabricator.binaryPath, + activeBakes.concat('--no-warnings', '-e', fileScript), + { stdio: ['ignore', 'ignore', stderr], env }, ); - }); + + if (result.error) { + return cb( + crossFabricatorError(fabricator, snap, wine, result.error.message), + ); + } + + if (result.status !== 0) { + let detail = ''; + try { + detail = readFileSync(errPath, 'utf8'); + } catch { + /* no error file */ + } + return cb( + crossFabricatorError( + fabricator, + snap, + wine, + detail || `exit code ${result.status}`, + ), + ); + } + + const out = readFileSync(outPath); + const sizeOfBlob = out.readInt32LE(0); + const blob = Buffer.alloc(sizeOfBlob); + out.copy(blob, 0, 4, 4 + sizeOfBlob); + return cb(undefined, blob); + } catch (error) { + return cb( + crossFabricatorError(fabricator, snap, wine, (error as Error).message), + ); + } finally { + cleanup(); + } +} + +export function fabricate( + bakes: string[], + fabricator: Target, + snap: string, + body: Buffer, + cb: (error?: Error, buffer?: Buffer) => void, +) { + // A Windows fabricator under Wine cannot use inherited stdin/stdout pipes; + // use the file-based protocol instead. Native and QEMU cross-arch builds keep + // the persistent-child pipe path below. + if (runsUnderWine(fabricator)) { + return fabricateViaFile(bakes, fabricator, snap, body, cb); + } + + const activeBakes = bytecodeBakes(bakes); const cmd = fabricator.binaryPath; const key = JSON.stringify([cmd, activeBakes]); diff --git a/lib/help.ts b/lib/help.ts index 31f0e613..955ec1e6 100644 --- a/lib/help.ts +++ b/lib/help.ts @@ -21,6 +21,7 @@ export default function help() { --no-bytecode skip bytecode generation and include source files as plain js --no-native-build skip native addons build --fallback-to-source if bytecode generation fails for a file, ship it as plain source instead of skipping it + --cross-bytecode run the win target's Node under Wine to make Windows bytecode when building win-x64 on Linux [off] --no-dict comma-separated list of packages names to ignore dictionaries. Use --no-dict * to disable all dictionaries -C, --compress [default=None] compression algorithm = Brotli, GZip, or Zstd (Zstd requires Node.js >= 22.15) --signature enable macOS binary signing (default; use to override signature:false in config) @@ -28,8 +29,8 @@ export default function help() { --sea (Experimental) compile given file using node's SEA feature. Requires node v20.0.0 or higher and only single file is supported All build-shaping flags above (compress, fallback-to-source, public, public-packages, - options, bytecode, native-build, no-dict, debug, signature, sea) can also be set in - the pkg config file (camelCase keys). CLI flags override config values. + options, bytecode, native-build, no-dict, debug, signature, sea, cross-bytecode) can + also be set in the pkg config file (camelCase keys). CLI flags override config values. ${pc.dim('Examples:')} diff --git a/lib/index.ts b/lib/index.ts index 4b224def..5e23343a 100644 --- a/lib/index.ts +++ b/lib/index.ts @@ -57,16 +57,36 @@ function buildMarker( const { hostArch, hostPlatform } = system; -function fabricatorForTarget({ nodeRange, arch }: NodeTarget) { - let fabPlatform = hostPlatform; +// Exported for unit testing. `host` defaults to the real build host but can be +// overridden in tests so the branch selection can be exercised as a pure +// function (mirrors how lib/sea.ts host matching is tested). +export function fabricatorForTarget( + { nodeRange, platform, arch }: NodeTarget, + crossBytecode: boolean, + host: { platform: string; arch: string } = { + platform: hostPlatform, + arch: hostArch, + }, +) { + let fabPlatform: string = host.platform; if ( - hostArch !== arch && - (hostPlatform === 'linux' || hostPlatform === 'alpine') + host.arch !== arch && + (host.platform === 'linux' || host.platform === 'alpine') ) { // With linuxstatic, it is possible to generate bytecode for different // arch with simple QEMU configuration instead of the entire sysroot. fabPlatform = 'linuxstatic'; + } else if ( + crossBytecode && + platform === 'win' && + (host.platform === 'linux' || host.platform === 'alpine') + ) { + // Same CPU arch but different OS: run the Windows target's own Node under + // Wine (an OS-ABI layer registered via binfmt_misc) so the bytecode is + // produced by the target's V8 and accepted at runtime on Windows. Opt-in + // via --cross-bytecode because it requires Wine + a binfmt MZ handler. + fabPlatform = 'win'; } return { @@ -223,14 +243,14 @@ export async function exec( // fetch targets - const { bytecode, nativeBuild } = flags; + const { bytecode, nativeBuild, crossBytecode } = flags; for (const target of targets) { target.forceBuild = forceBuild; await needWithDryRun(target); - target.fabricator = fabricatorForTarget(target) as Target; + target.fabricator = fabricatorForTarget(target, crossBytecode) as Target; if (bytecode) { await needWithDryRun({ diff --git a/lib/types.ts b/lib/types.ts index 8f1121f6..840e1359 100644 --- a/lib/types.ts +++ b/lib/types.ts @@ -66,6 +66,13 @@ export interface PkgOptions { debug?: boolean; signature?: boolean; sea?: boolean; + /** + * Run the Windows target's own Node under Wine to fabricate V8 bytecode when + * building a same-arch `win` target on a Linux/Alpine host. Requires Wine + + * a `binfmt_misc` `MZ` handler. Off by default. See the Windows-on-Linux + * cross-build guide. + */ + crossBytecode?: boolean; } export interface PackageJson { @@ -200,4 +207,10 @@ export interface PkgExecOptions { noDictionary?: string[]; /** Sign macOS binaries when applicable. Default `true`. */ signature?: boolean; + /** + * Run the Windows target's own Node under Wine to fabricate V8 bytecode when + * building a same-arch `win` target on a Linux/Alpine host. Default `false`. + * Requires Wine + a `binfmt_misc` `MZ` handler. + */ + crossBytecode?: boolean; } diff --git a/test/unit/config-parse.test.ts b/test/unit/config-parse.test.ts index 49b02293..27a1e9a1 100644 --- a/test/unit/config-parse.test.ts +++ b/test/unit/config-parse.test.ts @@ -194,6 +194,7 @@ describe('parseInput — CLI argv', () => { 'fallback-to-source', 'public', 'sea', + 'cross-bytecode', ]) { it(`--no-${flag} sets flag to false`, () => { assert.equal(parseInput([`--no-${flag}`, 'a.js']).flags[flag], false); @@ -371,6 +372,7 @@ describe('resolveFlags — CLI > config > default', () => { assert.equal(f.bakeOptions, undefined); assert.equal(f.publicPackages, undefined); assert.equal(f.noDictionary, undefined); + assert.equal(f.crossBytecode, false); }); it('config wins when CLI is absent', () => { @@ -527,6 +529,7 @@ describe('validatePkgConfig', () => { options: ['a'], publicPackages: 'x,y', noDictionary: ['*'], + crossBytecode: true, }); assert.equal(warned.length, 0, `unexpected warns: ${warned.join('|')}`); }); @@ -575,6 +578,48 @@ describe('validatePkgConfig', () => { }); }); +describe('crossBytecode flag', () => { + it('--cross-bytecode parses to true', () => { + assert.equal( + parseInput(['--cross-bytecode', 'a.js']).flags['cross-bytecode'], + true, + ); + }); + + it('--no-cross-bytecode parses to false', () => { + assert.equal( + parseInput(['--no-cross-bytecode', 'a.js']).flags['cross-bytecode'], + false, + ); + }); + + it('programmatic crossBytecode option round-trips', () => { + assert.equal( + parseInput({ input: 'a.js', crossBytecode: true }).flags[ + 'cross-bytecode' + ], + true, + ); + }); + + it('default is false; config can enable it; CLI overrides config', () => { + assert.equal(resolveFlags({}, {}).crossBytecode, false); + assert.equal(resolveFlags({}, { crossBytecode: true }).crossBytecode, true); + assert.equal( + resolveFlags({ 'cross-bytecode': false }, { crossBytecode: true }) + .crossBytecode, + false, + ); + }); + + it('non-boolean config value throws', () => { + assert.throws( + () => validatePkgConfig({ crossBytecode: 'yes' }), + /"crossBytecode" must be a boolean/, + ); + }); +}); + describe('isConfiguration', () => { it('true for package.json regardless of directory', () => { assert.equal(isConfiguration('package.json'), true); diff --git a/test/unit/fabricator-file-ipc.test.ts b/test/unit/fabricator-file-ipc.test.ts new file mode 100644 index 00000000..9d6b0c6e --- /dev/null +++ b/test/unit/fabricator-file-ipc.test.ts @@ -0,0 +1,56 @@ +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +import { system } from '@yao-pkg/pkg-fetch'; + +import { fabricateViaFile } from '../../lib/fabricator'; +import type { Target } from '../../lib/types'; + +// The file-based fabricator IPC exists so a cross-OS fabricator (a Windows Node +// under Wine) can exchange the snap/body and the resulting V8 cached data +// without inherited stdin/stdout pipes. Wine itself can't run here, so this +// proves the file protocol independently using the HOST Node as the fabricator +// (platform === hostPlatform, so no Wine path/env translation is applied). +describe('fabricateViaFile (host Node, no Wine)', () => { + const hostFabricator = { + nodeRange: `node${process.version.match(/^v(\d+)/)![1]}`, + platform: system.hostPlatform, + arch: system.hostArch, + binaryPath: process.execPath, + } as unknown as Target; + + it('round-trips a body into a non-empty cached-data blob', () => { + const snap = '/snapshot/test/app.js'; + const body = Buffer.from( + 'module.exports = function () { return 40 + 2; };\n', + ); + + let err: Error | undefined; + let blob: Buffer | undefined; + fabricateViaFile([], hostFabricator, snap, body, (e, b) => { + err = e; + blob = b; + }); + + assert.equal(err, undefined, err && err.message); + assert.ok( + blob && blob.length > 0, + 'expected a non-empty V8 cached-data blob', + ); + }); + + it('reports a descriptive error when the fabricator cannot execute', () => { + const broken = { + ...hostFabricator, + binaryPath: '/path/that/does/not/exist/node', + } as unknown as Target; + + let err: Error | undefined; + fabricateViaFile([], broken, '/snapshot/x.js', Buffer.from('1;\n'), (e) => { + err = e; + }); + + assert.ok(err, 'expected an error'); + assert.match(err!.message, /Failed to make bytecode/); + }); +}); diff --git a/test/unit/fabricator-target.test.ts b/test/unit/fabricator-target.test.ts new file mode 100644 index 00000000..ef33aa25 --- /dev/null +++ b/test/unit/fabricator-target.test.ts @@ -0,0 +1,93 @@ +import assert from 'node:assert/strict'; +import { describe, it } from 'node:test'; + +import { fabricatorForTarget } from '../../lib/index'; +import type { NodeTarget } from '../../lib/types'; + +// fabricatorForTarget picks which Node binary compiles app JS to V8 bytecode. +// The cross-OS Windows-under-Wine branch (opt-in via --cross-bytecode) must +// engage ONLY for a same-arch win target on a linux/alpine host, and must not +// disturb the existing host / cross-arch (linuxstatic/QEMU) selection. `host` +// is injected so the branch table can be pinned without depending on the +// machine running the suite. +const mk = (platform: string, arch: string, nodeRange = 'node22'): NodeTarget => + ({ nodeRange, platform, arch }) as unknown as NodeTarget; + +describe('fabricatorForTarget', () => { + it('win target, same arch, linux host, flag on → win (Wine)', () => { + const f = fabricatorForTarget(mk('win', 'x64'), true, { + platform: 'linux', + arch: 'x64', + }); + assert.equal(f.platform, 'win'); + assert.equal(f.arch, 'x64'); + assert.equal(f.nodeRange, 'node22'); + }); + + it('win target, same arch, alpine host, flag on → win (Wine)', () => { + const f = fabricatorForTarget(mk('win', 'x64'), true, { + platform: 'alpine', + arch: 'x64', + }); + assert.equal(f.platform, 'win'); + }); + + it('win target, same arch, linux host, flag OFF → host platform (no regression)', () => { + const f = fabricatorForTarget(mk('win', 'x64'), false, { + platform: 'linux', + arch: 'x64', + }); + assert.equal(f.platform, 'linux'); + }); + + it('win target, DIFFERENT arch, linux host, flag on → linuxstatic (cross-arch wins, Wine not used)', () => { + // win-arm64 from an x64 host needs CPU emulation and is out of scope; the + // cross-arch branch must take precedence over the Wine branch. + const f = fabricatorForTarget(mk('win', 'arm64'), true, { + platform: 'linux', + arch: 'x64', + }); + assert.equal(f.platform, 'linuxstatic'); + }); + + it('non-win target on linux host, flag on → host platform (unaffected)', () => { + const f = fabricatorForTarget(mk('linux', 'x64'), true, { + platform: 'linux', + arch: 'x64', + }); + assert.equal(f.platform, 'linux'); + }); + + it('win target on a win host → win (native build, no Wine branch)', () => { + const f = fabricatorForTarget(mk('win', 'x64'), true, { + platform: 'win', + arch: 'x64', + }); + assert.equal(f.platform, 'win'); + }); + + it('win target on a macos host, flag on → host platform (Wine branch is linux/alpine only)', () => { + const f = fabricatorForTarget(mk('win', 'x64'), true, { + platform: 'macos', + arch: 'x64', + }); + assert.equal(f.platform, 'macos'); + }); + + it('cross-arch on linux host (non-win target) still selects linuxstatic', () => { + const f = fabricatorForTarget(mk('linux', 'arm64'), false, { + platform: 'linux', + arch: 'x64', + }); + assert.equal(f.platform, 'linuxstatic'); + }); + + it('preserves nodeRange and arch through the win branch', () => { + const f = fabricatorForTarget(mk('win', 'x64', 'node24'), true, { + platform: 'linux', + arch: 'x64', + }); + assert.equal(f.nodeRange, 'node24'); + assert.equal(f.arch, 'x64'); + }); +});