Skip to content

tutorial-rust.md

Latest commit

 

History

History
451 lines (374 loc) · 17.7 KB

tutorial-rust.md

File metadata and controls

451 lines (374 loc) · 17.7 KB
Raw

subscript-gpu for Rust hosts

Your application owns the window, the frame loop, and the GPU objects that outlive a frame. A script owns encoding. The script programs against the WebGPU JS API shape — requestAdapter, createBuffer, beginRenderPass — and never names the C facade below it. The GPU work reads like WebGPU and reloads like a script.

A Rust host gets one thing a C host cannot get: the development tier runs in your process. ReloadSession compiles the program in memory, and your host calls its entry directly. No C compiler runs, and no generated artifact reaches the disk.

Read this first, plainly:

  • You supply the backend. Every command below needs SUBSCRIPT_GPU_BACKEND_LIB_DIR. The variable points at a directory that holds a yawgpu build. On macOS that is libyawgpu.dylib with libtint_shim.dylib beside it. The static library is unused. This repository neither vendors nor builds one.
  • The generation chain is fixed to this workspace. It reads gpu/engine/src/lib.rs and writes eight files to fixed paths here. The "you write it" row below means that file, in this checkout.
  • Lifetimes are manual. Create-owns plus explicit disposal, and no finalizers. Your host releases what your host created.
  • Scope. This document is enough to run the two examples this repository ships. It is not enough to build a host in your own crate. The closing section says why, and what to do instead.

Everything below is committed and gate-pinned: the compute program gpu/programs/a27-host-compute.ts and the windowed example gpu/examples/windowed-triangle/. The outputs come from a run of them.

The artifacts

Artifact What it is Who writes it
gpu/engine/src/lib.rs your host's script-visible surface, as extern "C" functions you
gpu/engine/engine.h the C header for that surface generated
gpu/mirror/engine.generated.d.ts the ambient declarations scripts see generated
gpu/mirror/sgpu.generated.d.ts the facade mirror, which is internal plumbing generated
gpu/api/webgpu.ts the WebGPU-shaped API layer a script imports generated

cargo run -p subscript-gpu-codegen --offline writes all five, plus the facade, its Rust surface, and the dev-JIT symbol table. Every output is gated byte-identical, so no output can drift from its source.

Two hosts, two divisions of labour

Example Who creates the GPU objects What the host does
compute_host over a27 the script step async, and nothing else
the windowed example the host own the window, the device, and the loop; push the handles into init and frame

Run them:

export SUBSCRIPT_GPU_BACKEND_LIB_DIR=<directory that holds the backend library>
cargo run -p subscript-gpu-harness --bin compute_host --features backend-yawgpu
cargo run --offline -p subscript-gpu-windowed-triangle --features backend-yawgpu

The minimal host: step the async, call the entry

A script that owns everything needs no host support beyond async. The entry is export async function main(). It suspends at the first await and only a step resumes it, so the host drains the pending work after the call:

    let mut actions = Vec::new();
    actions.push(HostAction::AsyncStep);
    session
        .async_step()
        .map_err(|error| format!("initial async step: {error}"))?;
    actions.push(HostAction::ScriptEntry);
    session
        .call_main()
        .map_err(|error| format!("call script entry: {error}"))?;

    while session.async_pending() != 0 {
        actions.push(HostAction::AsyncStep);
        session
            .async_step()
            .map_err(|error| format!("resume async entry: {error}"))?;
    }

This host pumps no GPU events, because it owns no instance. The API layer pumps the script's own instance inside every await. A host pumps what it owns, and nothing else.

session.take_output() drains what print wrote. The run prints:

dispatch:encoded-workgroups=4
completion:submitted=true:mapped=true
readback:observed=1,2,3,4
resources:disposed

The read-back bytes are the pre-dispatch contents. The headless substrate records a dispatch and does not execute the shader, so computed values need a real device (see "Select a real backend").

The script side disposes by scope. a27 binds its owned resources with using, so every exit path disposes them in reverse declaration order — including completion after an await:

    using adapter = adapterResult;
    using device = deviceResult;
    using queue = device.queue();

Two idioms carry it. Narrow first, then bind: a using initializer must be non-null. Settle the failure paths first, then bind: the early returns dispose child before parent by hand, and using starts after the device succeeds. Every owned wrapper declares [Symbol.dispose], and a host-side tsc setup needs ESNext.Disposable in lib, as this repository's tsconfig.json shows.

Who owns what — and how the types keep you honest

A script sees up to three lifecycle holders, and every GPU object belongs to exactly one of them:

You got it from Who owns it What you do
create*, request*, or queue() the script dispose() it, or bind it with using
an entry parameter, wrapped with hostOwnedGPU* the host use it — the wrapper has no dispose()
the per-frame view parameter the engine, for one frame use it this frame — the host releases it after present

The reason this is typed and not a comment: there is no GC and no finalizer, by design. A script that disposes a live host handle frees memory the host still uses, and across a C ABI that is a crash later, not an exception now. So the mistake moves to check time. GPUHostOwnedDevice exposes neither dispose() nor destroy(), and a call to either fails the check:

S100: `GPUHostOwnedDevice` has no method `dispose`

One question decides ownership: who calls the release? If the answer is not the script, the wrapper does not have the button. What you create through a host-owned device is still yours — queue() returns an owned GPUQueue wrapper (your reference to the host's queue), and createShaderModule returns your shader.

A frame host: push the handles into entries

A host that owns the loop pushes what it owns into exported script entries. The committed artifact for this shape is the windowed example (gpu/examples/windowed-triangle/), and this section walks its script. The script declares three entries and pulls nothing:

export function init(
  instance: SGPUInstance,
  device: SGPUDevice,
  format: GPUTextureFormat,
): void {
export function frame(view: SGPUTextureView, key: u32): void {

init builds the long-lived resources once. One class holds them with non-null fields, behind one nullable module global. The constructor wraps the host's device and creates what the script owns:

  constructor(
    instance: SGPUInstance,
    device: SGPUDevice,
    format: GPUTextureFormat,
  ) {
    this.device = hostOwnedGPUDevice(instance, device);
    this.queue = this.device.queue();

The format parameter feeds the pipeline directly — it crossed the boundary as a wire-mapped GPUTextureFormat, so the pipeline matches the surface the host actually configured, with no cast:

    // The host-selected wire value crosses as GPUTextureFormat, so the pipeline matches the surface exactly.
    this.pipeline = this.device.createRenderPipeline({
      label: "windowed-triangle-pipeline",
      vertex: { module: this.shader, entryPoint: "vs_main" },
      fragment: {
        module: this.shader,
        entryPoint: "fs_main",
        targets: [{ format }],
      },
    });

Its dispose() closes the script-owned references and nothing else — the device is not in the list, because the script cannot dispose it:

  dispose(): void {
    this.pipeline.dispose();
    this.shader.dispose();
    this.queue.dispose();
  }

frame encodes one frame against the pushed view. The view is the third holder from the table: engine-owned, valid for this frame only, so the script wraps it and never disposes it:

  // The view is frame-scoped because presentation invalidates the acquired surface texture.
  const frameView: GPUTextureView = new GPUTextureView(view);

The frame-scoped objects the script does create — the encoder, the pass, the command buffer — it disposes before the entry returns:

  command.dispose();
  pass.dispose();
  encoder.dispose();
  // frameView wraps the engine-owned per-frame handle; engineEndFrame releases it after present.

shutdown closes what outlived the frames. It disposes the FrameResources and clears the global, and it is an explicit entry because an honest teardown is a call, not a side effect:

export function shutdown(): void {
  // Static resources outlive frames, so this explicit entry closes their script-owned references.
  const current: FrameResources | null = resources;
  if (current === null) {
    return;
  }
  current.dispose();
  resources = null;
}

The host drives the entries. It acquires the handles from its own engine (host-side Rust — the script never sees those calls) and pushes them with ReloadSession::call_export_with:

    session
        .call_export_with(
            "init",
            &[
                EntryArg::Handle(instance.cast()),
                EntryArg::Handle(device.cast()),
                EntryArg::I32(format),
            ],
        )
        .expect("init accepts the host-owned instance, device, and surface format");
    session
        .call_export_with("frame", &[EntryArg::Handle(view.cast()), EntryArg::U32(0)])
        .expect("frame accepts the host-owned view and translated key");
    session
        .call_export("shutdown")
        .expect("shutdown disposes the script-owned frame resources");

Two rules guard the boundary:

  • The call is checked before any script code runs. call_export_with fails on a wrong name, a wrong arity, and a wrong argument kind.
  • The format is validated at the crossing. An unmapped wire value traps with the alias name before the entry body runs, on both tiers.

The host gates readiness and translates input, so frame(view, key) receives plain values, and the script decides what they mean.

The engine surface behind the entries

The entries carry the handles, and a real engine still has a small C surface of its own — scalar queries, input recording, its lifecycle. Three rules shape it.

Mark shared types external. The sgpu mirror already declares the handle types. A /// @subscript-external marker above your own declaration makes the engine mirror reference that declaration instead of emitting a second one.

Mark lifecycle functions host-only. A function the script must never call carries /// @subscript-host-only as the whole line:

/// Creates an engine for a host-owned instance and surface.
/// @subscript-host-only
#[no_mangle]

A host-only function leaves the generated header and the mirror, so a script that names the lifecycle of a host-owned object fails to compile. That is a stronger rule than a review comment.

Regenerate, never edit. One command (cargo run -p subscript-gpu-codegen --offline) turns the Rust declarations into the C header and the ambient mirror, both under byte-identical regeneration gates. What stays script-visible in the engine mirror serves scalar queries and the differential suite's programs — the frame script names none of the functions that return a host-owned handle, and a test pins that.

Ship the same program

The ship tier compiles and links a program instead of hosting it. a27 ships that way in the differential suite: its main belongs to the emitted program, and engine-linked suite programs run host code through two named hooks:

        run_c_aot_with_native_libraries_and_host_hooks(
            &files(),
            &libraries,
            Some(ENGINE_PRE_ENTRY_HOOK),
            Some(ENGINE_POST_RUN_HOOK),
        )

The pre-entry hook runs after subscript_init and before the script's entry, and it creates what the host owns. The post-run hook runs after the async pump and releases that. Post-run runs after a trap as well, so what the host creates is always released.

The frame script ships differently: each exported entry becomes one C function (subscript_export_init, subscript_export_frame, subscript_export_shutdown) that a C host calls directly. The C document's step 7 shows the emitted signatures from a measured run.

What a Rust host must know

  • One Context belongs to one thread. Calls on a session, and the entries it runs, come from one thread at a time.
  • A host pumps what it owns. A host with no GPU objects pumps nothing. A host that owns an instance pumps that instance.
  • Async completes only if the host steps it. A host that never steps leaves the script parked forever, by design.
  • Disposal order lives in the examples, not in prose. The scripts dispose in order — by using scope in a27, by explicit calls in frame.ts — because readers copy examples.
  • An entry call is checked before script code runs. call_export_with verifies the name, the arity, and each argument kind, and a wire-mapped alias parameter traps on an unmapped value with the alias name.
  • A headless run proves argument conversion, lifetimes, and completion delivery. It proves nothing about pixels, which is why the triangle prints samples:device-only.

Select a real backend

SUBSCRIPT_GPU_BACKEND takes default, metal, vulkan or gles. With the variable unset the run uses the headless substrate. On a real device the triangle's sample line becomes samples:covered=true:not-covered=true. An unsatisfiable request writes a diagnostic and returns a null instance. It never falls back to the headless substrate, because a device run that quietly became a headless run would prove nothing.

What this document does not cover, and what to do instead

  • How to obtain a backend library. This repository neither vendors an implementation nor documents a build of one.

  • A host in your own crate. The generator reads gpu/engine/src/lib.rs in this workspace and writes to fixed paths in it, and a downstream crate still owns its facade ABI block. Everything else a consumer needs now has a supported path:

    What you need Where it comes from
    the three script sources subscript-gpu-artifacts constants
    the facade include directory DEP_SGPU_INCLUDE
    the dev-JIT facade symbol table generate_facade_native_symbols

    Two things to know about those.

    DEP_SGPU_INCLUDE names the facade package root, not a header-only directory, so sgpu.h and src/generated.rs both sit under it. The second is the argument generate_facade_native_symbols takes. Cargo passes a DEP_* variable to the build scripts of direct dependents only. A consumer that reaches the facade through another crate receives nothing, and no diagnostic.

    Selecting the backend feature from outside this workspace does not work the obvious way. cargo build -p subscript-gpu-facade --features backend-yawgpu fails with "cannot specify features for packages outside of workspace". Select one of your own workspace members alongside the dependency, so the feature travels through the graph:

    cargo build --offline --release \
    -p <consumer-workspace-member> -p subscript-gpu-facade \
    --features backend-yawgpu --message-format=json

    Both measurements above come from the requesting project, not from this repository — no consumer crate exists here to run them.

    The suite plumbing stays private. This repository declined to publish gpu/harness/tests/support/backend.rs as a library: a public test-support API would be gated here forever, and a consumer's own copy is short. The refusal and its evidence are in specs/blocks/downstream-consumption.md. The recipe above is the part worth sharing, so it lives here rather than in an API.

  • Presentation details. The windowed example opens a window when you run it. The required gate does not: it drives the same entries in a headless session, because every required gate runs with no window and no device. The reasons are in specs/blocks/host-embedding.md (H3a).

Reading on