From e77a086aea19b7bc706c823084a22a0a119dd764 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 02:59:34 +0000 Subject: [PATCH 1/4] Initial plan From 8a8189ad6f2f28b27c906f863e8373cf2c3f146e Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 03:04:14 +0000 Subject: [PATCH 2/4] Add documentation on naga bundling and module composition capabilities Co-authored-by: bytesandwich <1048685+bytesandwich@users.noreply.github.com> --- docs/naga-bundling-capabilities.md | 207 +++++++++++++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 docs/naga-bundling-capabilities.md diff --git a/docs/naga-bundling-capabilities.md b/docs/naga-bundling-capabilities.md new file mode 100644 index 00000000000..3a2748d6a25 --- /dev/null +++ b/docs/naga-bundling-capabilities.md @@ -0,0 +1,207 @@ +# Naga WGSL Bundling and Module Composition Capabilities + +This document summarizes the current state of naga's support for shader modularity, bundling, and code organization features. + +## Executive Summary + +**Naga itself does not currently provide built-in import/include directives, module linking, or shader bundling capabilities.** The WGSL specification that naga implements does not include a module system or preprocessor. However, there are external solutions and potential extension points that enable these features. + +## Current Naga Architecture + +### Core Module Structure + +Naga's IR is organized around the `Module` struct (`naga/src/ir/mod.rs`), which contains: +- `types`: Arena of type definitions +- `constants`: Arena of constant values +- `overrides`: Arena of pipeline-overridable constants +- `global_variables`: Arena of global variables +- `global_expressions`: Arena of constant/override expressions +- `functions`: Arena of function definitions +- `entry_points`: Vector of entry point definitions + +Each module is a self-contained compilation unit with no built-in mechanism for referencing or importing from other modules. + +### Extension Mechanisms + +Naga does provide extension points via WGSL directives: + +1. **Enable Extensions** (`enable ;`) + - `f16` - Half-precision floating point support + - `dual_source_blending` - Dual source blending + - `clip_distances` - Clip distance arrays + - `wgpu_mesh_shader` - Mesh shader support (wgpu-specific) + - `wgpu_ray_query` - Ray query support (wgpu-specific) + +2. **Language Extensions** (`requires ;`) + - `readonly_and_readwrite_storage_textures` + - `packed_4x8_integer_dot_product` + - `pointer_composite_access` + +These extensions do **not** provide import/include functionality. + +### GLSL Preprocessor Support + +For GLSL input (not WGSL), naga uses `pp_rs` for preprocessing, which supports: +- `#define` macros +- `#include` directives +- Conditional compilation (`#ifdef`, `#ifndef`, etc.) + +This is only available for GLSL frontend, not WGSL. + +## External Solutions for Shader Modularity + +### 1. naga_oil (Recommended for Bevy/Rust Projects) + +[naga_oil](https://github.com/bevyengine/naga_oil) is the primary external solution for WGSL shader composition. It provides: + +**Features:** +- Module imports with path syntax: `#import my_module::function` +- Module definitions: `#define_import_path my_module` +- Preprocessor-style directives for conditional compilation +- Function and type sharing across modules +- Virtual/override functions + +**Example syntax:** +```wgsl +#define_import_path my_module + +fn my_func() -> f32 { return 1.0; } +``` + +```wgsl +#import my_module; + +fn main() -> f32 { + return my_module::my_func(); +} +``` + +**Limitations:** +- Uses regex/simple lexer for preprocessing (not naga's lexer) +- Separate tool from naga proper +- Import resolution happens before naga parsing + +### 2. WESL (WebGPU Extended Shader Language) + +[WESL](https://wesl-lang.dev) is an emerging standard for WGSL extensions: + +**Features:** +- Standardized import syntax: `import my::module::{ item1, item2 };` +- Package management concepts +- Language server support + +**Status:** Under active development, being considered for Bevy's future shader system. + +### 3. Manual Concatenation + +The simplest approach is to concatenate WGSL source files before parsing: + +```rust +let common = std::fs::read_to_string("common.wgsl")?; +let main = std::fs::read_to_string("main.wgsl")?; +let combined = format!("{}\n{}", common, main); +let module = naga::front::wgsl::parse_str(&combined)?; +``` + +**Limitations:** +- No namespace isolation +- Name collision risks +- Manual dependency ordering + +## Programmatic Module Composition + +While naga doesn't have built-in linking, its Rust API allows programmatic module manipulation: + +### Using naga's IR directly + +```rust +use naga::{Module, Function, Handle}; + +// Parse multiple modules +let module_a = naga::front::wgsl::parse_str(source_a)?; +let module_b = naga::front::wgsl::parse_str(source_b)?; + +// Manually merge (requires careful handle remapping) +let mut combined = Module::default(); +// ... copy types, functions, etc. with handle translation +``` + +**Challenges:** +- Handles are arena-specific and must be remapped +- Type deduplication is complex +- No built-in merge functionality + +### Module Compaction + +Naga provides compaction (`naga::compact::compact()`) which: +- Removes unused declarations +- Can filter to a specific entry point +- Useful for dead code elimination after manual merging + +## Relevant GitHub Issues + +1. **[#6250](https://github.com/gfx-rs/wgpu/issues/6250)** - Request to expose lexer/parser for tools like naga_oil +2. **[#5713](https://github.com/gfx-rs/wgpu/issues/5713)** - Request for compile-with-context for module linking (closed) +3. **[#4429](https://github.com/gfx-rs/wgpu/issues/4429)** - Discussion on error API for preprocessor tooling + +## Recommendations + +### For Production Use + +1. **Use naga_oil** if you need import/module functionality today, especially with Bevy +2. **Watch WESL development** for standardized future solutions +3. **Simple concatenation** works for basic use cases with careful naming + +### For Test Harness Separation + +To separate test code from production shaders: + +1. **Preprocessor approach** (requires external tool): + ```wgsl + #ifdef TEST + fn test_helper() { ... } + #endif + ``` + +2. **Multiple entry points** (built-in): + ```wgsl + @compute @workgroup_size(1) + fn production_main() { ... } + + @compute @workgroup_size(1) + fn test_main() { ... } + ``` + Use naga's `--entry-point` and `--compact` flags to extract specific entry points. + +3. **Module splitting** (with naga_oil): + ```wgsl + // production.wgsl + #define_import_path production + #import common; + + // test.wgsl + #define_import_path test + #import common; + #import production; // import production functions to test + ``` + +## CLI Usage for Module Processing + +```bash +# Validate a shader +naga shader.wgsl + +# Extract specific entry point with dead code elimination +naga shader.wgsl output.wgsl --entry-point main --compact + +# Convert to different formats +naga shader.wgsl output.spv # SPIR-V +naga shader.wgsl output.metal # Metal +naga shader.wgsl output.hlsl # HLSL +``` + +## Conclusion + +Naga is designed as a shader translation library, not a build system or module bundler. For module composition needs, external tools like naga_oil provide the necessary functionality. The WGSL specification itself does not include imports, and naga faithfully implements this specification while providing extension points for wgpu-specific features. + +Future developments in WESL may provide standardized import syntax, but for now, projects requiring shader modularity should use naga_oil or implement custom preprocessing solutions. From c82f2f3ab1d0044fd8264ecaa237c742007550d6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 03:05:33 +0000 Subject: [PATCH 3/4] Fix documentation review feedback: add missing Module fields and correct function path Co-authored-by: bytesandwich <1048685+bytesandwich@users.noreply.github.com> --- docs/naga-bundling-capabilities.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/naga-bundling-capabilities.md b/docs/naga-bundling-capabilities.md index 3a2748d6a25..1cbb6f86f17 100644 --- a/docs/naga-bundling-capabilities.md +++ b/docs/naga-bundling-capabilities.md @@ -12,12 +12,15 @@ This document summarizes the current state of naga's support for shader modulari Naga's IR is organized around the `Module` struct (`naga/src/ir/mod.rs`), which contains: - `types`: Arena of type definitions +- `special_types`: Dictionary of special type handles (e.g., ray query types) - `constants`: Arena of constant values - `overrides`: Arena of pipeline-overridable constants - `global_variables`: Arena of global variables - `global_expressions`: Arena of constant/override expressions - `functions`: Arena of function definitions - `entry_points`: Vector of entry point definitions +- `diagnostic_filters`: Arena for diagnostic filter rules +- `doc_comments`: Optional documentation comments Each module is a self-contained compilation unit with no built-in mechanism for referencing or importing from other modules. @@ -133,7 +136,7 @@ let mut combined = Module::default(); ### Module Compaction -Naga provides compaction (`naga::compact::compact()`) which: +Naga provides compaction via the `compact` function in `naga::compact` which: - Removes unused declarations - Can filter to a specific entry point - Useful for dead code elimination after manual merging From ec11827480d877823656dca5860f4b1476eb746f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 04:29:09 +0000 Subject: [PATCH 4/4] Add section on potential native import/export/linking implementation Co-authored-by: bytesandwich <1048685+bytesandwich@users.noreply.github.com> --- docs/naga-bundling-capabilities.md | 48 +++++++++++++++++++++++++++++- 1 file changed, 47 insertions(+), 1 deletion(-) diff --git a/docs/naga-bundling-capabilities.md b/docs/naga-bundling-capabilities.md index 1cbb6f86f17..d9cf90286d8 100644 --- a/docs/naga-bundling-capabilities.md +++ b/docs/naga-bundling-capabilities.md @@ -203,8 +203,54 @@ naga shader.wgsl output.metal # Metal naga shader.wgsl output.hlsl # HLSL ``` +## Potential Native Implementation + +Adding native import/export and linking to naga is technically feasible but would be a significant undertaking. Here's how it could be approached: + +### 1. Import/Export Pass (Pre-parse or AST level) + +New directive handling would be added in `naga/src/front/wgsl/parse/directive/`: +```wgsl +export fn myFunction() { ... } +import "module_path" { myFunction, MyType } +``` + +This would: +- Extend the lexer/parser to recognize `import`/`export` keywords +- Track exported symbols in a new `ExportTable` structure +- Track import references as unresolved dependencies + +### 2. Linking Pass (IR level) + +A new `naga/src/link/mod.rs` module would: +- Take multiple `naga::Module` instances +- Remap all `Handle` references when merging arenas (types, constants, functions, etc.) +- Resolve cross-module references using a symbol table +- Deduplicate equivalent types using `UniqueArena` +- Validate no circular dependencies exist + +The trickiest part is handle remapping—naga's arena-based IR uses `Handle` extensively, and merging modules requires translating all handles from source module arenas to the combined module. + +### 3. Source Mapping + +Naga already has `Span` infrastructure (`naga/src/span.rs`) that tracks byte offsets. To extend this: +- Add a `SourceMap` type mapping `Span → (file_path, original_span)` +- Modify parsing to accept file identifiers +- Preserve mappings through lowering and IR transformations +- Output source maps in a standard format (e.g., similar to JavaScript source maps) + +### Effort Estimate + +- **Import/Export syntax**: ~2-3 weeks (parser changes, AST extensions) +- **Linking pass**: ~4-6 weeks (handle remapping, type deduplication, validation) +- **Source mapping**: ~2-3 weeks (span tracking through all passes) + +### Alternative: Expose Parser API + +A lighter approach (as discussed in [#6250](https://github.com/gfx-rs/wgpu/issues/6250)) would be exposing naga's lexer/parser as a public API, letting tools like naga_oil handle composition at the source level before naga parsing. + ## Conclusion Naga is designed as a shader translation library, not a build system or module bundler. For module composition needs, external tools like naga_oil provide the necessary functionality. The WGSL specification itself does not include imports, and naga faithfully implements this specification while providing extension points for wgpu-specific features. -Future developments in WESL may provide standardized import syntax, but for now, projects requiring shader modularity should use naga_oil or implement custom preprocessing solutions. +Future developments in WESL may provide standardized import syntax, but for now, projects requiring shader modularity should use naga_oil or implement custom preprocessing solutions. Native import/export/linking support could be added to naga with significant engineering effort, primarily around handle remapping and source map preservation.