diff --git a/.gitignore b/.gitignore index 2e111dc..bfabda3 100644 --- a/.gitignore +++ b/.gitignore @@ -17,6 +17,7 @@ dist/ *.dylib .cthreads_cache.json .pytest_cache/ +.vscode/ # CMake / MSVC litter - must never land in the source tree src/cthreads/cpp/CMakeCache.txt diff --git a/docs/gpu_future_cpu_to_gpu.md b/docs/gpu_future_cpu_to_gpu.md new file mode 100644 index 0000000..2c7875d --- /dev/null +++ b/docs/gpu_future_cpu_to_gpu.md @@ -0,0 +1,111 @@ +# Future work: CPU kernels calling GPU + +**Status:** Planned after the public Python GPU package is in place (`@Gpu`, `gpu()` / `GpuJob`, marshal, launch, join, writeback). + +This note records the intended design so we do not bolt on a second launch stack later. Write it as if the GPU runtime and Python `gpu()` path already exist (or are about to land); this feature only adds a native caller on top. + +--- + +## Idea + +GPU work is started from **Python** today (`gpu(fn, *args) -> GpuJob`). + +A **CPU** `@Thread` kernel (native C++ in `kernels.dll`) should be able to launch the same GPU work and wait on it, without going back through Python for every dispatch. + +Example intent (shape, not final API): + +```text +@Thread +def step(...): + # CPU work + gpu_job = launch_gpu(some_gpu_fn, ...) # C++ handle + # more CPU work overlapping GPU + gpu_job.join() +``` + +Same GpuPack / descriptor / pipeline / writeback machinery as `gpu()`. Only the **caller** changes: Python host vs native CPU kernel. + +--- + +## Why this is a separate step + +1. Python `gpu()` is the place that proves the permanent launch types and memory model. +2. Native CPU->GPU needs a stable **C++ `GpuJob` handle** and a native launch entry that mirrors Python marshal. +3. Overlapping CPU threads calling GPU will stress **transfer** and **queue submit** concurrency. That is easier to size once the single-caller path is solid. + +--- + +## Assumptions (GPU system already done) + +Before this work, the GPU package already provides: + +- `Context`, TransferEngine (or an engine pool API), `memory::`, `GpuPack` +- Descriptor and pipeline path, `GpuJob.join`, writeback into caller-owned buffers / objects +- Kernel cache keyed by symbol (layout, pipeline, SPIR-V) +- Inflight state owned by each job (pack, descriptor set, fence) +- Python `gpu()` as a thin wrapper over that C++ surface + +--- + +## What this needs + +### A. C++ job handle (GPU side) + +This may already exist for Python bindings. Confirm it is first-class native, not Python-only. + +- Type like `cthreads::gpu::GpuJob`. +- API roughly: create/launch from packed args or from a native marshal helper; `join()`; optional `result()`; destroy / RAII. +- Safe to store and `join` from a CPU `@Thread` worker (OS thread; no GIL required for the wait itself). +- Overlapping jobs: two launches of the same GPU symbol get distinct inflight rows (symbol + inflight index), shared **kernel cache** entry. + +### B. Native launch entry (GPU side) + +Something CPU code can call without pybind, for example: + +```text +GpuJob launch(symbol_or_pipeline_key, GpuPack&& pack, /* writeback plan */) +``` + +or a small C ABI used by generated CPU kernels (same spirit as `Fn__call`). + +Marshal from C++ values into `GpuPack` (lists/scalars already in native memory). Do not go through `cthreads.marshal` / ctypes. + +### C. Transfer concurrency (GPU side) + +CPU threads will upload/download without Python sequencing. + +- Prefer a **pool of TransferEngines** (or a checkout API) on `Context`, not one global engine held for the whole job lifetime without a plan. +- Mutex around a single engine is a minimum; a pool is the better fit once CPU->GPU is real. +- Compute queue submit may need a mutex if multiple host threads submit at once. + +### D. Changes to the cthreads CPU / codegen system + +- Allow CPU kernels to **call into** the GPU runtime (link against `_ext` GPU symbols or a shared `cthreads_gpu` API exported for kernels). +- Codegen / validator: a controlled way to express "launch this `@Gpu` from `@Thread`" (name binding, arity, types). Reject anything that implies mid-run Python sync on the GPU job. +- Job lifetime: CPU `SpawnedKernel` may own or await a child `GpuJob`. Define destroy order (GPU join before CPU pack free if they share buffers). Prefer **separate** packs unless explicitly aliased. +- Pools: a CPU pool worker launching GPU must not assume the GIL; completion is fence-based like Python `join`. +- Optional: register GPU child jobs for debugging / `Job` trees. Not required for the first version of this feature. + +### E. What not to invent + +- A second descriptor/pipeline stack for "native only." +- A public `DeviceBuffer` type. +- GPU `__sync_state` / mid-run Python observe. +- Replacing Python `gpu()`. It stays the host entry; native launch is an additional caller. + +--- + +## Suggested sequencing + +1. Confirm `GpuJob` is a complete C++ type used by Python bindings. +2. Add TransferEngine pool / checkout if not already there. +3. Add native `launch` + C++ marshal-into-GpuPack for supported types. +4. Extend CPU codegen/validator for an approved call shape. +5. Tests: CPU `@Thread` launches `@Gpu`, overlaps, joins; two CPU workers launch GPU concurrently. +6. Docs: call rules, lifetime, no GIL assumptions. + +--- + +## Relation to other later work + +This is an **additive** product feature on top of the finished Python GPU path. It is not a rewrite of pack, descriptors, pipelines, or launch. Track it separately from atomics, MoltenVK, and shared IR (those stay in the Later list in `todo.md`). diff --git a/docs/index.md b/docs/index.md index deea0c7..668ade6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,4 +12,6 @@ | `cthreads documentation` | [link](./COMPILER.md) | | **Release**: GitHub Actions, TestPyPI, PyPI trusted publishing | [link](./release.md) | | End-to-end example (`@Thread` / `@Threadable` through codegen) | [link](./Example.md) | -| **Vulkan / GPU backend guide** (cthreads compute path, Issues 1+) | [link](./vk_guide/README.md) | +| **Vulkan / GPU backend guide** (cthreads compute path) | [link](./vk_guide/README.md) | +| **Internals:** GPU C++ modules (Context → launch/join) | [link](./internals/gpu/README.md) | +| **Future:** CPU `@Thread` launching GPU (after Python `gpu()` is done) | [link](./gpu_future_cpu_to_gpu.md) | diff --git a/docs/internals/gpu/README.md b/docs/internals/gpu/README.md new file mode 100644 index 0000000..1eb7cec --- /dev/null +++ b/docs/internals/gpu/README.md @@ -0,0 +1,76 @@ +# GPU internals (C++ Vulkan path) + +This folder documents the C++ GPU modules under `src/cthreads/cpp/gpu/`. It is written for newcomers who know C++ and maybe Python, but have never used Vulkan. Product-facing Python APIs such as `gpu()` and `@Gpu` are not finished yet. What you see here is the permanent substrate those APIs will call. + +Related reading: + +- Contributor Vulkan tutorial style notes: [docs/vk_guide/README.md](../../vk_guide/README.md) +- Future work (CPU `@Thread` calling GPU): [docs/gpu_future_cpu_to_gpu.md](../../gpu_future_cpu_to_gpu.md) +- Build flag: CMake `CTHREADS_GPU=ON` compiles these sources into `cthreads._ext` + +## What problem this stack solves + +CPU `@Thread` kernels run as normal operating system threads calling generated C++. GPU work is different. The GPU is a separate processor with its own memory. You cannot hand it a Python list pointer and expect a shader to read it. You must: + +1. Open a connection to a Vulkan-capable GPU (the [Context](./context.md)). +2. Allocate GPU memory and copy host bytes in and out ([memory](./memory.md)). +3. Group one launch's buffers into a GpuPack ([pack](./pack.md)). +4. Tell the shader which binding number maps to which buffer ([descriptors](./descriptors.md)). +5. Build a reusable compute pipeline from SPIR-V bytes ([shader](./shader.md)). +6. Launch: record bind+dispatch, submit with a fence, then `join` writeback ([module](./module.md)). + +## Mental model in one picture + +```text +Python / tests + | + v +Context (loader, device, queue, function pointers, TransferEngine) + | + +-- memory:: create/destroy GpuBuffer, upload/download via staging + | + +-- pack:: GpuPack (scalar SSBO + list SSBOs) + | + descriptor pool/set/update (same namespace, descriptors.hpp) + | + +-- shader:: ShaderCacheEntry (module, layouts, pipeline) + | ShaderCache (symbol -> entry) + | + +-- launch:: launch_gpu_kernel -> SpawnedGpuKernel::join (writeback) +``` + +## Binding convention + +cthreads packs shader arguments like this: + +| Binding | Contents | +|-|-| +| 0 | One storage buffer for all scalars (`int`, `float`, `bool`, POD fields) | +| 1 | First list argument | +| 2 | Second list argument | +| ... | More lists | + +There are no buffer device addresses (raw GPU pointers) stuffed inside the scalar struct. The descriptor set is the wiring table. + +## Module index + +| Doc | Namespace / types | Source | +|-|-|-| +| [Context](./context.md) | `cthreads::gpu::Context`, `TransferEngine` | `headers/context.hpp`, `impl/context.cpp` | +| [Memory](./memory.md) | `cthreads::gpu::memory` | `headers/memory.hpp`, `impl/memory.cpp` | +| [Pack](./pack.md) | `cthreads::gpu::pack` (GpuPack create/upload/download) | `headers/pack.hpp`, `impl/pack.cpp` | +| [Descriptors](./descriptors.md) | `cthreads::gpu::pack` (pool/set/update) | `headers/descriptors.hpp`, `impl/descriptors.cpp` | +| [Shader](./shader.md) | `cthreads::gpu::shader` | `headers/shader.hpp`, `shader_cache.hpp`, `impl/shader.cpp`, `shader_cache.cpp` | +| [Module / launch](./module.md) | `SpawnedGpuKernel`, `launch_gpu_kernel` | `headers/module.hpp`, `impl/module.cpp` | + +## Build and availability + +GPU code is compiled only when `CTHREADS_GPU` is on. The extension still loads `vulkan-1` (or `libvulkan.so.1`) dynamically at runtime. A machine without a Vulkan loader or GPU fails cleanly through Python `cthreads.gpu.available()` rather than crashing the import of CPU-only wheels. + +## What is intentionally not here yet + +- Public `gpu()` entry and `@Gpu` SPIR-V emit +- Threadable / schema marshal writeback (list[float]/list[int] writeback is in `SpawnedGpuKernel::join`) +- Inflight job store (header stub only) +- Dummy SSBOs for empty list slots in `update_descriptors` + +Those build on the modules documented here. diff --git a/docs/internals/gpu/context.md b/docs/internals/gpu/context.md new file mode 100644 index 0000000..c72b065 --- /dev/null +++ b/docs/internals/gpu/context.md @@ -0,0 +1,163 @@ +# Context and TransferEngine + +Source: `src/cthreads/cpp/gpu/headers/context.hpp`, `src/cthreads/cpp/gpu/impl/context.cpp`. + +Namespace: `cthreads::gpu`. + +This document explains the process-wide Vulkan connection that every other GPU helper needs. If you have never used Vulkan, start here. + +## Why a Context exists + +Vulkan is not a single library call like "run this kernel." It is a layered API: + +1. An operating system library (the Vulkan loader) finds GPU drivers. +2. You create an instance (your application's connection to the loader). +3. You pick a physical device (a GPU the driver can see). +4. You create a logical device (your opened session on that GPU). +5. You get a queue (a submission port where you send work). +6. You resolve dozens of function pointers by name, because cthreads loads the loader dynamically instead of linking `vulkan-1` into every wheel. + +The `Context` struct holds all of that state for the whole process. There is one Context, accessed through `context()`, guarded by a mutex for `init` / `shutdown` / `available`. + +## Public functions + +### `void init()` + +Opens the loader, creates instance and device, resolves entry points, marks `ready = true`, and creates the TransferEngine (command pool and fence). Throws typed-style error strings (for example `cthreads.gpu.VulkanLoaderNotFound`) that Python maps into exceptions in `cthreads.gpu`. + +Call this when you need the GPU. Python `cthreads.gpu.init()` ends up here. + +### `void shutdown()` + +Destroys children first, then parents: + +1. TransferEngine (staging buffer, fence, command pool) +2. Shader cache entries (pipelines and layouts) +3. Logical device +4. Instance +5. Unload the loader library +6. Clear all function pointers and handles + +Safe to call if the Context was never initialized. + +### `bool available()` + +Tries to ensure the Context is ready without throwing to the caller for "no GPU" style probes. Used by Python `available()` and by tests that skip when there is no device. + +### `const std::string& device_name()` + +Returns the human-readable GPU name from Vulkan device properties. Requires a ready Context. + +### `Context& context()` + +Returns the singleton. Most C++ helpers take `Context&` explicitly so ownership and testing stay clear. + +## Struct `Context` fields (grouped) + +### Loader and bootstrap + +- `loader_module`: operating system handle to `vulkan-1.dll` / `libvulkan.so.1`. +- `vkGetInstanceProcAddr`: the bootstrap function used to look up almost every other Vulkan function by name string. + +### Instance and device entry points + +Examples: `vkCreateInstance`, `vkEnumeratePhysicalDevices`, `vkCreateDevice`, `vkGetDeviceQueue`, `vkDestroyDevice`. + +These are stored as typed function pointers (`PFN_vk...`). A pointer-to-function (PFN) is simply a C function pointer with the Vulkan signature. cthreads assigns them during init after the loader is open. + +### Buffer and memory entry points + +Used by [memory](./memory.md): create/destroy buffers, allocate/free device memory, map host-visible memory, query memory properties. + +### Command, copy, and sync entry points + +Used by transfers and launch: command pools, command buffers, `vkCmdCopyBuffer`, fences, `vkQueueSubmit`, wait/reset fences. + +### Shader and pipeline entry points + +Used by [shader](./shader.md): create/destroy shader modules, descriptor set layouts, pipeline layouts, and compute pipelines. + +### Descriptor pool and update entry points + +Used by [descriptors](./descriptors.md): create/destroy descriptor pools, allocate/free sets, `vkUpdateDescriptorSets`. + +### Compute dispatch entry points + +Used by [module / launch](./module.md): `vkCmdBindPipeline`, `vkCmdBindDescriptorSets`, `vkCmdDispatch`, `vkCmdPipelineBarrier`. + +### Opaque handles + +- `instance`: connection to the loader for this application. +- `physical_device`: the chosen GPU. +- `device`: the logical device (opened GPU session). +- `queue`: compute queue used to submit copies and dispatches. +- `queue_family`: index of the queue family that supports compute (needed when creating command pools). +- `device_name`: string name for logging and Python. +- `ready`: true only after a fully successful init. + +### TransferEngine and mutex + +See the next section. `transfer_engine_mutex` serializes use of the single shared engine. + +## Technical terms + +- Vulkan loader: system library that discovers Installable Client Drivers (ICDs), which are the vendor GPU drivers. +- Instance: application-level Vulkan object. Not the GPU itself. +- Physical device: one GPU as seen by the driver. +- Logical device: your opened handle to use that GPU. +- Queue: port where the CPU submits recorded command buffers. +- Queue family: group of queues with the same capabilities (graphics, compute, transfer). +- Dynamic loading: open the DLL/SO at runtime and resolve symbols by name, instead of linking at build time. +- `VK_NULL_HANDLE`: sentinel meaning "no object." + +## TransferEngine + +### Purpose + +Uploading and downloading buffers needs a short GPU copy: host-visible staging memory to or from a device-local buffer. Creating a brand new command pool and fence for every copy would be slow and wasteful. The TransferEngine keeps reusable machinery on the Context for the process lifetime. + +### Fields + +- `command_pool`: Vulkan command pool for the compute queue family. Command buffers for copies are allocated from here and freed after each wait. +- `fence`: CPU-waitable fence. Created signaled so the first reset/wait path treats it as idle. After each copy, the CPU waits on this fence. +- `staging`: optional host-visible `GpuBuffer`. Empty until the first upload or download grows it. It only grows; it never shrinks until Context shutdown. + +### Lifecycle + +- Created in `init_transfer_engine` after the device is ready. +- Destroyed in `shutdown_transfer_engine` before the logical device is destroyed. +- Staging is grown by `memory::ensure_staging` under the transfer engine mutex. + +### Concurrency + +There is one engine today. All upload/download paths hold `transfer_engine_mutex` for the whole transfer (staging grow, memcpy, GPU copy, wait). That avoids races on the shared pool, fence, staging buffer, and queue submit. + +A future pool of engines is discussed for CPU threads launching GPU work. See [gpu_future_cpu_to_gpu.md](../../gpu_future_cpu_to_gpu.md). + +## Key Vulkan calls during init (story order) + +1. Load the loader library (`LoadLibrary` / `dlopen`). +2. Resolve `vkGetInstanceProcAddr`, then `vkCreateInstance`. +3. `vkCreateInstance` builds the instance. +4. Resolve instance-level functions. +5. `vkEnumeratePhysicalDevices` lists GPUs; pick one with a compute queue family (prefer discrete when possible). +6. `vkCreateDevice` opens the logical device; `vkGetDeviceQueue` gets the queue. +7. Resolve device-level functions (buffers, commands, shaders, descriptors). +8. Create TransferEngine pool and fence. +9. Set `ready = true`. + +## Key Vulkan calls during shutdown (story order) + +1. Wait on the transfer fence if needed; destroy staging, fence, command pool. +2. Clear the shader cache (destroy pipelines and layouts). +3. `vkDestroyDevice`. +4. `vkDestroyInstance`. +5. Unload the loader; null all pointers. + +## Relationship to other modules + +Every helper in memory, pack, descriptors, and shader takes a `Context&` and expects `ready == true` plus the entry points it needs. If init failed or shutdown already ran, those helpers throw `VulkanInitFailed`-style errors. + +## Python surface today + +`cthreads.gpu` exposes `init`, `shutdown`, `available`, and `device_name`, plus typed errors. It does not yet expose packs, shaders, or jobs. Those will wrap the C++ types documented in the sibling pages. diff --git a/docs/internals/gpu/descriptors.md b/docs/internals/gpu/descriptors.md new file mode 100644 index 0000000..0ed88e0 --- /dev/null +++ b/docs/internals/gpu/descriptors.md @@ -0,0 +1,118 @@ +# Descriptors (pack namespace) + +Source: `src/cthreads/cpp/gpu/headers/descriptors.hpp`, `src/cthreads/cpp/gpu/impl/descriptors.cpp`. + +Namespace: `cthreads::gpu::pack` (same as GpuPack; separate files for clarity). + +Depends on: [Context](./context.md), [Pack](./pack.md), [Shader](./shader.md) (for set layouts and `ShaderCacheEntry`). + +## Purpose in plain language + +A compute shader does not receive C++ pointers. It declares numbered bindings, for example "binding 0 is my scalar block" and "binding 1 is list x." On the CPU you own `VkBuffer` handles inside a `GpuPack`. Descriptors are the table that connects those two worlds: + +```text +shader binding 0 -> pack.scalar_buffer +shader binding 1 -> pack.container_slots[0] +shader binding 2 -> pack.container_slots[1] +... +``` + +There are two layers: + +1. Descriptor set layout: the schema (which binding numbers exist and that each is a storage buffer). Created once per kernel in `shader::create_entry` and stored on `ShaderCacheEntry`. +2. Descriptor set: one filled-in instance of that schema for one launch. Created from a pool, then updated to point at this launch's pack buffers. + +## Technical terms + +- Descriptor: one slot in the table (for us, always a storage buffer reference). +- Descriptor set layout: immutable schema of bindings for a pipeline. +- Descriptor set: concrete table instance you bind before dispatch. +- Descriptor pool: allocator that vends descriptor sets (similar in spirit to a memory pool). +- Storage buffer descriptor: a descriptor type that points at a `VkBuffer` used as an SSBO. +- `vkUpdateDescriptorSets`: Vulkan call that writes buffer handles into a set. +- Binding number: integer the shader and the CPU agree on (0 for scalars, then lists). + +## Struct `DescriptorPool` + +Fields: + +- `pool`: Vulkan `VkDescriptorPool` handle. +- `max_sets`: how many sets the pool was created to hold. +- `binding_count`: how many storage buffer descriptors each set needs (must match the shader cache entry). + +The pool is created with `VK_DESCRIPTOR_POOL_CREATE_FREE_DESCRIPTOR_SET_BIT` so individual sets can be returned with `free_set` when a job finishes. + +## Function `create_pool` + +Creates a pool for compute sets that follow the binding convention. + +Parameters: + +- `binding_count`: storage buffers per set (1 + number of list slots). +- `max_sets`: capacity. + +Internally it declares one pool size entry of type `STORAGE_BUFFER` with `descriptorCount = binding_count * max_sets`, then calls `vkCreateDescriptorPool`. + +## Function `destroy_pool` + +Destroys the Vulkan pool and clears the struct. All sets allocated from the pool become invalid. Prefer freeing live sets first when jobs still hold them. + +## Function `allocate_set` + +Allocates one `VkDescriptorSet` from the pool using a `VkDescriptorSetLayout` (normally `ShaderCacheEntry::set_layout`). The set is empty until `update_descriptors` runs. + +Key call: `vkAllocateDescriptorSets`. + +## Function `free_set` + +Returns a set to the pool via `vkFreeDescriptorSets`. Safe no-op if the set handle is already null. After success, the caller's set handle is set to `VK_NULL_HANDLE`. + +## Function `update_descriptors` (binding count overload) + +Writes the pack into the set. + +Rules: + +- `binding_count` must equal `1 + pack.container_slots.size()`. +- Binding 0 uses `pack.scalar_buffer`. +- Binding `i` for `i >= 1` uses `pack.container_slots[i - 1]`. +- Every written binding must have a non-null buffer and non-zero size. Empty pack slots are not supported yet. + +For each binding it builds a `VkDescriptorBufferInfo` (buffer, offset 0, range = size) and a `VkWriteDescriptorSet` of type `STORAGE_BUFFER`, then calls `vkUpdateDescriptorSets` once for all writes. + +## Function `update_descriptors` (entry overload) + +Convenience wrapper that uses `entry.binding_count` from a `ShaderCacheEntry`. + +## Who calls these + +The launch path (`launch_gpu_kernel` today via tests; later `gpu()`): + +```text +ShaderCache.get(symbol) -> entry +create_pool / allocate_set(entry.set_layout) +update_descriptors(set, entry, pack) +record CB: barrier -> bind pipeline -> bind set -> dispatch +vkQueueSubmit(..., fence) +// join: wait fence -> download ref lists -> release +``` + +The shader cache never updates descriptors. It only stores the layout schema. + +## Key Vulkan calls summary + +| Call | Role | +|-|-| +| `vkCreateDescriptorPool` | Create the allocator | +| `vkDestroyDescriptorPool` | Destroy the allocator | +| `vkAllocateDescriptorSets` | Get one set matching a layout | +| `vkFreeDescriptorSets` | Return a set to the pool | +| `vkUpdateDescriptorSets` | Point bindings at `VkBuffer`s | + +## Current limitation: empty lists + +`GpuPack` allows empty container slots without a buffer. `update_descriptors` throws if a required binding has a null buffer, because standard Vulkan does not accept a null buffer descriptor without special null-descriptor features. Until a process-wide dummy SSBO exists, test kernels should use non-empty buffers for every binding they declare. + +## What comes after update + +Descriptors alone do not run the shader. [Module / launch](./module.md) records the command buffer, submits with a fence, and `join` downloads ref lists into Python. diff --git a/docs/internals/gpu/memory.md b/docs/internals/gpu/memory.md new file mode 100644 index 0000000..0b10a09 --- /dev/null +++ b/docs/internals/gpu/memory.md @@ -0,0 +1,140 @@ +# Memory helpers + +Source: `src/cthreads/cpp/gpu/headers/memory.hpp`, `src/cthreads/cpp/gpu/impl/memory.cpp`. + +Namespace: `cthreads::gpu::memory`. + +Depends on: [Context and TransferEngine](./context.md). + +This module owns one idea: a contiguous byte region on the GPU (`GpuBuffer`), plus helpers to create it, destroy it, and copy bytes between host memory and device-local storage. + +## Host memory versus device memory + +- Host memory is ordinary process RAM. C++ `memcpy` and Python buffers live here. +- Device-local memory is GPU memory that shaders prefer. The CPU usually cannot keep a permanent pointer into it. +- Host-visible memory is a special GPU allocation the CPU can map. It is often slower for heavy shader traffic, so cthreads uses it only as a temporary staging mirror. + +cthreads's rule: shader-facing data lives in device-local buffers. Host traffic always goes through staging plus a GPU copy. + +## Technical terms + +- Storage buffer (SSBO): a buffer a compute shader can read and write through a descriptor binding. +- Staging buffer: host-visible buffer used only as the CPU side of an upload or download. +- `VkBuffer`: opaque Vulkan handle describing a buffer resource (size and usage). It is not a raw C pointer to bytes. +- `VkDeviceMemory`: opaque handle for an allocated memory slab. You bind a buffer to memory before using it. +- Memory type: one of the driver-exposed categories (device-local, host-visible, host-coherent, and so on). +- Fence: a GPU timeline object the CPU can wait on until submitted work finishes. +- Command buffer: a recorded list of GPU commands (for transfers, usually a single copy). +- Command pool: allocator that owns command buffers for one queue family. + +## Enum `BufferKind` + +One create API, two property sets. + +### `BufferKind::Staging` + +- Usage: transfer source and transfer destination. +- Memory: host-visible and host-coherent. +- After create, the buffer is mapped and `GpuBuffer::mapped` points at CPU-writable bytes. + +### `BufferKind::DeviceLocal` + +- Usage: storage buffer plus transfer source and destination (so shaders and copies both work). +- Memory: device-local. +- Never persistently mapped (`mapped` stays null). + +## Struct `GpuBuffer` + +Fields: + +- `buffer`: `VkBuffer` handle, or `VK_NULL_HANDLE` if empty. +- `memory`: `VkDeviceMemory` bound to that buffer, or null handle if empty. +- `size`: caller-facing byte count requested at create time. +- `mapped`: CPU pointer for staging only. +- `kind`: staging or device-local. + +Ownership: the struct owns the Vulkan objects until `destroy_buffer` runs. Moving the struct moves the handles; it does not clone GPU bytes. + +## Function `find_memory_type` + +Vulkan returns a bitmask of legal memory types for a new buffer (`type_bits`). You also request property flags (for example host-visible). This helper walks the physical device's memory types and returns the first index that is allowed by the bitmask and has every requested property. + +Used internally by `create_buffer`. You rarely call it from higher layers. + +## Function `create_buffer` + +Creates a `GpuBuffer` of the given kind and size. + +Steps in plain language: + +1. Reject size 0 and missing entry points. +2. Choose usage flags and memory properties from `BufferKind`. +3. Call `vkCreateBuffer` to create the buffer object. +4. Query memory requirements with `vkGetBufferMemoryRequirements`. +5. Pick a memory type with `find_memory_type`. +6. Call `vkAllocateMemory`, then `vkBindBufferMemory` at offset 0. +7. For staging, call `vkMapMemory` and store the pointer in `mapped`. + +Throws on failure. On partial failure it cleans up objects it already created. + +## Function `destroy_buffer` + +Unmaps staging memory if needed, destroys the `VkBuffer`, frees `VkDeviceMemory`, and clears the struct. Safe on an already-empty buffer. + +## Function `upload_buffer` + +Copies host bytes into a device-local `GpuBuffer`. + +Path: + +```text +host pointer + -> memcpy into TransferEngine staging (grown if needed) + -> GPU vkCmdCopyBuffer staging -> device-local + -> wait on TransferEngine fence +``` + +The whole path holds `context.transfer_engine_mutex` so two threads cannot share the engine's staging, pool, or fence unsafely. + +Requirements: + +- Target buffer must be device-local and non-null. +- `data` non-null; `size` in `(0, buffer.size]`. + +## Function `download_buffer` + +Opposite direction: + +```text +device-local + -> GPU copy into TransferEngine staging + -> wait + -> memcpy staging -> host pointer +``` + +Same mutex and validation rules as upload. + +## Internal helper `copy_buffer_and_wait` + +Not part of the public header. Records a one-time command buffer that copies `size` bytes from one `VkBuffer` to another, submits it on the Context queue with the TransferEngine fence, waits, then frees the command buffer. + +Important Vulkan calls: + +- `vkAllocateCommandBuffers` / `vkFreeCommandBuffers` +- `vkBeginCommandBuffer` / `vkEndCommandBuffer` +- `vkCmdCopyBuffer` +- `vkResetFences` / `vkQueueSubmit` / `vkWaitForFences` + +Caller must already hold the transfer engine mutex. + +## Internal helper `ensure_staging` + +Grows the TransferEngine staging buffer so it is at least `size` bytes. Never shrinks until Context shutdown. Caller must hold the transfer engine mutex. + +## Why not map device-local buffers? + +Many discrete GPUs cannot give the CPU a fast permanent pointer to device-local memory. Staging plus an explicit GPU copy is the portable model and matches how real engines move data. + +## How pack uses this + +[Pack](./pack.md) creates one device-local `GpuBuffer` for scalars and one per non-empty list. Upload and download helpers on the pack call `upload_buffer` / `download_buffer` for each region. diff --git a/docs/internals/gpu/module.md b/docs/internals/gpu/module.md new file mode 100644 index 0000000..e28edcd --- /dev/null +++ b/docs/internals/gpu/module.md @@ -0,0 +1,52 @@ +# Launch path (`SpawnedGpuKernel`) + +Sources: + +- `src/cthreads/cpp/gpu/headers/module.hpp` +- `src/cthreads/cpp/gpu/impl/module.cpp` + +Namespace: `cthreads::gpu`. + +Depends on: [Context](./context.md), [Pack](./pack.md), [Descriptors](./descriptors.md), [Shader](./shader.md). + +## Purpose + +`launch_gpu_kernel` is the GPU analogue of CPU `spawn_from_meta`: build a `GpuPack`, wire descriptors, record bind+dispatch, submit with a fence, and return a job handle. `SpawnedGpuKernel::join` waits on that fence, downloads ref lists into the same Python objects, then releases Vulkan state. There is no OS worker thread and no mid-run `sync_state`. + +Public `gpu()` / `@Gpu` (later) will call these same types. Tests exercise them today via `_ext.gpu.testing.smoke_launch_saxpy`. + +## Technical terms + +- Fence: CPU waits until the submitted dispatch has finished. +- Writeback: copy device list SSBOs back into the kept Python `list` objects (`pass_as` ref). +- Per-job command pool: short-lived pool that owns the launch command buffer (not the TransferEngine pool). + +## Struct `SpawnedGpuKernel` + +Owns per-launch GPU handles plus writeback inputs: + +| Field | Role | +|-|-| +| `pack` | Device-local scalar + list SSBOs | +| `descriptor_pool` / `descriptor_set` | Wired to this pack | +| `command_pool` / `command_buffer` | Recorded dispatch | +| `fence` | Signals when submit completes | +| `values_keep` | Python args kept alive for list writeback | +| `writeback_lists` | Plan of ref list slots to download on join | + +## Function `launch_gpu_kernel` + +Takes `meta` + `ordered_values` (same shape as the docstring on `module.hpp`). Registers nothing in the shader cache; the symbol must already exist. Returns `shared_ptr` without waiting. + +## Method `join` + +1. `vkWaitForFences` on the job fence +2. Compute → transfer barrier (TransferEngine) +3. For each ref list: `download_container` → fill the kept `py::list` in place +4. `release_inflight` (free CB/pool/set/fence/pack, drop `values_keep`) + +Value scalars are not written back. Threadable/schema marshal is later. + +## Testing + +`smoke_launch_saxpy` registers committed saxpy SPIR-V, launches, joins, and asserts `y` matches CPU saxpy. Pytest: `tests/unit/test_gpu_shader.py::test_live_smoke_launch_saxpy`. diff --git a/docs/internals/gpu/pack.md b/docs/internals/gpu/pack.md new file mode 100644 index 0000000..d57e8ae --- /dev/null +++ b/docs/internals/gpu/pack.md @@ -0,0 +1,100 @@ +# GpuPack + +Source: `src/cthreads/cpp/gpu/headers/pack.hpp`, `src/cthreads/cpp/gpu/impl/pack.cpp`. + +Namespace: `cthreads::gpu::pack`. + +Depends on: [Context](./context.md), [Memory](./memory.md). + +Descriptor pool and update helpers live in the same namespace but in separate files. See [Descriptors](./descriptors.md). + +## Purpose + +A `GpuPack` is the per-launch bag of GPU buffers for one compute dispatch under the binding convention: + +- One device-local scalar storage buffer (binding 0), or none if there are no scalar bytes. +- One device-local storage buffer per list/container argument (bindings 1..N). + +The pack does not know Python argument names or std430 field offsets. Marshal and codegen decide how to flatten scalars into the scalar blob and which list is slot `i`. The pack only owns buffers and moves bytes. + +## Technical terms + +- Binding convention: one scalar SSBO plus one SSBO per list, with no buffer device addresses inside the scalar block. +- SSBO: storage buffer object; a shader-readable and writable buffer bound through descriptors. +- Container slot: one list-like argument inside the pack. +- `numel`: number of elements in a slot. Zero means "no Vulkan buffer for this slot." +- std430: a GLSL/SPIR-V memory layout rule for how fields pack in a storage buffer. Marshal must match it; this module only stores opaque bytes. + +## Struct `ContainerSpec` + +Create-time size description for one list slot. + +- `elem_bytes`: bytes per element (4 for `float` or 32-bit `int`). +- `numel`: element count. If zero, create keeps an empty slot and does not call `vkCreateBuffer` with size 0. + +## Struct `ContainerSlot` + +- `buffer`: device-local `GpuBuffer` when `numel > 0`; empty handles otherwise. +- `spec`: the `ContainerSpec` used at create time (also used for upload size checks). + +## Struct `GpuPack` + +- `scalar_buffer`: device-local blob for all packed scalars. Empty if `scalar_bytes` was 0 at create. +- `container_slots`: vector in binding order for bindings 1..N. + +Returning a `GpuPack` by value moves handles only. It does not clone GPU memory. + +## Function `create_gpu_pack` + +Allocates the pack on a ready Context. + +Behavior: + +- If `scalar_bytes > 0`, create a device-local scalar buffer of that size. +- For each container spec, append a slot. If `numel > 0`, require `elem_bytes > 0` and create a device-local buffer of `elem_bytes * numel`. If `numel == 0`, keep a slot with no buffer. + +Zero-size Vulkan buffers are never created. + +## Upload helpers + +### `upload_scalars` + +Copies host bytes into `pack.scalar_buffer` via [memory upload](./memory.md). + +### `upload_container` + +Copies host bytes into one non-empty slot. Size must match the slot's byte size. + +### `upload_containers` + +Uploads every non-empty slot from parallel host pointers. Empty slots are skipped. + +## Download helpers + +### `download_scalars` / `download_container` / `download_containers` + +Mirror the upload helpers in the device-to-host direction. Results land in caller-owned host memory. `SpawnedGpuKernel::join` uses these for ref-list writeback (see [Module / launch](./module.md)). + +## Function `destroy_gpu_pack` + +Destroys every non-null buffer through `memory::destroy_buffer` and clears the pack. Safe on an already-empty pack. Does not shut down the Context. + +## Empty lists + +Empty lists are first-class in the pack: the slot exists so binding indices stay stable, but there is no `VkBuffer`. Descriptor update currently rejects null buffers (see [Descriptors](./descriptors.md)). A future dummy SSBO may fill empty bindings; until then, smoke launches should use non-empty lists for every binding the shader declares. + +## How this fits the launch path + +```text +create_gpu_pack +upload_scalars / upload_containers +allocate descriptor set + update_descriptors(pack) // descriptors.hpp +record CB: barrier + bind pipeline/set + dispatch // module.cpp +vkQueueSubmit(..., fence) +join: wait fence -> barrier -> download_* into kept Python lists -> release +destroy_gpu_pack +``` + +## Testing today + +`_ext.gpu.testing` exposes pack round-trip helpers (float and int packs) and `smoke_launch_saxpy` (full launch + join writeback). Pytest: `tests/unit/test_gpu_pack.py`, `tests/unit/test_gpu_shader.py`. diff --git a/docs/internals/gpu/shader.md b/docs/internals/gpu/shader.md new file mode 100644 index 0000000..7ead72c --- /dev/null +++ b/docs/internals/gpu/shader.md @@ -0,0 +1,133 @@ +# Shader cache and create_entry + +Sources: + +- `src/cthreads/cpp/gpu/headers/shader_cache.hpp` +- `src/cthreads/cpp/gpu/headers/shader.hpp` +- `src/cthreads/cpp/gpu/impl/shader_cache.cpp` +- `src/cthreads/cpp/gpu/impl/shader.cpp` + +Namespace: `cthreads::gpu::shader`. + +Depends on: [Context](./context.md). Used by: [Descriptors](./descriptors.md) (set layout), [Module / launch](./module.md). + +## Purpose + +Building a compute pipeline from SPIR-V is expensive. Doing it on every launch would waste time. The shader cache stores, per kernel symbol, the reusable Vulkan objects that stay identical across launches: + +- Shader module (SPIR-V wrapped for Vulkan) +- Descriptor set layout (binding convention schema) +- Pipeline layout (how sets attach to the pipeline) +- Compute pipeline (compiled program ready to bind) +- Binding count metadata + +Per-launch objects (GpuPack buffers, descriptor sets, fences) are not stored here. + +## Access rights + +- Writers (registry / testing, via `add` once friended): insert new entries. +- Everyone else: `get` returns a const reference. +- Context shutdown: `clear` destroys all Vulkan objects, then empties the map. + +Entries are not mutated in place after insert. Duplicate `add` of the same key throws. + +## Technical terms + +- SPIR-V: binary intermediate language for shaders. Vulkan drivers consume SPIR-V, not GLSL text, at runtime. +- Shader module: Vulkan object created from SPIR-V bytes (`VkShaderModule`). +- Compute pipeline: prepared compute program plus layout (`VkPipeline` with compute bind point). +- Pipeline layout: declares which descriptor set layouts (and optional push constants) a pipeline uses. +- Descriptor set layout: schema of bindings; see [Descriptors](./descriptors.md). +- Entry point name: function name inside the shader. cthreads uses `"main"`. +- Push constants: tiny values pushed in the command buffer without a buffer object. Not used in create_entry today; scalars live in binding 0. + +## Struct `ShaderCacheEntry` + +Move-only. Copying would duplicate Vulkan handles and double-destroy them. + +Fields: + +- `shader_module`: may remain non-null after pipeline create (kept for simplicity; `clear` destroys it). +- `set_layout`: storage buffer bindings `0 .. binding_count-1` (scalars then lists). +- `pipeline_layout`: layout used when creating the pipeline. +- `pipeline`: compute pipeline handle. +- `binding_count`: number of storage buffer bindings (at least 1). + +## Class `ShaderCache` + +Process-wide singleton via `getInstance()`. + +### `add(key, entry)` (private until registry friend exists) + +Moves the entry into an internal `unordered_map`. Returns a const reference to the map node. Throws if the key already exists. + +### `get(key)` + +Returns a const reference to an existing entry. Throws if missing. + +### `clear(context)` + +Destroys every entry's Vulkan objects through `destroy_entry`, then clears the map. Called from Context shutdown before the logical device is destroyed. + +### Destructor + +Only drops the map. Shutdown must have already cleared handles, because static destruction order versus Context is undefined. + +## Function `create_entry` + +Declared in `shader.hpp`, implemented in `shader.cpp`. + +Builds a complete `ShaderCacheEntry` from SPIR-V words and a binding count. Does not insert into the cache; the caller (registry or test) calls `add`. + +### Parameters + +- `context`: ready Context with create and destroy entry points. +- `spirv`: pointer to SPIR-V code as `uint32_t` words. +- `spirv_word_count`: number of words (byte size divided by 4). +- `binding_count`: storage buffer bindings (`>= 1`). + +### Steps + +1. Validate ready device, non-empty SPIR-V, binding count, and create PFNs. +2. `vkCreateShaderModule` from the SPIR-V bytes. +3. Build `binding_count` layout bindings, each `STORAGE_BUFFER`, compute stage, then `vkCreateDescriptorSetLayout`. +4. `vkCreatePipelineLayout` with that single set layout and no push constant ranges. +5. `vkCreateComputePipelines` with stage compute, module, entry `"main"`, and the pipeline layout. +6. On any failure after partial success, call `destroy_entry` and throw. + +### Return + +Owned `ShaderCacheEntry`. Move it into `ShaderCache::add`, or destroy it with `destroy_entry` if you abandon it. + +## Function `destroy_entry` + +Destroys pipeline, pipeline layout, set layout, and shader module in that order (children before parents), then nulls handles. Used by `ShaderCache::clear` and by fail paths in `create_entry`. + +If the device handle is already gone, it only nulls fields (shutdown edge case). + +## Key Vulkan calls + +| Call | Role | +|-|-| +| `vkCreateShaderModule` | Wrap SPIR-V | +| `vkCreateDescriptorSetLayout` | Binding convention schema | +| `vkCreatePipelineLayout` | Attach set layout to pipeline interface | +| `vkCreateComputePipelines` | Compile compute pipeline | +| Matching `vkDestroy*` | Tear down in `destroy_entry` / `clear` | + +## Where SPIR-V comes from + +Issue 3 feeds committed SPIR-V (or library-built smoke shaders) under `_ext.gpu.testing`. Later, `@Gpu` compilation will produce SPIR-V and call the same `create_entry` path. + +## What create_entry does not do + +- Allocate descriptor sets or pools +- Point bindings at a GpuPack +- Record dispatch +- Register the entry in the cache (caller must `add`) + +Those are separate steps on the launch timeline. + +## Relationship to the cache key + +The cache key is a string symbol (stable kernel name). Long term, content hashing of SPIR-V may be stored inside the entry for invalidation. Today the contract is: one symbol maps to one immutable entry for the process lifetime after `add`. diff --git a/docs/vk_guide/00-read-me-first.md b/docs/vk_guide/00-read-me-first.md index 1320c22..0c24347 100644 --- a/docs/vk_guide/00-read-me-first.md +++ b/docs/vk_guide/00-read-me-first.md @@ -76,7 +76,7 @@ Same idea (SSBO = storage buffer), more paperwork. ## Locked decisions in this project (so generic tutorials do not confuse contributors) -1. **Option 5 pack:** one scalar SSBO + one SSBO per `list`. +1. **Binding convention pack:** one scalar SSBO + one SSBO per `list`. 2. **Device-local** data for shaders; **staging** for CPU copies. 3. **Descriptors** bind buffers by binding index (not pointers in the scalar struct). 4. **Launch then join** — no mid-run Python `__sync_state` on GPU. diff --git a/docs/vk_guide/01-cthreads-gpu-big-picture.md b/docs/vk_guide/01-cthreads-gpu-big-picture.md index ef58c0e..892d034 100644 --- a/docs/vk_guide/01-cthreads-gpu-big-picture.md +++ b/docs/vk_guide/01-cthreads-gpu-big-picture.md @@ -48,7 +48,7 @@ Python list[float] <--writeback-- staging <--copy-- device-local SSBO ## Why not one giant buffer for everything? -cthreads uses **option 5**: +cthreads uses this binding convention: | Piece | Where it lives | |-------|----------------| @@ -81,7 +81,7 @@ That matches how GPUs want to work: record work, submit, wait once. |-------|----------------| | Context | Talk to Vulkan: instance, device, queue, entry points | | Memory helpers | Allocate buffers, staging upload/download | -| GpuPack | Option 5: scalar SSBO + per-list SSBOs; marshal/writeback | +| GpuPack | Scalar SSBO + per-list SSBOs; marshal/writeback | | Launch path | Descriptors, pipeline, dispatch, `GpuJob.join` | | `@Gpu` / `gpu()` | Compile and run user kernels (same typing model as CPU) | | Workloads / packaging | Real numeric steps, docs, CI, capability gates | diff --git a/docs/vk_guide/02-mental-model.md b/docs/vk_guide/02-mental-model.md index cb70a1d..0de4d2c 100644 --- a/docs/vk_guide/02-mental-model.md +++ b/docs/vk_guide/02-mental-model.md @@ -118,7 +118,7 @@ Python cthreads.gpu / future gpu() pybind _ext.gpu | v -GpuPack (option 5) +GpuPack (binding convention) | v memory:: create/upload/download diff --git a/docs/vk_guide/09-descriptors-ssbo.md b/docs/vk_guide/09-descriptors-ssbo.md index ba0d26a..468a859 100644 --- a/docs/vk_guide/09-descriptors-ssbo.md +++ b/docs/vk_guide/09-descriptors-ssbo.md @@ -36,7 +36,7 @@ That means: On the C++ side, binding 1's descriptor must reference the `VkBuffer` that holds those floats. -## cthreads binding convention (option 5) +## cthreads binding convention | Binding | Contents | |---------|----------| @@ -62,7 +62,7 @@ No pointers inside `Scalars`. Bindings do the wiring. | `STORAGE_BUFFER` | `buffer { ... }` (read/write) | | `UNIFORM_BUFFER` | `uniform` block (we avoid for mutable pack) | -cthreads uses **storage buffers for scalars and lists** (option 5 = all SSBO). +cthreads uses **storage buffers for scalars and lists** (binding convention = all SSBO). ## Lifecycle sketch (launch path) diff --git a/docs/vk_guide/10-std430-layouts.md b/docs/vk_guide/10-std430-layouts.md index ff81405..d031190 100644 --- a/docs/vk_guide/10-std430-layouts.md +++ b/docs/vk_guide/10-std430-layouts.md @@ -73,7 +73,7 @@ Host must insert the same padding or use `alignas`. Do not put `float x[];` inside the scalar block when you also need `y[]`. Unsized arrays in std430 must be last, and you only get one. -That is another reason cthreads option 5 uses **separate list SSBOs**. +That is another reason cthreads uses **separate list SSBOs**. ## How codegen will help later diff --git a/docs/vk_guide/12-gpupack-marshal.md b/docs/vk_guide/12-gpupack-marshal.md index 3ff6a05..7c2c1a2 100644 --- a/docs/vk_guide/12-gpupack-marshal.md +++ b/docs/vk_guide/12-gpupack-marshal.md @@ -1,4 +1,4 @@ -# 12 — GpuPack and marshal (option 5 end-to-end) +# 12 - GpuPack and marshal (binding convention end-to-end) ## Definition diff --git a/docs/vk_guide/13-map-to-our-code.md b/docs/vk_guide/13-map-to-our-code.md index ecedd62..16fcc38 100644 --- a/docs/vk_guide/13-map-to-our-code.md +++ b/docs/vk_guide/13-map-to-our-code.md @@ -7,21 +7,24 @@ | `src/cthreads/cpp/gpu/headers/context.hpp` | `Context` handles + all `PFN_*` | | `src/cthreads/cpp/gpu/impl/context.cpp` | Load loader, init/shutdown, resolve entry points | | `src/cthreads/cpp/gpu/headers/memory.hpp` | `BufferKind`, `GpuBuffer`, memory API | -| `src/cthreads/cpp/gpu/impl/memory.cpp` | find/create/destroy/upload/download | +| `src/cthreads/cpp/gpu/impl/memory.cpp` | find/create/destroy/upload/download + TransferEngine | +| `src/cthreads/cpp/gpu/headers/pack.hpp` | `GpuPack` create/upload/download | +| `src/cthreads/cpp/gpu/impl/pack.cpp` | Pack helpers | +| `src/cthreads/cpp/gpu/headers/descriptors.hpp` | Descriptor pool/set/update (namespace `pack`) | +| `src/cthreads/cpp/gpu/impl/descriptors.cpp` | Descriptor helpers | +| `src/cthreads/cpp/gpu/headers/shader.hpp` / `shader_cache.hpp` | `create_entry`, `ShaderCache` | +| `src/cthreads/cpp/gpu/impl/shader.cpp` / `shader_cache.cpp` | Pipeline build + cache | +| `src/cthreads/cpp/gpu/headers/module.hpp` | `SpawnedGpuKernel`, `launch_gpu_kernel` | +| `src/cthreads/cpp/gpu/impl/module.cpp` | Launch + join writeback | +| `src/cthreads/cpp/gpu/testing/` | Pack roundtrip + saxpy smoke SPIR-V | | `src/cthreads/cpp/bindings/gpu_module.cpp` | pybind `cthreads._ext.gpu` | +| `src/cthreads/cpp/bindings/gpu_testing_module.cpp` | Test-only `_ext.gpu.testing` | | `src/cthreads/cpp/bindings/module.cpp` | `#ifdef CTHREADS_WITH_GPU` calls `bind_gpu` | | `src/cthreads/cpp/CMakeLists.txt` | `CTHREADS_GPU` option + sources | | `src/cthreads/python/cthreads/gpu/` | Python façade + errors | -| `tests/unit/test_gpu_context.py` | Context / availability tests (skip if no GPU) | +| `tests/unit/test_gpu_*.py` | Context / pack / shader+launch tests | -Expected as the GPU stack grows: - -| Path (expected) | Role | -|-----------------|------| -| `gpu/headers/pack.hpp` (name may vary) | GpuPack | -| `gpu/.../transfer.*` | Reused command pool / staging | -| shader `.spv` / GLSL | Reference saxpy | -| more bindings | roundtrip / launch | +Newcomer-oriented C++ module docs: [docs/internals/gpu/README.md](../internals/gpu/README.md). ## How to study with the code open @@ -39,17 +42,18 @@ Expected as the GPU stack grows: 3. Walk `create_buffer` and `upload_buffer` in `memory.cpp` line by line. 4. Mentally simulate uploading 4 floats. -### Pass 3 — Descriptors and shaders +### Pass 3 — Pack, descriptors, shaders, launch 1. Read guide 09-12. -2. Sketch on paper the saxpy descriptor layout. -3. Sketch the command buffer for copy + dispatch + fence. +2. Open `pack.hpp` / `descriptors.hpp` / `shader_cache.hpp` / `module.hpp`. +3. Trace `smoke_launch_saxpy` in `gpu/testing/shader_smoke.cpp` end-to-end. +4. Run `tests/unit/test_gpu_shader.py::test_live_smoke_launch_saxpy`. ## Build flag reminder ```bat set CMAKE_ARGS=-DCTHREADS_GPU=ON -pip install -e . -v +pip install -e . ``` Wipe `build/` if CMake cached GPU off. @@ -62,6 +66,7 @@ C++ throws `std::runtime_error` with prefixes like: cthreads.gpu.VulkanLoaderNotFound: ... cthreads.gpu.VulkanInitFailed: ... cthreads.gpu.VulkanNoDevice: ... +cthreads.gpu.GpuInvalidArgument: ... ``` `cthreads.gpu` catches and raises typed exceptions. Keep prefixes stable when adding new errors. diff --git a/docs/vk_guide/14-glossary.md b/docs/vk_guide/14-glossary.md index baa145d..ae6df62 100644 --- a/docs/vk_guide/14-glossary.md +++ b/docs/vk_guide/14-glossary.md @@ -34,7 +34,7 @@ | **Pipeline (compute)** | Prepared compute shader + layout | | **Dispatch** | Launch compute workgroups (`vkCmdDispatch`) | | **Workgroup / local size** | Group of invocations that run together | -| **GpuPack** | Per-launch scalar SSBO + list SSBOs (option 5) | +| **GpuPack** | Per-launch scalar SSBO + list SSBOs (binding convention) | | **Marshal** | Copy Python values into native/GPU pack storage | | **Writeback** | Copy native/GPU results into the same Python objects | | **Join** | Wait for GPU job completion then writeback | diff --git a/docs/vk_guide/15-checklist.md b/docs/vk_guide/15-checklist.md index f424b7d..4769ed1 100644 --- a/docs/vk_guide/15-checklist.md +++ b/docs/vk_guide/15-checklist.md @@ -33,7 +33,7 @@ Use this after reading. If a concept cannot be explained in plain language, revi - [ ] cthreads binding 0 = scalars, 1..N = lists convention - [ ] What std430 padding is and why host/GPU must match - [ ] SPIR-V vs GLSL vs pipeline vs dispatch -- [ ] What GpuPack contains (option 5) +- [ ] What GpuPack contains (binding convention) - [ ] Why GPU jobs are launch/wait only (no mid-run sync) ## When stuck diff --git a/docs/vk_guide/README.md b/docs/vk_guide/README.md index 75b72c8..d8b7c49 100644 --- a/docs/vk_guide/README.md +++ b/docs/vk_guide/README.md @@ -6,7 +6,7 @@ shaders, and launch/wait — **not** the full graphics stack (swapchains, render passes, images, and so on). Architecture choices here match the cthreads GPU design (same Python types as the -CPU backend, GpuPack option 5, device-local + staging). Implementation status in +CPU backend, GpuPack binding convention, device-local + staging). Implementation status in the tree may move faster or slower than any particular roadmap document; treat this guide as the conceptual reference, and the source under `src/cthreads/cpp/gpu/` as ground truth for what is already landed. @@ -35,7 +35,7 @@ chapter as a reference. | 09 | [09-descriptors-ssbo.md](./09-descriptors-ssbo.md) | How shaders see buffers | | 10 | [10-std430-layouts.md](./10-std430-layouts.md) | Scalar struct packing | | 11 | [11-spirv-pipelines-dispatch.md](./11-spirv-pipelines-dispatch.md) | Shaders, pipelines, `dispatch` | -| 12 | [12-gpupack-marshal.md](./12-gpupack-marshal.md) | Option 5 pack end-to-end | +| 12 | [12-gpupack-marshal.md](./12-gpupack-marshal.md) | GpuPack end-to-end | | 13 | [13-map-to-our-code.md](./13-map-to-our-code.md) | Files in `src/cthreads/cpp/gpu/` | | 14 | [14-glossary.md](./14-glossary.md) | Terms in one place | | 15 | [15-checklist.md](./15-checklist.md) | Concepts a contributor should be able to explain | diff --git a/src/cthreads/cpp/CMakeLists.txt b/src/cthreads/cpp/CMakeLists.txt index 186014b..df89632 100644 --- a/src/cthreads/cpp/CMakeLists.txt +++ b/src/cthreads/cpp/CMakeLists.txt @@ -149,10 +149,15 @@ if(CTHREADS_GPU) "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/context.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/memory.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/pack.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/shader_cache.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/shader.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/descriptors.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/module.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/gpu/testing/pack_roundtrip.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/testing/shader_smoke.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/bindings/gpu_module.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/bindings/gpu_testing_module.cpp" - # GPU-02: memory + GpuPack marshal substrate (+ test-only roundtrip bindings) + # GPU: context, memory, GpuPack, shader cache/entry (+ test-only roundtrip) ) target_include_directories(_ext PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/gpu/headers diff --git a/src/cthreads/cpp/bindings/gpu_testing_module.cpp b/src/cthreads/cpp/bindings/gpu_testing_module.cpp index a4760ac..81984f0 100644 --- a/src/cthreads/cpp/bindings/gpu_testing_module.cpp +++ b/src/cthreads/cpp/bindings/gpu_testing_module.cpp @@ -5,6 +5,7 @@ #include "gpu_testing_module.hpp" #include "../gpu/testing/pack_roundtrip.hpp" +#include "../gpu/testing/shader_smoke.hpp" #include #include @@ -73,4 +74,40 @@ void bind_gpu_testing(py::module_& gpu_parent) { &cthreads::gpu::testing::probe_use_after_destroy_scalars, "Raises GpuUseAfterDestroy (upload into pack with no scalar buffer). Test-only." ); + + t.def( + "smoke_create_entry", + &cthreads::gpu::testing::smoke_create_entry, + "create_entry + destroy for committed smoke SPIR-V. Test-only." + ); + t.def( + "smoke_update_descriptors", + &cthreads::gpu::testing::smoke_update_descriptors, + "create_entry + pack + update_descriptors smoke. Test-only." + ); + t.def( + "smoke_cache_register_and_get", + &cthreads::gpu::testing::smoke_cache_register_and_get, + "ShaderCache add/get/clear smoke. Test-only." + ); + t.def( + "probe_cache_duplicate_add", + &cthreads::gpu::testing::probe_cache_duplicate_add, + "Raises on duplicate ShaderCache add. Test-only." + ); + t.def( + "probe_update_empty_list_slot", + &cthreads::gpu::testing::probe_update_empty_list_slot, + "Raises GpuInvalidArgument for empty list descriptor. Test-only." + ); + t.def( + "probe_create_entry_zero_bindings", + &cthreads::gpu::testing::probe_create_entry_zero_bindings, + "Raises GpuInvalidArgument for binding_count 0. Test-only." + ); + t.def( + "smoke_launch_saxpy", + &cthreads::gpu::testing::smoke_launch_saxpy, + "Register smoke SPIR-V, launch_gpu_kernel saxpy, join writeback, check y. Test-only." + ); } diff --git a/src/cthreads/cpp/bindings/gpu_testing_module.hpp b/src/cthreads/cpp/bindings/gpu_testing_module.hpp index df0276b..3f385ed 100644 --- a/src/cthreads/cpp/bindings/gpu_testing_module.hpp +++ b/src/cthreads/cpp/bindings/gpu_testing_module.hpp @@ -5,7 +5,7 @@ namespace py = pybind11; /** - * Register ``cthreads._ext.gpu.testing`` (GpuPack round-trip smoke API). - * Test-only — not part of the public ``cthreads.gpu`` package. + * Register `cthreads._ext.gpu.testing` (GpuPack round-trip smoke API). + * Test-only. Not part of the public `cthreads.gpu` package. */ void bind_gpu_testing(py::module_& gpu_parent); diff --git a/src/cthreads/cpp/gpu/headers/context.hpp b/src/cthreads/cpp/gpu/headers/context.hpp index 21e2401..36c778e 100644 --- a/src/cthreads/cpp/gpu/headers/context.hpp +++ b/src/cthreads/cpp/gpu/headers/context.hpp @@ -2,6 +2,7 @@ #include #include #include +#include #include "memory.hpp" @@ -69,6 +70,28 @@ struct Context { PFN_vkWaitForFences vkWaitForFences = nullptr; // blocks the CPU until the given fences signal PFN_vkResetFences vkResetFences = nullptr; // resets fences back to unsignaled for reuse + // Shader / pipeline create + destroy (entry build and ShaderCache::clear). + PFN_vkCreateShaderModule vkCreateShaderModule = nullptr; + PFN_vkDestroyShaderModule vkDestroyShaderModule = nullptr; + PFN_vkCreateDescriptorSetLayout vkCreateDescriptorSetLayout = nullptr; + PFN_vkDestroyDescriptorSetLayout vkDestroyDescriptorSetLayout = nullptr; + PFN_vkCreatePipelineLayout vkCreatePipelineLayout = nullptr; + PFN_vkDestroyPipelineLayout vkDestroyPipelineLayout = nullptr; + PFN_vkCreateComputePipelines vkCreateComputePipelines = nullptr; + PFN_vkDestroyPipeline vkDestroyPipeline = nullptr; + + // Descriptor pool / set / update (per-launch wiring of GpuPack buffers). + PFN_vkCreateDescriptorPool vkCreateDescriptorPool = nullptr; + PFN_vkDestroyDescriptorPool vkDestroyDescriptorPool = nullptr; + PFN_vkAllocateDescriptorSets vkAllocateDescriptorSets = nullptr; + PFN_vkFreeDescriptorSets vkFreeDescriptorSets = nullptr; + PFN_vkUpdateDescriptorSets vkUpdateDescriptorSets = nullptr; + + // Compute dispatch recording (launch_gpu_kernel command buffers). + PFN_vkCmdBindPipeline vkCmdBindPipeline = nullptr; + PFN_vkCmdBindDescriptorSets vkCmdBindDescriptorSets = nullptr; + PFN_vkCmdDispatch vkCmdDispatch = nullptr; + PFN_vkCmdPipelineBarrier vkCmdPipelineBarrier = nullptr; // Opaque Vulkan handles. VkInstance instance = VK_NULL_HANDLE; // connection to the loader/app @@ -82,7 +105,9 @@ struct Context { // True only after init() fully succeeded. bool ready = false; + // These are both temporary until the cpu side @gpu calls are implemented (this however is a future poject and not on the current timeline) TransferEngine transfer_engine; + std::mutex transfer_engine_mutex; }; // Process-wide singleton accessor. Context& context(); diff --git a/src/cthreads/cpp/gpu/headers/descriptors.hpp b/src/cthreads/cpp/gpu/headers/descriptors.hpp new file mode 100644 index 0000000..3a2ebbd --- /dev/null +++ b/src/cthreads/cpp/gpu/headers/descriptors.hpp @@ -0,0 +1,168 @@ +#pragma once + +#include +#include + +#include "pack.hpp" +#include "shader_cache.hpp" + +namespace cthreads::gpu { +struct Context; +} + +/** + * Per-launch descriptor helpers for GpuPack wiring (namespace pack). + * + * The set layout and pipeline live on ShaderCacheEntry (created once per + * symbol). Each job allocates a set, calls update_descriptors with that job's + * GpuPack, then binds the set at dispatch time. + * + * #### Technical terms: + * - descriptor set layout: schema of bindings (from create_entry / cache). + * - descriptor pool: allocator from which concrete descriptor sets are taken. + * - descriptor set: one instance of the layout; holds VkBuffer pointers for a launch. + * - update_descriptors: writes binding i -> GpuPack buffer i (scalars then lists). + */ +namespace cthreads::gpu::pack { + +/** + * Pool that can allocate descriptor sets for a fixed STORAGE_BUFFER binding count. + * + * Created with FREE_DESCRIPTOR_SET_BIT so sets can be returned with free_set + * when a job finishes. Destroy the pool only after all sets from it are freed + * (or the device is shutting down and no jobs remain). + * + * #### Fields: + * - pool: VkDescriptorPool = Vulkan pool handle; VK_NULL_HANDLE if empty. + * - max_sets: uint32_t = how many sets this pool can still hold at create time. + * - binding_count: uint32_t = STORAGE_BUFFER descriptors per set (binding convention). + */ +struct DescriptorPool { + VkDescriptorPool pool = VK_NULL_HANDLE; + uint32_t max_sets = 0; + uint32_t binding_count = 0; +}; + +/** + * Creates a descriptor pool for compute sets that follow the binding convention. + * + * Each set needs binding_count STORAGE_BUFFER descriptors. The pool can + * allocate up to max_sets such sets. Sets may be freed individually. + * + * #### Parameters: + * - context: Context& = initialized GPU context with descriptor pool entry points. + * - binding_count: uint32_t = bindings per set (must match ShaderCacheEntry). + * - max_sets: uint32_t = maximum sets this pool may allocate (>= 1). + * + * #### Returns: + * - DescriptorPool = owned pool; caller must destroy_pool when done. + * + * #### Throws: + * - runtime_error if context is not ready, counts are 0, PFNs are missing, or + * vkCreateDescriptorPool fails. + */ +DescriptorPool create_pool( + cthreads::gpu::Context& context, + uint32_t binding_count, + uint32_t max_sets +); + +/** + * Destroys a descriptor pool and resets the DescriptorPool fields. + * + * All sets allocated from this pool become invalid. Prefer free_set on each + * live set first when jobs may still hold sets. + * + * #### Parameters: + * - context: Context& = same device that created the pool. + * - pool: DescriptorPool& = pool to destroy; left empty on return. + */ +void destroy_pool( + cthreads::gpu::Context& context, + DescriptorPool& pool +); + +/** + * Allocates one descriptor set from the pool using the given set layout. + * + * The layout must match the pool's binding_count (same layout used in + * ShaderCacheEntry::set_layout). The set is empty until update_descriptors. + * + * #### Parameters: + * - context: Context& = initialized GPU context. + * - pool: DescriptorPool& = pool with remaining capacity. + * - set_layout: VkDescriptorSetLayout = layout from ShaderCacheEntry. + * + * #### Returns: + * - VkDescriptorSet = allocated set (not a owned C++ type; free with free_set). + * + * #### Throws: + * - runtime_error if the pool is empty/invalid, layout is null, or allocate fails. + */ +VkDescriptorSet allocate_set( + cthreads::gpu::Context& context, + DescriptorPool& pool, + VkDescriptorSetLayout set_layout +); + +/** + * Returns a set to its pool. Safe no-op if set is VK_NULL_HANDLE. + * + * #### Parameters: + * - context: Context& = same device as the pool. + * - pool: DescriptorPool& = pool that allocated the set. + * - set: VkDescriptorSet& = set to free; set to VK_NULL_HANDLE on return. + * + * #### Throws: + * - runtime_error if free fails (pool must have been created with free bit). + */ +void free_set( + cthreads::gpu::Context& context, + DescriptorPool& pool, + VkDescriptorSet& set +); + +/** + * Writes binding-convention buffer bindings from a GpuPack into a descriptor set. + * + * Binding 0 -> pack.scalar_buffer. Bindings 1..N -> pack.container_slots[0..N-1]. + * binding_count must equal 1 + pack.container_slots.size() and match the set + * layout. Every binding must have a non-null VkBuffer (empty list slots are + * not supported here yet; use a non-empty buffer or a future dummy SSBO). + * + * Called by the launch path after allocate_set and before recording bind/dispatch. + * + * #### Parameters: + * - context: Context& = initialized GPU context with vkUpdateDescriptorSets. + * - set: VkDescriptorSet = destination set from allocate_set. + * - binding_count: uint32_t = number of STORAGE_BUFFER bindings to write. + * - pack: const GpuPack& = source buffers in binding order (scalars then lists). + * + * #### Throws: + * - runtime_error if set is null, binding_count mismatches the pack, any + * required buffer handle is null, or update entry points are missing. + */ +void update_descriptors( + cthreads::gpu::Context& context, + VkDescriptorSet set, + uint32_t binding_count, + const GpuPack& pack +); + +/** + * Convenience: update_descriptors using entry.binding_count. + * + * #### Parameters: + * - context: Context& = initialized GPU context. + * - set: VkDescriptorSet = destination set. + * - entry: const ShaderCacheEntry& = cached layout metadata (binding_count). + * - pack: const GpuPack& = source buffers. + */ +void update_descriptors( + cthreads::gpu::Context& context, + VkDescriptorSet set, + const cthreads::gpu::shader::ShaderCacheEntry& entry, + const GpuPack& pack +); + +} // namespace cthreads::gpu::pack diff --git a/src/cthreads/cpp/gpu/headers/inflight_store.hpp b/src/cthreads/cpp/gpu/headers/inflight_store.hpp new file mode 100644 index 0000000..6f70f09 --- /dev/null +++ b/src/cthreads/cpp/gpu/headers/inflight_store.hpp @@ -0,0 +1 @@ +#pragma once diff --git a/src/cthreads/cpp/gpu/headers/memory.hpp b/src/cthreads/cpp/gpu/headers/memory.hpp index cfd0d65..88b4702 100644 --- a/src/cthreads/cpp/gpu/headers/memory.hpp +++ b/src/cthreads/cpp/gpu/headers/memory.hpp @@ -40,7 +40,7 @@ enum class BufferKind : uint8_t { /** * One contiguous byte region on the GPU (staging scratch, scalar SSBO, or one list). * - * Owns the Vulkan buffer object and the device memory bound to it. Option 5 packs + * Owns the Vulkan buffer object and the device memory bound to it. GpuPacks * use several device-local GpuBuffers: one for all scalars, then one per list. * * #### Fields: diff --git a/src/cthreads/cpp/gpu/headers/module.hpp b/src/cthreads/cpp/gpu/headers/module.hpp new file mode 100644 index 0000000..ca15fc7 --- /dev/null +++ b/src/cthreads/cpp/gpu/headers/module.hpp @@ -0,0 +1,212 @@ +#pragma once + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "pack.hpp" +#include "descriptors.hpp" + +namespace py = pybind11; + +namespace cthreads::gpu { + +struct Context; + +/** + * Per-launch GPU job handle (mirror of CPU SpawnedKernel, without an OS thread). + * + * CPU kernels run on a CThread. GPU kernels run on the device after + * vkQueueSubmit; this struct owns the host-side inflight state and waits on a + * fence in join(). There is no CGpuThread. + * + * Typical lifetime: + * 1. Launch path fills pack, descriptor set, records/submits with fence. + * 2. Caller may overlap other host work. + * 3. join() waits on the fence, downloads, writeback, then releases GPU objects. + * + * Methods are declared here; record/submit/join bodies land with the launch path. + * + * #### Fields: + * - pack: GpuPack = device-local scalar + list SSBOs for this launch. + * - descriptor_pool: DescriptorPool = pool that allocated descriptor_set (for free_set). + * - descriptor_set: VkDescriptorSet = bindings wired to pack buffers. + * - command_buffer: VkCommandBuffer = recorded dispatch (optional until submit path). + * - command_pool: VkCommandPool = pool that owns command_buffer (for free). + * - fence: VkFence = signals when the submitted dispatch has finished. + * - symbol: string = shader cache key for this kernel. + * - group_count_x/y/z: uint32_t = vkCmdDispatch workgroup counts. + * - values_keep: shared_ptr to py::list = Python args kept alive for list writeback. + * - writeback_lists: plan of ref list slots to download into values_keep on join. + * - finished: bool = true after join completed writeback / teardown. + * - eptr: exception_ptr = error captured during launch or join (rethrown on join). + * + * #### Technical terms: + * - Fence: GPU timeline object the CPU waits on until submitted work completes. + * - Descriptor set: per-launch table binding i -> pack buffer i. + * - GpuPack: binding-convention buffer bag (scalars at 0, lists at 1..N). + */ +struct SpawnedGpuKernel { + /** + * One list argument to download into the kept Python list on join. + * Built at launch from meta (pass_as ref) + live numel. + */ + struct WritebackListSlot { + size_t value_index = 0; // index into values_keep + size_t container_index = 0; // index into pack.container_slots + size_t numel = 0; + std::string elem_kind; // "float" / "int" / "double" + }; + + pack::GpuPack pack{}; + pack::DescriptorPool descriptor_pool{}; + VkDescriptorSet descriptor_set = VK_NULL_HANDLE; + VkCommandBuffer command_buffer = VK_NULL_HANDLE; + VkCommandPool command_pool = VK_NULL_HANDLE; + VkFence fence = VK_NULL_HANDLE; + + std::string symbol; + uint32_t group_count_x = 1; + uint32_t group_count_y = 1; + uint32_t group_count_z = 1; + + // Same Python list objects the caller passed (CPU SpawnedKernel values_keep). + std::shared_ptr values_keep; + // Ref list slots only; value scalars are not written back. + std::vector writeback_lists; + + bool finished = false; + std::mutex done_mu; + std::condition_variable done_cv; + bool done_flag = false; + std::exception_ptr eptr; + + SpawnedGpuKernel() = default; + SpawnedGpuKernel(const SpawnedGpuKernel&) = delete; + SpawnedGpuKernel& operator=(const SpawnedGpuKernel&) = delete; + SpawnedGpuKernel(SpawnedGpuKernel&&) = delete; + SpawnedGpuKernel& operator=(SpawnedGpuKernel&&) = delete; + + /** + * No-op for the default path (work is submitted at launch time). + * Kept so Python Job-shaped wrappers can share a start/join/done surface + * with CPU SpawnedKernel. + */ + void start(); + + /** + * Wait until the GPU fence signals, then download/writeback and release + * inflight GPU objects. Rethrows eptr if set. Idempotent after finished. + * + * #### Parameters: + * - context: Context& = same device that created pack / submitted work. + */ + void join(Context& context); + + /** + * Block until done_flag is set (join or failure path). Does not download. + * Prefer join() for the full writeback and teardown path. + */ + void wait(); + + /** + * True after the GPU work is complete (done_flag). Does not perform writeback. + */ + bool done(); + + /** + * Destroy remaining Vulkan objects if join was never called. + * Safe if already finished / empty. + */ + ~SpawnedGpuKernel(); +}; + +/** + * GPU analog of CPU spawn_from_meta: marshal args, submit compute, return a job. + * + * Ensures the process Context is ready, resolves the kernel from meta (shader + * cache symbol / SPIR-V), builds and uploads a GpuPack from ordered_values, + * allocates and updates a descriptor set, records bind+dispatch, submits with + * a fence, and returns an owned SpawnedGpuKernel. Does not wait; call + * job->join(context) for fence wait, download, and writeback. + * + * Unlike spawn_from_meta there is no CThread and no pool argument. Work runs on + * the GPU after vkQueueSubmit on the calling host thread. + * + * #### Parameters: + * - meta: py::dict = kernel metadata (symbol, binding layout / scalar size, + * container specs, dispatch size, and later types/schemas for writeback). + * Shape will align with @Gpu __kernel_meta__ when emit exists; smoke tests may + * pass a minimal dict. + * - ordered_values: py::list = Python args in parameter order matching meta + * (scalars flattened into the scalar SSBO; lists map to bindings 1..N). + * + * #### Returns: + * - shared_ptr = inflight job (pack, descriptors, fence). + * Caller owns the pointer until join/destroy. + * + * #### Throws: + * - runtime_error / type_error style errors if Context is not ready, meta is + * incomplete, arity mismatches, cache/SPIR-V is missing, or Vulkan submit fails. + * + * #### Example meta shape (saxpy-style @Gpu): + * ``py + * { + * "symbol": "saxpy", # ShaderCache key / kernel name + * "binding_count": 3, # STORAGE_BUFFER bindings (0 scalars + 2 lists) + * "scalar_bytes": 8, # std430 scalar SSBO size (e.g. int n + float a) + * "local_size_x": 64, # compute workgroup size (shader layout) + * "group_count_x": None, # optional override; else ceil(n / local_size_x) + * "group_count_y": 1, + * "group_count_z": 1, + * "params": [ + * { + * "name": "n", + * "kind": "int", # packed into scalar SSBO (binding 0) + * "pass_as": "value", + * }, + * { + * "name": "a", + * "kind": "float", + * "pass_as": "value", + * }, + * { + * "name": "x", + * "kind": "list", # binding 1 + * "pass_as": "ref", + * "elem_kind": "float", + * "elem_bytes": 4, + * }, + * { + * "name": "y", + * "kind": "list", # binding 2 + * "pass_as": "ref", + * "elem_kind": "float", + * "elem_bytes": 4, + * }, + * ], + * "types": {}, # optional: Python classes for writeback + * "schemas": {}, # optional: field layouts for Threadables + * } + * `` + * ordered_values for that meta would be like ``[n, a, x_list, y_list]``. + * Scalars are flattened into one SSBO in param order; each list gets its own + * binding 1..N. Smoke tests may omit types/schemas and pass a smaller dict. + * + * #### Technical terms: + * - spawn_from_meta: CPU launcher that builds SpawnedKernel + CThread/pool task. + * - Shader cache: process map of symbol -> reusable pipeline and set layout. + * - Descriptor set: per-launch binding table from GpuPack buffers to the shader. + */ +std::shared_ptr launch_gpu_kernel( + py::dict meta, + py::list ordered_values +); + +} // namespace cthreads::gpu diff --git a/src/cthreads/cpp/gpu/headers/pack.hpp b/src/cthreads/cpp/gpu/headers/pack.hpp index 67ea1ba..d989a82 100644 --- a/src/cthreads/cpp/gpu/headers/pack.hpp +++ b/src/cthreads/cpp/gpu/headers/pack.hpp @@ -11,8 +11,9 @@ struct Context; } /** - * Option 5 GpuPack: one device-local scalar SSBO plus one device-local SSBO per + * GpuPack: one device-local scalar SSBO plus one device-local SSBO per * list/container. Host traffic goes through memory:: upload/download helpers. + * Descriptor helpers wire a pack into a per-launch descriptor set for dispatch. * * This is a generic runtime bag of buffers. Per-kernel std430 layout and which * Python arg maps to which slot are marshal/codegen concerns, not this type. @@ -49,7 +50,7 @@ struct ContainerSlot { }; /** - * Per-launch GPU argument pack (option 5). + * Per-launch GPU argument pack (binding convention: scalars then lists). * * Owns Vulkan allocations until destroy_gpu_pack. Returning GpuPack by value * moves handles only; device bytes are not copied. diff --git a/src/cthreads/cpp/gpu/headers/shader.hpp b/src/cthreads/cpp/gpu/headers/shader.hpp new file mode 100644 index 0000000..960fae9 --- /dev/null +++ b/src/cthreads/cpp/gpu/headers/shader.hpp @@ -0,0 +1,85 @@ +#pragma once + +#include +#include + +#include "shader_cache.hpp" + +namespace cthreads::gpu { +struct Context; +} + +/** + * Stateless helpers that build and tear down ShaderCacheEntry Vulkan objects. + * + * create_entry turns SPIR-V + a binding count into the reusable pipeline row + * stored in ShaderCache. It does not allocate per-launch descriptor sets or + * bind a GpuPack; that is the launch / inflight path. + * + * Pass a ready Context with shader/pipeline create (and destroy) entry points. + * + * #### Technical terms: + * - SPIR-V: binary compute shader (uint32 words). Built for tests as committed + * bytes or via shaderc; @Gpu emit feeds the same helper later. + * - binding_count: STORAGE_BUFFER bindings for the binding convention + * (1 = scalars only, or 1 + number of list SSBOs). + * - ShaderCacheEntry: module + set layout + pipeline layout + compute pipeline. + */ +namespace cthreads::gpu::shader { + +/** + * Creates a ShaderCacheEntry from SPIR-V and a binding-convention binding count. + * + * Builds, in order: shader module, descriptor set layout (bindings 0 .. + * binding_count-1 as STORAGE_BUFFER), pipeline layout, compute pipeline. + * On failure, destroys any objects already created and throws. + * + * Does not insert into ShaderCache; the registry (or test harness) calls + * ShaderCache::add with the returned entry. + * + * #### Parameters: + * - context: Context& = initialized GPU context with device and create entry points. + * - spirv: const uint32_t* = SPIR-V code words (must be valid compute SPIR-V). + * - spirv_word_count: size_t = number of uint32 words in spirv (byte size / 4). + * - binding_count: uint32_t = number of SSBO bindings (must be >= 1). + * + * #### Returns: + * - ShaderCacheEntry = owned Vulkan handles; move into ShaderCache::add or + * destroy_entry when abandoning the entry. + * + * #### Throws: + * - runtime_error if context is not ready, spirv is null/empty, binding_count + * is 0, create entry points are missing, or any Vulkan create fails. + * + * #### Technical terms: + * - VkShaderModule: Vulkan wrapper around SPIR-V bytes. + * - VkDescriptorSetLayout: schema only; concrete descriptor sets are per launch. + * - VkPipeline: compiled compute program ready for vkCmdBindPipeline. + */ +ShaderCacheEntry create_entry( + cthreads::gpu::Context& context, + const uint32_t* spirv, + size_t spirv_word_count, + uint32_t binding_count +); + +/** + * Destroys Vulkan objects owned by a ShaderCacheEntry and resets handles to null. + * + * Safe to call on an already-empty entry. Used by ShaderCache::clear and by + * callers that abandon an entry before add. + * + * #### Parameters: + * - context: Context& = same device that created the entry (needs destroy PFNs). + * - entry: ShaderCacheEntry& = entry to destroy; left empty on return. + * + * #### Throws: + * - Does not throw on destroy failure paths that only null handles when the + * device is already gone (shutdown edge cases). + */ +void destroy_entry( + cthreads::gpu::Context& context, + ShaderCacheEntry& entry +); + +} // namespace cthreads::gpu::shader diff --git a/src/cthreads/cpp/gpu/headers/shader_cache.hpp b/src/cthreads/cpp/gpu/headers/shader_cache.hpp new file mode 100644 index 0000000..a120ea6 --- /dev/null +++ b/src/cthreads/cpp/gpu/headers/shader_cache.hpp @@ -0,0 +1,94 @@ +#pragma once + +#include +#include +#include +#include +#include + +namespace cthreads::gpu { +struct Context; +} + +namespace cthreads::gpu::testing { +struct ShaderCacheTestAccess; +} + +/** + * Shader cache: reusable per-symbol Vulkan pipeline objects. + * + * Binding schema and compute pipelines are fixed for a kernel symbol. Per-launch + * buffers and descriptor sets live on the job / inflight path, not here. + * + * #### Technical terms: + * - Context: process-wide Vulkan connection (device, queue, loaded entry points). + * - SPIR-V: binary shader IR the driver consumes (see create_entry in shader.hpp). + * - descriptor set layout: schema of SSBO bindings (0 scalars, 1..N lists). + * - compute pipeline: compiled shader + layout ready to bind and dispatch. + */ +namespace cthreads::gpu::shader { + +/** + * Per-kernel Vulkan objects reused across launches of the same symbol. + * + * Move-only: Vulkan handles must not be copied (would double-destroy). + * + * #### Fields: + * - shader_module: VkShaderModule = SPIR-V module (may be null after pipeline create). + * - set_layout: VkDescriptorSetLayout = bindings 0 scalars, 1..N lists. + * - pipeline_layout: VkPipelineLayout = layout used to create the compute pipeline. + * - pipeline: VkPipeline = compute pipeline ready to bind. + * - binding_count: uint32_t = number of STORAGE_BUFFER bindings (1 + list count). + */ +struct ShaderCacheEntry { + VkShaderModule shader_module = VK_NULL_HANDLE; + VkDescriptorSetLayout set_layout = VK_NULL_HANDLE; + VkPipelineLayout pipeline_layout = VK_NULL_HANDLE; + VkPipeline pipeline = VK_NULL_HANDLE; + uint32_t binding_count = 0; + + ShaderCacheEntry() = default; + ShaderCacheEntry(const ShaderCacheEntry&) = delete; + ShaderCacheEntry& operator=(const ShaderCacheEntry&) = delete; + ShaderCacheEntry(ShaderCacheEntry&& other) noexcept; + ShaderCacheEntry& operator=(ShaderCacheEntry&& other) noexcept; +}; + +/** + * Process-wide map of kernel symbol -> reusable pipeline objects. + * + * Writers (registry, later) call add. Everyone else only get / clear. + * Entries are immutable after insert; clear runs on Context shutdown. + */ +class ShaderCache { +private: + std::unordered_map _cache; + std::mutex _cache_mutex; + + ShaderCache() = default; + ~ShaderCache(); + + // Registry-only once a friend exists. Duplicate key throws. + const ShaderCacheEntry& add(const std::string& key, ShaderCacheEntry&& entry); + +public: + static ShaderCache& getInstance(); + + ShaderCache(const ShaderCache&) = delete; + ShaderCache& operator=(const ShaderCache&) = delete; + ShaderCache(ShaderCache&&) = delete; + ShaderCache& operator=(ShaderCache&&) = delete; + + // Throws if the symbol is not registered. + const ShaderCacheEntry& get(const std::string& key); + + // Destroy all Vulkan objects on the entry, then empty the map. + // Call from Context shutdown before destroying the logical device. + void clear(cthreads::gpu::Context& context); + + friend struct cthreads::gpu::Context; // shutdown / future access + // Test-only access to private add (see gpu/testing/shader_smoke.cpp). + friend struct cthreads::gpu::testing::ShaderCacheTestAccess; +}; + +} // namespace cthreads::gpu::shader diff --git a/src/cthreads/cpp/gpu/impl/context.cpp b/src/cthreads/cpp/gpu/impl/context.cpp index 4586ae0..60fe4ad 100644 --- a/src/cthreads/cpp/gpu/impl/context.cpp +++ b/src/cthreads/cpp/gpu/impl/context.cpp @@ -6,6 +6,7 @@ #include #include "../headers/memory.hpp" +#include "../headers/shader_cache.hpp" #if defined(_WIN32) // Windows (32-bit or 64-bit) @@ -25,7 +26,8 @@ namespace { // ------ Hidden TransferEngineHelpers ------ - void shutdown_transfer_engine(Context& c) { + // Caller must hold c.transfer_engine_mutex (also used from init while locked). + void shutdown_transfer_engine_unlocked(Context& c) { // Safe no-op if the engine was never created or already cleared. TransferEngine& te = c.transfer_engine; if (te.command_pool == VK_NULL_HANDLE && @@ -57,7 +59,13 @@ namespace { te = TransferEngine{}; } + void shutdown_transfer_engine(Context& c) { + std::lock_guard lock(c.transfer_engine_mutex); + shutdown_transfer_engine_unlocked(c); + } + void init_transfer_engine(Context& c) { + std::lock_guard lock(c.transfer_engine_mutex); // Pool + fence only. Staging is allocated later on demand so idle // contexts do not hold a fixed 1 MiB host-visible buffer. if (c.transfer_engine.command_pool != VK_NULL_HANDLE && @@ -76,7 +84,7 @@ namespace { if (c.transfer_engine.command_pool != VK_NULL_HANDLE || c.transfer_engine.fence != VK_NULL_HANDLE || c.transfer_engine.staging.buffer != VK_NULL_HANDLE) { - shutdown_transfer_engine(c); + shutdown_transfer_engine_unlocked(c); } VkCommandPoolCreateInfo pool_info{}; @@ -312,6 +320,41 @@ namespace { c, c.instance, "vkWaitForFences"); c.vkResetFences = get_fn( c, c.instance, "vkResetFences"); + c.vkCreateShaderModule = get_fn( + c, c.instance, "vkCreateShaderModule"); + c.vkDestroyShaderModule = get_fn( + c, c.instance, "vkDestroyShaderModule"); + c.vkCreateDescriptorSetLayout = get_fn( + c, c.instance, "vkCreateDescriptorSetLayout"); + c.vkDestroyDescriptorSetLayout = + get_fn( + c, c.instance, "vkDestroyDescriptorSetLayout"); + c.vkCreatePipelineLayout = get_fn( + c, c.instance, "vkCreatePipelineLayout"); + c.vkDestroyPipelineLayout = get_fn( + c, c.instance, "vkDestroyPipelineLayout"); + c.vkCreateComputePipelines = get_fn( + c, c.instance, "vkCreateComputePipelines"); + c.vkDestroyPipeline = get_fn( + c, c.instance, "vkDestroyPipeline"); + c.vkCreateDescriptorPool = get_fn( + c, c.instance, "vkCreateDescriptorPool"); + c.vkDestroyDescriptorPool = get_fn( + c, c.instance, "vkDestroyDescriptorPool"); + c.vkAllocateDescriptorSets = get_fn( + c, c.instance, "vkAllocateDescriptorSets"); + c.vkFreeDescriptorSets = get_fn( + c, c.instance, "vkFreeDescriptorSets"); + c.vkUpdateDescriptorSets = get_fn( + c, c.instance, "vkUpdateDescriptorSets"); + c.vkCmdBindPipeline = get_fn( + c, c.instance, "vkCmdBindPipeline"); + c.vkCmdBindDescriptorSets = get_fn( + c, c.instance, "vkCmdBindDescriptorSets"); + c.vkCmdDispatch = get_fn( + c, c.instance, "vkCmdDispatch"); + c.vkCmdPipelineBarrier = get_fn( + c, c.instance, "vkCmdPipelineBarrier"); c.ready = true; // After device + PFNs + ready: reusable copy pool/fence (staging grows later). @@ -319,8 +362,9 @@ namespace { } void shutdown_unlocked(Context& c) { - // Children before parents: transfer engine (pool/fence/staging) then device. + // Children before parents: transfer engine, shader cache, then device. shutdown_transfer_engine(c); + shader::ShaderCache::getInstance().clear(c); // 1) release logical device if (c.device != VK_NULL_HANDLE && c.vkDestroyDevice) { // check if device is set and if theres a destroy fn for it @@ -384,6 +428,23 @@ namespace { c.vkQueueSubmit = nullptr; c.vkWaitForFences = nullptr; c.vkResetFences = nullptr; + c.vkCreateShaderModule = nullptr; + c.vkDestroyShaderModule = nullptr; + c.vkCreateDescriptorSetLayout = nullptr; + c.vkDestroyDescriptorSetLayout = nullptr; + c.vkCreatePipelineLayout = nullptr; + c.vkDestroyPipelineLayout = nullptr; + c.vkCreateComputePipelines = nullptr; + c.vkDestroyPipeline = nullptr; + c.vkCreateDescriptorPool = nullptr; + c.vkDestroyDescriptorPool = nullptr; + c.vkAllocateDescriptorSets = nullptr; + c.vkFreeDescriptorSets = nullptr; + c.vkUpdateDescriptorSets = nullptr; + c.vkCmdBindPipeline = nullptr; + c.vkCmdBindDescriptorSets = nullptr; + c.vkCmdDispatch = nullptr; + c.vkCmdPipelineBarrier = nullptr; c.queue_family = 0; c.device_name.clear(); diff --git a/src/cthreads/cpp/gpu/impl/descriptors.cpp b/src/cthreads/cpp/gpu/impl/descriptors.cpp new file mode 100644 index 0000000..db43163 --- /dev/null +++ b/src/cthreads/cpp/gpu/impl/descriptors.cpp @@ -0,0 +1,211 @@ +#include "../headers/descriptors.hpp" +#include "../headers/context.hpp" +#include "../headers/shader_cache.hpp" + +#include +#include +#include + +namespace cthreads::gpu::pack { + +DescriptorPool create_pool( + Context& context, + uint32_t binding_count, + uint32_t max_sets +) { + if (!context.ready || context.device == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: create_pool needs an initialized " + "device"); + } + if (binding_count == 0 || max_sets == 0) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: create_pool binding_count and " + "max_sets must be >= 1"); + } + if (!context.vkCreateDescriptorPool) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: create_pool missing " + "vkCreateDescriptorPool"); + } + + VkDescriptorPoolSize pool_size{}; + pool_size.type = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; + pool_size.descriptorCount = binding_count * max_sets; + + VkDescriptorPoolCreateInfo pool_info{}; + pool_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO; + // Allow free_set per job when the launch completes. + pool_info.flags = VK_DESCRIPTOR_POOL_CREATE_FREE_DESCRIPTOR_SET_BIT; + pool_info.maxSets = max_sets; + pool_info.poolSizeCount = 1; + pool_info.pPoolSizes = &pool_size; + + DescriptorPool out{}; + out.binding_count = binding_count; + out.max_sets = max_sets; + if (context.vkCreateDescriptorPool( + context.device, &pool_info, nullptr, &out.pool) != VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateDescriptorPool failed"); + } + return out; +} + +void destroy_pool(Context& context, DescriptorPool& pool) { + if (pool.pool == VK_NULL_HANDLE) { + pool = DescriptorPool{}; + return; + } + if (context.device != VK_NULL_HANDLE && context.vkDestroyDescriptorPool) { + context.vkDestroyDescriptorPool(context.device, pool.pool, nullptr); + } + pool = DescriptorPool{}; +} + +VkDescriptorSet allocate_set( + Context& context, + DescriptorPool& pool, + VkDescriptorSetLayout set_layout +) { + if (!context.ready || context.device == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: allocate_set needs an initialized " + "device"); + } + if (pool.pool == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: allocate_set pool is empty"); + } + if (set_layout == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: allocate_set set_layout is null"); + } + if (!context.vkAllocateDescriptorSets) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: allocate_set missing " + "vkAllocateDescriptorSets"); + } + + VkDescriptorSetAllocateInfo alloc_info{}; + alloc_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO; + alloc_info.descriptorPool = pool.pool; + alloc_info.descriptorSetCount = 1; + alloc_info.pSetLayouts = &set_layout; + + VkDescriptorSet set = VK_NULL_HANDLE; + if (context.vkAllocateDescriptorSets(context.device, &alloc_info, &set) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkAllocateDescriptorSets failed"); + } + return set; +} + +void free_set(Context& context, DescriptorPool& pool, VkDescriptorSet& set) { + if (set == VK_NULL_HANDLE) { + return; + } + if (pool.pool == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: free_set pool is empty"); + } + if (!context.vkFreeDescriptorSets || context.device == VK_NULL_HANDLE) { + set = VK_NULL_HANDLE; + return; + } + if (context.vkFreeDescriptorSets(context.device, pool.pool, 1, &set) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkFreeDescriptorSets failed"); + } + set = VK_NULL_HANDLE; +} + +void update_descriptors( + Context& context, + VkDescriptorSet set, + uint32_t binding_count, + const GpuPack& pack +) { + if (!context.ready || context.device == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: update_descriptors needs an " + "initialized device"); + } + if (set == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: update_descriptors set is null"); + } + if (binding_count == 0) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: update_descriptors binding_count " + "must be >= 1"); + } + if (binding_count != 1u + static_cast(pack.container_slots.size())) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: update_descriptors binding_count " + "must equal 1 + container_slots.size()"); + } + if (!context.vkUpdateDescriptorSets) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: update_descriptors missing " + "vkUpdateDescriptorSets"); + } + + // One buffer info + write per binding; infos must stay alive for the call. + std::vector buffer_infos(binding_count); + std::vector writes(binding_count); + + for (uint32_t i = 0; i < binding_count; ++i) { + VkBuffer buffer = VK_NULL_HANDLE; + VkDeviceSize size = 0; + if (i == 0) { + buffer = pack.scalar_buffer.buffer; + size = pack.scalar_buffer.size; + } else { + const ContainerSlot& slot = pack.container_slots[i - 1]; + buffer = slot.buffer.buffer; + size = slot.buffer.size; + } + if (buffer == VK_NULL_HANDLE || size == 0) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: update_descriptors binding " + + std::to_string(i) + + " needs a non-empty GpuBuffer (empty pack slots not supported " + "yet)"); + } + + buffer_infos[i] = {}; + buffer_infos[i].buffer = buffer; + buffer_infos[i].offset = 0; + buffer_infos[i].range = size; + + writes[i] = {}; + writes[i].sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; + writes[i].dstSet = set; + writes[i].dstBinding = i; + writes[i].dstArrayElement = 0; + writes[i].descriptorCount = 1; + writes[i].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; + writes[i].pBufferInfo = &buffer_infos[i]; + } + + context.vkUpdateDescriptorSets( + context.device, + binding_count, + writes.data(), + 0, + nullptr); +} + +void update_descriptors( + Context& context, + VkDescriptorSet set, + const shader::ShaderCacheEntry& entry, + const GpuPack& pack +) { + update_descriptors(context, set, entry.binding_count, pack); +} + +} // namespace cthreads::gpu::pack diff --git a/src/cthreads/cpp/gpu/impl/memory.cpp b/src/cthreads/cpp/gpu/impl/memory.cpp index 6789fb9..1d4c2cb 100644 --- a/src/cthreads/cpp/gpu/impl/memory.cpp +++ b/src/cthreads/cpp/gpu/impl/memory.cpp @@ -4,6 +4,7 @@ #include #include #include +#include namespace cthreads::gpu::memory { @@ -28,6 +29,7 @@ void require_ready(const Context& context, const char* where) { } void require_transfer_engine(const Context& context, const char* where) { + // assumes the engine mutex is locked if (context.transfer_engine.command_pool == VK_NULL_HANDLE || context.transfer_engine.fence == VK_NULL_HANDLE) { throw std::runtime_error( @@ -37,6 +39,7 @@ void require_transfer_engine(const Context& context, const char* where) { } // GPU copy via Context TransferEngine pool + fence, then CPU wait. +// Caller must hold context.transfer_engine_mutex for the whole call. void copy_buffer_and_wait( Context& context, VkBuffer src, @@ -289,7 +292,7 @@ void destroy_buffer(Context& context, GpuBuffer& buffer) { namespace { // Grow-only host-visible scratch on the TransferEngine. Never shrinks until -// Context shutdown. Safe because upload/download wait on the engine fence. +// Context shutdown. Caller must hold context.transfer_engine_mutex. void ensure_staging(Context& context, VkDeviceSize size) { TransferEngine& te = context.transfer_engine; if (te.staging.buffer != VK_NULL_HANDLE && te.staging.size >= size && @@ -311,7 +314,6 @@ void upload_buffer( VkDeviceSize size ) { require_ready(context, "upload_buffer"); - require_transfer_engine(context, "upload_buffer"); if (buffer.kind != BufferKind::DeviceLocal || buffer.buffer == VK_NULL_HANDLE) { throw std::runtime_error( @@ -327,7 +329,9 @@ void upload_buffer( "cthreads.gpu.GpuInvalidArgument: upload_buffer size invalid"); } - // Host -> engine staging (memcpy) -> device-local (GPU copy). + // One lock for engine check, staging grow, host memcpy, and GPU copy. + std::lock_guard lock(context.transfer_engine_mutex); + require_transfer_engine(context, "upload_buffer"); ensure_staging(context, size); GpuBuffer& staging = context.transfer_engine.staging; std::memcpy(staging.mapped, data, static_cast(size)); @@ -341,7 +345,6 @@ void download_buffer( VkDeviceSize size ) { require_ready(context, "download_buffer"); - require_transfer_engine(context, "download_buffer"); if (buffer.kind != BufferKind::DeviceLocal || buffer.buffer == VK_NULL_HANDLE) { throw std::runtime_error( @@ -357,7 +360,9 @@ void download_buffer( "cthreads.gpu.GpuInvalidArgument: download_buffer size invalid"); } - // Device-local -> engine staging (GPU copy) -> host (memcpy). + // One lock for engine check, staging grow, GPU copy, and host memcpy. + std::lock_guard lock(context.transfer_engine_mutex); + require_transfer_engine(context, "download_buffer"); ensure_staging(context, size); GpuBuffer& staging = context.transfer_engine.staging; copy_buffer_and_wait(context, buffer.buffer, staging.buffer, size); diff --git a/src/cthreads/cpp/gpu/impl/module.cpp b/src/cthreads/cpp/gpu/impl/module.cpp new file mode 100644 index 0000000..53d257e --- /dev/null +++ b/src/cthreads/cpp/gpu/impl/module.cpp @@ -0,0 +1,772 @@ +#include "../headers/module.hpp" +#include "../headers/context.hpp" +#include "../headers/shader.hpp" +#include "../headers/shader_cache.hpp" +#include "../headers/pack.hpp" +#include "../headers/descriptors.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +namespace cthreads::gpu { +namespace { + +// Host byte sizes for the scalar SSBO (std430 / SPIR-V). Not C++ sizeof for +// bool/int: GLSL bool is stored as 32-bit; use int32_t so Win/Linux match. +// string is not a scalar-SSBO type on the GPU path. +static const std::unordered_map py_size_of = { + {"bool", 4}, // int32 0/1 + {"int", 4}, // int32_t + {"float", 4}, // float + {"double", 8}, // float64; align 8 when laying out +}; + +size_t align_up(size_t value, size_t alignment) { + return (value + alignment - 1) & ~(alignment - 1); +} + +size_t std430_align_of(const std::string& kind) { + // std430: scalar alignment equals its size for these types. + return py_size_of.at(kind); +} + +void release_inflight(Context& context, SpawnedGpuKernel& job) { + // Destroying the pool frees any CBs allocated from it; free first when we can. + if (job.command_buffer != VK_NULL_HANDLE && + job.command_pool != VK_NULL_HANDLE && + context.device != VK_NULL_HANDLE && + context.vkFreeCommandBuffers) { // free the cmd buffer when all relevant ressources are valid + context.vkFreeCommandBuffers( + context.device, job.command_pool, 1, &job.command_buffer); + } + job.command_buffer = VK_NULL_HANDLE; + + // Per-launch command pool (not the TransferEngine pool). + if (job.command_pool != VK_NULL_HANDLE && + context.device != VK_NULL_HANDLE && + context.vkDestroyCommandPool) { + context.vkDestroyCommandPool(context.device, job.command_pool, nullptr); + } + job.command_pool = VK_NULL_HANDLE; + + if (job.descriptor_set != VK_NULL_HANDLE) { // free the descriptors (if not freed yet) + pack::free_set(context, job.descriptor_pool, job.descriptor_set); + } + pack::destroy_pool(context, job.descriptor_pool); + // destroy the fence if not already done + if (job.fence != VK_NULL_HANDLE && context.device != VK_NULL_HANDLE && + context.vkDestroyFence) { + context.vkDestroyFence(context.device, job.fence, nullptr); + } + job.fence = VK_NULL_HANDLE; + // destroy the pack + pack::destroy_gpu_pack(context, job.pack); + job.symbol.clear(); // clear the symbol + job.writeback_lists.clear(); + job.values_keep.reset(); +} + +void mark_done(SpawnedGpuKernel& job) { + { // lock the done_mu mutex and set the done flag + std::lock_guard lock(job.done_mu); + job.done_flag = true; + } + job.done_cv.notify_all(); // notify all waiting threads (main thread currently, in later version also cthreads) +} + +// After dispatch fence: make SHADER_WRITE visible to TRANSFER_READ for downloads. +// Uses TransferEngine pool/fence under transfer_engine_mutex (same as uploads). +void compute_to_transfer_barrier(Context& context) { + std::lock_guard lock(context.transfer_engine_mutex); + if (context.transfer_engine.command_pool == VK_NULL_HANDLE || + context.transfer_engine.fence == VK_NULL_HANDLE || + !context.vkCmdPipelineBarrier || !context.vkAllocateCommandBuffers || + !context.vkBeginCommandBuffer || !context.vkEndCommandBuffer || + !context.vkQueueSubmit || !context.vkResetFences || + !context.vkWaitForFences || !context.vkFreeCommandBuffers || + !context.queue) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: compute_to_transfer_barrier missing " + "TransferEngine or entry points"); + } + + VkCommandBufferAllocateInfo alloc_info{}; + alloc_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; + alloc_info.commandPool = context.transfer_engine.command_pool; + alloc_info.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; + alloc_info.commandBufferCount = 1; + VkCommandBuffer cmd = VK_NULL_HANDLE; + if (context.vkAllocateCommandBuffers(context.device, &alloc_info, &cmd) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: compute_to_transfer_barrier " + "vkAllocateCommandBuffers failed"); + } + + VkCommandBufferBeginInfo begin_info{}; + begin_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; + begin_info.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; + if (context.vkBeginCommandBuffer(cmd, &begin_info) != VK_SUCCESS) { + context.vkFreeCommandBuffers( + context.device, context.transfer_engine.command_pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: compute_to_transfer_barrier " + "vkBeginCommandBuffer failed"); + } + + VkMemoryBarrier mem_barrier{}; + mem_barrier.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER; + mem_barrier.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT; + mem_barrier.dstAccessMask = VK_ACCESS_TRANSFER_READ_BIT; + context.vkCmdPipelineBarrier( + cmd, + VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, + VK_PIPELINE_STAGE_TRANSFER_BIT, + 0, + 1, + &mem_barrier, + 0, + nullptr, + 0, + nullptr); + + if (context.vkEndCommandBuffer(cmd) != VK_SUCCESS) { + context.vkFreeCommandBuffers( + context.device, context.transfer_engine.command_pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: compute_to_transfer_barrier " + "vkEndCommandBuffer failed"); + } + + VkFence te_fence = context.transfer_engine.fence; + if (context.vkResetFences(context.device, 1, &te_fence) != VK_SUCCESS) { + context.vkFreeCommandBuffers( + context.device, context.transfer_engine.command_pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: compute_to_transfer_barrier " + "vkResetFences failed"); + } + + VkSubmitInfo submit{}; + submit.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; + submit.commandBufferCount = 1; + submit.pCommandBuffers = &cmd; + if (context.vkQueueSubmit(context.queue, 1, &submit, te_fence) != + VK_SUCCESS) { + context.vkFreeCommandBuffers( + context.device, context.transfer_engine.command_pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: compute_to_transfer_barrier " + "vkQueueSubmit failed"); + } + if (context.vkWaitForFences( + context.device, 1, &te_fence, VK_TRUE, UINT64_MAX) != VK_SUCCESS) { + context.vkFreeCommandBuffers( + context.device, context.transfer_engine.command_pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: compute_to_transfer_barrier " + "vkWaitForFences failed"); + } + context.vkFreeCommandBuffers( + context.device, context.transfer_engine.command_pool, 1, &cmd); +} + +// Download each ref list SSBO into the kept Python list (in place). +void writeback_ref_lists(Context& context, SpawnedGpuKernel& job) { + if (job.writeback_lists.empty()) { + return; + } + if (!job.values_keep) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: join writeback missing values_keep"); + } + + py::gil_scoped_acquire gil; + py::list& values = *job.values_keep; + + for (const SpawnedGpuKernel::WritebackListSlot& slot : job.writeback_lists) { + if (slot.numel == 0) { + continue; + } + if (slot.value_index >= static_cast(values.size())) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: writeback value_index out of " + "range"); + } + py::list list_val = values[slot.value_index].cast(); + if (static_cast(list_val.size()) != slot.numel) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: writeback list length changed " + "during job"); + } + + if (slot.elem_kind == "float") { + std::vector host(slot.numel); + pack::download_container( + context, + job.pack, + slot.container_index, + host.data(), + host.size() * sizeof(float)); + for (size_t j = 0; j < slot.numel; ++j) { + list_val[j] = host[j]; + } + } else if (slot.elem_kind == "int") { + std::vector host(slot.numel); + pack::download_container( + context, + job.pack, + slot.container_index, + host.data(), + host.size() * sizeof(std::int32_t)); + for (size_t j = 0; j < slot.numel; ++j) { + list_val[j] = host[j]; + } + } else if (slot.elem_kind == "double") { + std::vector host(slot.numel); + pack::download_container( + context, + job.pack, + slot.container_index, + host.data(), + host.size() * sizeof(double)); + for (size_t j = 0; j < slot.numel; ++j) { + list_val[j] = host[j]; + } + } else { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: unsupported writeback " + "elem_kind: " + + slot.elem_kind); + } + } +} + +// Write one Python scalar into the host scalar blob at offset (std430 layout). +void write_scalar_bytes( + std::vector& blob, + size_t offset, + const std::string& kind, + const py::object& value +) { + if (kind == "int") { + const std::int32_t v = value.cast(); + std::memcpy(blob.data() + offset, &v, sizeof(v)); + } else if (kind == "float") { + const float v = value.cast(); + std::memcpy(blob.data() + offset, &v, sizeof(v)); + } else if (kind == "double") { + const double v = value.cast(); + std::memcpy(blob.data() + offset, &v, sizeof(v)); + } else if (kind == "bool") { + // GLSL bool in std430 is 32-bit; store 0/1 as int32. + const std::int32_t v = value.cast() ? 1 : 0; + std::memcpy(blob.data() + offset, &v, sizeof(v)); + } else { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: cannot pack scalar kind: " + kind); + } +} + +} // namespace + +void SpawnedGpuKernel::start() { + // Default path submits at launch time; start is a shared API no-op. +} + +void SpawnedGpuKernel::wait() { + std::unique_lock lock(done_mu); + done_cv.wait(lock, [this] { return done_flag; }); +} + +bool SpawnedGpuKernel::done() { + std::lock_guard lock(done_mu); + return done_flag; +} + +void SpawnedGpuKernel::join(Context& context) { + if (finished) { + if (eptr) { + std::rethrow_exception(eptr); + } + return; + } + + try { + if (fence != VK_NULL_HANDLE) { + if (!context.ready || context.device == VK_NULL_HANDLE || + !context.vkWaitForFences) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: SpawnedGpuKernel::join " + "needs a ready device and vkWaitForFences"); + } + // wait for the fence to be signaled + if (context.vkWaitForFences( + context.device, 1, &fence, VK_TRUE, UINT64_MAX) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkWaitForFences failed in " + "SpawnedGpuKernel::join"); + } + } + + // Permanent list writeback path (Threadable/schema marshal is later). + if (!writeback_lists.empty()) { + compute_to_transfer_barrier(context); + writeback_ref_lists(context, *this); + } + + mark_done(*this); // mark the job as done + release_inflight(context, *this); // release the inflight GPU state (clears all buffers, fences and cmd structures) + finished = true; + } catch (...) { + eptr = std::current_exception(); + mark_done(*this); + try { + release_inflight(context, *this); + } catch (...) { + // Prefer the original join error. + } + finished = true; + std::rethrow_exception(eptr); + } + + if (eptr) { + std::rethrow_exception(eptr); + } +} + +SpawnedGpuKernel::~SpawnedGpuKernel() { + if (finished) { + return; + } + try { + // try to await the fence and clear the inflight GPU state before destroying the object + Context& ctx = context(); + if (ctx.ready && ctx.device != VK_NULL_HANDLE) { + if (fence != VK_NULL_HANDLE && ctx.vkWaitForFences) { + ctx.vkWaitForFences( + ctx.device, 1, &fence, VK_TRUE, UINT64_MAX); + } + release_inflight(ctx, *this); + } + } catch (...) { + // Destructor must not throw. + } + mark_done(*this); // mark the job as done (doesnt mean the job finished successfully, just means it was terminated) + finished = true; +} + +std::shared_ptr launch_gpu_kernel( + py::dict meta, + py::list ordered_values +) { + // Launch path: + // Python args + gpu meta + // -> create/fill GpuPack (upload via staging) + // -> ShaderCache.get (create_entry/add is registry-only; missing => throw) + // -> allocate descriptor set, update_descriptors(pack) + // -> record CB: barrier, bind pipeline, bind set, dispatch + // -> create fence, vkQueueSubmit(..., fence) [no wait here] + // -> stash handles on SpawnedGpuKernel (pack, set, pool, CB, fence, symbol, groups) + // -> return SpawnedGpuKernel (no CThread) + // join(context): wait fence -> download/writeback -> release_inflight + + // Ensure Vulkan Context is ready (loader + device + TransferEngine). + init(); // init the context (noop if already initialized) + Context& context = cthreads::gpu::context(); + if (!context.ready) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: launch_gpu_kernel needs a ready " + "Context"); + } + + if (!meta.contains("symbol") || !meta.contains("params")) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: launch_gpu_kernel meta needs " + "'symbol' and 'params'"); + } + + const std::string symbol = meta["symbol"].cast(); // kernel / fn name + py::list params = meta["params"]; // list of input params (see docs string for example) + // ensure this fn call mathces the number of params in the kernels meta + if (static_cast(ordered_values.size()) != static_cast(params.size())) { + throw py::type_error( + "cthreads.gpu: expected " + std::to_string(params.size()) + + " args for '" + symbol + "', got " + + std::to_string(ordered_values.size())); + } + + // Walk params: scalars -> binding 0 blob; lists -> ContainerSpec (bindings 1..N). + // build the container specs + scalar layout (std430 align while summing) + std::vector container_specs; + std::vector scalar_host; // filled after we know scalar_bytes + struct ScalarSlot { // private helper struct to record metadata of scalar params in the kernel call + size_t value_index = 0; + size_t offset = 0; + std::string kind; + }; + std::vector scalar_slots; + struct ContainerSlotPlan { // private helper struct to record metadata of list params in the kernel call + size_t value_index = 0; + size_t elem_bytes = 0; + size_t numel = 0; + std::string elem_kind; + bool writeback = true; // pass_as ref (default for lists) + }; + std::vector container_plans; + + size_t scalar_bytes = 0; + // iter all the params and build the container specs + scalar layout (std430 align while summing) + for (size_t i = 0; i < static_cast(params.size()); ++i) { + // each param is a dict from meta (see launch_gpu_kernel docstring example) + py::dict param = params[i].cast(); // this param is a dict with meta like name, kind, pass_as, numel, elem_kind, elem_bytes, ... + std::string name = param["name"].cast(); // the var name / identifier (set by usr code) + std::string kind = param["kind"].cast(); // the type + const bool is_list = (kind == "list"); + // Scalars default value; lists default ref (join writeback). + std::string pass_as = param.contains("pass_as") + ? param["pass_as"].cast() + : (is_list ? std::string("ref") : std::string("value")); + (void)name; + + if (is_list) { // lists get theri own ssbo (single buffer) + // List SSBO: elem size from meta; numel from the Python list length. + std::string elem_kind = param.contains("elem_kind") //type (default float) + ? param["elem_kind"].cast() + : std::string("float"); + size_t elem_bytes = 0; + // check if the bytes are set in the meta, otherwise try to infer from the type or throw an err + if (param.contains("elem_bytes")) { + elem_bytes = param["elem_bytes"].cast(); + } else if (py_size_of.find(elem_kind) != py_size_of.end()) { + elem_bytes = py_size_of.at(elem_kind); + } else { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: unknown list elem type: " + + elem_kind); + } + // Lists default to pass_as ref (in-place writeback on join). + if (pass_as != "ref" && pass_as != "value") { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: unsupported pass_as for " + "list '" + + name + "': " + pass_as); + } + const bool do_writeback = (pass_as != "value"); + // gte the actuall value from the kernel call (py side) + py::list list_val = ordered_values[i].cast(); + const size_t numel = static_cast(list_val.size()); + // optional meta numel must match the live list if both present + if (param.contains("numel") && param["numel"].cast() != numel) { // check that sizes match the expectations form the meta + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: meta numel does not match " + "list length for '" + + name + "'"); + } + container_specs.push_back(pack::ContainerSpec{elem_bytes, numel}); + container_plans.push_back(ContainerSlotPlan{ + i, elem_bytes, numel, std::move(elem_kind), do_writeback}); + continue; + } + // --- handle scalar args --- + + // Scalar field into the binding-0 SSBO (not a separate descriptor). + if (py_size_of.find(kind) == py_size_of.end()) { // check if we can interpret this type + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: unknown variable type: " + + kind); + } + if (pass_as != "value" && pass_as != "ref") { + // Scalars are always packed by value into the SSBO; pass_as is + // reserved for future semantics. Reject unknown tags early. + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: unsupported pass_as for " + "scalar '" + + name + "': " + pass_as); + } + const size_t size = py_size_of.at(kind); + const size_t alignment = std430_align_of(kind); + scalar_bytes = align_up(scalar_bytes, alignment); + scalar_slots.push_back(ScalarSlot{i, scalar_bytes, kind}); + scalar_bytes += size; + } + + // Prefer meta scalar_bytes when present (codegen truth); must match walk. + if (meta.contains("scalar_bytes")) { + const size_t meta_bytes = meta["scalar_bytes"].cast(); + if (meta_bytes != scalar_bytes) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: meta scalar_bytes (" + + std::to_string(meta_bytes) + ") != layout sum (" + + std::to_string(scalar_bytes) + ")"); + } + } + + // Job owns GPU objects from here on so failures can release_inflight. + auto job = std::make_shared(); + job->symbol = symbol; + // Keep the same Python arg objects for join writeback (list identity). + job->values_keep = std::make_shared(ordered_values); + for (size_t c = 0; c < container_plans.size(); ++c) { + const ContainerSlotPlan& plan = container_plans[c]; + if (!plan.writeback || plan.numel == 0) { + continue; + } + job->writeback_lists.push_back(SpawnedGpuKernel::WritebackListSlot{ + plan.value_index, + c, + plan.numel, + plan.elem_kind, + }); + } + // collect launch group data (required to ensure the correct num threads a re launched and the correct thread block shape is used) + if (meta.contains("group_count_x") && !meta["group_count_x"].is_none()) { + job->group_count_x = meta["group_count_x"].cast(); + } + if (meta.contains("group_count_y") && !meta["group_count_y"].is_none()) { + job->group_count_y = meta["group_count_y"].cast(); + } + if (meta.contains("group_count_z") && !meta["group_count_z"].is_none()) { + job->group_count_z = meta["group_count_z"].cast(); + } + + try { + // init the gpu pack (device-local scalar blob + one buffer per list) + job->pack = pack::create_gpu_pack( + context, + scalar_bytes, + container_specs + ); + + // Pack Python scalars into a host byte blob, then upload through staging. + if (scalar_bytes > 0) { + scalar_host.assign(scalar_bytes, 0); + for (const ScalarSlot& slot : scalar_slots) { + write_scalar_bytes( + scalar_host, + slot.offset, + slot.kind, + ordered_values[slot.value_index].cast()); + } + // upload the scalars + pack::upload_scalars( + context, job->pack, scalar_host.data(), scalar_bytes); + } + + // Upload each list container (binding 1..N) from ordered_values. + for (size_t c = 0; c < container_plans.size(); ++c) { + const ContainerSlotPlan& plan = container_plans[c]; + if (plan.numel == 0) { + continue; // empty slot: no VkBuffer; update_descriptors still rejects empty for now + } + py::list list_val = ordered_values[plan.value_index].cast(); // get the py side list that was passed in the kernel call + if (plan.elem_kind == "float") { + std::vector host(plan.numel); + for (size_t j = 0; j < plan.numel; ++j) { + host[j] = list_val[j].cast(); + } + pack::upload_container( + context, + job->pack, + c, + host.data(), + host.size() * sizeof(float)); + } else if (plan.elem_kind == "int") { + std::vector host(plan.numel); + for (size_t j = 0; j < plan.numel; ++j) { + host[j] = list_val[j].cast(); + } + pack::upload_container( + context, + job->pack, + c, + host.data(), + host.size() * sizeof(std::int32_t)); + } else if (plan.elem_kind == "double") { + std::vector host(plan.numel); + for (size_t j = 0; j < plan.numel; ++j) { + host[j] = list_val[j].cast(); + } + pack::upload_container( + context, + job->pack, + c, + host.data(), + host.size() * sizeof(double)); + } else { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: unsupported list elem_kind: " + + plan.elem_kind); + } + } + + // get the shader cache entry (must already be registered) + const shader::ShaderCacheEntry& entry = + shader::ShaderCache::getInstance().get(symbol); + + // binding_count on the entry must match 1 + number of list slots + const uint32_t expected_bindings = + 1u + static_cast(container_specs.size()); + if (entry.binding_count != expected_bindings) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: ShaderCacheEntry binding_count (" + + std::to_string(entry.binding_count) + ") != 1 + list count (" + + std::to_string(expected_bindings) + ")"); + } + if (entry.pipeline == VK_NULL_HANDLE || + entry.pipeline_layout == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: ShaderCacheEntry missing " + "pipeline for '" + + symbol + "'"); + } + + // get the descriptor pool to allocate the descriptor set next + job->descriptor_pool = + pack::create_pool(context, entry.binding_count, 1); + // allocate the descriptor set + job->descriptor_set = + pack::allocate_set(context, job->descriptor_pool, entry.set_layout); + // wire binding i -> pack buffer i (schema from entry, buffers from this pack) + pack::update_descriptors( + context, job->descriptor_set, entry, job->pack); + + // Need bind/dispatch/barrier + the usual CB/submit PFNs. + if (!context.vkCreateCommandPool || !context.vkDestroyCommandPool || + !context.vkAllocateCommandBuffers || !context.vkFreeCommandBuffers || + !context.vkBeginCommandBuffer || !context.vkEndCommandBuffer || + !context.vkCmdPipelineBarrier || !context.vkCmdBindPipeline || + !context.vkCmdBindDescriptorSets || !context.vkCmdDispatch || + !context.vkCreateFence || !context.vkDestroyFence || + !context.vkQueueSubmit || !context.queue) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: launch_gpu_kernel missing " + "dispatch/command/fence entry points or queue"); + } + + // Per-job command pool: own lifetime, no TransferEngine mutex needed. + VkCommandPoolCreateInfo pool_info{}; + pool_info.sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO; + pool_info.queueFamilyIndex = context.queue_family; + pool_info.flags = VK_COMMAND_POOL_CREATE_TRANSIENT_BIT; + if (context.vkCreateCommandPool( + context.device, &pool_info, nullptr, &job->command_pool) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateCommandPool failed in " + "launch_gpu_kernel"); + } + + VkCommandBufferAllocateInfo alloc_info{}; + alloc_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; + alloc_info.commandPool = job->command_pool; + alloc_info.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; + alloc_info.commandBufferCount = 1; + if (context.vkAllocateCommandBuffers( + context.device, &alloc_info, &job->command_buffer) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkAllocateCommandBuffers failed " + "in launch_gpu_kernel"); + } + + VkCommandBufferBeginInfo begin_info{}; + begin_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; + begin_info.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; + if (context.vkBeginCommandBuffer(job->command_buffer, &begin_info) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkBeginCommandBuffer failed in " + "launch_gpu_kernel"); + } + + // Uploads already waited on the TransferEngine fence, but Vulkan still + // needs a barrier so compute sees TRANSFER_WRITE results. + VkMemoryBarrier mem_barrier{}; + mem_barrier.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER; + mem_barrier.srcAccessMask = VK_ACCESS_TRANSFER_WRITE_BIT; + mem_barrier.dstAccessMask = + VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT; + context.vkCmdPipelineBarrier( + job->command_buffer, + VK_PIPELINE_STAGE_TRANSFER_BIT, + VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, + 0, + 1, + &mem_barrier, + 0, + nullptr, + 0, + nullptr); + + // Bind compute pipeline + this launch's descriptor set, then dispatch. + context.vkCmdBindPipeline( + job->command_buffer, + VK_PIPELINE_BIND_POINT_COMPUTE, + entry.pipeline); + context.vkCmdBindDescriptorSets( + job->command_buffer, + VK_PIPELINE_BIND_POINT_COMPUTE, + entry.pipeline_layout, + 0, + 1, + &job->descriptor_set, + 0, + nullptr); + context.vkCmdDispatch( + job->command_buffer, + job->group_count_x, + job->group_count_y, + job->group_count_z); + + if (context.vkEndCommandBuffer(job->command_buffer) != VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkEndCommandBuffer failed in " + "launch_gpu_kernel"); + } + + // Per-job fence (not TransferEngine.fence). Unsignaled until submit done. + VkFenceCreateInfo fence_info{}; + fence_info.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO; + if (context.vkCreateFence( + context.device, &fence_info, nullptr, &job->fence) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateFence failed in " + "launch_gpu_kernel"); + } + + VkSubmitInfo submit{}; + submit.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; + submit.commandBufferCount = 1; + submit.pCommandBuffers = &job->command_buffer; + if (context.vkQueueSubmit( + context.queue, 1, &submit, job->fence) != VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkQueueSubmit failed in " + "launch_gpu_kernel"); + } + // Do not wait here — join() waits on job->fence. + } catch (...) { + // Tear down any handles already stashed; then rethrow to Python. + try { + release_inflight(context, *job); + } catch (...) { + // Prefer the original launch error. + } + throw; + } + + return job; +} + +} // namespace cthreads::gpu diff --git a/src/cthreads/cpp/gpu/impl/shader.cpp b/src/cthreads/cpp/gpu/impl/shader.cpp new file mode 100644 index 0000000..ad5011f --- /dev/null +++ b/src/cthreads/cpp/gpu/impl/shader.cpp @@ -0,0 +1,119 @@ +#include "../headers/shader.hpp" +#include "../headers/shader_cache.hpp" +#include "../headers/context.hpp" + +#include +#include + +namespace cthreads::gpu::shader { + +ShaderCacheEntry create_entry( + Context& context, + const uint32_t* spirv, + size_t spirv_word_count, + uint32_t binding_count +) { + if (!context.ready || context.device == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: create_entry needs an initialized " + "device"); + } + if (spirv == nullptr || spirv_word_count == 0) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: create_entry spirv is empty"); + } + if (binding_count == 0) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: create_entry binding_count must " + "be >= 1"); + } + if (!context.vkCreateShaderModule || !context.vkCreateDescriptorSetLayout || + !context.vkCreatePipelineLayout || !context.vkCreateComputePipelines) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: create_entry missing shader/" + "pipeline create entry points"); + } + + ShaderCacheEntry entry{}; + entry.binding_count = binding_count; + + // 1) SPIR-V -> shader module + VkShaderModuleCreateInfo module_info{}; + module_info.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO; + module_info.codeSize = spirv_word_count * sizeof(uint32_t); + module_info.pCode = spirv; + if (context.vkCreateShaderModule( + context.device, &module_info, nullptr, &entry.shader_module) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateShaderModule failed"); + } + + // 2) set layout: binding i is one STORAGE_BUFFER (compute). + std::vector bindings(binding_count); + for (uint32_t i = 0; i < binding_count; ++i) { + bindings[i] = {}; + bindings[i].binding = i; + bindings[i].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; + bindings[i].descriptorCount = 1; + bindings[i].stageFlags = VK_SHADER_STAGE_COMPUTE_BIT; + bindings[i].pImmutableSamplers = nullptr; + } + + VkDescriptorSetLayoutCreateInfo layout_info{}; + layout_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO; + layout_info.bindingCount = binding_count; + layout_info.pBindings = bindings.data(); + if (context.vkCreateDescriptorSetLayout( + context.device, &layout_info, nullptr, &entry.set_layout) != + VK_SUCCESS) { + destroy_entry(context, entry); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateDescriptorSetLayout failed"); + } + + // 3) Pipeline layout (one set, no push constants. IF OPTIMIZATION REQUIRES THEM ADD PUSH CONSTS HERE). + VkPipelineLayoutCreateInfo pipe_layout_info{}; + pipe_layout_info.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO; + pipe_layout_info.setLayoutCount = 1; + pipe_layout_info.pSetLayouts = &entry.set_layout; + pipe_layout_info.pushConstantRangeCount = 0; + pipe_layout_info.pPushConstantRanges = nullptr; + if (context.vkCreatePipelineLayout( + context.device, &pipe_layout_info, nullptr, &entry.pipeline_layout) != + VK_SUCCESS) { + destroy_entry(context, entry); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreatePipelineLayout failed"); + } + + // 4) Compute pipeline from module + layout (entry point "main"). + VkPipelineShaderStageCreateInfo stage{}; + stage.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO; + stage.stage = VK_SHADER_STAGE_COMPUTE_BIT; + stage.module = entry.shader_module; + stage.pName = "main"; + + VkComputePipelineCreateInfo pipe_info{}; + pipe_info.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO; + pipe_info.stage = stage; + pipe_info.layout = entry.pipeline_layout; + pipe_info.basePipelineHandle = VK_NULL_HANDLE; + pipe_info.basePipelineIndex = -1; + + if (context.vkCreateComputePipelines( + context.device, + VK_NULL_HANDLE, + 1, + &pipe_info, + nullptr, + &entry.pipeline) != VK_SUCCESS) { + destroy_entry(context, entry); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateComputePipelines failed"); + } + + return entry; +} + +} // namespace cthreads::gpu::shader diff --git a/src/cthreads/cpp/gpu/impl/shader_cache.cpp b/src/cthreads/cpp/gpu/impl/shader_cache.cpp new file mode 100644 index 0000000..288a078 --- /dev/null +++ b/src/cthreads/cpp/gpu/impl/shader_cache.cpp @@ -0,0 +1,123 @@ +#include "../headers/shader_cache.hpp" +#include "../headers/context.hpp" + +#include +#include + +namespace cthreads::gpu::shader { + +void destroy_entry(Context& context, ShaderCacheEntry& entry) { + if (context.device == VK_NULL_HANDLE) { + // Cannot destroy without a device; drop handle values only. + entry.shader_module = VK_NULL_HANDLE; + entry.set_layout = VK_NULL_HANDLE; + entry.pipeline_layout = VK_NULL_HANDLE; + entry.pipeline = VK_NULL_HANDLE; + entry.binding_count = 0; + return; + } + // Pipeline before layouts/module (children before parents). + if (entry.pipeline != VK_NULL_HANDLE && context.vkDestroyPipeline) { + context.vkDestroyPipeline(context.device, entry.pipeline, nullptr); + entry.pipeline = VK_NULL_HANDLE; + } + if (entry.pipeline_layout != VK_NULL_HANDLE && + context.vkDestroyPipelineLayout) { + context.vkDestroyPipelineLayout( + context.device, entry.pipeline_layout, nullptr); + entry.pipeline_layout = VK_NULL_HANDLE; + } + if (entry.set_layout != VK_NULL_HANDLE && + context.vkDestroyDescriptorSetLayout) { + context.vkDestroyDescriptorSetLayout( + context.device, entry.set_layout, nullptr); + entry.set_layout = VK_NULL_HANDLE; + } + if (entry.shader_module != VK_NULL_HANDLE && + context.vkDestroyShaderModule) { + context.vkDestroyShaderModule( + context.device, entry.shader_module, nullptr); + entry.shader_module = VK_NULL_HANDLE; + } + entry.binding_count = 0; +} + +ShaderCacheEntry::ShaderCacheEntry(ShaderCacheEntry&& other) noexcept + : shader_module(other.shader_module), + set_layout(other.set_layout), + pipeline_layout(other.pipeline_layout), + pipeline(other.pipeline), + binding_count(other.binding_count) { + other.shader_module = VK_NULL_HANDLE; + other.set_layout = VK_NULL_HANDLE; + other.pipeline_layout = VK_NULL_HANDLE; + other.pipeline = VK_NULL_HANDLE; + other.binding_count = 0; +} + +ShaderCacheEntry& ShaderCacheEntry::operator=(ShaderCacheEntry&& other) noexcept { + if (this == &other) { + return *this; + } + // Assumes this entry's handles are already null or ownership was transferred. + // clear() destroys before erase; move-assign is only for empty or stolen rows. + shader_module = other.shader_module; + set_layout = other.set_layout; + pipeline_layout = other.pipeline_layout; + pipeline = other.pipeline; + binding_count = other.binding_count; + other.shader_module = VK_NULL_HANDLE; + other.set_layout = VK_NULL_HANDLE; + other.pipeline_layout = VK_NULL_HANDLE; + other.pipeline = VK_NULL_HANDLE; + other.binding_count = 0; + return *this; +} + +ShaderCache& ShaderCache::getInstance() { + static ShaderCache instance; + return instance; +} + +ShaderCache::~ShaderCache() { + // Static teardown order vs Context is undefined. Shutdown must clear first + // so handles are already null here; only drop the map. + std::lock_guard lock(_cache_mutex); + _cache.clear(); +} + +const ShaderCacheEntry& ShaderCache::add( + const std::string& key, ShaderCacheEntry&& entry) { + std::lock_guard lock(_cache_mutex); + auto [it, inserted] = + _cache.emplace(key, std::move(entry)); + if (!inserted) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: shader cache entry already " + "exists: " + + key); + } + return it->second; +} + +const ShaderCacheEntry& ShaderCache::get(const std::string& key) { + std::lock_guard lock(_cache_mutex); + const auto it = _cache.find(key); + if (it == _cache.end()) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: shader not found in cache: " + + key); + } + return it->second; +} + +void ShaderCache::clear(Context& context) { + std::lock_guard lock(_cache_mutex); + for (auto& [key, entry] : _cache) { + (void)key; + destroy_entry(context, entry); + } + _cache.clear(); +} + +} // namespace cthreads::gpu::shader diff --git a/tests/unit/test_gpu_shader.py b/tests/unit/test_gpu_shader.py new file mode 100644 index 0000000..bb69cc2 --- /dev/null +++ b/tests/unit/test_gpu_shader.py @@ -0,0 +1,108 @@ +""" +GPU shader / launch smoke (test-only _ext.gpu.testing). + +Covers create_entry, ShaderCache, update_descriptors, and launch_gpu_kernel +via smoke_launch_saxpy (join writeback). Live checks skip when CTHREADS_GPU +is off or no Vulkan device is available. +""" + +from __future__ import annotations + +import pytest + +from cthreads import gpu +from cthreads.gpu.errors import GpuInvalidArgument + + +def _ext_gpu(): + return gpu._gpu + + +def _require_gpu_testing(): + ext = _ext_gpu() + if ext is None: + pytest.skip("cthreads built without CTHREADS_GPU (_ext.gpu missing)") + if not hasattr(ext, "testing"): + pytest.skip("_ext.gpu.testing missing (rebuild with CTHREADS_GPU=ON)") + if not gpu.available(): + pytest.skip("Vulkan loader/device not available in this environment") + return ext.testing + + +def _map_probe(exc: BaseException): + return gpu._map_error(exc) + + +def test_public_gpu_has_no_shader_smoke_exports(): + assert not hasattr(gpu, "smoke_create_entry") + assert not hasattr(gpu, "smoke_update_descriptors") + assert not hasattr(gpu, "smoke_launch_saxpy") + assert not hasattr(gpu, "testing") + + +def test_live_smoke_create_entry(): + testing = _require_gpu_testing() + try: + testing.smoke_create_entry() + finally: + gpu.shutdown() + + +def test_live_smoke_update_descriptors(): + testing = _require_gpu_testing() + try: + testing.smoke_update_descriptors() + finally: + gpu.shutdown() + + +def test_live_smoke_cache_register_and_get(): + testing = _require_gpu_testing() + try: + testing.smoke_cache_register_and_get() + finally: + gpu.shutdown() + + +def test_live_smoke_launch_saxpy(): + testing = _require_gpu_testing() + if not hasattr(testing, "smoke_launch_saxpy"): + pytest.skip("rebuild with latest gpu testing (smoke_launch_saxpy)") + try: + testing.smoke_launch_saxpy() + finally: + gpu.shutdown() + + +def test_live_probe_cache_duplicate_add(): + testing = _require_gpu_testing() + try: + with pytest.raises(Exception) as ei: + testing.probe_cache_duplicate_add() + mapped = _map_probe(ei.value) + assert isinstance(mapped, GpuInvalidArgument) + assert "already exists" in str(mapped) or "GpuInvalidArgument" in str(mapped) + finally: + gpu.shutdown() + + +def test_live_probe_update_empty_list_slot(): + testing = _require_gpu_testing() + try: + with pytest.raises(Exception) as ei: + testing.probe_update_empty_list_slot() + mapped = _map_probe(ei.value) + assert isinstance(mapped, GpuInvalidArgument) + finally: + gpu.shutdown() + + +def test_live_probe_create_entry_zero_bindings(): + testing = _require_gpu_testing() + try: + with pytest.raises(Exception) as ei: + testing.probe_create_entry_zero_bindings() + mapped = _map_probe(ei.value) + assert isinstance(mapped, GpuInvalidArgument) + finally: + gpu.shutdown()