Add GPU launch path and SpawnedGpuKernel join (#22).

.gitignore

@@ -17,6 +17,7 @@ dist/
 *.dylib
 .cthreads_cache.json
 .pytest_cache/
+.vscode/
 
 # CMake / MSVC litter - must never land in the source tree
 src/cthreads/cpp/CMakeCache.txt

docs/gpu_future_cpu_to_gpu.md

@@ -0,0 +1,111 @@
+# Future work: CPU kernels calling GPU
+
+**Status:** Planned after the public Python GPU package is in place (`@Gpu`, `gpu()` / `GpuJob`, marshal, launch, join, writeback).
+
+This note records the intended design so we do not bolt on a second launch stack later. Write it as if the GPU runtime and Python `gpu()` path already exist (or are about to land); this feature only adds a native caller on top.
+
+---
+
+## Idea
+
+GPU work is started from **Python** today (`gpu(fn, *args) -> GpuJob`).
+
+A **CPU** `@Thread` kernel (native C++ in `kernels.dll`) should be able to launch the same GPU work and wait on it, without going back through Python for every dispatch.
+
+Example intent (shape, not final API):
+
+```text
+@Thread
+def step(...):
+    # CPU work
+    gpu_job = launch_gpu(some_gpu_fn, ...)   # C++ handle
+    # more CPU work overlapping GPU
+    gpu_job.join()
+```
+
+Same GpuPack / descriptor / pipeline / writeback machinery as `gpu()`. Only the **caller** changes: Python host vs native CPU kernel.
+
+---
+
+## Why this is a separate step
+
+1. Python `gpu()` is the place that proves the permanent launch types and memory model.
+2. Native CPU->GPU needs a stable **C++ `GpuJob` handle** and a native launch entry that mirrors Python marshal.
+3. Overlapping CPU threads calling GPU will stress **transfer** and **queue submit** concurrency. That is easier to size once the single-caller path is solid.
+
+---
+
+## Assumptions (GPU system already done)
+
+Before this work, the GPU package already provides:
+
+- `Context`, TransferEngine (or an engine pool API), `memory::`, `GpuPack`
+- Descriptor and pipeline path, `GpuJob.join`, writeback into caller-owned buffers / objects
+- Kernel cache keyed by symbol (layout, pipeline, SPIR-V)
+- Inflight state owned by each job (pack, descriptor set, fence)
+- Python `gpu()` as a thin wrapper over that C++ surface
+
+---
+
+## What this needs
+
+### A. C++ job handle (GPU side)
+
+This may already exist for Python bindings. Confirm it is first-class native, not Python-only.
+
+- Type like `cthreads::gpu::GpuJob`.
+- API roughly: create/launch from packed args or from a native marshal helper; `join()`; optional `result()`; destroy / RAII.
+- Safe to store and `join` from a CPU `@Thread` worker (OS thread; no GIL required for the wait itself).
+- Overlapping jobs: two launches of the same GPU symbol get distinct inflight rows (symbol + inflight index), shared **kernel cache** entry.
+
+### B. Native launch entry (GPU side)
+
+Something CPU code can call without pybind, for example:
+
+```text
+GpuJob launch(symbol_or_pipeline_key, GpuPack&& pack, /* writeback plan */)
+```
+
+or a small C ABI used by generated CPU kernels (same spirit as `Fn__call`).
+
+Marshal from C++ values into `GpuPack` (lists/scalars already in native memory). Do not go through `cthreads.marshal` / ctypes.
+
+### C. Transfer concurrency (GPU side)
+
+CPU threads will upload/download without Python sequencing.
+
+- Prefer a **pool of TransferEngines** (or a checkout API) on `Context`, not one global engine held for the whole job lifetime without a plan.
+- Mutex around a single engine is a minimum; a pool is the better fit once CPU->GPU is real.
+- Compute queue submit may need a mutex if multiple host threads submit at once.
+
+### D. Changes to the cthreads CPU / codegen system
+
+- Allow CPU kernels to **call into** the GPU runtime (link against `_ext` GPU symbols or a shared `cthreads_gpu` API exported for kernels).
+- Codegen / validator: a controlled way to express "launch this `@Gpu` from `@Thread`" (name binding, arity, types). Reject anything that implies mid-run Python sync on the GPU job.
+- Job lifetime: CPU `SpawnedKernel` may own or await a child `GpuJob`. Define destroy order (GPU join before CPU pack free if they share buffers). Prefer **separate** packs unless explicitly aliased.
+- Pools: a CPU pool worker launching GPU must not assume the GIL; completion is fence-based like Python `join`.
+- Optional: register GPU child jobs for debugging / `Job` trees. Not required for the first version of this feature.
+
+### E. What not to invent
+
+- A second descriptor/pipeline stack for "native only."
+- A public `DeviceBuffer` type.
+- GPU `__sync_state` / mid-run Python observe.
+- Replacing Python `gpu()`. It stays the host entry; native launch is an additional caller.
+
+---
+
+## Suggested sequencing
+
+1. Confirm `GpuJob` is a complete C++ type used by Python bindings.
+2. Add TransferEngine pool / checkout if not already there.
+3. Add native `launch` + C++ marshal-into-GpuPack for supported types.
+4. Extend CPU codegen/validator for an approved call shape.
+5. Tests: CPU `@Thread` launches `@Gpu`, overlaps, joins; two CPU workers launch GPU concurrently.
+6. Docs: call rules, lifetime, no GIL assumptions.
+
+---
+
+## Relation to other later work
+
+This is an **additive** product feature on top of the finished Python GPU path. It is not a rewrite of pack, descriptors, pipelines, or launch. Track it separately from atomics, MoltenVK, and shared IR (those stay in the Later list in `todo.md`).

docs/index.md

@@ -12,4 +12,6 @@
 | `cthreads documentation` | [link](./COMPILER.md) |
 | **Release**: GitHub Actions, TestPyPI, PyPI trusted publishing | [link](./release.md) |
 | End-to-end example (`@Thread` / `@Threadable` through codegen) | [link](./Example.md) |
-| **Vulkan / GPU backend guide** (cthreads compute path, Issues 1+) | [link](./vk_guide/README.md) |
+| **Vulkan / GPU backend guide** (cthreads compute path) | [link](./vk_guide/README.md) |
+| **Internals:** GPU C++ modules (Context → launch/join) | [link](./internals/gpu/README.md) |
+| **Future:** CPU `@Thread` launching GPU (after Python `gpu()` is done) | [link](./gpu_future_cpu_to_gpu.md) |

docs/internals/gpu/README.md

@@ -0,0 +1,76 @@
+# GPU internals (C++ Vulkan path)
+
+This folder documents the C++ GPU modules under `src/cthreads/cpp/gpu/`. It is written for newcomers who know C++ and maybe Python, but have never used Vulkan. Product-facing Python APIs such as `gpu()` and `@Gpu` are not finished yet. What you see here is the permanent substrate those APIs will call.
+
+Related reading:
+
+- Contributor Vulkan tutorial style notes: [docs/vk_guide/README.md](../../vk_guide/README.md)
+- Future work (CPU `@Thread` calling GPU): [docs/gpu_future_cpu_to_gpu.md](../../gpu_future_cpu_to_gpu.md)
+- Build flag: CMake `CTHREADS_GPU=ON` compiles these sources into `cthreads._ext`
+
+## What problem this stack solves
+
+CPU `@Thread` kernels run as normal operating system threads calling generated C++. GPU work is different. The GPU is a separate processor with its own memory. You cannot hand it a Python list pointer and expect a shader to read it. You must:
+
+1. Open a connection to a Vulkan-capable GPU (the [Context](./context.md)).
+2. Allocate GPU memory and copy host bytes in and out ([memory](./memory.md)).
+3. Group one launch's buffers into a GpuPack ([pack](./pack.md)).
+4. Tell the shader which binding number maps to which buffer ([descriptors](./descriptors.md)).
+5. Build a reusable compute pipeline from SPIR-V bytes ([shader](./shader.md)).
+6. Launch: record bind+dispatch, submit with a fence, then `join` writeback ([module](./module.md)).
+
+## Mental model in one picture
+
+```text
+Python / tests
+    |
+    v
+Context (loader, device, queue, function pointers, TransferEngine)
+    |
+    +-- memory::  create/destroy GpuBuffer, upload/download via staging
+    |
+    +-- pack::    GpuPack (scalar SSBO + list SSBOs)
+    |               + descriptor pool/set/update (same namespace, descriptors.hpp)
+    |
+    +-- shader::  ShaderCacheEntry (module, layouts, pipeline)
+    |             ShaderCache (symbol -> entry)
+    |
+    +-- launch::  launch_gpu_kernel -> SpawnedGpuKernel::join (writeback)
+```
+
+## Binding convention
+
+cthreads packs shader arguments like this:
+
+| Binding | Contents |
+|-|-|
+| 0 | One storage buffer for all scalars (`int`, `float`, `bool`, POD fields) |
+| 1 | First list argument |
+| 2 | Second list argument |
+| ... | More lists |
+
+There are no buffer device addresses (raw GPU pointers) stuffed inside the scalar struct. The descriptor set is the wiring table.
+
+## Module index
+
+| Doc | Namespace / types | Source |
+|-|-|-|
+| [Context](./context.md) | `cthreads::gpu::Context`, `TransferEngine` | `headers/context.hpp`, `impl/context.cpp` |
+| [Memory](./memory.md) | `cthreads::gpu::memory` | `headers/memory.hpp`, `impl/memory.cpp` |
+| [Pack](./pack.md) | `cthreads::gpu::pack` (GpuPack create/upload/download) | `headers/pack.hpp`, `impl/pack.cpp` |
+| [Descriptors](./descriptors.md) | `cthreads::gpu::pack` (pool/set/update) | `headers/descriptors.hpp`, `impl/descriptors.cpp` |
+| [Shader](./shader.md) | `cthreads::gpu::shader` | `headers/shader.hpp`, `shader_cache.hpp`, `impl/shader.cpp`, `shader_cache.cpp` |
+| [Module / launch](./module.md) | `SpawnedGpuKernel`, `launch_gpu_kernel` | `headers/module.hpp`, `impl/module.cpp` |
+
+## Build and availability
+
+GPU code is compiled only when `CTHREADS_GPU` is on. The extension still loads `vulkan-1` (or `libvulkan.so.1`) dynamically at runtime. A machine without a Vulkan loader or GPU fails cleanly through Python `cthreads.gpu.available()` rather than crashing the import of CPU-only wheels.
+
+## What is intentionally not here yet
+
+- Public `gpu()` entry and `@Gpu` SPIR-V emit
+- Threadable / schema marshal writeback (list[float]/list[int] writeback is in `SpawnedGpuKernel::join`)
+- Inflight job store (header stub only)
+- Dummy SSBOs for empty list slots in `update_descriptors`
+
+Those build on the modules documented here.

docs/internals/gpu/context.md

@@ -0,0 +1,163 @@
+# Context and TransferEngine
+
+Source: `src/cthreads/cpp/gpu/headers/context.hpp`, `src/cthreads/cpp/gpu/impl/context.cpp`.
+
+Namespace: `cthreads::gpu`.
+
+This document explains the process-wide Vulkan connection that every other GPU helper needs. If you have never used Vulkan, start here.
+
+## Why a Context exists
+
+Vulkan is not a single library call like "run this kernel." It is a layered API:
+
+1. An operating system library (the Vulkan loader) finds GPU drivers.
+2. You create an instance (your application's connection to the loader).
+3. You pick a physical device (a GPU the driver can see).
+4. You create a logical device (your opened session on that GPU).
+5. You get a queue (a submission port where you send work).
+6. You resolve dozens of function pointers by name, because cthreads loads the loader dynamically instead of linking `vulkan-1` into every wheel.
+
+The `Context` struct holds all of that state for the whole process. There is one Context, accessed through `context()`, guarded by a mutex for `init` / `shutdown` / `available`.
+
+## Public functions
+
+### `void init()`
+
+Opens the loader, creates instance and device, resolves entry points, marks `ready = true`, and creates the TransferEngine (command pool and fence). Throws typed-style error strings (for example `cthreads.gpu.VulkanLoaderNotFound`) that Python maps into exceptions in `cthreads.gpu`.
+
+Call this when you need the GPU. Python `cthreads.gpu.init()` ends up here.
+
+### `void shutdown()`
+
+Destroys children first, then parents:
+
+1. TransferEngine (staging buffer, fence, command pool)
+2. Shader cache entries (pipelines and layouts)
+3. Logical device
+4. Instance
+5. Unload the loader library
+6. Clear all function pointers and handles
+
+Safe to call if the Context was never initialized.
+
+### `bool available()`
+
+Tries to ensure the Context is ready without throwing to the caller for "no GPU" style probes. Used by Python `available()` and by tests that skip when there is no device.
+
+### `const std::string& device_name()`
+
+Returns the human-readable GPU name from Vulkan device properties. Requires a ready Context.
+
+### `Context& context()`
+
+Returns the singleton. Most C++ helpers take `Context&` explicitly so ownership and testing stay clear.
+
+## Struct `Context` fields (grouped)
+
+### Loader and bootstrap
+
+- `loader_module`: operating system handle to `vulkan-1.dll` / `libvulkan.so.1`.
+- `vkGetInstanceProcAddr`: the bootstrap function used to look up almost every other Vulkan function by name string.
+
+### Instance and device entry points
+
+Examples: `vkCreateInstance`, `vkEnumeratePhysicalDevices`, `vkCreateDevice`, `vkGetDeviceQueue`, `vkDestroyDevice`.
+
+These are stored as typed function pointers (`PFN_vk...`). A pointer-to-function (PFN) is simply a C function pointer with the Vulkan signature. cthreads assigns them during init after the loader is open.
+
+### Buffer and memory entry points
+
+Used by [memory](./memory.md): create/destroy buffers, allocate/free device memory, map host-visible memory, query memory properties.
+
+### Command, copy, and sync entry points
+
+Used by transfers and launch: command pools, command buffers, `vkCmdCopyBuffer`, fences, `vkQueueSubmit`, wait/reset fences.
+
+### Shader and pipeline entry points
+
+Used by [shader](./shader.md): create/destroy shader modules, descriptor set layouts, pipeline layouts, and compute pipelines.
+
+### Descriptor pool and update entry points
+
+Used by [descriptors](./descriptors.md): create/destroy descriptor pools, allocate/free sets, `vkUpdateDescriptorSets`.
+
+### Compute dispatch entry points
+
+Used by [module / launch](./module.md): `vkCmdBindPipeline`, `vkCmdBindDescriptorSets`, `vkCmdDispatch`, `vkCmdPipelineBarrier`.
+
+### Opaque handles
+
+- `instance`: connection to the loader for this application.
+- `physical_device`: the chosen GPU.
+- `device`: the logical device (opened GPU session).
+- `queue`: compute queue used to submit copies and dispatches.
+- `queue_family`: index of the queue family that supports compute (needed when creating command pools).
+- `device_name`: string name for logging and Python.
+- `ready`: true only after a fully successful init.
+
+### TransferEngine and mutex
+
+See the next section. `transfer_engine_mutex` serializes use of the single shared engine.
+
+## Technical terms
+
+- Vulkan loader: system library that discovers Installable Client Drivers (ICDs), which are the vendor GPU drivers.
+- Instance: application-level Vulkan object. Not the GPU itself.
+- Physical device: one GPU as seen by the driver.
+- Logical device: your opened handle to use that GPU.
+- Queue: port where the CPU submits recorded command buffers.
+- Queue family: group of queues with the same capabilities (graphics, compute, transfer).
+- Dynamic loading: open the DLL/SO at runtime and resolve symbols by name, instead of linking at build time.
+- `VK_NULL_HANDLE`: sentinel meaning "no object."
+
+## TransferEngine
+
+### Purpose
+
+Uploading and downloading buffers needs a short GPU copy: host-visible staging memory to or from a device-local buffer. Creating a brand new command pool and fence for every copy would be slow and wasteful. The TransferEngine keeps reusable machinery on the Context for the process lifetime.
+
+### Fields
+
+- `command_pool`: Vulkan command pool for the compute queue family. Command buffers for copies are allocated from here and freed after each wait.
+- `fence`: CPU-waitable fence. Created signaled so the first reset/wait path treats it as idle. After each copy, the CPU waits on this fence.
+- `staging`: optional host-visible `GpuBuffer`. Empty until the first upload or download grows it. It only grows; it never shrinks until Context shutdown.
+
+### Lifecycle
+
+- Created in `init_transfer_engine` after the device is ready.
+- Destroyed in `shutdown_transfer_engine` before the logical device is destroyed.
+- Staging is grown by `memory::ensure_staging` under the transfer engine mutex.
+
+### Concurrency
+
+There is one engine today. All upload/download paths hold `transfer_engine_mutex` for the whole transfer (staging grow, memcpy, GPU copy, wait). That avoids races on the shared pool, fence, staging buffer, and queue submit.
+
+A future pool of engines is discussed for CPU threads launching GPU work. See [gpu_future_cpu_to_gpu.md](../../gpu_future_cpu_to_gpu.md).
+
+## Key Vulkan calls during init (story order)
+
+1. Load the loader library (`LoadLibrary` / `dlopen`).
+2. Resolve `vkGetInstanceProcAddr`, then `vkCreateInstance`.
+3. `vkCreateInstance` builds the instance.
+4. Resolve instance-level functions.
+5. `vkEnumeratePhysicalDevices` lists GPUs; pick one with a compute queue family (prefer discrete when possible).
+6. `vkCreateDevice` opens the logical device; `vkGetDeviceQueue` gets the queue.
+7. Resolve device-level functions (buffers, commands, shaders, descriptors).
+8. Create TransferEngine pool and fence.
+9. Set `ready = true`.
+
+## Key Vulkan calls during shutdown (story order)
+
+1. Wait on the transfer fence if needed; destroy staging, fence, command pool.
+2. Clear the shader cache (destroy pipelines and layouts).
+3. `vkDestroyDevice`.
+4. `vkDestroyInstance`.
+5. Unload the loader; null all pointers.
+
+## Relationship to other modules
+
+Every helper in memory, pack, descriptors, and shader takes a `Context&` and expects `ready == true` plus the entry points it needs. If init failed or shutdown already ran, those helpers throw `VulkanInitFailed`-style errors.
+
+## Python surface today
+
+`cthreads.gpu` exposes `init`, `shutdown`, `available`, and `device_name`, plus typed errors. It does not yet expose packs, shaders, or jobs. Those will wrap the C++ types documented in the sibling pages.