Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/clippy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ jobs:
uses: actions/checkout@v4

- name: Install CUDA toolkit
uses: Jimver/cuda-toolkit@3d45d157f327c09c04b50ee6ccdea2d9d017ec76 # v0.2.35
uses: Jimver/cuda-toolkit@0ca15bb027fc96f7f61ddbb9c5221d699da06bdb # v0.2.36
id: cuda-toolkit
with:
cuda: "13.1.1"
cuda: "13.3.1"
method: network
sub-packages: '["nvcc", "cudart-dev"]'
non-cuda-sub-packages: '["libcurand-dev"]'
Expand All @@ -49,7 +49,7 @@ jobs:
# Point all of those invocations at one shared CARGO_TARGET_DIR under
# ${RUNNER_TEMP}. Cargo fingerprints each unit by package + features
# + workspace_root + toolchain, so co-located workspaces using the
# same pinned nightly deduplicate on identical deps and compile each
# same pinned toolchain deduplicate on identical deps and compile each
# one once. This is intentionally scoped *not* to cover the
# `cargo clippy --workspace` step above, which already has a
# well-defined `target/` under cuda-oxide.
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@ jobs:
- name: Install CUDA toolkit
# `cargo doc --workspace` compiles cuda-bindings' build-time bindgen,
# which needs `cuda.h`. Same toolkit pin as the unit-tests job.
uses: Jimver/cuda-toolkit@3d45d157f327c09c04b50ee6ccdea2d9d017ec76 # v0.2.35
uses: Jimver/cuda-toolkit@0ca15bb027fc96f7f61ddbb9c5221d699da06bdb # v0.2.36
id: cuda-toolkit
with:
cuda: "13.1.1"
cuda: "13.3.1"
method: network
sub-packages: '["nvcc", "cudart-dev"]'
non-cuda-sub-packages: '["libcurand-dev"]'
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/examples-compile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,10 @@ jobs:
# Needed for cuda.h and curand.h: bindgen runs in the shared
# cuda-bindings build script. Same setup as unit-tests.yml / clippy.yml.
- name: Install CUDA toolkit
uses: Jimver/cuda-toolkit@3d45d157f327c09c04b50ee6ccdea2d9d017ec76 # v0.2.35
uses: Jimver/cuda-toolkit@0ca15bb027fc96f7f61ddbb9c5221d699da06bdb # v0.2.36
id: cuda-toolkit
with:
cuda: "13.1.1"
cuda: "13.3.1"
method: network
sub-packages: '["nvcc", "cudart-dev"]'
# nvJitLink is packaged separately from cuda-nvcc on Linux. Keep it
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/fmt.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:

# rustfmt comes from cuda-oxide/rust-toolchain.toml's component list,
# like clippy,
# so `cargo fmt` resolves against the same nightly as the rest of CI.
# so `cargo fmt` resolves against the same toolchain as the rest of CI.

- name: Check stable host workspace formatting
run: cargo fmt --all -- --check
Expand Down Expand Up @@ -61,7 +61,7 @@ jobs:
example_dir="$(dirname "$manifest")"
echo "::group::cargo fmt --check ($example_dir)"
# Run inside the component so rustup discovers CUDA Oxide's
# directory-scoped nightly pin.
# directory-scoped toolchain pin.
(cd "$example_dir" && cargo fmt -- --check)
echo "::endgroup::"
done
4 changes: 2 additions & 2 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,10 @@ jobs:

- name: Install CUDA toolkit
if: matrix.needs_cuda
uses: Jimver/cuda-toolkit@3d45d157f327c09c04b50ee6ccdea2d9d017ec76 # v0.2.35
uses: Jimver/cuda-toolkit@0ca15bb027fc96f7f61ddbb9c5221d699da06bdb # v0.2.36
id: cuda-toolkit
with:
cuda: "13.1.1"
cuda: "13.3.1"
method: network
sub-packages: '["nvcc", "cudart-dev"]'
non-cuda-sub-packages: '["libcurand-dev"]'
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ resolver = "3"
members = [
# Stable host-side crates shared by the Tile and SIMT components.
# Package metadata and dependencies stay in each crate so this workspace
# does not impose CUDA Oxide's nightly toolchain or edition.
# does not impose CUDA Oxide's workspace toolchain pin or edition.
"cuda-bindings",
"cuda-core",
"cuda-core-derive",
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ shape its direction by sharing feedback on your experience.
| Safety | Data-race freedom for disjoint slice accesses; you control thread synchronization and shared memory | Data-race freedom by construction within a kernel: no explicit thread indexing, shared memory, or synchronization |
| Indexing | Explicit thread/block indices | Implicit via partitions |
| Compiles | Ahead of time, Rust → PTX | JIT at first launch, Rust → Tile IR → cubin |
| Toolchain | Pinned nightly, managed by `cargo oxide` | Stable Rust 1.89 or newer |
| Toolchain | Pinned stable Rust (`1.99.1`), managed by `cargo oxide` | Stable Rust 1.89 or newer |
| Portability | Produces PTX that the driver compiles for compatible GPUs; you control tuning per GPU | CUDA architecture-agnostic; the compiler tunes per GPU |
| Reach for it when | You need direct control over threads, warps, shared memory, TMA, or cluster ops | Your problem is naturally expressed as tensor computations and you want the compiler to handle scheduling details |

Expand Down Expand Up @@ -81,7 +81,7 @@ are documented in the cutile-rs book.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for commit sign-off requirements, pull requests, and CI. Toolchain requirements differ between the two: cutile-rs builds on stable Rust, cuda-oxide needs its pinned nightly via `cargo oxide`; each book covers its own.
See [CONTRIBUTING.md](CONTRIBUTING.md) for commit sign-off requirements, pull requests, and CI. Both components build on stable Rust and pin their toolchains in their respective `rust-toolchain.toml` files (with `rustc-dev` and `llvm-tools` for `cuda-oxide`, managed via `cargo oxide`); each book covers its own setup.

## License

Expand Down
1 change: 1 addition & 0 deletions cuda-oxide/.devcontainer/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ RUN apt-get update \
pkg-config \
python3 \
python3-pip \
python3-venv \
xz-utils \
&& curl -fsSL https://apt.llvm.org/llvm-snapshot.gpg.key \
| gpg --dearmor -o /usr/share/keyrings/apt.llvm.org.gpg \
Expand Down
5 changes: 3 additions & 2 deletions cuda-oxide/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,9 @@ authors = ["Nihal Pasham <nihalp@nvidia.com>"]
license = "Apache-2.0"
repository = "https://github.com/NVIDIA/cuda-rust"

# Note: crates/rustc-codegen-cuda is NOT a workspace member because it requires
# special rustc nightly features and a different build process.
# Note: crates/rustc-codegen-cuda is NOT a workspace member because it links
# against rustc_private (via rustc-dev and RUSTC_BOOTSTRAP=1) and uses a
# separate build process.
# Build and run examples with: cargo oxide run <example>
# Examples are in: crates/rustc-codegen-cuda/examples/ and cute-rs-preview/examples/
# Complete model workloads are in: cute-rs-preview/models/
Expand Down
12 changes: 6 additions & 6 deletions cuda-oxide/crates/cargo-oxide/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,7 @@ merged repository it also formats the parent shared-host workspace. With

### `cargo oxide doctor`

Validates that your environment is correctly set up: Rust nightly toolchain,
Validates that your environment is correctly set up: pinned Rust toolchain,
CUDA headers (`cuda.h`), CUDA toolkit (`nvcc`, libNVVM, nvJitLink,
libdevice), LLVM (`llc`), clang/libclang, the NVIDIA driver / GPU, and the
codegen backend `.so`. Optional probes cover `cuda-gdb` (for `debug`) and
Expand Down Expand Up @@ -510,11 +510,11 @@ Cargo.lock cuda-device = git+https://github.com/NVIDIA/cuda-rust.git#<sha>
The kernels compile against the crates at that commit and the backend that
lowers them is built from the same commit, so the two cannot drift. Changing
the dependency (`cargo update`, a new `rev = ...`) rebuilds the cache on the
next build. The checkout's `rust-toolchain.toml` names the nightly that commit
needs; if the project pins a different one, the build stops before compiling
and tells you which channel to set. A path dependency on a local checkout
builds in place, like step 3. Projects with no cuda-oxide dependency fall back
to a one-time clone of `main`. The backend's build tree lives at
next build. The checkout's `rust-toolchain.toml` names the toolchain channel
that commit needs; if the project pins a different one, the build stops before
compiling and tells you which channel to set. A path dependency on a local
checkout builds in place, like step 3. Projects with no cuda-oxide dependency
fall back to a one-time clone of `main`. The backend's build tree lives at
`~/.cargo/cuda-oxide/target` (shared by every commit's builds; delete it to
reclaim the space).

Expand Down
2 changes: 1 addition & 1 deletion cuda-oxide/crates/cargo-oxide/src/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -813,7 +813,7 @@ fn report_unhealable_toolchain_mismatch(current: &str, recorded: &str) {
toolchain_identity_line(recorded)
);
eprintln!(
"Your project's rust-toolchain.toml pins a different nightly than the \
"Your project's rust-toolchain.toml pins a different toolchain than the \
cuda-oxide backend source it depends on. Align the pins (and run \
`cargo oxide update` after changing them), or point CUDA_OXIDE_BACKEND \
at a backend built with your project's toolchain."
Expand Down
12 changes: 3 additions & 9 deletions cuda-oxide/crates/cargo-oxide/src/commands/doctor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ pub(super) fn backend_source_check(

/// Validate the development environment.
///
/// Checks for: Rust nightly toolchain, `rust-toolchain.toml`, the codegen
/// Checks for: Rust toolchain (`rustc`), `rust-toolchain.toml`, the codegen
/// backend `.so` (informational), CUDA headers (`cuda.h`), CUDA toolkit
/// (`nvcc`, libNVVM, nvJitLink, libdevice), LLVM (`llc`), clang/libclang,
/// the NVIDIA driver / GPU (informational), and optionally `cuda-gdb` /
Expand All @@ -379,17 +379,11 @@ pub fn doctor(ctx: &Context) {
let mut ok = true;

// 1. Rust toolchain
print!("Rust nightly toolchain... ");
print!("Rust toolchain... ");
match Command::new("rustc").args(["--version"]).output() {
Ok(output) if output.status.success() => {
let version = String::from_utf8_lossy(&output.stdout);
let version = version.trim();
if version.contains("nightly") {
println!("✓ {}", version);
} else {
println!("✗ expected nightly, got: {}", version);
ok = false;
}
println!("✓ {}", version.trim());
}
_ => {
println!("✗ rustc not found");
Expand Down
8 changes: 4 additions & 4 deletions cuda-oxide/crates/cuda-host/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@
// that would silently reject non-`'static` borrowing closures).
//
// We accept the `internal_features` warning here: cuda-oxide already ships
// `rustc_codegen_cuda` against `rustc_private` and pins a nightly
// toolchain, so the broader project is firmly inside rustc's internal API
// surface. Anyone trying to lift this helper to a stable crate will hit
// the same gate and have to make the same trade-off there.
// `rustc_codegen_cuda` against `rustc_private` and pins its Rust toolchain
// with `RUSTC_BOOTSTRAP=1`, so the broader project is firmly inside rustc's
// internal API surface. Anyone trying to lift this helper to a standalone
// crate will hit the same gate and have to make the same trade-off there.
#![feature(core_intrinsics)]
#![allow(internal_features)]

Expand Down
2 changes: 1 addition & 1 deletion cuda-oxide/crates/cuda-oxide-codegen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ lowering, and LLVM tooling as the rustc frontend uses. The only supported
public surface is the `experimental` module.

The point is the absence of a dependency: this crate has **no rustc linkage**.
It needs neither `rustc_private` nor a nightly toolchain matched to
It needs neither `rustc_private` nor a toolchain pin matched to
`rustc_driver`, so a caller that can build the IR itself can reach PTX without
the compiler-plugin machinery `rustc-codegen-cuda` requires.

Expand Down
2 changes: 1 addition & 1 deletion cuda-oxide/crates/cuda-oxide-codegen/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
//! the rustc frontend.
//!
//! This crate intentionally has no rustc linkage. It does not require
//! `rustc_private` or a nightly toolchain matched to `rustc_driver`.
//! `rustc_private` or a toolchain pin matched to `rustc_driver`.

#![warn(missing_docs)]

Expand Down
2 changes: 1 addition & 1 deletion cuda-oxide/crates/rustc-codegen-cuda/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ readme = "README.md"
[lib]
crate-type = ["dylib"]

# Keep out of workspace - requires special nightly build
# Keep out of workspace - links against rustc_private (rustc-dev)
[workspace]

[dependencies]
Expand Down
15 changes: 8 additions & 7 deletions cuda-oxide/cuda-oxide-book/appendix/building-from-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ crates (`cuda-device`, `cuda-host`, `cuda-macros`, etc.) and the build tooling
(`cargo-oxide`). The host runtime (`cuda-bindings`, `cuda-core`, `cuda-async`)
is shared with cutile-rs and lives at the repository root; the SIMT workspace
depends on those crates by path. This section, "Verify the installation", and
"Common commands" run from `cuda-oxide/`, where the nightly pin and the
"Common commands" run from `cuda-oxide/`, where the SIMT toolchain pin and the
`cargo oxide` workspace alias apply:

```bash
Expand All @@ -148,8 +148,9 @@ cargo build

```{note}
The codegen backend (`cuda-oxide/crates/rustc-codegen-cuda/`) is intentionally **not** a
workspace member because it requires special nightly features and a different
build process. `cargo-oxide` handles building it transparently.
workspace member because it links against `rustc_private` (via `rustc-dev` and
`RUSTC_BOOTSTRAP=1`) and uses a separate build process. `cargo-oxide` handles
building it transparently.
```

## Install cargo-oxide
Expand Down Expand Up @@ -248,8 +249,8 @@ repository/
├── cuda-async/ # Shared async runtime
├── cutile-rs/ # Tile DSL (nested import)
└── cuda-oxide/
├── Cargo.toml # Nightly SIMT workspace
├── rust-toolchain.toml # Pinned nightly + components
├── Cargo.toml # SIMT workspace
├── rust-toolchain.toml # Pinned stable toolchain (1.99.1) + components
├── crates/
│ ├── cuda-device/ # Device intrinsics (#![no_std])
│ ├── cuda-host/ # Host launch APIs
Expand Down Expand Up @@ -295,5 +296,5 @@ CUDA driver version mismatch

`cargo oxide doctor` fails on codegen backend
: The backend is built on first use. If the build fails, check that
`rust-src` is installed and that the nightly version matches
`rust-toolchain.toml`.
`rust-src` and `rustc-dev` are installed and that the active toolchain
matches `rust-toolchain.toml`.
Original file line number Diff line number Diff line change
Expand Up @@ -488,19 +488,19 @@ cargo oxide doctor

Doctor checks:

| Check | What it verifies |
|:----------------|:-----------------------------------------------|
| Rust toolchain | Nightly compiler with required components |
| Codegen backend | `librustc_codegen_cuda.so` built |
| CUDA headers | `cuda.h` present under the toolkit root |
| CUDA toolkit | `nvcc` found and version compatible |
| libNVVM | `libnvvm.so` loadable (libdevice math kernels) |
| nvJitLink | `libnvJitLink.so` loadable (same) |
| libdevice | `libdevice.10.bc` discoverable (same) |
| LLVM | `llc` (21+) available for PTX generation |
| Driver / GPU | `nvidia-smi` reports a GPU and its compute cap |
| cuda-gdb | Optional; only needed for `cargo oxide debug` |
| compute-sanitizer | Optional; only needed for `cargo oxide sanitize` |
| Check | What it verifies |
|:----------------|:----------------------------------------------------|
| Rust toolchain | Pinned compiler (`1.99.1`) with required components |
| Codegen backend | `librustc_codegen_cuda.so` built |
| CUDA headers | `cuda.h` present under the toolkit root |
| CUDA toolkit | `nvcc` found and version compatible |
| libNVVM | `libnvvm.so` loadable (libdevice math kernels) |
| nvJitLink | `libnvJitLink.so` loadable (same) |
| libdevice | `libdevice.10.bc` discoverable (same) |
| LLVM | `llc` (21+) available for PTX generation |
| Driver / GPU | `nvidia-smi` reports a GPU and its compute cap |
| cuda-gdb | Optional; only needed for `cargo oxide debug` |
| compute-sanitizer | Optional; only needed for `cargo oxide sanitize` |

The libNVVM / nvJitLink / libdevice checks fire only when a kernel calls
CUDA libdevice math (`sin`, `cos`, `exp`, `pow`, `sqrt`, ...). If your
Expand Down
2 changes: 1 addition & 1 deletion cuda-oxide/cute-rs-preview/ctm-rs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ expansions name it.

## Check

From `cuda-oxide/` with its pinned nightly:
From `cuda-oxide/` with its pinned toolchain:

```sh
cargo test -p ctm-rs --all-targets
Expand Down
2 changes: 1 addition & 1 deletion cuda-oxide/cute-rs-preview/cute-layout/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ documented arithmetic and shape preconditions.

## Validation

Run from `cuda-oxide/` with its pinned nightly:
Run from `cuda-oxide/` with its pinned toolchain:

```sh
cargo test -p cute-layout
Expand Down
2 changes: 1 addition & 1 deletion cuda-oxide/cute-rs-preview/cute-rs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ items are the `epilogue::sm120` items.

## Checks

Run from the `cuda-oxide/` workspace with its pinned nightly:
Run from the `cuda-oxide/` workspace with its pinned toolchain:

```sh
cargo test -p cute-rs --no-default-features
Expand Down
4 changes: 2 additions & 2 deletions cuda-oxide/cute-rs-preview/examples/ffn_cute/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ input work and completes all four kernels before returning.

## Run it

From `cuda-oxide/`, with the pinned toolchain, CUDA toolkit, CUTLASS compiler,
and an RTX 5090:
From `cuda-oxide/`, with the pinned toolchain, CUDA toolkit, the CUTLASS compiler
(`cargo oxide toolchain install cutlass`) and an RTX 5090:

```bash
CUDA_OXIDE_DEVICE_BACKEND=cutlass-mlir \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@ Q/K/V tensor views → typed copies → FP32 dot products and local softmax
- The cache contains only visible tokens. Cache management, paging and grouped-query
attention are outside this example.

From `cuda-oxide/`, install CUTLASS 4.7 (`cargo oxide toolchain install cutlass`)
and prepare the shared Python environment once for prefill and decode:
From `cuda-oxide/`, once per checkout:

```bash
cargo oxide toolchain install cutlass
python3 -m venv cute-rs-preview/.venv
cute-rs-preview/.venv/bin/python -m pip install -r cute-rs-preview/examples/fmha_prefill_sm120a_cute/reference/requirements.txt
cute-rs-preview/.venv/bin/pip install -r cute-rs-preview/requirements.txt
```

Compare runtime and static specialization on RTX 5090:
Expand All @@ -45,10 +45,8 @@ Add `--json PATH` to save the comparison, `--iters 0` to verify without timing,
or `--rust-only` for a standalone Rust report. `--all-cases` cannot be combined
with `--rust-only`. Pass `--help` after `--` for all options.

The comparison needs CUDA-enabled PyTorch and CUTLASS DSL 4.7; see
[reference setup](reference/README.md). The shared virtualenv is found
automatically. Use `--python PATH` to select another interpreter and
`--device N` to select another visible GPU; device 0 is the default.
Use `--python PATH` to select another interpreter and `--device N` to select
another visible GPU; device 0 is the default.
The CuTeDSL reference is independently authored here using the same split-KV
algorithm; it is **not an upstream CUTLASS decode kernel**.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,16 @@ stride and merge bounds dynamic. The comparison requires the same effective
mode on both sides, so neither compiler receives an extra shape advantage.

Use `nvidia-cutlass-dsl[cu13]==4.7.0` with a compatible CUDA-enabled PyTorch
build and driver. `requirements.txt` pins the DSL; dependencies retain their
own licenses. All code in this directory has the indicated first-party
Apache-2.0 license.
build and driver. The shared [`requirements.txt`](../../../requirements.txt)
pins the DSL; dependencies retain their own licenses. All code in this
directory has the indicated first-party Apache-2.0 license.

Prepare the shared Python environment once for both prefill and decode,
from `cuda-oxide/`:

```bash
python3 -m venv cute-rs-preview/.venv
cute-rs-preview/.venv/bin/python -m pip install -r cute-rs-preview/examples/fmha_prefill_sm120a_cute/reference/requirements.txt
cute-rs-preview/.venv/bin/pip install -r cute-rs-preview/requirements.txt
```

Select a CUDA-enabled PyTorch build that supports SM120 and matches your CUDA
Expand Down
Loading
Loading