Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions concepts/cuda-catalog-and-cache.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

This article covers how Flox and Nix users can consume pre-built CUDA packages from the Flox Catalog and Flox binary cache. **Flox users** can skip ahead to [Flox users: Getting pre-built CUDA dependencies from the Flox Catalog](#flox-users-getting-pre-built-cuda-dependencies-from-the-flox-catalog). **Nix users** can skip ahead to [Nix users: Getting CUDA packages from Flox's public binary cache](#nix-users-getting-cuda-packages-from-floxs-public-binary-cache).

**Note**: The Flox binary cache depends on the voluntary work of the [Nixpkgs CUDA maintainers](https://nixos.org/community/teams/cuda/), who package and maintain NVIDIA's CUDA software for Nixpkgs. This team also tests PyTorch and other critical CUDA-accelerated packages against core NVIDIA CUDA dependencies, working tirelessly to make sure these packages build and link correctly against the evolving NVIDIA CUDA stack. CUDA-on-Nix would not be a reality without them.
**Note**: The Flox binary cache depends on the voluntary work of the [Nixpkgs CUDA maintainers](https://nixos.org/community/teams/cuda/), who package and maintain NVIDIA's CUDA software for Nixpkgs. This team also tests PyTorch and other critical CUDA-accelerated packages against core NVIDIA CUDA dependencies, working tirelessly to make sure these packages build and link correctly against the ever-evolving NVIDIA CUDA stack. CUDA-on-Nix would not be a reality without them.


## CUDA dependencies and Nixpkgs
Expand Down Expand Up @@ -52,7 +52,7 @@
$ flox install flox-cuda/python3Packages.torch
```

Similarly, to declaratively define PyTorch in a Flox environment, run [`flox edit`](/man/flox-edit):

Check warning on line 55 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L55

Did you really mean 'declaratively'?

```toml
[install]
Expand All @@ -77,7 +77,7 @@
flox-cuda/python3Packages.pytorch-lightning Lightweight PyTorch wrapper for machine learning researchers
```

**Note**: Nix-defined Python packages like Torchvision, Torchaudio, or Transformers *already include* their transitive runtime dependencies as part of their closures. As a result, a package like `flox-cuda/python3Packages.torchvision` *automatically pulls in* the version of PyTorch against which it was built. So defining `flox-cuda/python3Packages.torchvision` and `flox-cuda/python3Packages.torch` in the same environment can fetch multiple PyTorch store paths if the explicitly defined `torch` resolves to a different store path from the version pulled in by Torchvision, along with their respective transitive dependencies. This substantially increases the size of the Flox environment's closure.

Check warning on line 80 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L80

Did you really mean 'Torchvision'?

Check warning on line 80 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L80

Did you really mean 'Torchaudio'?

Check warning on line 80 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L80

Did you really mean 'Torchvision'?

To learn more about working with CUDA packages in the Flox Catalog, refer to the [Flox + CUDA tutorial](/tutorials/cuda).

Expand All @@ -86,16 +86,16 @@

## Nix users: Getting CUDA packages from Flox's public binary cache

Flox builds [Nixpkgs' `release-cuda` jobset](https://github.com/nixos/nixpkgs/blob/master/pkgs/top-level/release-cuda.nix) and publishes the built store paths to its public binary cache.

Check warning on line 89 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L89

Did you really mean 'jobset'?

| | |
| ---------- | ---------------------------------------------------------------- |
| Cache URL | https://cache.flox.dev |
| Public Key | flox-cache-public-1:7F4OyH7ZCnFhcze3fJdfyXYLQw/aV7GEed86nQ7IsOs= |

Which store paths are available in the cache depends on the specific Nixpkgs revisions that Flox builds against. Flox currently evaluates and builds CUDA packages against revisions from [its own Nixpkgs fork](https://github.com/flox/nixpkgs). This repo tracks the upstream `nixpkgs-unstable` branch across four distinct [Flox-maintained branches](/concepts/base-catalog#flox-branches): `unstable`, `staging`, `stable`, and `lts`.
Which store paths are available in the cache depends on the specific Nixpkgs revisions that Flox builds against. Flox currently evaluates and builds CUDA packages against revisions from [its own Nixpkgs fork](https://github.com/flox/nixpkgs). This repo tracks the upstream `nixos-unstable` branch across four distinct [Flox-maintained branches](/concepts/base-catalog#flox-branches): `unstable`, `staging`, `stable`, and `lts`.

These branch names describe Flox's Nixpkgs update schedules, *not* the stability of the software they reference. In general the `unstable` branch tracks upstream `nixpkgs-unstable` most closely (i.e., on a daily update schedule). Notably, Flox does not presently build against Nixpkgs release channels, so users that pin to revisions from these channels are unlikely to find matching CUDA store paths in the Flox cache. Because different Nixpkgs revisions can produce different store paths for the same package, **choosing a revision that Flox builds against significantly increases the chances of a cache hit**.
These branch names describe Flox's Nixpkgs update schedules, *not* the stability of the software they reference. In general the `unstable` branch tracks upstream `nixos-unstable` most closely (i.e., on a daily update schedule). Notably, Flox does not presently build against Nixpkgs release channels, so users that pin to revisions from these channels are unlikely to find matching CUDA store paths in the Flox cache. Because different Nixpkgs revisions can produce different store paths for the same package, **choosing a revision that Flox builds against significantly increases the chances of a cache hit**.

<Warning>
NixOS users who do not want system builds to query the Flox cache for NVIDIA driver derivations and other CUDA package derivations should read [Control which builds query the Flox cache](#control-which-builds-query-the-flox-cache) before adding it to `extra-substituters`.
Expand Down Expand Up @@ -133,7 +133,7 @@
$ sudo nixos-rebuild switch
```

Check that the cache is configured as a substituter:

Check warning on line 136 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L136

Did you really mean 'substituter'?

```console
$ nix config show | grep substituters
Expand Down Expand Up @@ -163,9 +163,9 @@

### Control which builds query the Flox cache

When Nix needs a store path, it queries substituters in priority order, starting with higher-priority substituters and falling back to lower-priority ones if it can't find a match. By adding Flox's cache to Nix's substituter set, it therefore becomes one of the sources that Nix queries for matches. On NixOS, this behavior also applies to any builds that use the Nix daemon's default substituter set, including `nixos-rebuild`. These builds may fetch matching NVIDIA driver store paths from Flox *in addition to* user-space CUDA package store paths.

Check warning on line 166 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L166

Did you really mean 'substituter'?

If you do not want NixOS system builds to obtain NVIDIA driver store paths or other CUDA package store paths from the Flox cache, authorize the cache without adding it to the Nix daemon's default substituter set:

Check warning on line 168 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L168

Did you really mean 'substituter'?

```nix
{
Expand All @@ -189,4 +189,4 @@
github:flox/nixpkgs/stable#cudaPackages.cuda_nvcc
```

This way, a `nixos-rebuild` operation queries the Flox cache *only* when the command that starts the build explicitly specifies it as a substituter.

Check warning on line 192 in concepts/cuda-catalog-and-cache.mdx

View check run for this annotation

Mintlify / Mintlify Validation (flox) - vale-spellcheck

concepts/cuda-catalog-and-cache.mdx#L192

Did you really mean 'substituter'?
Loading