Skip to content

enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions#1862

Merged
rapids-bot[bot] merged 31 commits intorapidsai:release/26.04from
jameslamb:nvjitlink-floor
Mar 19, 2026
Merged

enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions#1862
rapids-bot[bot] merged 31 commits intorapidsai:release/26.04from
jameslamb:nvjitlink-floor

Conversation

@jameslamb
Copy link
Member

@jameslamb jameslamb commented Mar 2, 2026

The changes from #1405 introduced linking against nvJitLink. nvJitLink has versioned symbols that are added in each new CTK release, and some of those are exposed in libcuvs.so.

libcuvs wheels are built against the latest CTK supported in RAPIDS (CUDA 13.1.1 as of this writing), so when those wheels are used in environments with older nvJitLink, runtime errors like this can happen:

libcugraph.so: undefined symbol: __nvJitLinkGetErrorLog_13_1, version libnvJitLink.so.13

For more details, see rapidsai/cugraph#5443

This tries to fix that.

Contributes to rapidsai/build-planning#257

  • builds CUDA 13 wheels with the 13.0 CTK
  • ensures CUDA 13 wheels ship with a runtime dependency of nvidia-nvjitlink>={whatever-minor-version-they-were-built-against}

Contributes to rapidsai/build-planning#256

  • updates wheel tests to cover a range of CTK versions (we previously, accidentally, were only testing the latest 12.x and 13.x)

Other changes

  • ensures conda packages also take on floors of libnvjitlink>={whatever-minor-version-they-were-built-against}

Notes for Reviewers

How I tested this

This uses wheels from similar PRs from RAPIDS dependencies, at build and test time:

Other Options

  1. avoiding those versioned symbols with a build-time shim (WIP: [NOT READY FOR REVIEW] avoid including versioned nvJitLink functions, start testing oldest supported CTK in CI #1855 does this, but hasn't been successful yet)
  2. statically linking libnvJitLink (hasn't been successful yet)

@jameslamb jameslamb added breaking Introduces a breaking change improvement Improves an existing functionality labels Mar 2, 2026
@copy-pr-bot
Copy link

copy-pr-bot bot commented Mar 2, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@jameslamb jameslamb changed the title WIP: enforce a floor on libnvjitlink, test wheels against older CTKs WIP: [NOT READY FOR REVIEW] enforce a floor on libnvjitlink, test wheels against older CTKs Mar 2, 2026
@jameslamb jameslamb changed the title WIP: [NOT READY FOR REVIEW] enforce a floor on libnvjitlink, test wheels against older CTKs WIP: [NOT READY FOR REVIEW] enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions Mar 5, 2026
@jameslamb jameslamb changed the base branch from main to release/26.04 March 12, 2026 19:14
@jameslamb
Copy link
Member Author

/ok to test

@jameslamb jameslamb changed the title WIP: [NOT READY FOR REVIEW] enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions enforce a floor on libnvjitlink, build wheels with CUDA 13.0.x, test wheels against mix of CTK versions Mar 18, 2026
@jameslamb jameslamb marked this pull request as ready for review March 18, 2026 17:21
@jameslamb jameslamb requested review from a team as code owners March 18, 2026 17:21
@jameslamb jameslamb requested review from AyodeAwe and divyegala March 18, 2026 17:21
- nccl
- if: cuda_major == "13"
then:
- libnvjitlink
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added this, per this conversation w/ @bdice : rapidsai/cudf#21671 (comment)

Having libnvjitlink-dev in the host: environment should get these packages the correct run: dependency via run_exports.

Copy link
Contributor

@gforsyth gforsyth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Confirming that libnvjitlink shows up via the run exports of the libnvjitlink-dev


 │ │ Finalized run dependencies (libcuvs-headers-26.04.00a124-cuda13_260319_ade1af6a):
 │ │ ╭──────────────────┬──────────────────────────────────────────────────────────╮
 │ │ │ Name             ┆ Spec                                                     │
 │ │ ╞══════════════════╪══════════════════════════════════════════════════════════╡
 │ │ │ Run dependencies ┆                                                          │
 │ │ │ __glibc          ┆ >=2.28,<3.0.a0 (RE of [build: sysroot_linux-64])         │
 │ │ │                  ┆ >=2.28,<3.0.a0 (RE of [cache-build: sysroot_linux-64])   │
 │ │ │ cuda-cudart      ┆ *                                                        │
 │ │ │ cuda-version     ┆ >=13,<14.0a0 (PC)                                        │
 │ │ │ libcublas        ┆ *                                                        │
 │ │ │ libcurand        ┆ *                                                        │
 │ │ │ libcusolver      ┆ *                                                        │
 │ │ │ libcusparse      ┆ *                                                        │
 │ │ │ libgcc           ┆ >=14 (RE of [cache-build: gcc_linux-64])                 │
 │ │ │                  ┆ >=14 (RE of [cache-build: gxx_linux-64])                 │
 │ │ │ libnvjitlink     ┆ >=13.2.51,<14.0a0 (RE of [host: libnvjitlink-dev])       │
 │ │ │                  ┆ >=13.2.51,<14.0a0 (RE of [cache-host: libnvjitlink-dev]) │
 │ │ │ libraft-headers  ┆ 26.4.*                                                   │
 │ │ │ librmm           ┆ 26.4.*                                                   │
 │ │ │ libstdcxx        ┆ >=14 (RE of [cache-build: gxx_linux-64])                 │
 │ │ │ nccl             ┆ *                                                        │
 │ │ ╰──────────────────┴──────────────────────────────────────────────────────────╯

@gforsyth
Copy link
Contributor

/merge

@rapids-bot rapids-bot bot merged commit df5cd00 into rapidsai:release/26.04 Mar 19, 2026
79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Introduces a breaking change improvement Improves an existing functionality

Development

Successfully merging this pull request may close these issues.

3 participants