Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -416,6 +416,38 @@ jobs:
with:
is-release: ${{ github.ref_type == 'tag' }}

precommit-windows:
Comment thread
mdboom marked this conversation as resolved.
name: Pre-commit on Windows
runs-on: windows-latest
if: ${{ github.repository_owner == 'nvidia' && !fromJSON(needs.should-skip.outputs.skip) }}
needs:
- should-skip
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 1
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: '3.13'

- name: Install pre-commit
shell: bash
run: |
set -euxo pipefail
python -m pip install --upgrade pip pre-commit

- name: Run pre-commit
shell: bash
run: |
set -euxo pipefail
SKIP=lychee pre-commit run --all-files

checks:
name: Check job status
if: always()
Expand All @@ -429,6 +461,7 @@ jobs:
- test-linux-aarch64
- test-windows
- doc
- precommit-windows
steps:
- name: Exit
run: |
Expand Down Expand Up @@ -461,6 +494,7 @@ jobs:
check_result "should-skip" "success" "${{ needs.should-skip.result }}"
check_result "detect-changes" "success" "${{ needs.detect-changes.result }}"
check_result "doc" "success" "${{ needs.doc.result }}"
check_result "precommit-windows" "success" "${{ needs.precommit-windows.result }}"

# [doc-only] flips these from 'success' to 'skipped'
if [[ "$doc_only" == "true" ]]; then expected="skipped"; else expected="success"; fi
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ repos:

- id: stubgen-pyx-cuda-core
name: Generate .pyi stubs for cuda_core
entry: stubgen-pyx cuda_core/cuda --continue-on-error --include-private
entry: python ./toolshed/run_stubgen_pyx.py
language: python
files: ^cuda_core/cuda/.*\.(pyx|pxd)$
pass_filenames: false
Expand Down Expand Up @@ -88,7 +88,7 @@ repos:
- id: check-yaml
- id: debug-statements
- id: end-of-file-fixer
exclude: &gen_exclude '^(?:cuda_python/README\.md|cuda_bindings/cuda/bindings/.*\.in?|cuda_bindings/docs/source/module/.*\.rst?|.*\.pyi)$'
exclude: &gen_exclude '^(?:cuda_python/README\.md|(?:^|/)CLAUDE\.md|(?:^|/)\.git_archival\.txt|cuda_bindings/cuda/bindings/.*\.in?|cuda_bindings/docs/source/module/.*\.rst?|.*\.pyi)$'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still misses eight nested symlink placeholders. Because the complete expression is anchored with ^, the inner (?:^|/) alternatives are evaluated at offset 0 and cannot match paths such as cuda_core/CLAUDE.md or cuda_pathfinder/.git_archival.txt. In a core.symlinks=false checkout, end-of-file-fixer rewrote all eight nested placeholders. Could we use something like (?:.*/)?(?:CLAUDE\.md|\.git_archival\.txt) inside the outer anchored alternation (or list the tracked paths explicitly), and ideally exercise a core.symlinks=false checkout in CI?

- id: mixed-line-ending
- id: trailing-whitespace
exclude: |
Expand Down
12 changes: 12 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Thank you for your interest in contributing to CUDA Python! Based on the type of
- [Table of Contents](#table-of-contents)
- [Type stubs for cuda.core](#type-stubs-for-cudacore)
- [Pre-commit](#pre-commit)
- [Pre-commit on Windows](#pre-commit-on-windows)
- [Signing Your Work](#signing-your-work)
- [Code signing](#code-signing)
- [Developer Certificate of Origin (DCO)](#developer-certificate-of-origin-dco)
Expand Down Expand Up @@ -74,6 +75,17 @@ between commits, leaving stale headers or out-of-date stubs in the history.
If the hook isn't installed, `pre-commit run` (and CI) will print a visible
warning reminding you to run `pre-commit install`.

### Pre-commit on Windows

For development on Windows (not WSL), the `lychee` pre-commit task will not work
when running `pre-commit run --all-files`. This problem does not occur if you
install the pre-commit hook and run it automatically as part of your `git
commit` workflow. To resolve this, you can either:

1. Run `pre-commit` it in Git Bash, rather directly in PowerShell or cmd

2. Skip it by setting the environment variable `SKIP` to `lychee`. This would
be `$env:SKIP = "lychee"` in PowerShell or `SKIP=lychee` in cmd.
Comment on lines +85 to +88

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Command Prompt requires set SKIP=lychee (plain SKIP=lychee is not cmd syntax).


## Signing Your Work

Expand Down
58 changes: 35 additions & 23 deletions cuda_pathfinder/cuda/pathfinder/_dynamic_libs/load_dl_linux.py

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Touching this file seems unnecessary, why does pre-commit care on Windows?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because mypy is now checking this on Windows, where things like os.RTLD_NOW don't exist.

mypy doesn't/can't know that this is only imported on Linux. It does have special support for parsing if sys.platform == "linux" (which is why this works), but it can't follow that between modules. (And we use slightly different logic there anyway).

IME, this stuff is necessary to make type-checking complete, annoying as it is.

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

from __future__ import annotations
Expand All @@ -7,14 +7,18 @@
import ctypes
import ctypes.util
import os
import sys
from typing import TYPE_CHECKING, cast

from cuda.pathfinder._dynamic_libs.load_dl_common import LoadedDL

if TYPE_CHECKING:
from cuda.pathfinder._dynamic_libs.lib_descriptor import LibDescriptor

CDLL_MODE = os.RTLD_NOW | os.RTLD_GLOBAL
if sys.platform == "linux":
CDLL_MODE = os.RTLD_NOW | os.RTLD_GLOBAL
else:
CDLL_MODE = 0


def _load_libdl() -> ctypes.CDLL:
Expand Down Expand Up @@ -132,27 +136,35 @@ def _candidate_sonames(desc: LibDescriptor) -> list[str]:
return candidates


def check_if_already_loaded_from_elsewhere(desc: LibDescriptor, _have_abs_path: bool) -> LoadedDL | None:
for soname in _candidate_sonames(desc):
try:
handle = ctypes.CDLL(soname, mode=os.RTLD_NOLOAD)
except OSError:
continue
else:
return LoadedDL(
abs_path_for_dynamic_library(desc.name, handle),
True,
handle._handle,
"was-already-loaded-from-elsewhere",
)
return None


def _load_lib(desc: LibDescriptor, filename: str) -> ctypes.CDLL:
cdll_mode = CDLL_MODE
if desc.requires_rtld_deepbind:
cdll_mode |= os.RTLD_DEEPBIND
return ctypes.CDLL(filename, cdll_mode)
if sys.platform == "linux":

def check_if_already_loaded_from_elsewhere(desc: LibDescriptor, _have_abs_path: bool) -> LoadedDL | None:
for soname in _candidate_sonames(desc):
try:
handle = ctypes.CDLL(soname, mode=os.RTLD_NOLOAD)
except OSError:
continue
else:
return LoadedDL(
abs_path_for_dynamic_library(desc.name, handle),
True,
handle._handle,
"was-already-loaded-from-elsewhere",
)
return None

def _load_lib(desc: LibDescriptor, filename: str) -> ctypes.CDLL:
cdll_mode = CDLL_MODE
if desc.requires_rtld_deepbind:
cdll_mode |= os.RTLD_DEEPBIND
return ctypes.CDLL(filename, cdll_mode)
else:

def check_if_already_loaded_from_elsewhere(_desc: LibDescriptor, _have_abs_path: bool) -> LoadedDL | None:
raise RuntimeError(f"check_if_already_loaded_from_elsewhere() is not supported on platform {sys.platform!r}")

def _load_lib(_desc: LibDescriptor, _filename: str) -> ctypes.CDLL:
raise RuntimeError(f"_load_lib() is not supported on platform {sys.platform!r}")


def load_with_system_search(desc: LibDescriptor) -> LoadedDL | None:
Expand Down
17 changes: 12 additions & 5 deletions cuda_pathfinder/cuda/pathfinder/_dynamic_libs/load_dl_windows.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

from __future__ import annotations
Expand All @@ -21,7 +21,10 @@
POINTER_ADDRESS_SPACE = 2 ** (struct.calcsize("P") * 8)

# Set up kernel32 functions with proper types
kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined]
windll = getattr(ctypes, "windll", None)
if windll is None:
raise RuntimeError("ctypes.windll is required on Windows")
kernel32 = windll.kernel32

# GetModuleHandleW
kernel32.GetModuleHandleW.argtypes = [ctypes.wintypes.LPCWSTR]
Expand All @@ -47,6 +50,10 @@
kernel32.AddDllDirectory.argtypes = [ctypes.wintypes.LPCWSTR]
kernel32.AddDllDirectory.restype = ctypes.c_void_p # DLL_DIRECTORY_COOKIE

# GetLastError
kernel32.GetLastError.argtypes = []
kernel32.GetLastError.restype = ctypes.wintypes.DWORD


def ctypes_handle_to_unsigned_int(handle: ctypes.wintypes.HMODULE) -> int:
"""Convert ctypes HMODULE to unsigned int."""
Expand Down Expand Up @@ -87,15 +94,15 @@ def abs_path_for_dynamic_library(libname: str, handle: ctypes.wintypes.HMODULE)
length = kernel32.GetModuleFileNameW(handle, buffer, len(buffer))

if length == 0:
error_code = ctypes.GetLastError() # type: ignore[attr-defined]
error_code = kernel32.GetLastError()
raise RuntimeError(f"GetModuleFileNameW failed for {libname!r} (error code: {error_code})")

# If buffer was too small, try with larger buffer
if length == len(buffer):
buffer = ctypes.create_unicode_buffer(32768) # Extended path length
length = kernel32.GetModuleFileNameW(handle, buffer, len(buffer))
if length == 0:
error_code = ctypes.GetLastError() # type: ignore[attr-defined]
error_code = kernel32.GetLastError()
raise RuntimeError(f"GetModuleFileNameW failed for {libname!r} (error code: {error_code})")

return buffer.value
Expand Down Expand Up @@ -161,7 +168,7 @@ def load_with_abs_path(desc: LibDescriptor, found_path: str, found_via: str | No
handle = kernel32.LoadLibraryExW(found_path, None, flags)

if not handle:
error_code = ctypes.GetLastError() # type: ignore[attr-defined]
error_code = kernel32.GetLastError()
raise RuntimeError(f"Failed to load DLL at {found_path}: Windows error {error_code}")

return LoadedDL(found_path, False, ctypes_handle_to_unsigned_int(handle), found_via)
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Platform loader seam for OS-specific dynamic linking.
Expand All @@ -16,11 +16,11 @@

from __future__ import annotations

import sys
from typing import Protocol

from cuda.pathfinder._dynamic_libs.lib_descriptor import LibDescriptor
from cuda.pathfinder._dynamic_libs.load_dl_common import LoadedDL
from cuda.pathfinder._utils.platform_aware import IS_WINDOWS


class PlatformLoader(Protocol):
Expand All @@ -31,7 +31,7 @@ def load_with_system_search(self, desc: LibDescriptor) -> LoadedDL | None: ...
def load_with_abs_path(self, desc: LibDescriptor, found_path: str, found_via: str | None = None) -> LoadedDL: ...


if IS_WINDOWS:
if sys.platform == "win32":
from cuda.pathfinder._dynamic_libs import load_dl_windows as _impl
else:
from cuda.pathfinder._dynamic_libs import load_dl_linux as _impl
Expand Down
8 changes: 3 additions & 5 deletions cuda_pathfinder/cuda/pathfinder/_utils/driver_info.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@

import ctypes
import functools
import sys
from collections.abc import Callable
from dataclasses import dataclass

from cuda.pathfinder._dynamic_libs.load_nvidia_dynamic_lib import (
load_nvidia_dynamic_lib as _load_nvidia_dynamic_lib,
)
from cuda.pathfinder._utils.platform_aware import IS_WINDOWS


class QueryDriverCudaVersionError(RuntimeError):
Expand Down Expand Up @@ -63,10 +63,8 @@ def query_driver_cuda_version() -> DriverCudaVersion:
def _query_driver_cuda_version_int() -> int:
"""Return the encoded CUDA driver version from ``cuDriverGetVersion()``."""
loaded_cuda = _load_nvidia_dynamic_lib("cuda")
if IS_WINDOWS:
# `ctypes.WinDLL` exists on Windows at runtime. The ignore is only for
# Linux mypy runs, where the platform stubs do not define that attribute.
loader_cls: Callable[[str], ctypes.CDLL] = ctypes.WinDLL # type: ignore[attr-defined]
if sys.platform == "win32":

@rwgk rwgk Jul 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direct sys.platform check is useful here because mypy understands the platform narrowing, but removing IS_WINDOWS leaves the existing tests patching a deleted attribute (test_utils_driver_info.py:49 and :96), which is why every current Windows test job fails with AttributeError.

I asked my agent about teaching tests to keep IS_WINDOWS and sys.platform synchronized, but it said that's brittle. As a safer alternative, could we make the selected implementation the injectable seam?

if sys.platform == "win32":
    _DRIVER_LIB_LOADER: Callable[[str], ctypes.CDLL] = ctypes.WinDLL
else:
    _DRIVER_LIB_LOADER = ctypes.CDLL

and then:

driver_lib = _DRIVER_LIB_LOADER(loaded_cuda.abs_path)

Unit tests can patch _DRIVER_LIB_LOADER directly, while a small native-platform test can assert that it is ctypes.WinDLL when IS_WINDOWS and ctypes.CDLL otherwise. This keeps direct sys.platform confined to the mypy-recognized selection boundary, avoids globally pretending the process is another OS, and lets Linux/Windows CI verify the real selection.

loader_cls: Callable[[str], ctypes.CDLL] = ctypes.WinDLL
else:
loader_cls = ctypes.CDLL
driver_lib = loader_cls(loaded_cuda.abs_path)
Expand Down
63 changes: 63 additions & 0 deletions toolshed/run_stubgen_pyx.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

"""Run stubgen-pyx for cuda_core and normalize line endings in the generated
stubs across platforms.
"""

from __future__ import annotations

import os
import pathlib
import subprocess


def _normalize_generated_stubs(root: pathlib.Path) -> None:
for stub in root.rglob("*.pyi"):
data = stub.read_bytes()
if not data:
continue

# Normalize line endings to LF so Windows runs do not churn tracked files.
normalized_data = data.replace(b"\r\n", b"\n").replace(b"\r", b"\n")

split_at = normalized_data.find(b"\n")
if split_at == -1:
first_line = normalized_data
remainder = b""
newline = b""
else:
first_line = normalized_data[:split_at]
remainder = normalized_data[split_at + 1 :]
newline = b"\n"

if not first_line.startswith(b"# This file was generated by stubgen-pyx"):
continue

normalized_header = first_line.replace(b"\\", b"/")
updated_data = normalized_header + newline + remainder
if updated_data != data:
stub.write_bytes(updated_data)


def main() -> int:
env = os.environ.copy()
env.setdefault("PYTHONUTF8", "1")
env.setdefault("PYTHONIOENCODING", "utf-8")

cmd = [
"stubgen-pyx",
"cuda_core/cuda",
"--continue-on-error",
"--include-private",
]
result = subprocess.run(cmd, env=env) # noqa: S603
if result.returncode != 0:
return result.returncode

_normalize_generated_stubs(pathlib.Path("cuda_core/cuda"))
return 0


if __name__ == "__main__":
raise SystemExit(main())
Loading