Skip to content

Fix Windows pip packaging and cross-platform libHalide shadowing#9233

Merged
alexreinking merged 2 commits into
mainfrom
alexreinking/fix-pip-windows
Jul 22, 2026
Merged

Fix Windows pip packaging and cross-platform libHalide shadowing#9233
alexreinking merged 2 commits into
mainfrom
alexreinking/fix-pip-windows

Conversation

@alexreinking

Copy link
Copy Markdown
Member

Summary

  • Fixes Linux Python bindings can load foreign libHalide.so. #8866: the Python bindings could load a foreign libHalide.so/Halide.dll that shadows the one bundled in the wheel. halide_ depends on it by soname/module name alone, and the dynamic linker's search order (RUNPATH < LD_LIBRARY_PATH on Linux; PATH/default dirs on Windows) lets a same-named library elsewhere on the system take precedence over the bundled copy. Confirmed via readelf that halide_.so has RUNPATH (not RPATH), and reproduced the failure end-to-end in WSL: a garbage libHalide.so on LD_LIBRARY_PATH broke import halide before this fix, and no longer does after it.
  • The fix explicitly loads our own copy via ctypes with an absolute path before importing halide_. Both Linux's and Windows's loaders recognize a module already mapped into the process by its soname/module name and reuse it instead of searching again, so halide_'s implicit dependency resolves to our copy regardless of what else is on the search path.
  • Separately, cibuildwheel 4.0 made delvewheel repair the Windows default, and it was erroring out trying to locate Halide.dll itself (it doesn't search inside the not-yet-repaired wheel), breaking the Windows pip packaging workflow. Re-enabled with --ignore-existing, which tells delvewheel to treat DLLs already bundled in the wheel as satisfying their own dependencies rather than erroring out. Verified against the current published win_amd64 wheel: with this flag, delvewheel leaves Halide.dll alone but still correctly vendors msvcp140.dll, a genuine external MSVC runtime dependency that isn't currently bundled at all -- so this isn't just a no-op safety net, it closes a real gap in the Windows wheel's self-containedness.

Test plan

  • Reproduced Linux Python bindings can load foreign libHalide.so. #8866 in WSL against the published halide==21.0.0 manylinux wheel (garbage libHalide.so on LD_LIBRARY_PATH breaks import)
  • Verified the ctypes-preload fix flips that repro from fail to pass, and that basic Halide functionality (Func/realize) still works
  • Verified delvewheel repair --ignore-existing against the published win_amd64 wheel: no error, Halide.dll untouched, msvcp140.dll correctly vendored
  • CI: Build PyPI package workflow green on all platforms

🤖 Generated with Claude Code

Fixes #8866. The Python bindings could load a foreign libHalide.so/
Halide.dll that shadows the one bundled in the wheel, because halide_
depends on it by soname/module name alone, and the dynamic linker's
search order (RUNPATH < LD_LIBRARY_PATH on Linux; PATH/default dirs on
Windows) lets a same-named library elsewhere on the system take
precedence. Confirmed via readelf (halide_.so has RUNPATH, not RPATH)
and reproduced end-to-end in WSL: a garbage libHalide.so on
LD_LIBRARY_PATH broke `import halide` before this fix, and no longer
does after it.

The fix is to explicitly load our own copy via ctypes with an absolute
path before importing halide_. Both Linux's and Windows's loaders
recognize a module already mapped into the process by its
soname/module name and reuse it instead of searching again, so
halide_'s implicit dependency resolves to our copy regardless of what
else is on the search path.

Separately, cibuildwheel 4.0 made delvewheel repair the Windows
default, and it was erroring out trying to locate Halide.dll itself
(it doesn't search inside the not-yet-repaired wheel), breaking the
Windows pip packaging workflow. Re-enable it with --ignore-existing,
which tells delvewheel to treat DLLs already bundled in the wheel as
satisfying their own dependencies rather than erroring out trying to
locate them itself. Verified against the current published win_amd64
wheel: with this flag, delvewheel leaves Halide.dll alone but still
correctly vendors msvcp140.dll, a genuine external MSVC runtime
dependency that isn't currently bundled at all -- so this isn't just a
no-op safety net, it fixes a real gap in the Windows wheel's
self-containedness.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alexreinking
alexreinking requested a review from shoaibkamil July 22, 2026 16:42
@alexreinking

Copy link
Copy Markdown
Member Author

No CI actually touches these files pre-merge. I'll merge upon approval.

@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@a3bc832). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #9233   +/-   ##
=======================================
  Coverage        ?   70.21%           
=======================================
  Files           ?      255           
  Lines           ?    78939           
  Branches        ?    18882           
=======================================
  Hits            ?    55424           
  Misses          ?    17897           
  Partials        ?     5618           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking
alexreinking merged commit 0561c0d into main Jul 22, 2026
22 of 23 checks passed
@alexreinking
alexreinking deleted the alexreinking/fix-pip-windows branch July 22, 2026 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linux Python bindings can load foreign libHalide.so.

2 participants