Skip to content

Build a compiled .so and create a Python binding - #43

Draft
timholy wants to merge 3 commits into
mainfrom
teh/lib
Draft

Build a compiled .so and create a Python binding#43
timholy wants to merge 3 commits into
mainfrom
teh/lib

Conversation

@timholy

@timholy timholy commented Aug 19, 2026

Copy link
Copy Markdown
Member

This PR wraps MatrixCovers as a compiled Python package (no Julia needed at runtime) using JuliaLibWrapping and JLWInterop. It's intended as a real-world "teaching example," though it will eventually be merged.

Everything lives under lib/, plus one workflow and small README/.gitignore additions. Nothing in the package proper (src/) changed.

Hand-written

  • lib/src/matrixcovers.jl — the binding layer, and the main thing to look at. One @api entrypoint per exposed call (15 of them), each with the docstring that gets carried into the generated Python and C interfaces. Two @enums (Penalty, Linsolve) become Python enum classes. Arguments are Matrix{Float64}/Vector{Float64}, which arrive as zero-copy views of the caller's memory. Note that the Julia package supports arbitrary element type; the Python version will be Float64-specific.

  • lib/src/trimmability.jl — whether you need something like this depends on the contents of your package. juliac --trim=safe compiles only calls it can resolve statically, and a handful of MatrixCovers calls aren't resolvable in their natural form: kwargs... forwarded through several layers, optional (Union{T,Nothing}) keywords, non-tail-position calls. Each function here reshapes one such call, and the comment above it states the constraint that forces the shape. How much effort this requires is highly dependent on the package.

  • lib/python/_facade.py — the Python API. Splits vcat(a, b) returns back into (a, b) tuples, collapsing iscover_sym/iscover_ab into one iscover(a, A, b=None), and one gramcover dispatching over w/W. The C ABI can't yet express tuple returns (planned for JLW 0.3) or Julia's multiple dispatch, so this is where the Python-facing API gets to look like Python.

  • lib/build.jl — the build. Mostly a standard_build call; the extra machinery writes a temporary Project.toml with an absolute [sources] path to the repo (juliac copies the project to a tmpdir, so relative paths don't survive) and picks up the version from the top-level Project.toml.

  • lib/Project.toml / lib/build-env/Project.toml — runtime and build dependencies kept separate. The wrapper module depends on JLWInterop (ABI types + @api); only the build depends on JuliaLibWrapping and JuliaC.

  • lib/test/python/test_smoke.py — smoke test run against the built wheel in CI. Numerical results against references, enums accepted as members or strings, and the error paths (bad enum, shape mismatch, extension-only penalty) — that last group matters, since errors have to survive the trip through JLWStatus rather than being thrown.

  • .github/workflows/python-wheel.yml — 5 lines calling JuliaLibWrapping's reusable workflow. Builds and attaches wheels on release.

Generated (gitignored, lib/out/)

Nothing here is committed; it's what the build produces:

  • matrixcovers/_lowlevel.py — ctypes bindings, struct layouts, ABI checks
  • matrixcovers/_facade.py — generated facade (overwritten by ours)
  • matrixcovers/bundle/matrixcovers.so (~5 MB) plus libjulia and artifacts (~100 MB total unpacked)
  • matrixcovers.h — C header, if you'd rather call it from C
  • matrixcovers.jlw.json, matrixcovers.abi.json — the interface metadata the generators consume
  • pyproject.toml

Scope

Only the AbsLog{2} solvers are compiled in. The AbsLog{1} and AbsLinear paths live in JuMP/HiGHS and JuMP/Ipopt extensions, which aren't linked into the library; asking for them raises an error naming the missing extension.

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.91%. Comparing base (03d948c) to head (a3f96c5).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #43   +/-   ##
=======================================
  Coverage   98.91%   98.91%           
=======================================
  Files          15       15           
  Lines        3785     3785           
=======================================
  Hits         3744     3744           
  Misses         41       41           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Build a shared library with juliac and expose the MatrixCovers API
through ctypes and NumPy. Copy inputs at the ABI boundary and report
failures through structured status values.

The compiled library includes solvers that do not require package
extensions. Unsupported penalty and solver combinations return an
error.

Assisted-by: Claude Sonnet 5 <noreply@anthropic.com>

Assisted-by: Claude Fable 5 <noreply@anthropic.com>
Use JuliaLibWrapping's reusable workflow to build wheels for releases
and manual runs. Document installation and basic Python usage.

Assisted-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant