Conversation
…asmtime
Replace the ctypes-based Python binding (which loaded a per-platform
libformulon.{so,dylib,dll}) with a pure-Python py3-none-any wheel that
ships a standalone WebAssembly module driven by the wasmtime runtime.
This removes the manylinux build complexity that blocked PyPI publishing
and collapses the 3-platform wheel matrix to a single artifact.
WASM build variant (cmake/FormulonWasm.cmake):
- Add FM_WASM_VARIANT cache variable: embind (default, npm) | capi (PyPI)
- capi variant: compiles src/c_api/wasm_entry.cpp + formulon_static,
links with --no-entry + -sSTANDALONE_WASM=1, exports the curated fm_*
list from tools/wasm/capi_exports.txt plus malloc / free; no -pthread
(scheduler degrades to serial; acceptable for the Python distribution)
- Add tools/wasm/capi_exports.txt with 37 fm_* symbols used by the
Python binding
- New make wasm-capi target; outputs build-wasm-capi/formulon_capi.wasm
(1.5 MiB / 414 KiB Brotli)
Python binding rewrite:
- packages/python/formulon/_c.py: replace ctypes CDLL / Structure / Union
with a wasmtime-driven _WasmInstance class; lazy-init on first use,
shared process-wide; every WASM call serialised by an internal RLock
- _WasmInstance exposes read_bytes / write_bytes / read_u32 / read_i32 /
read_f64 / read_cstr / alloc / free / alloc_utf8 / alloc_bytes for
explicit linear-memory management
- packages/python/formulon/workbook.py: replace ctypes Structure / POINTER
byref patterns with explicit WASM scratch-buffer alloc/free; Value._from_c
becomes Value._from_wasm reading fm_value_t directly from WASM memory via
struct.unpack_from; Workbook._handle changes from fm_workbook_t_p to int
Packaging:
- pyproject.toml: add wasmtime>=20,<45 runtime dependency; update packages
list from formulon._lib to formulon._wasm; set OS Independent classifier
- setup.py: root_is_pure = True; get_tag() returns ("py3", "none", "any")
- MANIFEST.in: swap recursive-include from _lib *.so/*.dylib/*.dll/*.lib
to _wasm *.wasm
- Delete packages/python/formulon/_lib/ directory and platform_tag.py
- Simplify scripts/stage.py from a cmake-invoking build script to a
single shutil.copy2 of formulon_capi.wasm into formulon/_wasm/
- Remove FM_BUILD_PYTHON_SHARED cmake option from CMakeLists.txt
CI (.github/workflows/release.yml):
- Replace 3-runner python-wheels matrix (darwin-arm64 / linux-x64 /
linux-arm64) with a single python-wheel job on ubuntu-latest
- python-wheel job installs Emscripten 3.1.74, runs make wasm-capi then
make python-wheel, and verifies with python -m unittest discover
- Fix pre-existing publish-npm bug: add CMAKE=/usr/bin/cmake to npm-package
and npm-test steps to prevent emsdk's PATH-shadowing non-executable cmake
from causing EACCES failures
- publish-pypi: download single formulon-wheel artifact (no merge-multiple)
Documentation:
- CHANGELOG.md: document the three distribution channels (npm / PyPI / CLI)
and describe the py3-none-any WASM-based wheel approach
- packages/python/README.md: update install, architecture, and
"Building from source" sections to reflect the new WASM transport
Local verification:
- make wasm: embind WASM still builds, npm smoke test passes
- make wasm-capi: formulon_capi.wasm builds at 1.5 MiB / 414 KiB Brotli
- make python-wheel: produces formulon-0.9.0-py3-none-any.whl (558 KB)
- pip install in fresh venv: 22 unittest smoke tests pass
- make build + ctest -LE "SLOW|LOAD|BENCH": 10114 tests passed, 0 failed
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment Thanks for integrating Codecov - We've got you covered ☂️ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Switch the Python distribution from ctypes-based (
libformulon.{so,dylib,dll}) to a pure-Pythonpy3-none-anywheel that ships a standalone WebAssembly module driven by wasmtime. One wheel works on every platform wasmtime supports.Why
The previous v0.9.0 release failed PyPI upload because plain
linux_x86_64/linux_aarch64wheels are rejected; PyPI requires manylinux tags. cibuildwheel + manylinux containers would have added significant CI complexity. The WASM approach removes the per-platform build entirely.Changes
cmake/FormulonWasm.cmake:FM_WASM_VARIANToption switches betweenembind(npm) andcapi(PyPI). The capi variant emits a standalone reactor WASM with the curatedfm_*export list fromtools/wasm/capi_exports.txt._c.pynow drives the WASM through wasmtime;workbook.pyuses explicit linear-memoryalloc/free.pyproject.tomladdswasmtime>=20,<45runtime dep;setup.pymarks the build as pure-Python.CMAKE=/usr/bin/cmakeoverride was missing onmake npm-package/make npm-test.Test plan
make wasm— embind WASM builds,make npm-testpasses (7 tests)make wasm-capi—formulon_capi.wasmbuilds at 1.5 MiB / 414 KiB Brotlimake python-wheel— producesformulon-0.9.0-py3-none-any.whl(558 KB)pip installin fresh venv + 22 unittest smoke tests passmake build+ctest -LE "SLOW|LOAD|BENCH": 10114 tests passed, 0 failedNext steps after merge
v0.9.0tag from originv0.9.0on the mergedmainrelease.ymldrives the full pipeline (npm + CLI + PyPI + GitHub Release)