tree-sitter-languages: add build-tree-sitter-languages.yml for riscv64 wheels - #467
Merged
Conversation
…4 wheels Builds tree-sitter-languages 1.10.2 riscv64 wheels for cp312/cp313/cp314/cp314t, mirroring upstream's release.yml (cibuildwheel over a checkout, before-build runs build.py to clone the 48 pinned grammar repos and link them into languages.so). Upstream pins nothing but Cython, so three build inputs are pinned here: tree-sitter below 0.22 (build.py needs Language.build_library and core.pyx needs Language(path, name), both removed there), Cython 3.3.0 (upstream's 3.0.8 predates free threading and cannot compile core.pyx under cp314t), and setuptools below 79 (79.0.0 dropped -shared from distutils' C++ link command, so languages.so links as an executable and fails on a missing main). Carries one patch adding the vendored grammars' licence files to the wheel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
.github/workflows/build-tree-sitter-languages.yml, building tree-sitter-languages 1.10.2 riscv64 wheels for cp312/cp313/cp314/cp314t. Upstream publishes real per-interpreter compiled wheels (cp37–cp312, ~8.5 MB each) for x86_64/i686/aarch64/musl/macOS/Windows but nothing for riscv64.The workflow mirrors upstream's release.yml: cibuildwheel over a checkout of the tag, with
before-buildrunningbuild.py, which shallow-clones the 48 grammar repositories pinned inrepos.txtand links theirparser.c/scanner.ccinto a singletree_sitter_languages/languages.so(~88 MB unstripped) that the Cythoncoreextension dlopens through py-tree-sitter. Tests are upstream's ownpytest --showlocals {package}/tests, which parses Python with the built parser and instantiates all 48 grammars.Upstream pins only Cython, and three build inputs had to be pinned to build the tag at all today — all three failures are arch-independent and reproduce on x86/arm:
tree-sitter<0.22—build.pycallsLanguage.build_libraryandcore.pyxcallsLanguage(path, name); both were removed in tree-sitter 0.22, so an unpinned resolve (upstream'spip install -e .) gets 0.26 and neither the build nor the test suite can work. The pin covers the test phase too.cython==3.3.0— upstream'scython==3.0.8predates free threading and fails to compilecore.pyxunder cp314t; 3.3.0 builds cleanly on all four interpreters (core.pyxis 15 lines of plain Python, so there is no codegen-drift exposure of the kind gotcha 23 describes).setuptools<79— setuptools 79.0.0 regresseddistutils' C++ link command: the class default forlinker_exe_cxxis["c++", "-shared"], and_linker_paramsstrips that whole prefix offlinker_so_cxx, so-sharedis dropped andbuild.py's link oflanguages.sodies withcrt1.o: undefined reference to 'main'.build.pyusesnew_compiler()withoutcustomize_compiler(), soLDCXXSHAREDnever gets a look-in and an env var cannot work around it. Bisected in the manylinux image: 78.1.1 emitsc++ -shared …, 79.0.1 emitsc++ ….patches/tree-sitter-languages/1.10.2/0001-ship-the-vendored-grammar-licences-in-the-wheel.patchcloses a licensing gap:languages.sois 48 third-party grammars compiled in, each under its own MIT/Apache-2.0 terms, while the wheel ships only tree-sitter-languages' ownLICENSE. The patch addslicense_files=['LICENSE', 'vendor/*/LICENSE*']tosetup.py(the default glob only looks at the project root), which lands 46 licence files indist-info/licenses/vendor/— the two exceptions, tree-sitter-ql and tree-sitter-fixed-form-fortran, carry no licence file at the commitsrepos.txtpins. The test command asserts the count so the patch cannot silently stop applying.Local validation before pushing: a full
cibuildwheel --only cp312-manylinux_aarch64run againstquay.io/pypa/manylinux_2_39_aarch64(same Rocky 10 image family) — clone, compile, wheel, auditwheel repair tomanylinux_2_34_aarch64, licence assertion, 3 tests passed — plus the same build-and-test loop by hand in that image for cp313, cp314 and cp314t. cp314t works but re-enables the GIL on import (Cython does not declarePy_mod_gilfor this module), which is a runtime property of upstream's source, not of the port.