Expand languages#20
Open
casualjim wants to merge 35 commits intoRubixDev:mainfrom
Open
Conversation
…at any of this with the agent)
Co-authored-by: casualjim <456109+casualjim@users.noreply.github.com>
Co-authored-by: casualjim <456109+casualjim@users.noreply.github.com>
Co-authored-by: casualjim <456109+casualjim@users.noreply.github.com>
Co-authored-by: casualjim <456109+casualjim@users.noreply.github.com>
Fix CI: reorder WASM target setup to prevent pcre2 JIT compilation errors
Tree-sitter 0.26+ added custom stdlib implementations for all wasm32-unknown* targets, which conflict with Emscripten's libc. Downgrading to 0.25.2 (matching upstream RubixDev/syntastica) resolves the duplicate symbol errors. Changes: - tree-sitter: 0.26.3 -> 0.25.2 - tree-sitter-generate: 0.26.3 -> 0.25.1 - tree-sitter-loader: 0.26.3 -> 0.25.1 - tree-sitter-tags: 0.26.3 -> 0.25.1 - Revert progress_callback API change (ControlFlow -> bool) - Revert generate_parser_in_directory to 6 arguments
Restore the VM setup script and grammar priming script from commits after eb66bd4, without the gitcommit OOM workarounds.
The gitcommit parser has massive character tables (84K lines with 18 locales) that cause emscripten to use 80GB+ RAM during compilation. Set wasm = false in languages.toml to exclude it from JS/WASM builds. The parser will still work for native Rust builds. Changes: - syntastica-macros/languages.toml: wasm = false for gitcommit - Removed syntastica-js/langs/gitcommit/ directory
Exclude tmux from wasm language generation to avoid CI build failures.\nThe emcc compile for tree-sitter-tmux is being SIGKILLed on the runner,\nso this matches the existing gitcommit wasm exclusion. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exclude zsh from wasm language generation to avoid CI build failures.\nThe emcc compile for tree-sitter-zsh is being SIGKILLed on the runner,\nmatching the tmux/gitcommit wasm exclusions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Exclude gnuplot from wasm language generation to avoid CI build failures.\nThe emcc compile for tree-sitter-gnuplot is being SIGKILLed on the runner,\nmatching the tmux/zsh/gitcommit wasm exclusions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
parser.c is too large and causes emcc to be OOM-killed (SIGKILL) on the Hetzner CI runner. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…zner runner registration Co-authored-by: casualjim <456109+casualjim@users.noreply.github.com>
…d-error Fix CI 401 Unauthorized error on Hetzner runner creation
Co-authored-by: casualjim <456109+casualjim@users.noreply.github.com>
Revert broken CI config introduced in PR #3
Deduping by ffi symbol kept one alias gate and could drop the\nextern/build path for mixed-group aliases like c_sharp/csharp\nand diff/gitdiff.\n\nBuild and FFI codegen now gate shared symbols on the union of\nalias features and test the mixed-group cases.
casualjim
force-pushed
the
expand-languages
branch
2 times, most recently
from
0d4cf9d to
81cb0af
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
So I don't know if you're going to want this. it's pretty challenging getting this to build in a reasonable amount of time.
Anyway this brings language support up to 340 languages. I used it to create this: https://github.com/casualjim/umber
I wanted to make something that would do more languages and formats than bat and would be faster, it's both those things.
I also joined hyperpolyglot (github's linguist in rust) with your tft. https://github.com/casualjim/palate it also detects all those languages.
I really liked this syntastica project when I found it, but I also have embeders and chunkers and such that were using the full set of tree sitter parsers that I found. I was using syntastica for syntax highlighting and I thought it's a shame they can't share these grammars easily, now they can.