Skip to content

Add an installable native extra-language pack - #53

Closed
thesiti92 wants to merge 4 commits into
mainfrom
native-extra-languages
Closed

thesiti92 wants to merge 4 commits into
mainfrom
native-extra-languages

Conversation

@thesiti92

@thesiti92 thesiti92 commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Move Fortran, F#, Verilog, OCaml (implementation and interface), Julia, Haskell, VHDL, Salesforce Apex, and QML into one optional native pack. LaTeX and general SQL remain bundled. diffr recognizes every language and prefers a compiled parser, then a verified installed parser, then a text diff with a structured fallback reason.

languages/extra.json is the single optional-language registry: it generates Rust parser configuration and drives packaging, availability reporting, and test discovery. Optional entries leave the built-in tree_sitter_parser.rs configuration. The builder reads grammar sources and highlight queries from locked Cargo dependencies; upstream highlight files are not copied into the repository. Query files ship with the pack and are hash-verified before use. Existing Julia/Verilog query customizations and compile-time lang-*/all-languages support remain intact.

Adds diffr languages list/install extra, offline installation, atomic repair, lazy loading, and protected release validation. Runtime downloads use exact catalog pins. The migration steps are documented in languages/ADDING.md.

Validation for the current registry/query layout:

  • Full default Rust workspace: 290 passed, one ignored. All-languages bin/CLI tests: 271 passed, one ignored.
  • Signed hardened-runtime macOS ARM64 installation and native/static output parity for all ten grammars, including corrupt-query detection and repair.
  • Linux ARM64 Docker integration and minimal unprivileged runtime installation with no network or development tools.
  • Registry, publication-gate, formatting, spelling, workflow, and package-content checks pass. Clippy retains six pre-existing warnings.

The previous pack also passed signed x64/Rosetta tests and clean Linux x64 Docker installation, plus all four native CI targets. Four-target CI will rerun for this query relocation. Measurements and evidence boundaries are recorded in languages/VALIDATION.md.

Draft release gate: the embedded catalog remains empty until all four final archives are published and downloaded back with matching hashes. Public npm artifacts are not yet available. Publication requires the protected release environment and signing/npm credentials; this PR does not publish packages or disable macOS library validation.

AI-assisted implementation and validation with Codex.

License packaging: upstream license text is no longer copied into the repository. extra.json pins license paths and hashes; packaging collects Cargo source licenses or fetches them at Cargo's exact upstream VCS revision. All published archives retain licenses. Nine packaging tests pass, and offline regeneration produces an identical macOS ARM64 archive. This packaging-only follow-up is AI-assisted.

Preserve compile-time parser features and add pinned native loading, atomic installation, metadata-driven packaging, and protected release validation.

AI-assisted implementation and validation with Codex.
Verify signed macOS x64 loading under Rosetta and Linux x64 installation without development tools. Grant the publication guard read access to environment protection metadata and fix an error-message spelling.

AI-assisted implementation and validation with Codex.
Generate optional Rust configurations from languages/extra.json and remove their arms from the built-in configuration. Build highlight assets from locked upstream sources, verify them before use, and include shared query licenses. Preserve compiled language features and existing Julia/Verilog queries.

Validated default/all-language tests, signed macOS installation parity, and clean Linux Docker installation with corrupt-query repair. AI-assisted implementation and validation with Codex.
AI-assisted implementation and validation with Codex.

This branch was successfully deployed

1 active deployment
languages-ci — b3b45fe4 Deployed Sep 29, 2026 by thesiti92 via build (aarch64-apple-darwin, macos-14) #4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant