arm PR106.x: EXL3 DFlash2 kit reference series (PR106.0 external / PR106.1 native 3.5bpw single-GPU) - #114
Open
ddvnguyen wants to merge 1 commit into
Open
arm PR106.x: EXL3 DFlash2 kit reference series (PR106.0 external / PR106.1 native 3.5bpw single-GPU)#114ddvnguyen wants to merge 1 commit into
ddvnguyen wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Arm series spec derived from the MiaAI-Lab Qwen3.8-27B-DFlash2-EXL3-5.0bpw deployment kit (forked exllamav3 engine; EXL3 3.5 bpw target + MTP or DFlash2 5.0 bpw draft + ~4.5-bit KV lane; batch-1, always-reasoning). llama.cpp cannot load EXL3, so the family splits into an external reference and a llama.cpp-native translation.
What this PR contains
Docs only (
docs/arms/pr106-exl3-dflash2-reference.md):Test plan
Independent of PR103.0/PR104.x. Risks noted: sm_120 EXL3 kernel build unverified on the 5060 Ti; fork engine maturity.
AI usage disclosure: YES — arm spec drafted by AI (opencode) from the kit README and local flag verification; launch line flagged for on-rig confirmation.