arm PR103.0: CUDA GDN cache-cpy fusion (arm spec + implementation PR spec) - #112
Open
ddvnguyen wants to merge 1 commit into
Open
arm PR103.0: CUDA GDN cache-cpy fusion (arm spec + implementation PR spec)#112ddvnguyen wants to merge 1 commit into
ddvnguyen wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Arm spec for porting the Metal
GGML_METAL_FUSE_GDN_CACHEfusion (upstream d011a21) to CUDA: GDN kernel writes recurrent-state snapshots directly into the state/cache buffer, trailing cpy elided. Qwen3.8-27B is GDN-heavy (48 of 64 layers) — 48 copies + launch overhead removed per token.What this PR contains
Docs only (
docs/arms/pr103-gdn-cache-cpy-fusion.md): implementation PR spec (kernel epilogue write, fusion-table entry,GGML_CUDA_FUSE_GDN_CACHEenv toggle for single-binary A/B), edge cases (other consumers of GDN output, chunked prefill, MTP/nextn draft layer), A/B test matrix, bars, correctness gates. The fusion implementation lands as a separate code PR after review.Bars (vs arm102/PR105.0 reference)
Test plan
Sequencing: after PR105.0 (reference topology); PR104.x builds on this base.
AI usage disclosure: YES — arm spec drafted by AI (opencode) from the Metal fusion commit semantics and local source verification; implementation code to be human-authored/owned.