[gfx950][FlyDSL] Add direct dense A4W4 MXFP4 GEMM

[LiuYinfeng01]

Summary

Add a native gfx950 FlyDSL dense A4W4 GEMM that quantizes BF16 activations inline to MXFP4 and consumes packed E2M1 weights with E8M0 scales directly in scaled FP4 MFMA.

This is intentionally independent from #4772:

• this PR implements native A4W4 semantics: BF16 activation -> inline MXFP4 quant -> MXFP4 x MXFP4 MFMA
• [gfx950][FlyDSL] Add dense BF16 x MXFP4 GEMM #4772 implements A16WFP4 semantics: BF16 activation x decoded MXFP4 weight -> BF16 MFMA
• no A16WFP4 source or API dependency is included here

Implementation

• public prepare_gemm_a4w4_weight / flydsl_gemm_a4w4 API
• one-time packed E2M1 weight and E8M0 scale preshuffle
• dense direct-output path in the shared FlyDSL GEMM1 body
• BM16 for small M and BM64 for larger M
• BK128 support for K=4224; existing BK256 MoE behavior is preserved
• M-tail masking and ceil-based scale accounting
• benchmark and gfx950 regression coverage

Kernel microbenchmark

MI355X (gfx950), latest AITER main base 878d60d77. Median microseconds; speedup is versus existing Triton dynamic quant + A4W4. Weight preparation/JIT excluded.

N x K	M	Direct us	Triton us	Speedup
8448 x 7168	1	23.201	86.881	3.74x
8448 x 7168	8	26.940	86.620	3.22x
8448 x 7168	32	30.240	71.361	2.36x
8448 x 7168	128	60.121	77.680	1.29x
8448 x 7168	512	89.480	111.921	1.25x
8448 x 7168	2048	237.863	184.681	0.78x
7168 x 4224	1	15.960	75.121	4.71x
7168 x 4224	8	23.041	76.281	3.31x
7168 x 4224	32	23.520	67.040	2.85x
7168 x 4224	128	23.280	67.960	2.92x
7168 x 4224	512	50.041	79.881	1.60x
7168 x 4224	2048	107.041	117.721	1.10x
1536 x 7168	1	18.420	89.821	4.88x
1536 x 7168	8	23.320	90.081	3.86x
1536 x 7168	32	26.181	67.601	2.58x
1536 x 7168	128	28.300	69.020	2.44x
1536 x 7168	512	29.621	81.461	2.75x
1536 x 7168	2048	65.081	117.481	1.81x
7168 x 768	1	12.580	77.060	6.13x
7168 x 768	8	11.921	76.660	6.43x
7168 x 768	32	12.120	72.281	5.96x
7168 x 768	128	12.380	71.401	5.77x
7168 x 768	512	15.520	69.101	4.45x
7168 x 768	2048	31.700	78.920	2.49x

The direct output is numerically equal to the existing Triton A4W4 semantic path within the expected reduction-order error. Across these points, relative L2 was 2.42%-4.15% against the cached BF16/QDQ emulation reference, matching Triton's activation-quantization error.

The one measured crossover is (N=8448,K=7168,M>=2048), where Triton is faster. The downstream vLLM integration uses a shape-aware Triton fallback for that region instead of forcing this kernel.

ATT findings

Wave-state analysis guided the BK/BM choices:

• (7168,4224,M=512), BM64/BK128: quant/decode was 4.95% of sampled latency; wait/barrier was 60.01%
• (8448,7168,M=512), BM64/BK256: quant/decode was 5.59%; VMEM was 23.01%; wait/barrier was 39.26%

This indicates inline quantization is not the dominant cost; the remaining large-shape opportunity is load/wait scheduling.

Model validation

Kimi-K3, TP8, validation image local/k3-direct-a4w4-validation:20260815:

• 5 x 512 decode: 59.521 s mean E2E, 116.135 ms TPOT
• versus #50814 cached BF16 fallback: -2.04% E2E
• versus tuned A16WFP4: -3.59% E2E
• GSM8K 8-shot, first 100: 100/100 strict, 100/100 flexible, zero malformed/invalid
• per-rank device memory: 190.649 GiB average, 4.531 GiB lower than the cached-BF16 #50814 path

The experimental downstream hybrid retained Triton weights only for (8448,7168,M>=2048). That duplicated 32,169,984 bytes per affected layer, or 2.786 GiB/rank across 93 layers; this is a downstream routing trade-off, not an allocation made by the AITER kernel itself.

Validation

• direct gfx950 A4W4: 41 passed
• K=4224 BK128, BM16/BM64, M-tail and scale-edge coverage
• focused existing MXFP4 MoE regression: 11 passed
• strict existing BK256 GEMM1 execution; logits diff 4.05e-06
• latest-main 24-point microbenchmark rerun
• Ruff, Black, py_compile, and git diff --check

Scope

• gfx950 only
• N divisible by 256; K divisible by 128
• packed E2M1 weights and E8M0 scales are preshuffled once by the caller
• this PR does not change vLLM backend selection

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4782 --add-label <label>