arm PR104.x: mixed-quant row sharding arm series (PR104.0 roofline / PR104.1 spike / PR104.2 production)

[ddvnguyen]

Arm series spec for per-device quantization of row-split weight slices: Q4_K rows on the 3060, Q5/Q6_K rows on the 5060 Ti. Row ratios unchanged; bytes-per-token rebalanced onto bandwidth-proportional lines. Attacks the rig's actual wall (3060 bandwidth x VRAM-forced share) instead of porting htsglang-style asymmetric TP.

What this PR contains

Docs only (docs/arms/pr104-mixed-quant-row-sharding.md):

• Roofline: current ~41.5/58.5 VRAM-pinned split -> ~31 ms/token (~32 t/s raw); balanced mixed-quant -> ~21 ms (~47 t/s raw ceiling, +30-45%), plus 3-4 GB freed VRAM
• PR104.0: paper roofline from measured per-device BW (no code) — go/no-go >= 45 t/s predicted, else kill-switch
• PR104.1: spike via LLAMA_ARG_SPLIT_ROW_QUANT=q4_k,q6_k (default = model type, rebuild-free A/B); bars >= 43 ship-min / >= 45 target
• PR104.2: production — imatrix-aware requant, llama-perplexity KL quality gate vs unsharded baseline, CLI + docs, MATRIX_ROW_PADDING per slice type
• Implementation spec: per-device slice type in ggml_tensor_extra_gpu, init_tensor/set_tensor changes (ggml-cuda.cu:929/979), mul_mat dispatch from slice_type[id]; no graph/kernel/ggml-core changes

Test plan

• PR104.0 roofline measured and go/no-go decided in writing
• PR104.1 spike A/B on PR105.0 topology, bars checked
• PR104.2 quality gate (KL) agreed threshold pass
• Results into arm report

Sequencing: PR104.0 can run in parallel with PR103.0 coding; PR104.1/2 build on PR103.0's landed base.

AI usage disclosure: YES — arm spec drafted by AI (opencode) from local row-split source verification (split buffer single-type limitation at ggml-cuda.cu:929/950/979); implementation code to be human-authored/owned.