Skip to content

fix(tuned_gemm): gate skinny GEMM default to archs with a real kernel - #4791

Open
yichiche wants to merge 1 commit into
ROCm:mainfrom
yichiche:yichiche/skinny-gemm-arch-gate
Open

fix(tuned_gemm): gate skinny GEMM default to archs with a real kernel#4791
yichiche wants to merge 1 commit into
ROCm:mainfrom
yichiche:yichiche/skinny-gemm-arch-gate

Conversation

@yichiche

Copy link
Copy Markdown
Contributor

Summary

  • is_skinny_default_shape() could select the skinny GEMM path on architectures where those kernels are compiled as assert(false) stubs, aborting at runtime.
  • Restrict the skinny default to the archs that actually have a kernel body: gfx90a / gfx942 / gfx950.
  • Unsupported archs (e.g. gfx1250) now fall through to the regular GEMM dispatch instead of aborting.

Motivation

get_GEMM_A16W16_config() falls back to a default config when a shape is not in bf16_tuned_gemm.csv. One of those fallbacks is the skinny path:

elif is_skinny_default_shape(M, N, K, dtype, cu_num):
    default_config["libtype"] = "skinny"

The skinny GEMM kernels — wvSpltK, LLMM1, wv_splitk_small_fp16_bf16 — only have a real body inside #if defined(__HIP__MI350_MI300_MI250__) in csrc/kernels/custom_kernels.cu. That macro is defined only for __gfx90a__ || __gfx942__ || __gfx950__ (custom_kernels.cu:28-30); every other arch compiles the #else branch, whose body is UNREACHABLE_CODE, i.e. assert(false);.

is_skinny_default_shape() itself only looked at M/N/K/dtype/cu_num, never at the arch. So on an arch without the kernel body, any untuned skinny-shaped GEMM (small M, N <= cu_num, K % 8 == 0) selects a stub and aborts. This is reachable on gfx1250 with a bf16 model whose small-M shapes are not yet in the tuned config.

Modifications

aiter/tuned_gemm.py: add an arch allowlist and check it first in is_skinny_default_shape().

_SKINNY_GEMM_ARCHS = {"gfx90a", "gfx942", "gfx950"}

def is_skinny_default_shape(M, N, K, dtype, cu_num=None):
    if get_gfx() not in _SKINNY_GEMM_ARCHS:
        return False
    ...

The set mirrors the __HIP__MI350_MI300_MI250__ definition exactly, and follows the same pattern already used a few lines above for _OPUS_WS_ARCHS (aiter/tuned_gemm.py:77).

Impact

  • gfx90a / gfx942 / gfx950: no behaviour change — get_gfx() returns one of the allowlisted values, so the predicate result is identical to before.
  • Other archs: the skinny default is no longer selected; those shapes fall through to the existing non-skinny dispatch. This replaces a runtime abort with a working kernel.
  • Tuned configs are unaffected — this only touches the untuned-shape default path.

Test

  • Functional fix, no perf claim: the change removes an abort on archs that never had a working skinny kernel, and is a no-op on the three archs that do.
  • Verified get_gfx() returns a bare arch string from GFX_MAP (aiter/jit/utils/chip_info.py:66-68), so the membership test matches the existing get_gfx() == "gfx950" style comparisons elsewhere in the repo.
  • ruff / black / isort clean.

The skinny GEMM kernels (wvSpltK / LLMM1 / wv_splitk_small_fp16_bf16) are
only compiled with a real body under __HIP__MI350_MI300_MI250__ in
csrc/kernels/custom_kernels.cu. On every other arch the assert(false) stub
is compiled instead, so whenever is_skinny_default_shape() picks skinny as
the default path the call aborts at runtime.

Restrict the skinny default to gfx90a / gfx942 / gfx950 so unsupported
archs (e.g. gfx1250) fall through to the regular GEMM dispatch.
@yichiche
yichiche requested a review from a team August 17, 2026 05:45
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4791 --add-label <label>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant