fix(tuned_gemm): gate skinny GEMM default to archs with a real kernel - #4791
Open
yichiche wants to merge 1 commit into
Open
fix(tuned_gemm): gate skinny GEMM default to archs with a real kernel#4791yichiche wants to merge 1 commit into
yichiche wants to merge 1 commit into
Conversation
The skinny GEMM kernels (wvSpltK / LLMM1 / wv_splitk_small_fp16_bf16) are only compiled with a real body under __HIP__MI350_MI300_MI250__ in csrc/kernels/custom_kernels.cu. On every other arch the assert(false) stub is compiled instead, so whenever is_skinny_default_shape() picks skinny as the default path the call aborts at runtime. Restrict the skinny default to gfx90a / gfx942 / gfx950 so unsupported archs (e.g. gfx1250) fall through to the regular GEMM dispatch.
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
|
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
is_skinny_default_shape()could select the skinny GEMM path on architectures where those kernels are compiled asassert(false)stubs, aborting at runtime.gfx90a/gfx942/gfx950.gfx1250) now fall through to the regular GEMM dispatch instead of aborting.Motivation
get_GEMM_A16W16_config()falls back to a default config when a shape is not inbf16_tuned_gemm.csv. One of those fallbacks is the skinny path:The skinny GEMM kernels —
wvSpltK,LLMM1,wv_splitk_small_fp16_bf16— only have a real body inside#if defined(__HIP__MI350_MI300_MI250__)incsrc/kernels/custom_kernels.cu. That macro is defined only for__gfx90a__ || __gfx942__ || __gfx950__(custom_kernels.cu:28-30); every other arch compiles the#elsebranch, whose body isUNREACHABLE_CODE, i.e.assert(false);.is_skinny_default_shape()itself only looked atM/N/K/dtype/cu_num, never at the arch. So on an arch without the kernel body, any untuned skinny-shaped GEMM (smallM,N <= cu_num,K % 8 == 0) selects a stub and aborts. This is reachable ongfx1250with a bf16 model whose small-M shapes are not yet in the tuned config.Modifications
aiter/tuned_gemm.py: add an arch allowlist and check it first inis_skinny_default_shape().The set mirrors the
__HIP__MI350_MI300_MI250__definition exactly, and follows the same pattern already used a few lines above for_OPUS_WS_ARCHS(aiter/tuned_gemm.py:77).Impact
gfx90a/gfx942/gfx950: no behaviour change —get_gfx()returns one of the allowlisted values, so the predicate result is identical to before.Test
get_gfx()returns a bare arch string fromGFX_MAP(aiter/jit/utils/chip_info.py:66-68), so the membership test matches the existingget_gfx() == "gfx950"style comparisons elsewhere in the repo.ruff/black/isortclean.