feat(autotune): separate general and decode attention warmup - #1549

Merged

hiworldwzj merged 16 commits into

mainfrom

Sep 9, 2026

Commits on Sep 8, 2026

fix(autotune): use representative decode KV lengths and bucket run keys
hiworldwzj
committed
c287bbc
feat(autotune): tune GQA decode with representative KV lengths
hiworldwzj
committed
e9c6a31
fix(autotune): tune MTP decode stages with representative inputs
hiworldwzj
committed
af860ba
fix(autotune): tune int4 KV decode with representative lengths
hiworldwzj
committed
43b9649
fix(autotune): tune int8 KV decode with representative lengths
hiworldwzj
committed
1c7a77f
feat(autotune): tune MTP linear attention and allow skipping cached config warmup
hiworldwzj
committed
361fdcd

Commits on Sep 9, 2026

fix(autotune): reuse MTP stage2 intermediate inputs
hiworldwzj
committed
8d5b790
perf(autotune): avoid cloning decode stage outputs
hiworldwzj
committed
5cca61c
docs(autotune): identify stateful warmup exception
hiworldwzj
committed
b202a1a
test(autotune): use unique pytest module names
shihaobai
committed
2404154
fix(api): cap CUDA graph batch size
shihaobai
committed
a011d28
fix(autotune): use 32K default decode length
hiworldwzj
committed
00426f5