[AMD][AgentX] GLM-5.2 FP4 MI355X sglang-disagg HiCache MTP / [AMD][AgentX] MI355X 上 GLM-5.2 FP4 分离式 HiCache MTP

[charxwu]

Summary

• Add glm5.2-fp4-mi355x-sglang-disagg-agentic-hicache to amd-master.yaml: GLM-5.2 MXFP4 on MI355X, 1P1D TP8 disaggregated SGLang (sglang-disagg, MoRI P2P), agentic-coding, conc-list [2, 4, 8, 16, 32], DRAM HiCache KV offload, built-in MTP (spec-decoding: mtp, DECODE_MTP_SIZE=2).
• Add launcher benchmarks/multi_node/agentic/glm5.2_fp4_mi355x_sglang-disagg.sh (CI-style sibling of the DSv4 MI355X disagg recipe): env-driven submit via submit.sh, HiCache defaults, MTP gate + simulated acceptance for non-eval throughput.
• Extend AMD multi-node utils for this path:
  • models.yaml GLM-5.2-MXFP4-AgentX: EAGLE mtp_flags (--speculative-algorithm EAGLE --speculative-eagle-topk 1); decode mem_fraction_static 0.85.
  • env.sh: ionic MSN-safe MoRI IO QP tuning scoped to GLM-5.2-MXFP4 + DISAGG=true.
  • job.slurm / recipe: default model root /it-share/data (mia1*); forward IBDEVICES / MORI_RDMA_TC into hicache_mc.env.
• Pin validated HiCache L2 tunables in master-config prefill additional-settings: HICACHE_RATIO=1.5, write_through_selective, page_first_direct, direct IO, page_size=1.
• MTP throughput path uses synthetic acceptance from golden_al_distribution/glm5.2_mtp.yaml (thinking_on, num_speculative_tokens=2 → AL 2.50: SGLANG_SIMULATE_ACC_LEN=2.50); EVAL_ONLY=true runs use real draft-model MTP.
• Append perf-changelog.yaml entry for glm5.2-fp4-mi355x-sglang-disagg-agentic-hicache.

Image: lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260729
Model: amd/GLM-5.2-MXFP4
KV offload: SGLang HiCache (DRAM, kv-offload-backend: hicache, dram-utilization: 0.80)
Topology: 1P1D TP8 / EP1, MoRI disagg transfer; MTP size 2 (EAGLE)

Test plan

• Label full-sweep-fail-fast (or sweep-enabled for trim-conc); green sweep on this PR
• Confirm conc ladder [2, 4, 8, 16, 32] with HiCache + MTP; eval path uses real MTP when enabled
• /reuse-sweep-run after full green with evals

Related

• Golden AL table: golden_al_distribution/glm5.2_mtp.yaml (from [Klaud Cold] feat(golden-al): add GLM-5.2 MTP golden AL distribution / 新增 GLM-5.2 MTP 黄金接受长度(AL)分布 #2460)
• PR: [AMD][AgentX] GLM-5.2 FP4 MI355X sglang-disagg HiCache MTP / [AMD][AgentX] MI355X 上 GLM-5.2 FP4 分离式 HiCache MTP #2679

---

中文说明

• 在 amd-master.yaml 新增 glm5.2-fp4-mi355x-sglang-disagg-agentic-hicache：MI355X 上 GLM-5.2 MXFP4 的 1P1D TP8 分离式 SGLang（sglang-disagg，MoRI P2P），agentic-coding 场景，conc-list 为 [2, 4, 8, 16, 32]，DRAM HiCache KV 卸载，内置 MTP（spec-decoding: mtp，DECODE_MTP_SIZE=2）。
• 新增启动脚本 benchmarks/multi_node/agentic/glm5.2_fp4_mi355x_sglang-disagg.sh（对标 DSv4 MI355X 分离式 recipe 的 CI 入口）：由环境变量驱动、经 submit.sh 提交；含 HiCache 默认参数，以及 MTP 门控与非 eval 吞吐的模拟接受长度。
• 扩展 AMD 多节点工具链：
  • models.yaml 的 GLM-5.2-MXFP4-AgentX：补充 EAGLE mtp_flags（--speculative-algorithm EAGLE --speculative-eagle-topk 1）；decode mem_fraction_static 调至 0.85。
  • env.sh：仅在 GLM-5.2-MXFP4 且 DISAGG=true 时启用 ionic MSN-safe MoRI IO QP 调参。
  • job.slurm / recipe：默认模型根目录 /it-share/data（mia1*）；经 hicache_mc.env 向容器转发 IBDEVICES / MORI_RDMA_TC。
• 在 master config 的 prefill additional-settings 中固化经验证的 HiCache L2 参数：HICACHE_RATIO=1.5、write_through_selective、page_first_direct、direct IO、page_size=1。
• 吞吐路径使用 golden_al_distribution/glm5.2_mtp.yaml 的合成接受长度（thinking_on、num_speculative_tokens=2 → AL 2.50：SGLANG_SIMULATE_ACC_LEN=2.50）；EVAL_ONLY=true 时走真实 draft-model MTP。
• 在 perf-changelog.yaml 末尾追加 glm5.2-fp4-mi355x-sglang-disagg-agentic-hicache 条目。

镜像： lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260729
模型： amd/GLM-5.2-MXFP4
KV 卸载： SGLang HiCache（DRAM，kv-offload-backend: hicache，dram-utilization: 0.80）
拓扑： 1P1D TP8 / EP1，MoRI 分离式传输；MTP size 2（EAGLE）

测试计划

• 打上 full-sweep-fail-fast（或 sweep-enabled 做 trim-conc）；在本 PR 上跑出绿 sweep
• 确认 conc 阶梯 [2, 4, 8, 16, 32] 在 HiCache + MTP 下通过；启用 eval 时走真实 MTP
• 全绿且 evals 通过后评论 /reuse-sweep-run

相关

• Golden AL 表：golden_al_distribution/glm5.2_mtp.yaml（来自 [Klaud Cold] feat(golden-al): add GLM-5.2 MTP golden AL distribution / 新增 GLM-5.2 MTP 黄金接受长度(AL)分布 #2460）
• PR：[AMD][AgentX] GLM-5.2 FP4 MI355X sglang-disagg HiCache MTP / [AMD][AgentX] MI355X 上 GLM-5.2 FP4 分离式 HiCache MTP #2679

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档