feat(config): add GLM-5.2 NVFP4 B200 Dynamo-SGLang AgentX recipes / 新增 GLM-5.2-NVFP4 B200 Dynamo-SGLang AgentX 配方 - #2673
feat(config): add GLM-5.2 NVFP4 B200 Dynamo-SGLang AgentX recipes / 新增 GLM-5.2-NVFP4 B200 Dynamo-SGLang AgentX 配方#2673Ankur-singh wants to merge 5 commits into
Conversation
新增 GLM-5.2-NVFP4 B200 Dynamo-SGLang AgentX 聚合式与分离式配方：覆盖并发度 2/4/8/12 的聚合 TP8，以及并发度 113 的 1P2D 和 1P3D 分离式配置，使用 EAGLE MTP、HiCache DRAM 卸载和 NIXL KV 传输，吞吐任务注入 golden synthetic acceptance。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
在性能变更日志中补充 PR #2673 链接。
将 GLM-5.2 B200 AgentX 聚合式与分离式配置改由 Nscale 集群运行：注册新的 cluster:b200-nscale 精确机群标签，扩展 launch_b200-nscale-slurm.sh 支持该模型/框架组合并对本地 NVMe 模型路径跳过 srtctl preflight，同时撤销此前对 launch_b200-dgxc.sh 的改动。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32224799614 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32225293298 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32225293298 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32225293298 |
sbatch_directives.cpus-per-task was ported verbatim from NVIDIA/srt-slurm#314 (a DGXC-tuned value) and does not match the b200-nscale cluster's registered per-node CPU count, causing every job routed there to fail at submission with 'sbatch: error: CPU count per node can not be satisfied'. All three recipes already request the whole node via use_exclusive_sbatch_directive: true, so the explicit cpus-per-task constraint is redundant on top of --exclusive and safe to drop entirely (precedent: sglang/glm5.2/agentic/disagg-h200-2p2d-*.yaml already omits it).
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32311694482 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32311923080 |
1 participant
Description
Adds Dynamo-SGLang AgentX coverage for GLM-5.2-NVFP4 on B200: one aggregated
TP8 config and one disaggregated config carrying two topologies (1P2D, 1P3D).
glm5.2-fp4-b200-dynamo-sglang-agentic-agg: single TP8 worker doing bothprefill and decode, swept at concurrency 2, 4, 8, and 12.
glm5.2-fp4-b200-dynamo-sglang-agentic-disagg: EP8/DP-attention prefillagainst either 2 or 3 TP8 decode workers (1P2D / 1P3D), both at concurrency
113, using NIXL for KV transfer between prefill and decode.
acceptance to the golden values in
golden_al_distribution/glm5.2_mtp.yaml(2.99 for the aggregate 3-step config, 2.5 for the disaggregated 2-step
decode config) for throughput runs only —
EVAL_ONLYruns keep realtarget-model verification.
benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b200-fp4/agentic/and copied into the srt-slurm checkout by a new
launch_b200-dgxc.shbranch, pinned to srt-slurm
v1.0.53(
217f94387abeddfed7149a71955dc523e07cd765).SRT_SLURM_MODEL_PREFIXfor this model/precision fromglm5.2-fp4toglm-5.2-fp4so it matches themodel.paththe recipesdeclare — this model-prefix alias was previously unused by any multi-node
config, so the fix has no effect on existing entries.
中文说明
为 B200 上的 GLM-5.2-NVFP4 新增 Dynamo-SGLang AgentX 覆盖:一个聚合式 TP8 配置,
以及一个包含两种拓扑(1P2D、1P3D)的分离式配置。
glm5.2-fp4-b200-dynamo-sglang-agentic-agg:单个 TP8 worker 同时承担 prefill和 decode,在并发度 2、4、8、12 下扫描。
glm5.2-fp4-b200-dynamo-sglang-agentic-disagg:EP8/DP-attention 的 prefill分别对接 2 个或 3 个 TP8 decode worker(1P2D / 1P3D),均在并发度 113 下运行,
prefill 与 decode 之间使用 NIXL 传输 KV。
acceptance 固定为
golden_al_distribution/glm5.2_mtp.yaml中的 golden 值(聚合3-step 配置为 2.99,分离式 2-step decode 配置为 2.5);
EVAL_ONLY运行则保留真实目标模型验证。
benchmarks/multi_node/srt-slurm-recipes/sglang/glm5.2/b200-fp4/agentic/,由新增的
launch_b200-dgxc.sh分支复制进 srt-slurm 检出目录,固定至 srt-slurmv1.0.53(217f94387abeddfed7149a71955dc523e07cd765)。SRT_SLURM_MODEL_PREFIX从glm5.2-fp4修正为glm-5.2-fp4,使其与配方声明的model.path一致——此模型前缀别名此前未被任何多机配置使用,因此该修正不影响现有条目。
The serving image is the upstream SGLang image
lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642. TRT-LLM is only used byselected SGLang kernel backends in this recipe.
服务镜像为上游 SGLang 镜像
lmsysorg/sglang:nightly-dev-cu13-20260805-211ee642。本配方中的 TRT-LLM 仅用于部分 SGLang kernel backend。
Related Issue
N/A
Type of Change
Validation
process_changelog.pyagainst this diff generates six throughput jobs(four aggregate concurrency points, two disaggregated topologies) and two
AgentX eval jobs (one per disaggregated topology).
generate_sweep_configs.py test-configresolves both new config keys withno errors.
YAML parsing,
bash -non the launch script, andgit diff --checkallpass.
Golden synthetic-acceptance lengths (2.99 aggregate / 2.5 disaggregated)
match
golden_al_distribution/glm5.2_mtp.yaml.Synthetic acceptance is injected through the shared
inject_synthetic_acceptance.pylauncher helper, which is a no-op wheneverEVAL_ONLYis set, so eval runs keep real target-model verification.针对本次 diff 运行
process_changelog.py,生成六个吞吐任务(四个聚合并发点、两个分离式拓扑)和两个 AgentX eval 任务(每个分离式拓扑一个)。
generate_sweep_configs.py test-config可正确解析两个新增 config key,无报错。YAML 解析、launch 脚本的
bash -n,以及git diff --check均通过。Golden synthetic acceptance 长度(聚合 2.99 / 分离式 2.5)与
golden_al_distribution/glm5.2_mtp.yaml一致。Synthetic acceptance 通过共享的
inject_synthetic_acceptance.pylauncherhelper 注入,在设置
EVAL_ONLY时为 no-op,因此 eval 运行仍保留真实目标模型验证。
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.