perf(gb300): Add more AgentX vLLM MTP aggregate and P/D recipes / 增加更多 GB300 AgentX vLLM MTP 聚合与 P/D 配方 - #2665
perf(gb300): Add more AgentX vLLM MTP aggregate and P/D recipes / 增加更多 GB300 AgentX vLLM MTP 聚合与 P/D 配方#2665ivanium wants to merge 25 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
185f2e3 to
ad2c1a8
Compare
ivanium
changed the title
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline finding on the checked-in synthetic MTP config, I also checked the new 1P3D GB300 recipe (disagg-gb300-1p3d-dep4-tp8-c3-mtp-agentic.yaml) for missing multi-node NCCL/UCX env vars on the cross-node TP8 decode workers — this was examined and ruled out.
Extended reasoning...
Bugs were found in this run and are already posted as inline comments; this note only records an additional item that was examined and ruled out, not a full re-review.
be4e8ef to
023bf49
Compare
ivanium
added
agentx
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32226635322 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32228682229 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32228916968 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32231829727 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32239067095 |
65b0ed7 to
4c06b82
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32240961682 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32244949807 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32244949807 |
刷新 GB300 AgentX vLLM MTP P/D 配方。
新增 GB300 AgentX 并发度 512 的 P/D 测试点。
恢复 GB300 AgentX 聚合 TP4/TP8 扫描，并在高并发点启用 vLLM Simple CPU KV 卸载。
将全部 GB300 AgentX 配方回退到 426e59f 镜像，禁用前缀缓存保留间隔，并对齐聚合 TP 的低延迟归约配置。
Restore the GB300 DeepSeek-V4-Pro aggregate TP4 and TP8 recipes to 0.94 GPU memory utilization. The 0.92 setting belongs to the separate low-latency 1P3D decode topology. 将 GB300 DeepSeek-V4-Pro 聚合式 TP4 和 TP8 配方的 GPU 显存利用率恢复为 0.94。0.92 设置仅适用于独立的低延迟 1P3D 解码拓扑。
Remove checked-in synthetic acceptance fields from the four GB300 aggregate recipes. Throughput jobs inject acceptance length 2.49 from nvidia-master.yaml, while eval-only runs retain real target-model verification. 从四个 GB300 聚合式配方中移除检入的合成接受率字段。吞吐任务由 nvidia-master.yaml 注入 2.49 的接受长度，而仅评测任务保留真实目标模型验证。
为聚合 TP4/TP8 配方启用 Mooncake Store，并调整 Simple CPU Offload 调度容量。
将聚合 Simple CPU Offload 的主机 DRAM 预算提高到 80%。
将 TP4 并发 6、8 和 TP8 并发 12 调整为无 CPU 卸载运行。
将聚合高并发卸载路径从 vLLM Simple CPU Offload 更正为 Mooncake Store。
在两个聚合式 Mooncake Store 配置中固定 PYTHONHASHSEED，以确保跨进程和节点的前缀块键保持一致。
将聚合式 TP4 和 TP8 扫描上限统一为并发 16，并添加经过当前分支运行时配置适配的 1P1D DEP8/DEP32 并发 388 P/D 点。
将 c388 DEP32 解码 worker 的 Mooncake 接收线程数从 20 降至 4，并保持 DEP8 预填充 worker 为 20。
将 GB300 AgentX 的 1P1D DEP8/DEP32 数据点从并发 388 调整为并发 384，并同步配方名称、路径和 JIT 缓存标识。
将 GB300 AgentX 的 1P1D DEP8/DEP32 数据点调整到并发 448，并将解码侧 max-num-seqs 提高到 8、max-num-batched-tokens 提高到 32。
添加 GB300 AgentX DEP8 预填充和 DEP32 解码的 c256 配置点，并复用 c448 运行时调优。
保留现有聚合与解聚配置不变，仅新增 TP4 并发点、1P4D c4 以及 DEP16/DEP32 配置，并将性能变更日志标记为 append-only。
2454f22 to
2a8bf7b
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32337992643 |
ivanium
changed the title
2 participants
Summary / 摘要
English
中文
Validation / 验证