Perf tuning for GB300 agentx Kimi K3 - #2669
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Extend the GB300 AgentX aggregate sweep with concurrency 2, 8, 16, 24, 32, 40 and 56, alongside the existing 1, 4, 48, 70 and 74 points. Each concurrency gets its own recipe: max-num-seqs is sized at 2x concurrency and the cudagraph ladder is built from the draft length, which steps down with concurrency -- DSpark K=7 at golden AL 3.84 up to 8, K=3 at 3.00 for 16, no drafting above. gpu-memory-utilization is 0.92 and derates to 0.90 from concurrency 56 up. The Mooncake DRAM tier stays on the GB300 constraint, a 150 GB per-rank segment at dram-utilization 0.1664. The changelog entry lists every Kimi-K3 GB300 key, aggregate and disaggregated, so the sweep re-measures all 14 points in one run rather than mixing fresh results with older ones. Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ff7bdee to
8511297
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32218025306 |
1 participant
No description provided.