Skip to content

[module_mla_reduce] detorch reduce.cu + delete dead no_redundant Python port - #4796

Open
amd-ruitang3 wants to merge 1 commit into
ROCm:mainfrom
amd-ruitang3:module_mla_reduce_refactor
Open

[module_mla_reduce] detorch reduce.cu + delete dead no_redundant Python port#4796
amd-ruitang3 wants to merge 1 commit into
ROCm:mainfrom
amd-ruitang3:module_mla_reduce_refactor

Conversation

@amd-ruitang3

@amd-ruitang3 amd-ruitang3 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Clean-rebuild wall for module_mla_reduce (2 TUs): ~243s (torch-included) → ~222–231s de-torched (~5–9%; device codegen of the 1269-line opus/CK kernel dominates, so torch-header removal is a modest slice)

1. csrc/kernels/mla/reduce.cu — remove torch dependency

The Python side (develop=True), the C++ signature (mla_reduce.h uses aiter_tensor_t) and the pybind (rocm_ops.hpp is torch-free) were already de-torched in #4729; only reduce.cu still pulled torch. Fixed the two remaining torch deps:

  • TORCH_WARN (torch macro, host-side occupancy warning) → fprintf(stderr, ...). Same trigger frequency as before (the original used TORCH_WARN, not _ONCE), and it only fires in the degraded high-splits regime — no kernel/GPU perf impact.
  • std::optional in the mla_reduce_v1 signature was satisfied only transitively by the torch headers; after dropping <ATen/hip/HIPContext.h> / <ATen/hip/impl/HIPGuardImplMasqueradingAsCUDA.h> / <torch/python.h> it needs an explicit #include <optional> (+ <cstdio> for the fprintf).

2. aiter/ops/attention.py — delete dead code

get_mla_metadata_v1_no_redundant was reimplemented as a pure-Python port in #4729 but has zero in-repo callers and no tests — dead code. Removed it (and its now-unused import heapq).

Why it matters (latent build break)

mla_reduce_v1 is @compile_ops(develop=True), whose production build sets torch_exclude=True — this strips pytorch's -isystem .../torch/include from the compile. The original reduce.cu fails under that path:

fatal error: 'ATen/hip/HIPContext.h' file not found

So this isn't only a build-time win — the de-torched TU is what makes the module actually buildable in the torch-excluded develop path.

Verification (in-container, gfx942 / MI308X)

  • op_tests/test_mla_reduce.pyALL PASSED (all splits / head / dv combos)
  • black + ruff check — clean
  • Clean-rebuild wall for module_mla_reduce (2 TUs): ~243s (torch-included) → ~222–231s de-torched (~5–9%; device codegen of the 1269-line opus/CK kernel dominates, so torch-header removal is a modest slice)

🤖 Generated with Claude Code

@amd-ruitang3
amd-ruitang3 requested a review from a team August 17, 2026 07:50
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4796 --add-label <label>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant