[Common] Split grouped activation build

[harryzhou2000]

Description

Split five monolithic grouped-activation CUDA translation units into smaller
per-activation and per-direction translation units:

• GELU and QGELU forward, backward, and dbias
• ReLU and SReLU forward, backward, and dbias
• SiLU and SwiGLU forward and backward

The existing function bodies and exported symbols are moved without changing
kernel logic or public APIs. swiglu_grouped_dbias.cu remains unchanged because
it contains only one instantiation.

Build performance

The comparison used separate empty ccache directories with zero hits on the same
full-node GCP-NRT allocation (224 logical CPUs), container, venv, source base,
and submodules. MAX_JOBS and CMAKE_BUILD_PARALLEL_LEVEL were unset, so Ninja
used machine parallelism. Both builds used NVTE_BUILD_THREADS_PER_JOB=4,
NVTE_CUDA_ARCHS=100, and the PyTorch-only configuration. The split build ran
first, so any system cache warming favored the baseline.

Metric	Baseline	Split	Reduction	Speedup
Maximum translation unit	932.579 s	445.884 s	52.2%	2.09x
Common-library build	935.905 s	448.261 s	52.1%	2.09x
Full editable install	1,038.390 s	551.548 s	46.9%	1.88x

The split increases aggregate compiler work by 13.7% because template parsing
is repeated across more translation units. With machine parallelism, that work
moves off the critical path: the longest translation unit changes from
gelu_grouped.cu to the unrelated fused_topk_with_score_function.cu.

Validation

• Focused pre-commit checks: merge-conflict, large-file, EOF, trailing-whitespace,
  and clang-format checks passed
• git diff --check passed
• Confirmed all 14 moved functions remain exported
• Expanded grouped MXFP8 activation test matrix: 915 passed, 810 expected skips,
  0 failures

[greptile-apps]

Greptile Summary

The PR splits grouped activation CUDA implementations into smaller per-activation and per-direction translation units to improve parallel build performance while preserving existing entry points.

• Registers all new CUDA translation units in the architecture-specific source list.
• Applies the existing optional fast-math configuration to the split files.
• Moves GELU, QGELU, ReLU, SReLU, SiLU, and grouped SwiGLU definitions without changing their function bodies.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
transformer_engine/common/CMakeLists.txt	Registers each split grouped-activation translation unit in the common CUDA build and optional fast-math source set.
transformer_engine/common/activation/gelu_grouped.cu	Retains grouped GELU forward while moving backward and QGELU entry points into dedicated translation units.
transformer_engine/common/activation/relu_grouped.cu	Retains grouped ReLU forward while moving backward and SReLU entry points into dedicated translation units.
transformer_engine/common/activation/swiglu_grouped.cu	Retains grouped SiLU forward while moving backward and scaled SwiGLU entry points into dedicated translation units.
transformer_engine/common/activation/qgelu_grouped_dbias.cu	Provides the relocated grouped QGELU dbias-backward entry point without changing its dispatch behavior.
transformer_engine/common/activation/srelu_grouped_dbias.cu	Provides the relocated grouped SReLU dbias-backward entry point without changing its dispatch behavior.
transformer_engine/common/activation/scaled_clamped_swiglu_grouped.cu	Provides the relocated grouped scaled-clamped SwiGLU entry point with its existing parameter and dispatch contract.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Grouped activation API symbols] --> B[Per-activation CUDA translation units]
  B --> C[CMake architecture-specific source list]
  C --> D[Transformer Engine common shared library]
  E[Optional fast-math configuration] --> B

Loading

_{Reviews (2): Last reviewed commit: "Grouped scaled swiglu" | Re-trigger Greptile}

[ptrendx]

/te-ci