Skip to content

feat: support GLM-5.2#4737

Draft
CUHKSZzxy wants to merge 17 commits into
InternLM:mainfrom
CUHKSZzxy:feat/glm-5-2-support
Draft

feat: support GLM-5.2#4737
CUHKSZzxy wants to merge 17 commits into
InternLM:mainfrom
CUHKSZzxy:feat/glm-5-2-support

Conversation

@CUHKSZzxy

@CUHKSZzxy CUHKSZzxy commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Add GLM-5.2 PyTorch serving support and improve FP8 decode efficiency and stability. The DSA optimizations are shared with DeepSeek-V3.2 where applicable.

Changes

  • Add GLM-5.2 model/config integration, DSA indexing and cache handling, tool-call parsing, expert/indexer replay, and MoE-only FP8 support.
  • Replace the memory-heavy bitonic sparse top-k path with radix top-k, avoiding a score-sized temporary index buffer.
  • Fuse DSA indexer preparation: remove redundant Hadamard transforms; fuse Q RoPE, FP8 quantization, and gate scaling; fuse K LayerNorm, RoPE, FP8 quantization, and cache writes; and merge K/head-gate projections.
  • Support both GLM interleaved RoPE and DeepSeek-V3.2 NeoX RoPE. LMDEPLOY_DISABLE_DSA_INDEXER_FUSION=1 preserves the original path.
  • Use native Transformers configs where available, with local fallback registration for DeepSeek-V3.2.

Validation

  • Pre-commit passed; 25 config/model tests and 8 CUDA indexer tests passed.
  • Real 5-layer FP8 smoke tests passed for GLM-5.2 and DeepSeek-V3.2.
  • Short batch-8 decode timelines reduced median cycle time by 3.41% for GLM-5.2 and 4.66% for DeepSeek-V3.2, removing 13 CUDA graph child kernels per DSA layer. These are profiling results, not full-model throughput claims.

Assistance

Assisted with Codex + GPT-5.6-Sol xHigh, reviewed manually

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant