Skip to content

Support vLLM 0.30.0 with CUDA graph validation - #24

Open
zhaoyboo wants to merge 2 commits into
ProjectDMX:mainfrom
zhaoyboo:vllm-0.30-support
Open

zhaoyboo wants to merge 2 commits into
ProjectDMX:mainfrom
zhaoyboo:vllm-0.30-support

Conversation

@zhaoyboo

@zhaoyboo zhaoyboo commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Scope

Port the external integration from official vLLM 0.29.0 to 0.30.0
(ced6857afa0ea7b2e3f0846a62e1394e90f15607), based on integration main
c17b9f2e69b4f330a617896f759313eb331cb6dd.
Qualified core: DMI 1.2.0 / API v1 at
c222f18c4db55f6f2d7136b1d6608c0b540b273a, rebuilt against torch 2.13 / CUDA 13.

Production changes are limited to version metadata/router guard and the V2
dispatch contract: forward num_ubatches, retain DBO rejection, and reject the
new KV-sharing fast-prefill layout before device initialization.
No model arithmetic, residual-hook rewrite, V1 adapter change, core change,
or performance optimization.
This PR does not update the DMI core submodule
pointer or create a release tag.

Validation

Bounded GPU cell: official wheel, Qwen3-0.6B revision
c1899de289a04d12100db370d81485cdf75e47ca, BF16, RTX 4090, TP1/PP1,
offline generation, three ragged requests, eight output tokens each,
ClickHouse 25.12.2.54. Stock and monitored run in separate processes.

  • V1 and V2 eager, V1 and V2 CUDA graph, V2 AOT cache reload: exact public
    outputs and all eight raw-logit tensors, with 744 exact-once stored rows per cell.
  • Graph cells retain combo kernels and verify actual PIECEWISE prefill + FULL
    decode
    , rejecting eager fallback. Reload is checked in both logs and uses
    fresh request IDs.
  • V1 eager residual oracle: 1,416 stored rows, including 1,368 residual rows
    checked against independent pre-norm snapshots. This is not a graph residual
    value oracle.
  • CPU: 695 passed, 4 skipped, 11 deselected. Three process-global router skips
    pass individually; one skip requires a newer core configuration API.
  • 865 interface occurrences / 555 mapped groups; no static audit errors.
    All 21 registered model targets import with the real native extension.
    Wheel/sdist build and package metadata checks pass.

Reproduce:

CUDA_VISIBLE_DEVICES=1 DMI_V030_PYTHON="$PWD/.venv/bin/python" \
  DMI_V030_KV_CACHE_MEMORY_BYTES=1073741824 \
  DMI_V030_CLEAR_LD_LIBRARY_PATH=1 bash tests/run_v030_smoke.sh

Clearing the library path is a host-specific opt-in, not a general requirement.
The harness requires cached model weights, a matching native build and ClickHouse.

Important numerical qualification

Exact graph comparison uses identical explicit compiler controls in both
processes
: deterministic=True, combo_kernels=True,
benchmark_combo_kernel=False. The integration does not override production
compiler defaults. CUDA graphs/compilation remain enabled; tolerances are not relaxed.

The original uncontrolled-autotuning failure is retained, not relabeled a pass.
Stock and monitored had byte-identical Q/K kernel source but different upstream
XBLOCK choices (8 vs 1). A fresh stock-only control with the same recorded
kernel tuning choice reproduced the entire original monitored output and all
eight logits bitwise, without DMI. This attributes the observed V2 discrepancy
to upstream numerical kernel selection; it does not promise universal bitwise
stability under independent production autotuning.

See docs/v030-port.md, docs/evidence/v030-autotune-control.json and
docs/evidence/v030-graph-qualification.json for exact configuration, limits,
artifact hashes, historical failures and final receipts.

Not qualified by this PR

Other checkpoints/models, TP/PP/DP/EP beyond 1, quantization, multimodal input,
speculative decoding, serving/streaming/cancellation and broad performance claims.
Import coverage is not model-wide GPU qualification. Existing unsupported-mode
guards remain in effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant