Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Port the external integration from official vLLM 0.29.0 to 0.30.0
(
ced6857afa0ea7b2e3f0846a62e1394e90f15607), based on integration mainc17b9f2e69b4f330a617896f759313eb331cb6dd.Qualified core: DMI 1.2.0 / API v1 at
c222f18c4db55f6f2d7136b1d6608c0b540b273a, rebuilt against torch 2.13 / CUDA 13.Production changes are limited to version metadata/router guard and the V2
dispatch contract: forward
num_ubatches, retain DBO rejection, and reject thenew KV-sharing fast-prefill layout before device initialization.
No model arithmetic, residual-hook rewrite, V1 adapter change, core change,
or performance optimization. This PR does not update the DMI core submodule
pointer or create a release tag.
Validation
Bounded GPU cell: official wheel, Qwen3-0.6B revision
c1899de289a04d12100db370d81485cdf75e47ca, BF16, RTX 4090, TP1/PP1,offline generation, three ragged requests, eight output tokens each,
ClickHouse 25.12.2.54. Stock and monitored run in separate processes.
outputs and all eight raw-logit tensors, with 744 exact-once stored rows per cell.
decode, rejecting eager fallback. Reload is checked in both logs and uses
fresh request IDs.
checked against independent pre-norm snapshots. This is not a graph residual
value oracle.
pass individually; one skip requires a newer core configuration API.
All 21 registered model targets import with the real native extension.
Wheel/sdist build and package metadata checks pass.
Reproduce:
CUDA_VISIBLE_DEVICES=1 DMI_V030_PYTHON="$PWD/.venv/bin/python" \ DMI_V030_KV_CACHE_MEMORY_BYTES=1073741824 \ DMI_V030_CLEAR_LD_LIBRARY_PATH=1 bash tests/run_v030_smoke.shClearing the library path is a host-specific opt-in, not a general requirement.
The harness requires cached model weights, a matching native build and ClickHouse.
Important numerical qualification
Exact graph comparison uses identical explicit compiler controls in both
processes:
deterministic=True,combo_kernels=True,benchmark_combo_kernel=False. The integration does not override productioncompiler defaults. CUDA graphs/compilation remain enabled; tolerances are not relaxed.
The original uncontrolled-autotuning failure is retained, not relabeled a pass.
Stock and monitored had byte-identical Q/K kernel source but different upstream
XBLOCK choices (8 vs 1). A fresh stock-only control with the same recorded
kernel tuning choice reproduced the entire original monitored output and all
eight logits bitwise, without DMI. This attributes the observed V2 discrepancy
to upstream numerical kernel selection; it does not promise universal bitwise
stability under independent production autotuning.
See
docs/v030-port.md,docs/evidence/v030-autotune-control.jsonanddocs/evidence/v030-graph-qualification.jsonfor exact configuration, limits,artifact hashes, historical failures and final receipts.
Not qualified by this PR
Other checkpoints/models, TP/PP/DP/EP beyond 1, quantization, multimodal input,
speculative decoding, serving/streaming/cancellation and broad performance claims.
Import coverage is not model-wide GPU qualification. Existing unsupported-mode
guards remain in effect.