fix(SPEC-MTP): --speculative-config DROPPED every key it did not read, so probabilistic drafting ran GREEDY in silence (#1160) - #1175
Merged
Conversation
…, so probabilistic drafting ran GREEDY in silence (#1160) The parser read five keys and discarded the rest, with no unknown-key check anywhere in `src/vllm/config/speculative.cpp:14-131`. So `{"method":"dspark","model":"...","num_speculative_tokens":7, "draft_sample_method":"probabilistic"}` parsed cleanly, started a server that drafted greedily, logged nothing and exited 0, and a misspelled `"num_speculatve_tokens"` quietly took the resolved default instead of the value that was typed. Upstream gets this refusal for free. `SpeculativeConfig` carries `extra="forbid"` through its `@config` decorator (`vllm/config/speculative.py:81-83` @ the pinned oracle `555967922`), and the hand-written C++ parser reproduced the reads without the guard. The method value WAS validated and an unrecognised method refused (`speculative.cpp:43,48-54`), so the strictness already existed for one field and was absent for the object. This is not an inert flag. Draft sampling and verify are greedy here (`include/vllm/v1/worker/gpu/spec_decode/dspark/speculator.h:36-38`, `include/vllm/v1/spec_decode/rejection_sampler.h:53-57`), so a dropped `probabilistic` produced a DETERMINISTIC run when a sampled draft was requested. A deterministic run is adjudicable by the token-exact greedy gate while the configuration the user actually asked for is not, so a parity or benchmark number could be taken under a configuration nobody chose, and every downstream figure would inherit it. The object is now admitted key by key against the `SpeculativeConfig` field set at the pin (`vllm/config/speculative.py:85-283`), in three classes. The five honoured keys pass. `draft_sample_method` (:283) and `rejection_sample_method` (:216) pass at their upstream defaults `greedy` and `standard`, which are exactly what this engine implements, so a vLLM config that spells a default explicitly still runs, and any other value is refused with the missing part named and row `SPEC-ACCEPT-VARIANTS` cited. Every other name is refused, worded differently for a field vLLM declares than for a typo, because telling a user that `quantization` is "unknown" sends them hunting for a spelling mistake that is not there. The method check still runs first, so an unsupported method stays the error a user sees. Scope item 2 of the issue, probabilistic draft sampling itself, is NOT here. It stays owned by `SPEC-ACCEPT-VARIANTS` (`.agents/engine-matrix.md:191`, `INVENTORIED`). Landing the refusal first converts a silent wrong answer into a named one, which is the order the porting rules already require. Gated red-first. `tests/vllm/config/test_speculative_unknown_keys.cpp` failed 6 of 9 cases and 7 assertions before the change, every refusal case returning an empty message because the config parsed, and passes 9/9 with 63 assertions after. That file calls the parser directly, so it localizes a failure and proves nothing about reach. The reach proof is three black-box cases in `examples/CMakeLists.txt` that enter through the server binary on its default configuration, including a control asserting a valid config still gets PAST admission to the model load. With the production call site at `server_main.cpp:1115-1117` deleted in a scratch copy, those go RED 2 of 3 while the unit file stays fully green, which is the finding that made them necessary. The tree was restored byte-for-byte, sha256 `93a77f2774104496b28f8fa9755a1898e5cffabcc7eeb45a34bee9ccf359c9b1`. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
Brings the branch level with origin/main before the gate run, so both commit-range blocks of the preflight execute against a current base. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
origin/main advanced again during the gate run, which left the trailer and style gates SKIPPED because the branch was behind. Re-level so both commit-range blocks execute. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
origin/main advanced again with two more issue-index appends. Locally the union driver merges them cleanly, but GitHub does not apply that driver and reported the pull request CONFLICTING, so the branch has to carry the rows rather than leave them to the forge. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
FOLLOWING_AGENTS_PROTOCOL Brings the branch up to `origin/main` so the committed-range and trailer gates examine this tree rather than skipping. A skipped gate reports nothing about the tree, and exit status alone cannot carry that. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 18, 2026
…erlapped Two more commits arrived while this branch was in flight: the `--speculative-config` key drop (#1175) and the inert `--gpu-memory-utilization` budget (#1176). Neither touches the vt graph seam, the capture path, or the `ENG-CUDAGRAPH-DEDUP` row, and the keyed records they move are `.agents/issue-index.md` (append-only) and rows this branch never wrote, so the automatic merge was verified rather than trusted: the engine matrix still carries this row and its summary counts, and `check-agent-record.py` re-runs clean. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The parser read five keys and discarded the rest, with no unknown-key check anywhere in
src/vllm/config/speculative.cpp:14-131. So{"method":"dspark","model":"...","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}parsed cleanly, started a server that drafted greedily, logged nothing and exited 0, and a misspelled"num_speculatve_tokens"quietly took the resolved default instead of the value that was typed.Upstream gets this refusal for free.
SpeculativeConfigcarriesextra="forbid"through its@configdecorator (vllm/config/speculative.py:81-83@ the pinned oracle555967922), and the hand-written C++ parser reproduced the reads without the guard. The method value WAS validated and an unrecognised method refused (speculative.cpp:43,48-54), so the strictness already existed for one field and was absent for the object.This is not an inert flag. Draft sampling and verify are greedy here (
include/vllm/v1/worker/gpu/spec_decode/dspark/speculator.h:36-38,include/vllm/v1/spec_decode/rejection_sampler.h:53-57), so a droppedprobabilisticproduced a DETERMINISTIC run when a sampled draft was requested. A deterministic run is adjudicable by the token-exact greedy gate while the configuration the user actually asked for is not, so a parity or benchmark number could be taken under a configuration nobody chose, and every downstream figure would inherit it. That is the reason this is a bug rather than an ergonomics complaint.What changed
The object is admitted key by key against the
SpeculativeConfigfield set at the pin (vllm/config/speculative.py:85-283), in three classes.method,num_speculative_tokens,model,prompt_lookup_min,prompt_lookup_max. Parsed and used, exactly as before.draft_sample_method(:283) acceptsgreedy,rejection_sample_method(:216) acceptsstandard. Both are upstream's own defaults and both are exactly what this engine implements, so a vLLM config that spells a default explicitly still runs. Any other value is refused with the missing part named and rowSPEC-ACCEPT-VARIANTScited.quantizationormax_model_len, is reported as a real vLLM field this engine does not implement. Anything else is reported as unknown, with the accepted list. The two messages differ on purpose, because telling a user thatquantizationis "unknown" sends them hunting for a spelling mistake that is not there.Presence is what is judged, including an explicit
null, since a null on a key we do not implement still names a capability we do not have. The method check still runs first, so an unsupported method stays the error a user sees, and both spellings of that ordering are pinned by a test.Scope item 2 of the issue, probabilistic draft sampling itself, is NOT here. It stays owned by
SPEC-ACCEPT-VARIANTS(.agents/engine-matrix.md:191,INVENTORIED). Landing the refusal first converts a silent wrong answer into a named one, which is the order the porting rules already require.Evidence
RED first.
tests/vllm/config/test_speculative_unknown_keys.cppfailed 6 of 9 cases and 7 assertions, every refusal case returning an empty message because the config parsed:GREEN after, 9/9 cases and 63 assertions. Full CPU suite 510/510 passed, 0 failed, 2 pre-existing checkpoint-gated skips.
That unit file calls the parser directly, so it localizes a failure and proves nothing about reach. The reach proof is three black-box cases in
examples/CMakeLists.txtentering through the server binary on its default configuration, including a control asserting a valid config still gets PAST admission to the model load. With the production call site atserver_main.cpp:1115-1117deleted in a scratch copy, those go RED 2 of 3 while the unit file stays fully green, which is the finding that made them necessary. The tree was restored byte-for-byte, sha25693a77f2774104496b28f8fa9755a1898e5cffabcc7eeb45a34bee9ccf359c9b1.scripts/agent-preflight.sh: 79 ok, 0 FAIL, 0 SKIP,All gates green., exit 0, with both commit-range blocks executing againstorigin/main825a61e65. An earlier run showedtest_cpu_x86_llamacpp_floorred at loadavg 22 withexit=4(NO_QUIET_WINDOW) instead of2, which is the known load-dependent flake tracked by #618. It passes 10/10 on the quiet box and this change touches nothing it reads.Closes #1160
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]