Skip to content

fix(SPEC-MTP): --speculative-config DROPPED every key it did not read, so probabilistic drafting ran GREEDY in silence (#1160) - #1175

Merged
localai-bot merged 5 commits into
mainfrom
row/FIX-SPEC-CONFIG-UNKNOWN-KEYS
Aug 18, 2026
Merged

fix(SPEC-MTP): --speculative-config DROPPED every key it did not read, so probabilistic drafting ran GREEDY in silence (#1160)#1175
localai-bot merged 5 commits into
mainfrom
row/FIX-SPEC-CONFIG-UNKNOWN-KEYS

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

The parser read five keys and discarded the rest, with no unknown-key check anywhere in src/vllm/config/speculative.cpp:14-131. So {"method":"dspark","model":"...","num_speculative_tokens":7,"draft_sample_method":"probabilistic"} parsed cleanly, started a server that drafted greedily, logged nothing and exited 0, and a misspelled "num_speculatve_tokens" quietly took the resolved default instead of the value that was typed.

Upstream gets this refusal for free. SpeculativeConfig carries extra="forbid" through its @config decorator (vllm/config/speculative.py:81-83 @ the pinned oracle 555967922), and the hand-written C++ parser reproduced the reads without the guard. The method value WAS validated and an unrecognised method refused (speculative.cpp:43,48-54), so the strictness already existed for one field and was absent for the object.

This is not an inert flag. Draft sampling and verify are greedy here (include/vllm/v1/worker/gpu/spec_decode/dspark/speculator.h:36-38, include/vllm/v1/spec_decode/rejection_sampler.h:53-57), so a dropped probabilistic produced a DETERMINISTIC run when a sampled draft was requested. A deterministic run is adjudicable by the token-exact greedy gate while the configuration the user actually asked for is not, so a parity or benchmark number could be taken under a configuration nobody chose, and every downstream figure would inherit it. That is the reason this is a bug rather than an ergonomics complaint.

What changed

The object is admitted key by key against the SpeculativeConfig field set at the pin (vllm/config/speculative.py:85-283), in three classes.

  • Honoured. method, num_speculative_tokens, model, prompt_lookup_min, prompt_lookup_max. Parsed and used, exactly as before.
  • Accepted at the implemented value. draft_sample_method (:283) accepts greedy, rejection_sample_method (:216) accepts standard. Both are upstream's own defaults and both are exactly what this engine implements, so a vLLM config that spells a default explicitly still runs. Any other value is refused with the missing part named and row SPEC-ACCEPT-VARIANTS cited.
  • Refused. Every other name. A field vLLM declares, such as quantization or max_model_len, is reported as a real vLLM field this engine does not implement. Anything else is reported as unknown, with the accepted list. The two messages differ on purpose, because telling a user that quantization is "unknown" sends them hunting for a spelling mistake that is not there.

Presence is what is judged, including an explicit null, since a null on a key we do not implement still names a capability we do not have. The method check still runs first, so an unsupported method stays the error a user sees, and both spellings of that ordering are pinned by a test.

Scope item 2 of the issue, probabilistic draft sampling itself, is NOT here. It stays owned by SPEC-ACCEPT-VARIANTS (.agents/engine-matrix.md:191, INVENTORIED). Landing the refusal first converts a silent wrong answer into a named one, which is the order the porting rules already require.

Evidence

RED first. tests/vllm/config/test_speculative_unknown_keys.cpp failed 6 of 9 cases and 7 assertions, every refusal case returning an empty message because the config parsed:

[doctest] test cases:  9 |  3 passed | 6 failed | 0 skipped
[doctest] assertions: 38 | 31 passed | 7 failed |
[doctest] Status: FAILURE!

GREEN after, 9/9 cases and 63 assertions. Full CPU suite 510/510 passed, 0 failed, 2 pre-existing checkpoint-gated skips.

That unit file calls the parser directly, so it localizes a failure and proves nothing about reach. The reach proof is three black-box cases in examples/CMakeLists.txt entering through the server binary on its default configuration, including a control asserting a valid config still gets PAST admission to the model load. With the production call site at server_main.cpp:1115-1117 deleted in a scratch copy, those go RED 2 of 3 while the unit file stays fully green, which is the finding that made them necessary. The tree was restored byte-for-byte, sha256 93a77f2774104496b28f8fa9755a1898e5cffabcc7eeb45a34bee9ccf359c9b1.

scripts/agent-preflight.sh: 79 ok, 0 FAIL, 0 SKIP, All gates green., exit 0, with both commit-range blocks executing against origin/main 825a61e65. An earlier run showed test_cpu_x86_llamacpp_floor red at loadavg 22 with exit=4 (NO_QUIET_WINDOW) instead of 2, which is the known load-dependent flake tracked by #618. It passes 10/10 on the quiet box and this change touches nothing it reads.

Closes #1160

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

mudler added 5 commits August 18, 2026 00:44
…, so probabilistic drafting ran GREEDY in silence (#1160)

The parser read five keys and discarded the rest, with no unknown-key check
anywhere in `src/vllm/config/speculative.cpp:14-131`. So
`{"method":"dspark","model":"...","num_speculative_tokens":7,
"draft_sample_method":"probabilistic"}` parsed cleanly, started a server that
drafted greedily, logged nothing and exited 0, and a misspelled
`"num_speculatve_tokens"` quietly took the resolved default instead of the value
that was typed.

Upstream gets this refusal for free. `SpeculativeConfig` carries
`extra="forbid"` through its `@config` decorator
(`vllm/config/speculative.py:81-83` @ the pinned oracle `555967922`), and the
hand-written C++ parser reproduced the reads without the guard. The method value
WAS validated and an unrecognised method refused (`speculative.cpp:43,48-54`), so
the strictness already existed for one field and was absent for the object.

This is not an inert flag. Draft sampling and verify are greedy here
(`include/vllm/v1/worker/gpu/spec_decode/dspark/speculator.h:36-38`,
`include/vllm/v1/spec_decode/rejection_sampler.h:53-57`), so a dropped
`probabilistic` produced a DETERMINISTIC run when a sampled draft was requested.
A deterministic run is adjudicable by the token-exact greedy gate while the
configuration the user actually asked for is not, so a parity or benchmark number
could be taken under a configuration nobody chose, and every downstream figure
would inherit it.

The object is now admitted key by key against the `SpeculativeConfig` field set
at the pin (`vllm/config/speculative.py:85-283`), in three classes. The five
honoured keys pass. `draft_sample_method` (:283) and `rejection_sample_method`
(:216) pass at their upstream defaults `greedy` and `standard`, which are exactly
what this engine implements, so a vLLM config that spells a default explicitly
still runs, and any other value is refused with the missing part named and row
`SPEC-ACCEPT-VARIANTS` cited. Every other name is refused, worded differently for
a field vLLM declares than for a typo, because telling a user that `quantization`
is "unknown" sends them hunting for a spelling mistake that is not there. The
method check still runs first, so an unsupported method stays the error a user
sees.

Scope item 2 of the issue, probabilistic draft sampling itself, is NOT here. It
stays owned by `SPEC-ACCEPT-VARIANTS` (`.agents/engine-matrix.md:191`,
`INVENTORIED`). Landing the refusal first converts a silent wrong answer into a
named one, which is the order the porting rules already require.

Gated red-first. `tests/vllm/config/test_speculative_unknown_keys.cpp` failed 6
of 9 cases and 7 assertions before the change, every refusal case returning an
empty message because the config parsed, and passes 9/9 with 63 assertions after.
That file calls the parser directly, so it localizes a failure and proves nothing
about reach. The reach proof is three black-box cases in `examples/CMakeLists.txt`
that enter through the server binary on its default configuration, including a
control asserting a valid config still gets PAST admission to the model load.
With the production call site at `server_main.cpp:1115-1117` deleted in a scratch
copy, those go RED 2 of 3 while the unit file stays fully green, which is the
finding that made them necessary. The tree was restored byte-for-byte, sha256
`93a77f2774104496b28f8fa9755a1898e5cffabcc7eeb45a34bee9ccf359c9b1`.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
Brings the branch level with origin/main before the gate run, so both
commit-range blocks of the preflight execute against a current base.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
origin/main advanced again during the gate run, which left the trailer and
style gates SKIPPED because the branch was behind. Re-level so both
commit-range blocks execute.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
origin/main advanced again with two more issue-index appends. Locally the
union driver merges them cleanly, but GitHub does not apply that driver and
reported the pull request CONFLICTING, so the branch has to carry the rows
rather than leave them to the forge.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
FOLLOWING_AGENTS_PROTOCOL

Brings the branch up to `origin/main` so the committed-range and trailer gates
examine this tree rather than skipping. A skipped gate reports nothing about the
tree, and exit status alone cannot carry that.

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
@localai-bot
localai-bot merged commit 84b9660 into main Aug 18, 2026
@localai-bot
localai-bot deleted the row/FIX-SPEC-CONFIG-UNKNOWN-KEYS branch August 18, 2026 01:14
localai-bot pushed a commit that referenced this pull request Aug 18, 2026
…erlapped

Two more commits arrived while this branch was in flight: the
`--speculative-config` key drop (#1175) and the inert `--gpu-memory-utilization`
budget (#1176). Neither touches the vt graph seam, the capture path, or the
`ENG-CUDAGRAPH-DEDUP` row, and the keyed records they move are
`.agents/issue-index.md` (append-only) and rows this branch never wrote, so the
automatic merge was verified rather than trusted: the engine matrix still carries
this row and its summary counts, and `check-agent-record.py` re-runs clean.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--speculative-config silently drops unknown keys, so draft_sample_method=probabilistic runs GREEDY with no warning

2 participants