fix(FIX-GPU-MEM-UTIL-INERT): --gpu-memory-utilization stops reporting success for a budget it discards (#1165)

[localai-bot]

The flag reached six files and changed nothing. ResolveNumBlocks fell through knob 1 and knob 2 to a bare return 256, so --gpu-memory-utilization 0.85 produced the same pool as passing nothing, and the engine said so nowhere. A user who typed it believed they had sized the KV pool.

The flag is NOT refused, and the resolution is unchanged. .agents/roadmap_v1.md:71 records the intent that it keeps vLLM's exact name and fraction semantics so a published vllm serve line ports unchanged, and refusing it would break that and contradict the mirror rule. What changes is that the line which discards the value now says so, naming the 256-block fallback that resolved instead, the two knobs that do bind today, and the row and issue that own the real fix.

vllm.cpp: WARNING --gpu-memory-utilization 0.85 was accepted but did NOT size the KV cache.
vllm.cpp:   The profile run that turns a free-memory fraction into a block count is not
vllm.cpp:   implemented yet (ROAD-V1-MEM M3, https://github.com/mudler/vllm.cpp/issues/83).
vllm.cpp:   The pool fell back to 256 blocks. To size it today, pass
vllm.cpp:   --kv-cache-memory <bytes> for an absolute KV budget, or --num-blocks <n> for an
vllm.cpp:   exact block count.

Only a CHOSEN value warns. EngineParams::gpu_memory_utilization becomes std::optional<double>, mirroring the enable_prefix_caching tri-state in the same struct, because a plain double pre-filled with 0.92 cannot tell "the user typed 0.92" from "nobody touched it", and a line on every start is noise rather than a warning. The server's Args field becomes the same tri-state. examples/cli now leaves the ABI field at 0.0, the sentinel vllm.h already documents as unset, instead of overwriting it with 0.92 on every run.

Knobs 1 and 2 return before the notice, so sizing the pool with --num-blocks or --kv-cache-memory is silent. Under --kv-cache-memory that also mirrors vLLM, which ignores the fraction there (cache.py:189), so such a caller has nothing to be warned about.

This does not close #83

ROAD-V1-MEM M3 still owns the utilization path itself, it still needs a device profile run and an oracle-matched pool on dgx.casa, and nothing here stubs one. The spec states that boundary explicitly, because the obvious misreading of a landed warning is that the feature arrived.

The C ABI is unchanged

vllm_model_params_default() still pre-fills 0.92, so a C caller who never touched the field does get the notice: that ABI has no "flag not typed" state, and a struct carrying 0.92 into an engine that ignores it is the case the notice exists for. vllm.h now says so and names 0.0 as the opt-out.

Records

docs/USAGE.md documented neither sizing knob and called --num-blocks a 256 default when it defaults to 0. All three entries are corrected, and the GB10 unified-memory hazard lands there rather than in the notice, because the notice disappears exactly when that hazard becomes real.

.agents/feature-matrix.md:92 and docs/FEATURES.md:365 were checked and deliberately not edited: both already record the row as PARTIAL with M3 dgx-gated, neither states anything this change makes false, and editing the matrix would pull FEATURES.md in behind it with nothing new to say.

Evidence

Red first, on the merged tree at 469f3839, with the notice block removed and the tri-state kept so the test still compiled (BUILD_RC=0, so the red is not a compile failure wearing a pass):

test_loaded_engine_dense.cpp:710: FATAL ERROR: REQUIRE( logged.find(kInertNotice) != std::string::npos )
[doctest] test cases: 19 | 18 passed | 1 failed | 0 skipped
[doctest] assertions: 81 | 80 passed | 1 failed |

Restored, sha256sum -c OK, green after: 19/19 cases, 87/87 assertions.

Three mutations, each built clean and each restored byte-for-byte:

Mutation	Expected red	Result
Delete the ResolveNumBlocks call in MakeKVCacheResolved	the explicit case	RED 18/19, 81 assertions
Force the guard to if (true)	the unset case	RED 18/19, 87 assertions
Hoist the notice above the early returns	both precedence cases	RED 17/19, 87 assertions

The first is the reachability mutation. The test enters through the loader rather than the resolver, which is private, so deleting the production call site is what makes it fail.

scripts/agent-preflight.sh --fail-on-skip at c5b198e5: ok=79, FAIL=0, SKIP=0, --=0, verdict All gates green., exit 0. Counted after stripping ANSI and slicing at the banner, because .agents/NOW.md follows it and contains those words. Per block: role 1, record gates 27, mutation suites 46, committed range 3, commit trailers 2.

No checker, CI workflow, or mutation-suite file is touched, and no budget is raised.

test_cpu_x86_llamacpp_floor failed twice during this work and passed three times on an unmodified tree at loadavg 5 to 33, with failures taking 18 s and 170 s against 3 s for passes. That is the NO_QUIET_WINDOW retry path of #618. Its harness and script are byte-identical to origin/main, so it is the box, not this diff.

CPU only. No GPU lease was taken and rc was not called.

Closes #1165.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]