Upgrade llama.cpp from b10456 to b10679

[bernardladenthin]

Summary

Upgrades the pinned llama.cpp from b10456 to b10679 across the four pin sites
(llama/CMakeLists.txt GIT_TAG, LlamaCppVersion.LLAMA_CPP_VERSION, the README badge, CLAUDE.md),
walked in individually reviewed steps sized by git diff bytes. Upstream split, gated and renamed
several things across that window, so the bump carries the adaptations with it:

• getMetrics() rebuilt from two upstream tasks. b10408 (server: refactor + correctness fixes for metrics ggml-org/llama.cpp#26920) reduced
  server_task_result_metrics::to_json() to a bare slot array and b10519 (server: refactor sleep handling, allow access /metrics during sleep ggml-org/llama.cpp#27376)
  split the task in two. No signature moved, everything still compiled and linked, and the Java
  payload was quietly the wrong shape for hundreds of builds. server_metrics_to_json() merges
  SERVER_TASK_TYPE_METRICS and SERVER_TASK_TYPE_SLOT_GET back into the shape the Java API has
  always documented, converting upstream microseconds to the milliseconds the payload uses, and
  surfacing counters that existed only inside the Prometheus text.
• Two new local patches. 0010 casts vocab_type at upstream's GET /models emit site — since
  the common_json switch (common: add json.h abstraction ggml-org/llama.cpp#27511) an unscoped enum binds to the bool overload
  and serialises as true/false. 0011 makes common_peg_until_parser respect leniency on an
  invalid UTF-8 byte the way it already does on an incomplete one: a single malformed byte
  anywhere in a model's output was turning a finished generation into an HTTP 500. Both are
  upstream-submittable, and both were re-checked at b10679 against the pristine upstream file —
  upstream has fixed neither, so both stay.
• RouterClient API-key support (server : make models endpoints private when authentication is enabled ggml-org/llama.cpp#26347 dropped /models + /v1/models from
  the public set), ModelParameters.setMmprojDevice() (b10541), and repeat-last-n /
  dry-penalty-last-n rejecting -1, the sentinel upstream dropped at b10273
  (Samplers: Remove ability to specify "full-context windows" from history-based samplers ggml-org/llama.cpp#26524).
• The b10644 → b10649 step broke the compile, in exactly the place the priority review table
  warns about. Upstream threaded a new mtmd_helper_init_opt (video-decode settings) through every
  media-ingesting helper, changing mtmd_helper_bitmap_init_from_file, tokenize_input_prompts and
  format_prompt_rerank. Four call sites adapted; each passes mtmd_helper_init_opt_default(), since
  all of them use mctx = nullptr or handle audio. Only a real compile finds this — server-common.h
  is a same-repo header the project #includes directly rather than one reachable through the
  documented dependency graph.
• Patch applier made idempotent via a stamp file recording the checked-out llama.cpp commit plus
  each patch's SHA-256, so a reconfigure of an existing build directory no longer aborts.

What changes for consumers

Three things here are visible to callers. Each is expanded below, and CHANGELOG.md carries the same
list for the release notes.

1. Slot state files written by an earlier build are rejected. LLAMA_SESSION_VERSION 9 → 10 and
   LLAMA_STATE_SEQ_VERSION 2 → 3 (b10642). This reaches LlamaModel.saveSlot / restoreSlot and the
   server's /slots/{id}?action=save; existing files have to be regenerated. The in-memory Session
   snapshot/fork feature is unaffected — it never writes a file. Detail: One consumer-visible break.
2. ModelParameters.setSleepIdleSeconds now throws IllegalArgumentException on 0 and on values
   below -1. Upstream rejects both, and emitting them aborted the entire argv parse into a bare
   "Failed to parse model parameters" that named neither the flag nor the reason. -1 (disable) and
   any positive count are unchanged.
3. Six InferenceParameters methods are deprecated as silent no-ops — withTfsZ,
   withPenalizeNl, both withPenaltyPrompt overloads, withUseChatTemplate and withChatTemplate.
   Upstream reads none of those keys from a request body, so they never had an effect; they still
   compile and still do nothing. Three production call sites that used withUseChatTemplate(true) to
   "enable jinja" were removed.

Not a break, but worth knowing: with setSleepIdleSeconds(> 0) every request now waits out an
in-progress idle sleep before the task is built (wake_server()). With idle sleep off — the default
— it is a no-op.

The final step: b10649 → b10679

No project-source change was required, and that is a checked result rather than an assumption.
server-schema.cpp, server-task.cpp, server-common.cpp and all six tools/server/*.h
headers are byte-identical across the range (compared by blob SHA), as are common/chat.h and
tools/mtmd/mtmd-helper.h — so the request-field set, its bounds, the emitted response keys, the
getMetrics()-class of silent contract break and the tts_engine.cpp surface are all provably out
of scope, not merely "checked and unchanged". The in-scope delta is 172 changed lines over 8
files; the rest of the 159-file range is tools/ui (rebuilt from GIT_TAG by CI) and backends
this project does not build.

Three patch-target files were touched (common/arg.cpp, server-context.cpp, server.cpp) and all
eight patches still apply with zero fuzz. Patch 0007's standing invariant survives because the
new KV-pool-sizing block in llama_server() sits before the extracted route table, not inside it.

Patch 0001 shrank from 37 to 36 files at this step, then grew back to 37 in the audit below.
Upstream rewrote tests/test-save-load-state.cpp's main() to strip a --models DIR option into its
own filtered_argv and call common_params_parse(fargc, filtered_argv.data(), …). By that patch's own
rule — an embedded caller that builds its own argv must call common_params_parse directly — that
site no longer wants the _main() flip, so the hunk was dropped, not refreshed. The patch itself
is still required, verified rather than assumed: common_params_parse in pristine
b10679:common/arg.cpp still carries the #ifdef _WIN32 count-guarded argv = utf8.ptrs.data()
override, and common_params_parse_main appears nowhere in b10679:common/arg.h — upstream has
adopted neither direction proposed in
ggml-org/llama.cpp#26416.

Two additive upstream features arrived, and both are exposed:

Flag	Java API	Why it was exposed rather than refused
--kv-unified-per-slot (b10662)	ModelParameters.setKvUnifiedPerSlot(int)	Registered set_examples({LLAMA_EXAMPLE_SERVER}) and jllama.cpp parses with LLAMA_EXAMPLE_SERVER, so it binds. Its cap half is applied in server-context.cpp's new n_ctx_slot(), whose value this binding reads through server_context_meta::slot_n_ctx — it becomes every slot.n_ctx and is the context budget handed to format_prompt_infill.
--tensor-read-lazy (b10653)	ModelParameters.setTensorReadLazy(TensorReadLazyMode) + the new args.TensorReadLazyMode enum (OFF/AUTO/ON, mirroring llama_lazy_mode)	No set_examples restriction, and common_model_params_to_llama copies lazy_mode into llama_model_params, so it reaches a plain LlamaModel load.

The --kv-unified-per-slot Javadoc is deliberately narrower than upstream's help text. Upstream's
second effect — sizing the shared KV pool to n_parallel * N when no context size is given — lives in
llama_server(), which a ModelParameters-loaded model never enters, so it applies to NativeServer
only. Stating that half unconditionally would have shipped an accurate-for-upstream sentence that is
wrong for the API it documents; the first draft did exactly that and it was caught by reading
server.cpp rather than the help string.

Two more upstream changes needed no adaptation, each for a specific reason:
llama_model_quantize_params gained max_buf_size, but LlamaQuantizer builds its params from
llama_model_quantize_default_params(), so upstream initialises the new field; and the
get_slot_n_ctx() → n_ctx_slot() rename is a private member of the server impl that the project
never calls — it reads the value through the unchanged server_context_meta::slot_n_ctx.

New upstream flags across the whole range: seven exposed, two refused

Nine flags were added across b10456 → b10679; none was removed or renamed. The decision was made per
flag, not by default:

Flag	Decision
--n-cpu-ffn / -ncffn (b10645)	Exposed as ModelParameters.setCpuFfnLayers(int)
--video-fps, --video-timestamp-interval, --video-ffmpeg-dir (b10647)	Exposed as setVideoFps / setVideoTimestampInterval / setVideoFfmpegDir
--mmproj-device / -mmdev (b10541)	Exposed as setMmprojDevice(String)
--tensor-read-lazy (b10653), --kv-unified-per-slot (b10662)	Exposed as setTensorReadLazy / setKvUnifiedPerSlot (see above)
--spec-synth-len, --spec-synth-rates (b10649)	Refused — upstream marks both "benchmarking only"

--n-cpu-moe / -ncmoe is not new — it has existed upstream since b6089
(ggml-org/llama.cpp#15077) and b10649 only refactored its lambda onto a shared helper. It had simply
never been exposed here, so setCpuMoeLayers(int) is added alongside setCpuFfnLayers(int). Both are
the companions to setGpuLayers: where that moves whole layers, these move only the weight class that
dominates a model's size, usually fitting a much larger model into the same VRAM at a smaller speed
cost.

The three --video-* flags were initially refused as "inert without a ContentPart video
factory". The follow-up audit showed that reasoning was wrong twice over, and they are now exposed:
server_context::load_model copies them into its own mtmd_helper_init_opt when the projector loads,
and that init_opt is what server-context.cpp hands to process_mtmd_prompt on the task.cli_files
path this binding drives — so they take effect for any media a caller attaches. Video decoding is also
genuinely compiled in: MTMD_VIDEO defaults ON (gated only on LLAMA_SUBPROCESS, also ON) and the
shipped libjllama.so carries the ffmpeg invocation strings. setVideoFfmpegDir is the one that earns
its place — upstream otherwise resolves ffmpeg/ffprobe from PATH, which a JVM process frequently
does not have them on. The content part itself — upstream's input_video, which takes raw base64
rather than a data: URI — remains in TODO.md.

The --spec-synth-* pair synthesises fake per-position acceptance probabilities so llama.cpp's own
speculative harness can be measured without a draft model — an instrument for benchmarking llama.cpp,
not a knob for an application. That refusal stands, with its reasoning in TODO.md.

One consumer-visible break

include/llama.h bumped LLAMA_SESSION_VERSION 9 → 10 and LLAMA_STATE_SEQ_VERSION 2 → 3 (at
b10642, inside the b10639 → b10644 step), following from llama_kv_cell_ext gaining a tok field
that must survive a state save/restore.

This is a state-file format break, not an API break, and it is reachable from public API: the
public LlamaModel.saveSlot(int, String) / restoreSlot(int, String) reach
llama_state_seq_save_file, which is stamped with that constant. Every slot file written by an earlier
build is rejected. Both methods now document the version lock, and that a mismatch surfaces as
upstream's misleading "Unable to restore slot: No available space in KV cache or invalid slot save file". The in-memory Session snapshot/fork feature is unaffected — it never writes a file.

Post-upgrade audits

Four rounds. The first covered b10456→b10644 (completeness, adaptation correctness, test integrity).
The second re-audited what the first could not see: its own fixes, the b10644→b10649 adaptation, and
the CI changes. The third mutation-tested every guard the first two added and swept for untested
surfaces. The fourth re-ran all of it against the b10679 target with five parallel agents.
Each round found mistakes made by the previous one, which is the argument for having run all of
them.

No missed adaptation, confirmed repeatedly. The mechanical server-contract checks are identical at
both ends of every range (b10644→b10649: 68 request fields and 23 bounds, unchanged; b10649→b10679: the
question does not arise, since the emitting files are byte-identical). libjllama.so links with
zero undefined upstream symbols — 531 undefined, all of them versioned libc/libstdc++/libgomp plus
three _ITM_*/__gmon_start__ stubs — so there is no latent macOS-ld64/MSVC link error of the
server_mcp kind. Patch 0007's route-table invariant was re-verified byte-for-byte at b10679: the
extracted helper, the patch's removal block and the pristine upstream lines all hash identically.

Round 1 findings (all fixed)

Finding	Fix
patches/0010 had no guard — CommonJsonEnumTrap builds its own JSON and calls no project code, so reverting the cast left it green.	NativeServerAttachIntegrationTest drives GET /v1/models and asserts vocab_type is an integer — and doubles as the signal to drop the patch if upstream ever casts it themselves.
train_engine.cpp carried the same postprocess_cpu_params pair as tts_params.hpp, inline, and its only test never runs in CI.	Both call one shared jllama::resolve_cpu_params (cpu_params.hpp).
testGetMetrics checked 8 of the 21 merged keys, so a rename in the C++ helper alone still shipped.	Asserts the full set, with the right JSON type per key.
Two gates still bypassed the resolver whose absence muted the whole model-backed suite.	Both routed through it, and both modules gained a rule that fails if a new test reintroduces one.
The langchain4j streaming test had been weakened to "content or thinking", which a content-routing regression passes.	Budget raised past Qwen3-0.6B's thinking block so it asserts real content again.
withTfsZ, withPenalizeNl, both withPenaltyPrompt overloads do nothing — those fields appear nowhere upstream and the schema discards unknown fields.	Deprecated with an explicit "ignored by the server" note.
getModelMeta() emitted only vision and audio; upstream has tracked has_inp_video for releases.	modalities.video emitted, ModelMeta.supportsVideo() added.
The -1 sentinel was cited as b10275 in 4 Javadoc blocks, 4 exception messages and ~9 docs. It is b10273.	All corrected.

Round 2 findings (all fixed)

The round-1 fix for idle-sleep was itself wrong, and two of the guards round 1 added did not actually
guard anything — both established by mutation, not by reading.

Finding	Fix
Runtime defect. With setSleepIdleSeconds(> 0) the model became permanently unusable after the first idle period. server_queue::post() does not leave the sleeping state — it notifies the condition variable, whose predicate tests req_stop_sleeping, so the loop woke, re-tested and slept again with the task still queued. Upstream wakes for the caller in server_res_generator's constructor, but only for readers from create_response(); this layer uses the CLI-facing get_response_reader(), and nothing here called wait_until_no_sleep() at all. Round 1's is_sleeping() predicate turned the hang into a thrown "No result" — it did not fix the cause.	All six post sites go through wake_and_post(). The predicate stays as a backstop, with its comment corrected.
The resolve_cpu_params tests were claimed to guard the trainer. They do not: train_engine.cpp is compiled into jllama only, never jllama_test, and LlamaTrainerIntegrationTest is gated on a property no CI job sets. Deleting the call left the suite green.	Assembly extracted to train_params.hpp (sibling of tts_params.hpp). Mutation-verified.
The new source-scanning rule matched only a bare string literal, so it could not see the house idiom (a PROP_* constant) or a wrapped call — it had zero true positives and could never fire.	Rewritten to inspect each call's own argument list. Mutation-verified against both forms. Its directory check was assumeTrue — a rule against silent skipping that silently skipped; now assertTrue, plus an empty-scan failure.
The parent-directory resolver test passed with the branch deleted (System.setProperty("user.dir") does not move the process CWD).	Dropped in favour of the pre-existing test that does fail; the langchain4j mirror gained the branch test it never had, also mutation-verified.
withUseChatTemplate / withChatTemplate are silent no-ops — upstream reads neither key from a request body — and three production call sites used withUseChatTemplate(true) to "enable jinja for tools", which it cannot do.	Both deprecated, the three calls removed. Tool calling was unaffected only because upstream defaults use_jinja to true.
The modality fixtures all set audio and video alike, so a getter wired to the wrong key passed; and nothing asserted the native side emits modalities.video.	Discriminating fixtures, plus the first assertion against the real emitter.
Version claims wrong in shipped Javadoc: the /metrics idle-timer exemption is b10519 (ggml-org/llama.cpp#27376), not b10644; the LLAMA_STATE_SEQ_VERSION bump is b10642, not b10644; --n-cpu-moe is b6089, not b10649.	All corrected.
The langchain4j integration job is model-backed and crosses JNI but had neither a crash-log echo nor a dump upload.	Both added, matching the six test-java-* jobs.

Round 3 findings (all fixed)

A third pass — a 25-agent workflow scanning Java API coverage, C++ helper coverage, diff correctness
and cheap pre-existing gaps, then adversarially verifying each finding — returned 39 findings, 18
upheld. Two were defects in code round 2 had just added, both re-confirmed against pristine
upstream before anything was changed.

Finding	Fix
setVideoFps rejected every value <= 0 — but mtmd-helper.h documents fps_target as "desired output fps; <= 0 means use the video's native fps", and the decoder resolves it as fps_target = arg > 0 ? arg : orig_fps. The guard deleted the only way to say "match this clip's own rate": the target is fixed when the projector loads, long before a clip is attached, so a caller cannot supply the rate instead.	Rejects only NaN/infinity now — infinity is the value that genuinely breaks, reaching ffmpeg as fps=inf. The same header also showed the test comment claiming interval = 0 meant "a timestamp every frame" was backwards: <= 0 means no timestamps.
setMmprojOffload dropped --mmproj-device unconditionally. Only some combinations actually clash: a named device and --mmproj-offload both resolve to (use_gpu=true, device=named) in either argv order, so clearing there discarded a real multi-GPU pin in order to set a field that already defaults to true.	Clearing restricted to the pairs whose meaning depends on argv order. The README paragraph that still claimed the two are mutually exclusive was replaced with a truth table re-derived from the two upstream handlers.
setVideoTimestampInterval accepted values above INT_MAX. The upstream field is int64_t, but the flag is registered with an int handler dispatched through std::stoi, so a larger value throws out_of_range, common_params_parse returns false, and the caller sees only "Failed to parse model parameters" — naming neither the flag nor the reason.	Bounded at INT_MAX, with the boundary itself asserted so the guard cannot drift to >=.

Six test groups were added, each checked to fail on the regression it guards: ModelParametersTest
(sentinel passthrough, non-finite rejection, the INT_MAX boundary, both directions of the mmproj
rule), test_tts_params.cpp (TrainParams, mutation-checked), test_json_helpers.cpp
(parse_positive_int_config above/at INT_MAX), test_tts_wav.cpp (the put_u16 fmt fields — no
test read a u16 from that header, so the s390x big-endian gate could not observe a byte-order
regression at all), TrainingParametersTest (LR-schedule wire keys, whose native fallbacks are
byte-identical to the Java defaults, so a renamed key silently reverts the schedule), and
LangChain4jMappingTest (pins that no "use_jinja" is emitted).

Seven items were deferred to TODO.md rather than widening a version bump — most notably that
ModelParameters emits five CLI flags the server arg parser rejects (--grp-attn-n/-w are
set_examples-scoped away from LLAMA_EXAMPLE_SERVER, so a grep sweep is structurally blind to
them, and four existing tests pin the dead literals happily). That list was re-verified at b10679 and
is still accurate and complete; this branch adds no new dead flag.

Round 4 (b10679) — five agents, complete

Five parallel agents audited the b10679 target: upstream-completeness over the full b10456→b10679
diff, Java API semantics, test integrity by mutation, patches/build/CI, and documentation
fact-checking. Each was given this branch's complete diff, the upstream diff, and a tagged llama.cpp
checkout as the authority — with the explicit instruction that this repo's own docs are under audit,
never evidence.

No blockers. Two agents independently confirmed the adaptation is complete: across the whole
range 9 CLI flags were added, 0 removed, 0 renamed, and all 9 are either exposed or refused with a
written rationale — nothing silently missed. The mechanical contract checks come out identical end to
end, and for the final step the question does not arise at all: the emitting files are byte-identical.

Test honesty, measured rather than asserted. The mutation agent applied 27 mutations and 26 went
red on the test that claims them. The one that did not is host-dependent by design (a thread-count
fixture that coincides with a 4-core runner), is covered by a sibling test that derives its probe from
the host, and the file's own comment anticipates it. Independently, PIT: 318 generated, 318 killed,
0 survived, 0 no-coverage, test strength 100%. Rounds 1–3 each found tests that passed with their
subject deleted; this round found none.

Three real defects, two of them introduced by this PR:

Finding	Fix
patches/0001 never flipped tools/tokenize/tokenize.cpp. The patch removes the Windows UTF-8 argv recovery from common_params_parse, so every unflipped standalone main() loses it — and llama-tokenize is a real binary passing the process argv. Impact here is zero (LLAMA_BUILD_TOOLS is OFF, so the file never compiles), but the patch is documented as the complete upstream change and submitting it verbatim would have regressed that binary on Windows.	Flip added: 36 → 37 files, all 8 patches re-verified against a pristine b10679 checkout with zero fuzz.
The concurrency block added earlier in this PR did not do what its comment claimed. cancel-in-progress: false does not protect a release run: GitHub cancels a pending run whenever a newer run joins the same group, independent of that setting — so a queued publish_to_central dispatch on main could be silently dropped by a later push to main.	Every non-PR run now gets its own group via run_id, so it can never be queued behind a sibling. PR runs still share a group per ref and supersede each other — verified empirically: the next push cancelled all 62 jobs of the previous run.
setMmprojDevice("none") cleared --no-mmproj-offload, although that pair resolves to (use_gpu=false, device=null) in either argv order. The code comment listed three of the four combinations while the README table written earlier in this PR listed four — the two disagreed.	Exactly one flag is contradicted per device value, never both. The missing fourth-row test is mutation-verified.

Nine wrong upstream tag attributions, seven of which ship in the javadoc jar. Every one named a tag
this bump chunked through rather than the tag that introduced the change; two were written during
this round. Each corrected value was bisected against the tags: --n-cpu-ffn b10645,
the three --video-* flags and mtmd_helper_init_opt b10647, --kv-unified-per-slot b10662,
--tensor-read-lazy b10653, dedup-cache-models b10505, and the server-schema.h
n_ctx_slot drop b10273 — the same commit as the -1 sentinel removal, not the unrelated b10275
break it had been filed under.

Three wrong mechanisms, all in text a future bump would act on — including the justification for
exposing --kv-unified-per-slot, which claimed the cap reaches eval_llama_cmpl_schema and moves the
repeat_last_n / dry_penalty_last_n sentinel expansion. Both false at b10679: that function lost its
n_ctx_slot parameter at b10273, and both fields carry set_hard_limits(0, INT32_MAX), so no sentinel
expands to anything. The conclusion held; the reasoning did not.

One number was removed rather than corrected. An earlier revision of this description claimed "286
response keys identical across the range". Swept four ways, the count comes out 229 / 282 / 254 / 317
depending on the extraction — never 286, and the figure could not be reconstructed. The claim now
states only the part that is both load-bearing and checkable: the key set does not move.

Three coverage gaps, two closed here:

• wake_and_post had no guard of any kind — the largest new runtime behaviour in jllama.cpp.
  jllama.cpp is not in the jllama_test target, and the only setSleepIdleSeconds reference
  asserted the flag string. Closed with IdleSleepWakeIntegrationTest, and that test then found
  two real JVM crashes — see below. It is the most valuable thing this audit produced.
• jsonSchemaToGrammar needs no model but was tested only inside a model-gated class, so on a
  host without GGUFs the native call was never made. Both directions moved into
  NativeLibraryLoadSmokeTest (now 4 tests, 0 skipped). The LlamaSystemProperties.PREFIX guard hole
  fixed earlier in the core module turned out to exist in the langchain4j copy too; closed there as
  well.
• patches/0010 still has no guard that runs on a model-free host — reverting its cast leaves
  ctest at a clean 520/520. Its model-gated guard does run on all six CI Java jobs, so this is a
  coverage gap rather than a shipping risk; recorded in TODO.md with the reproduction.

A documentation claim was corrected along the way that matters more than it looks: CLAUDE.md said a
class whose @BeforeAll assumption fails "reported as skipped". It does not — Surefire records
tests="0" errors="0" skipped="0", i.e. nothing at all. Any future guard of the form "did this run
skip anything?" would be structurally blind to it, which is exactly how the model-backed suite stayed
silently muted for months. The durable fix — a floor on tests actually executed — is in TODO.md.

Also recorded rather than folded in, both pre-existing: two all-*-aarch64 fat jars are attached to
releases with no smoke job (violating this repo's own "no release asset that CI has not run" rule,
on runners the workflow already uses), and the patch applier silently accepts a partially-reverted
source tree (CI is unaffected — every job configures into a fresh build directory).

The two crashes the new idle-sleep test found

Adding one model-backed test to an unguarded path immediately produced a deterministic SIGSEGV on
all six CI platforms. Both defects are pre-existing — reachable from public API for as long as
--sleep-idle-seconds has existed — and neither is caused by the version bump. Neither could be
reproduced locally: this sandbox has no GGUF models, so CI was the only instrument.

The mechanism behind both: upstream's handle_sleeping_state(true) calls destroy(), which frees
the model and context and nulls ctx_tgt/model_tgt; resuming calls load_model() again and
produces a new model. Three things in this layer assumed otherwise.

Crash 1 — llama_context::get_model(), on main. server_context::get_meta() dereferences
ctx_tgt/model_tgt and is read on every request to build the task. Upstream says as much of the
sibling accessor in its own header: "get the underlaying llama_context, can return nullptr if
sleeping". Separately, jctx->vocab is captured once after the initial load and used on ~16
tokenize/detokenize/rerank/infill paths — the reload replaces the model, so it dangles. (Its comment
claimed "valid for the lifetime of this context", true only while idle sleep is off, which is the
default and why this was never seen.) The round-2 wake_and_post() fix was necessary but not
sufficient: it wakes at post time and both reads happen before the post. Both now go through one
wake_server() choke point that waits out the sleep and re-reads the vocab.

Crash 2 — load_progress_callback(), on the worker thread, mid-reload — and it was ours. Fixing
crash 1 made the resume actually happen, and the SIGSEGV moved, which is how we knew the first fix
worked. Upstream declares the progress user data as a stack local of load_model() and stores its
address into a field that outlives the call — sound only because they re-assign both fields
unconditionally every load. patches/0002 made that conditional on == nullptr so a caller's
LoadProgressCallback survives; on resume the guard saw our own callback from the first load, skipped the
re-assignment, and left user_data pointing into a dead stack frame. A latent defect in that
patch since the day it was written, reachable only via a second load_model(). The guard now accepts
its own callback too, and CLAUDE.md records why the second disjunct exists — it otherwise reads as
redundant and would be simplified straight back into the crash.

Confirmed by CI, which is the only place it could be: all six Java jobs failed on b32485f, all
six failed differently on 9eb94b7, and all six pass on cce1911.

A smaller sibling defect surfaced on the way: setSleepIdleSeconds emitted 0, which upstream's
handler throws on — aborting the entire argv parse into a bare "Failed to parse model parameters".
A test was pinning that value as correct. Both fixed, with the -1 boundary asserted.

Test plan

• Local verification at the b10679 pin, from a fresh build directory through the real
  FetchContent path so the fail-loud PATCH_COMMAND ran for real — checked first, because three
  patch-target files changed in the final range. All 8 patches applied with zero fuzz (stamp
  pinned to 50f068ff), Release build clean, ctest 520/520, NativeLibraryLoadSmokeTest 4/4
  with 0 skipped (including nativeBuildInfoMatchesPinnedVersionConstant, the cross-check that
  the four pin sites and the linked build-info agree), full Java suite 1475 run / 0 failures /
  17 model-gated skips, llama-langchain4j 46 run / 0 failures, whole-reactor mvn verify
  clean (SpotBugs 0 bugs, enforcer incl. dependency convergence, javadoc jar), clang-format
  22.1.8 clean across the enforced set, publish.yml parses with 62 jobs.
• Every guard added by this PR mutation-tested. Round 4 applied 27 mutations; 26 went red
  on the test that claims them. The one that did not is host-dependent by design and covered by a
  sibling test that derives its probe from the host. PIT: 318 generated, 318 killed, 0 survived,
  0 no-coverage, test strength 100% — the value/args/exception/json gate at threshold 100.
• CI complete and green on the b10649 head (run 33124949458): 64 jobs — 56 success, 2
  failure, 6 skipped. The only two failures were the Verify GPG signing key jobs below; the six
  skips are the publish/release jobs, which correctly do not run on a PR branch.
  • Tests: all six Java Tests jobs (Ubuntu, macOS 14, macOS 15 Metal and no-Metal, Windows
    Ninja and MSVC), Integration Test llama-langchain4j (model-backed), C++ Tests, and
    Build and Test llama-kotlin.
  • Release gating: Package JARs, Package all-backends fat jars,
    Package + Validate Android AARs, and all three artifact smokes —
    Smoke test packaged natives (macOS), Smoke test all-backends fat jar (Linux) and
    (Windows). These are the checks that actually exercise the published artifact, and the macOS
    one exists because a corrupt dylib once shipped in three releases under a fully green pipeline.
  • Builds: every GPU/backend cross-compile, including both CUDA jobs.
• CI on the b10679 head confirms the idle-sleep fixes (run 33243049437, head cce1911):
  all six Java Tests jobs pass — Ubuntu, macOS 14 Metal, macOS 15 Metal and no-Metal, Windows
  Ninja and MSVC. That matters because it is the only instrument available: the crashing path needs
  a real GGUF, which this sandbox does not have. The progression is the evidence — 6/6 failed on
  b32485f at llama_context::get_model(), 6/6 failed differently on 9eb94b7 at
  load_progress_callback(), 6/6 pass here. Also green: C++ Tests, the spotless + SpotBugs style
  gate, llama-langchain4j, llama-kotlin, vmlens, the WebUI build, verify-model-cache on all
  three OSes, and every completed native build (Linux x86-64/aarch64/s390x, Windows x86-64/x86/arm64,
  both Android, Vulkan ×2, ROCm ×2, CUDA ×2, OpenVINO ×2, SYCL ×3, OpenCL ×2). The run is now
  complete: 64 jobs — 56 success, 2 failure, 6 skipped, the same shape as the b10649 run. The two
  failures are the Verify GPG signing key pair below; the six skips are the publish/release jobs,
  which correctly do not run on a PR branch. The release gate came in green after the long
  Build Windows 2025 x86_64 CUDA job (1 h 56 m — nvcc is uncached on Windows by design):
  Package JARs, Package all-backends fat jars, Package + Validate Android AARs,
  Smoke test packaged natives (macOS), Smoke test all-backends fat jar (Linux) and (Windows).
  Coveralls reports 87.011 %; SonarCloud, CodeQL, osv-scanner and clang-format are all green.
• Red checks, none of them this PR's code:
  • License Compliance — a FOSSA-style GitHub App status, not a workflow. Pre-existing: the identical
    finding count is recorded against PRs Add Windows Ninja Multi-Config build with sccache caching #248 and Feature wave: Android AAR + Kotlin facade, server attach/router modes, LangChain4j streaming, GGUF tooling (llama.cpp b9878) #298 in TODO.md. It is what keeps
    mergeable_state at blocked. The scanner's host is outside this environment's egress allowlist,
    so the findings could not be read.
  • Verify GPG signing key ×2 — fail in ~2 s before any step runs, because they declare
    environment: maven-central, whose deployment-branch policy admits only main and tags. Not
    re-runnable to a different outcome from a feature branch.
  • claude-review — API-side: the log shows the SDK returning is_error: true after 243 ms with
    num_turns: 1, total_cost_usd: 0 and modelUsage: {}, i.e. the model call never happened. No
    fix exists in this repo.
• Docs / CHANGELOG updated — CLAUDE.md, CHANGELOG.md, README.md,
  docs/history/llama-cpp-breaking-changes.md (a change-inventory row and a paired verification row
  per range, including b10649→b10679), docs/upgrade/llama-cpp-version-bump.md, TODO.md.

Related issues / PRs

Upstream references are fully qualified (ggml-org/llama.cpp#N); a bare #N anywhere in this
description is a PR in this repository.

• b6089 (llama : add --n-cpu-moe option ggml-org/llama.cpp#15077) — added --n-cpu-moe, exposed here for the first time
• b10273 (Samplers: Remove ability to specify "full-context windows" from history-based samplers ggml-org/llama.cpp#26524) — dropped -1 as the context-size sentinel for the penalty
  windows, and dropped eval_llama_cmpl_schema's n_ctx_slot parameter in the same commit
• b10408 (server: refactor + correctness fixes for metrics ggml-org/llama.cpp#26920) / b10519 (server: refactor sleep handling, allow access /metrics during sleep ggml-org/llama.cpp#27376) — split the metrics
  task; the latter also made /metrics stop resetting the idle timer
• b10505 (server: add dedup-cache-models preset option ggml-org/llama.cpp#27346) — dedup-cache-models, which hides a cache model from
  GET /models
• b10519 (server : make models endpoints private when authentication is enabled ggml-org/llama.cpp#26347) — removed /models + /v1/models from the public-endpoint
  set
• b10541 — added --mmproj-device
• b10585 (common: add json.h abstraction ggml-org/llama.cpp#27511) — switched the server's json alias to common_json
• b10642 — llama_kv_cell_ext.tok, and the session/state-seq version bumps that follow
• b10645 — --n-cpu-ffn
• b10647 — the mtmd_helper_init_opt threading plus the three --video-* flags
• b10649 — the two --spec-synth-* flags (refused)
• b10653 — llama_lazy_mode + --tensor-read-lazy
• b10662 — --kv-unified-per-slot
• b10679 — llama_model_quantize_params::max_buf_size (no adaptation needed)

patches/0011 is not tied to a version in that list — the invalid-UTF-8 branch is long-standing
upstream behaviour. It surfaced here only because this is the first CI run in which the model-backed
suite actually executed.

Checklist

• I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
• My commits follow Conventional Commits — every subject rewritten to the prefixes
  CONTRIBUTING.md lists, each under 72 characters. The rewrite touched messages only: the tree at
  the rewritten head was byte-identical to the pre-rewrite backup, verified with an empty git diff.
• No security-sensitive changes

https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH

[sonarqubecloud]

Quality Gate passed

Issues
35 New issues
0 Accepted issues

Measures
0 Security Hotspots
99.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[bernardladenthin]

Final CI result — run 33243049437, head cce1911

64 jobs — 56 success, 2 failure, 6 skipped. Identical shape to the b10649 run. The two failures
and the six skips are both accounted for below; nothing red is this PR's code.

The six Java jobs are the whole point of this run

All six pass: Ubuntu x86-64, macOS 14 arm64 (Metal), macOS 15 arm64 (Metal), macOS 15 arm64 (no
Metal), Windows x86-64 (Ninja) and Windows x86-64 (MSVC classifier).

They are the only instrument that could confirm the two idle-sleep crash fixes — the crashing path
needs a real GGUF, and the sandbox this branch was developed in has none. The evidence is the
progression, not a single green run:

Head	Result	Crash frame
b32485f	6/6 fail	llama_context::get_model(), on main
9eb94b7	6/6 fail, differently	server_context_impl::load_progress_callback(), on the server worker
cce1911	6/6 pass	—

The frame moving after the first fix is what showed the fix worked rather than merely relocating the
symptom.

Release gating — all green

The long pole was Build Windows 2025 x86_64 CUDA at 1 h 56 m (nvcc is deliberately not wrapped
by sccache on Windows, so the device passes are uncached). Everything downstream of it then came in
green:

• Package JARs, Package all-backends fat jars, Package + Validate Android AARs
• Smoke test packaged natives (macOS), Smoke test all-backends fat jar (Linux), (Windows) —
  the three checks that exercise the published artifact rather than a job-local build
• Android emulator on-device test (x86_64), Integration Test llama-langchain4j (model-backed),
  C++ Tests, Build and Test llama-kotlin, vmlens
• every native build: Linux x86-64/aarch64/s390x, Windows x86-64/x86/arm64, both Android,
  Vulkan ×2, ROCm ×2, CUDA ×2, OpenVINO ×2, SYCL ×3, OpenCL ×2

Also green outside the workflow: SonarCloud, CodeQL, osv-scanner, clang-format, and Coveralls at
87.011 %.

The three red checks, and why none of them is this branch

• Verify GPG signing key ×2 — both fail in ~2 s, before any step runs. They declare
  environment: maven-central, whose deployment-branch policy admits only main and tags, so a
  feature branch is rejected at job start. Not re-runnable to a different outcome from here.
• claude-review — fails API-side: the log shows the SDK returning is_error: true after
  243 ms with num_turns: 1, total_cost_usd: 0 and an empty modelUsage, i.e. the model call
  never happened. No fix for this exists in this repository.
• License Compliance (a commit status from a GitHub App, not a workflow job) — error,
  17 issues found. Pre-existing: the same finding count is recorded against PRs Add Windows Ninja Multi-Config build with sccache caching #248 and Feature wave: Android AAR + Kotlin facade, server attach/router modes, LangChain4j streaming, GGUF tooling (llama.cpp b9878) #298 in
  TODO.md, and the components it flags are third-party test-scope dependencies (JMH, Logback,
  jcstress, Lincheck, jqwik, JUnit Jupiter). This is what holds mergeable_state at blocked —
  not CI. The scanner's host is outside this environment's egress allowlist, so the individual
  findings could not be read from here.

The six skipped jobs are Check: main branch / SNAPSHOT, Check: v* tag, Publish Snapshot to Central, Publish Release to Central, Update Snapshot Pre-release on GitHub and Attach Signed Binaries to GitHub Release — the publish path, which correctly does not run on a PR branch.

One note on the concurrency change in this PR

Run 33215659259, on the now-superseded head 267d975, was never cancelled. That is expected and
one-time: GitHub reads the concurrency group from each run's own ref, and that run predates the
group-expression change. It is self-healing — the very next push did cancel all 62 jobs of run
33217708855. Cancelling the stale run returns 403 from this session, so its status is not a
statement about this PR.

---

Generated by Claude Code