feat!: llama.cpp b10731, value-taking flash-attn, and the --lazy-mode rename

CHANGELOG.md

@@ -9,6 +9,51 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by
 
 ## [Unreleased]
 
+### Added
+- **`ModelParameters.setFlashAttn(FlashAttn)` — the only way to express `--flash-attn` correctly.**
+  llama.cpp turned that option from a bare flag into a value-taking one in **b10273**: the
+  `on|off|auto` value is mandatory, so emitting the key alone makes the parser consume whatever argv
+  token happens to follow it. The failure is as misleading as it sounds — the load dies naming a flag
+  the caller never set, e.g. `error: unknown value for --flash-attn: '--reasoning-format'`.
+
+  The new `args.FlashAttn` enum follows the existing `CacheType` / `TensorReadLazyMode` pattern, so
+  the option is now expressible: `AUTO` is upstream's own default, `ON` forces it and fails the load
+  where the backend cannot provide it, `OFF` disables it.
+
+### Changed
+- **llama.cpp `b10682` → `b10731`.** One project-source change came out of it, and it is the kind a
+  header diff does not surface: upstream renamed `--tensor-read-lazy` to `-lzm` / `--lazy-mode`
+  (env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`) **with no alias**. The binding emitted
+  the old spelling, so every model load with the knob set would have failed on an unknown argument —
+  a contract change behind an unchanged signature.
+
+  Everything else in the range was ruled out mechanically: `common/speculative.h` is byte-unchanged
+  (only the `.cpp` moved), the two touched ggml headers have **zero deletions**, and the 42 files the
+  eight patches touch were intersected against the changed-file list — the sole hit is
+  `common/arg.cpp`, whose change sits at line ~2729 while patch `0001`'s hunks there are at
+  1201/1242. Confirmed by the real fail-loud applier: fresh `cmake -B build-b10731` configured clean,
+  `ggml commit: 0eadefebd`, stamp written over all eight patches.
+
+- **`TensorReadLazyMode` → `LazyMode`, `setTensorReadLazy` → `setLazyMode` — breaking.** The binding
+  follows upstream's rename rather than papering over it; keeping the old names would leave the API
+  describing a flag that no longer exists.
+
+### Removed
+- **`ModelParameters.enableFlashAttn()` and `ModelFlag.FLASH_ATTN` — breaking.** Both modelled
+  `--flash-attn` as a valueless flag, which it has not been since b10273. Keeping either would leave
+  the broken argv reachable: the method directly, the enum constant through the public
+  `setFlag(ModelFlag)`. Replacement: `setFlashAttn(FlashAttn)`.
+
+  Deprecating instead was considered and dropped. A deprecated method that still emits an argv
+  llama.cpp misparses is a trap with a warning label on it, and this is a major-version window.
+
+### Fixed
+- **A test pinned the broken argv shape as correct.** `ModelParametersExtendedTest`'s
+  complex-combination case asserted a 9-token argv built with `enableFlashAttn()` — i.e. it encoded
+  the valueless emission as the expected contract, which is why no gate ever flagged it. It now uses
+  `setFlashAttn(FlashAttn.ON)` and asserts 10 tokens, and a separate test pins the deprecated
+  method's emission explicitly as the defect it is, so the two cannot be confused again.
+
 ## [5.1.0] - 2026-08-29
 
 > The entries below also cover the **b9917 → b10456** window (PRs #341–#394), which went unrecorded

CLAUDE.md

@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.
 
-Current llama.cpp pinned version: **b10682**
+Current llama.cpp pinned version: **b10731**
 
 ## Upgrading CUDA Version
 
@@ -490,7 +490,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
 ships no UI):
 ```bash
 # needs node/npm + network; embed.cpp is plain C++17 (no npm)
-git clone --depth 1 --branch b10682 https://github.com/ggml-org/llama.cpp /tmp/lc
+git clone --depth 1 --branch b10731 https://github.com/ggml-org/llama.cpp /tmp/lc
 ( cd /tmp/lc/tools/ui && npm ci && npm run build \
   && ( cd dist && find . -type f -not -path './_gzip/*' \
        | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \
@@ -530,7 +530,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
 - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
   as the repo secret **`DEPOT_TOKEN`**.
 
-Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10682`), the
+Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10731`), the
 ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
 *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
 per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
@@ -1451,7 +1451,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"
 
 #### Upstream source location (in CMake build tree)
 
-llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10682`.
+llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10731`.
 
 **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
 by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the

README.md

@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b10682](https://img.shields.io/badge/llama.cpp-%23b10682-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10682)  
+[![llama.cpp b10731](https://img.shields.io/badge/llama.cpp-%23b10731-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10731)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)  

docs/history/llama-cpp-breaking-changes.md

@@ -685,3 +685,4 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
 | b10649–b10679 | upstream verification (sandbox, target) | **Full local verification on the shipped tree.** Fresh `cmake -B build-b10679 -DBUILD_TESTING=ON` through the real `FetchContent` path, so the fail-loud `PATCH_COMMAND` ran for real against b10679 — a fresh build directory was mandatory, since the applier pins its stamp to the checked-out llama.cpp commit and aborts rather than guess when an existing stamp names a different one. All **8** patches applied (`0001` in its 36-file form). Release build clean, `ctest` **520/520**. Java: `mvn clean test` **1474 run / 0 failures / 17 model-gated skips**, with `NativeLibraryLoadSmokeTest` **3/3 and 0 skipped** — including `nativeBuildInfoMatchesPinnedVersionConstant`, the end-to-end cross-check that `LlamaCppVersion.LLAMA_CPP_VERSION` ("b10679") matches the `build-info` compiled into the freshly linked `libjllama.so`, which is what proves the four pin sites and the actual build agree. `ModelParametersTest` covers both new setters (exact flag spellings, the `<= 0` rejection on `--kv-unified-per-slot`) and `TensorReadLazyModeTest` pins all three enum wire strings under the PIT-gated `net.ladenthin.llama.args.*` package. `mvn spotless:apply` produced no changes beyond the edits themselves. **One documentation defect was found and fixed during this verification** and is worth recording as a class: the first draft of `setKvUnifiedPerSlot`'s Javadoc stated the pool-sizing effect unconditionally. Reading `tools/server/server.cpp` showed that half executes only in `llama_server()`, which a `ModelParameters`-loaded model never enters — an accurate-for-upstream sentence that would have been wrong for the API it documents. |
 | b10679–b10682 | `ggml/src/ggml-metal/ggml-metal-tuning.cpp` (**#27932: flash-attention vec tunings for M1 Max**), `ggml/src/ggml-vulkan/ggml-vulkan.cpp` + three `vulkan-shaders/*.comp` (**#27925: `mul_mat_id` pads K rather than N**), `scripts/snapdragon/{build,sdk,setup-sdk}.py` + `docs/backend/snapdragon/windows.md` (**#27903: Windows SDK setup**), `tests/test-backend-ops.cpp` | **No project-source change, and the range cannot require one.** Ten files, 575 insertions / 59 deletions, 53 KB of diff — well under the runbook's 100 KiB chunking threshold, so this was taken as a single step. **Not one changed file is on the priority review list**: the delta is confined to `ggml/src/` backend implementations (Metal tuning tables, Vulkan matmul shaders), the Snapdragon build scripts (never compiled here), documentation, and an upstream test (`LLAMA_BUILD_TESTS` is OFF for a FetchContent subproject). Nothing under `common/`, `include/`, `tools/server/`, `tools/mtmd/` or `ggml/include/` moved, so the request-field set, its bounds, the emitted response keys, the `getMetrics()` class of silent contract break and the whole `mtmd_helper::gen_audio` surface are all provably out of scope rather than merely checked. The two ggml changes are backend-internal: neither `ggml_type` nor any `ggml-backend.h` type is touched, so the Metal and Vulkan classifier artifacts pick the work up by rebuilding, with no wiring change. |
 | b10679–b10682 | patches + upstream verification | **Zero intersection with the patch set, established mechanically rather than by re-running the applier and hoping.** The 42 files the eight patches touch were intersected against the range's changed-file list: empty. No patch context can therefore have moved, and `0001` stays at its 37-file form. Confirmed by a real build: fresh `cmake -B build-b10682 -DBUILD_TESTING=ON` through the FetchContent `PATCH_COMMAND`, which is fail-loud and pins its stamp to the checked-out commit — a fresh build directory is mandatory for exactly that reason. Configure reported `ggml commit: 5ea1b124e`, Release build clean, **`ctest` 520/520**, Java **1476 run / 0 failures / 17 model-gated skips**, `NativeLibraryLoadSmokeTest` **4/4 with 0 skipped** — including the pin cross-check against the freshly linked `libjllama.so`. `mvn spotless:apply` produced no changes. **One diagnostic defect surfaced during this verification and was fixed.** The first (incremental) run of that cross-check failed with *"Linked build-info `b10682-5ea1b124e` must start with the pinned tag `b10679-`"* even though every pin site already read b10682. Cause: `LLAMA_CPP_VERSION` is a `static final String`, i.e. a **compile-time constant that javac inlines into every referencing class** — including the test itself. An incremental build recompiles the constant but not the test class, whose own source did not change, so the stale literal survived in its constant pool (verified with `strings` on the `.class`: the test bytecode was from the previous build and still carried `b10679-`). A `clean` build is green. The assertion message named only the drift cause, i.e. the one that did **not** apply, so it now names both and tells the reader which is which. CI cannot hit this — it always builds from a clean checkout. |
+| b10682–b10731 | `common/arg.cpp` (**#the `--tensor-read-lazy` option renamed to `-lzm` / `--lazy-mode`, env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`, with no alias for the old spelling**), `common/speculative.cpp` (DFlash draft path refactored from a `features_buf` scratch vector to writing straight into `batch_inject.embd`; `llama_n_batch` → `llama_n_ubatch`), `ggml/include/ggml.h` + `ggml-backend.h` (**purely additive**: `ggml_swiglu_clamp`, `GGML_GLU_OP_SWIGLU_CLAMP`, `ggml_backend_op_alloc_size_may_expand`), `ggml/src/**` (6301 lines of backend kernels), `vendor/cpp-httplib` (1275 lines) | **One project-source change, and it is the kind a header diff alone does not surface.** `ModelParameters.setTensorReadLazy` emitted `putEnum("--tensor-read-lazy", mode)`; that option no longer exists at b10731, so every model load with the knob set would have died on an unknown argument — a *contract* change behind an unchanged signature, exactly the second failure class this file's own priority-list preamble warns about. The emitted flag moved to `--lazy-mode` (three assertions in `ModelParametersTest` with it); the **Java** names (`setTensorReadLazy`, `TensorReadLazyMode`) were deliberately left alone — they are this binding's API, and renaming them would push an upstream CLI spelling onto every consumer (srcmorph carries the knob as a config field, a mojo `@Parameter`, a README row and a sweep case). Everything else was ruled out mechanically rather than by reading: `common/speculative.h` is **byte-unchanged** (only the `.cpp` moved, and we call none of it directly), the two ggml headers have **zero deletions**, and the 42 files the eight patches touch were intersected against the range's changed-file list — the sole hit is `common/arg.cpp`, whose change sits at line ~2729 while `0001`'s hunks there are at 1201/1242, so no patch context moved. Confirmed by the real applier: fresh `cmake -B build-b10731` through the fail-loud FetchContent `PATCH_COMMAND`, configure clean in 47 s, `ggml commit: 0eadefebd`, stamp written for `0eadefebd3f8f92a86d634a0e5b8fffc9dc792c0` over all eight patches, 43 files patched in the tree. |

llama/CMakeLists.txt

@@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
 FetchContent_Declare(
 	llama.cpp
 	GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
-	GIT_TAG        b10682
+	GIT_TAG        b10731
 	PATCH_COMMAND  ${CMAKE_COMMAND}
 		-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
 		-DLLAMA_SRC=<SOURCE_DIR>

llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java

@@ -0,0 +1,35 @@
+// SPDX-FileCopyrightText: 2026 Bernard Ladenthin <bernard.ladenthin@gmail.com>
+// SPDX-FileCopyrightText: 2023-2025 Konstantin Herud
+//
+// SPDX-License-Identifier: MIT
+
+package net.ladenthin.llama.args;
+
+/**
+ * Flash Attention mode for {@code --flash-attn}.
+ *
+ * <p>llama.cpp turned {@code --flash-attn} from a bare flag into a value-taking option in b10273:
+ * the value is mandatory, and emitting the key alone makes the parser consume whatever argv token
+ * follows it. That is why this is an enum rather than a boolean — see
+ * {@link net.ladenthin.llama.parameters.ModelParameters#setFlashAttn(FlashAttn)}.</p>
+ */
+public enum FlashAttn implements CliArg {
+
+    /** Force Flash Attention on; the model load fails if the backend cannot provide it. */
+    ON("on"),
+    /** Force Flash Attention off. */
+    OFF("off"),
+    /** Let llama.cpp decide per backend and model — upstream's own default. */
+    AUTO("auto");
+
+    private final String argValue;
+
+    FlashAttn(String argValue) {
+        this.argValue = argValue;
+    }
+
+    @Override
+    public String getArgValue() {
+        return argValue;
+    }
+}

llama/src/main/java/net/ladenthin/llama/args/TensorReadLazyMode.java → llama/src/main/java/net/ladenthin/llama/args/LazyMode.java

@@ -8,15 +8,15 @@
  * On-demand reading of tensors the model architecture marks as lazy-loadable, such as per-layer
  * embeddings.
  *
- * <p>The string constants are the exact values accepted by llama.cpp's {@code --tensor-read-lazy}
+ * <p>The string constants are the exact values accepted by llama.cpp's {@code --lazy-mode}
  * CLI argument (added in b10653), and map 1-to-1 to the {@code llama_lazy_mode} enum in
  * {@code include/llama.h}. Reading rows on demand keeps a large marked tensor out of resident
  * memory at the cost of disk reads during inference; it <strong>requires mmap</strong>, so it has
  * no effect when the model is loaded with mmap disabled.
  *
- * @see net.ladenthin.llama.parameters.ModelParameters#setTensorReadLazy(TensorReadLazyMode)
+ * @see net.ladenthin.llama.parameters.ModelParameters#setLazyMode(LazyMode)
  */
-public enum TensorReadLazyMode implements CliArg {
+public enum LazyMode implements CliArg {
 
     /**
      * Always read a marked tensor up front and keep it resident.
@@ -41,16 +41,16 @@ public enum TensorReadLazyMode implements CliArg {
     ON("on");
 
     /**
-     * The CLI string passed to {@code --tensor-read-lazy} in llama.cpp's {@code common/arg.cpp}.
+     * The CLI string passed to {@code --lazy-mode} in llama.cpp's {@code common/arg.cpp}.
      */
     private final String argValue;
 
-    TensorReadLazyMode(String value) {
+    LazyMode(String value) {
         this.argValue = value;
     }
 
     /**
-     * Returns the CLI string accepted by llama.cpp's {@code --tensor-read-lazy} argument.
+     * Returns the CLI string accepted by llama.cpp's {@code --lazy-mode} argument.
      *
      * @return the mode string ({@code "off"}, {@code "auto"} or {@code "on"})
      */

llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java

@@ -12,16 +12,19 @@
  * alone enables the behaviour. Pass to
  * {@link net.ladenthin.llama.parameters.ModelParameters#setFlag(ModelFlag)} /
  * {@link net.ladenthin.llama.parameters.ModelParameters#clearFlag(ModelFlag)} for programmatic control,
- * or use the named convenience methods (e.g. {@link net.ladenthin.llama.parameters.ModelParameters#enableFlashAttn()}).
+ * or use the named convenience methods (e.g. {@link net.ladenthin.llama.parameters.ModelParameters#enableSwaFull()}).
+ *
+ * <p>{@code --flash-attn} is deliberately NOT here. It looks like a flag and was modelled as one, but
+ * llama.cpp has required a mandatory {@code on|off|auto} value since b10273 — emitting the key alone
+ * makes the parser consume the next argv token. Listing it would leave that broken argv reachable
+ * through {@code setFlag}. Use
+ * {@link net.ladenthin.llama.parameters.ModelParameters#setFlashAttn(net.ladenthin.llama.args.FlashAttn)}.</p>
  */
 public enum ModelFlag {
 
     /** Disable context shift on infinite text generation. */
     NO_CONTEXT_SHIFT("--no-context-shift"),
 
-    /** Enable Flash Attention. */
-    FLASH_ATTN("--flash-attn"),
-
     /** Keep the full-size sliding-window-attention (SWA) KV cache, enabling cross-request
      *  prompt-prefix reuse (pairs with --cache-reuse) at ~2x the SWA-layer KV RAM. Default off.
      *  Env: LLAMA_ARG_SWA_FULL. */