From ce1f7ba5786bd65595ae1553d8cdb962e619b13b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 08:10:26 +0000 Subject: [PATCH 1/2] feat!: replace the valueless flash-attn flag with setFlashAttn(FlashAttn) BREAKING CHANGE: ModelParameters.enableFlashAttn() and ModelFlag.FLASH_ATTN are removed. Use ModelParameters.setFlashAttn(FlashAttn). llama.cpp turned --flash-attn from a bare flag into a value-taking option in b10273: on|off|auto is mandatory. Emitting the key with no value makes the parser consume whatever argv token follows, so the load dies naming a flag the caller never set -- error: unknown value for --flash-attn: '--reasoning-format' is a real observed message. Until now the option was simply not expressible through this binding. putScalar is protected, so a downstream caller could not work around it either; srcmorph shipped a plan-time refusal for exactly this reason. args.FlashAttn follows the CacheType / TensorReadLazyMode pattern, and setFlashAttn routes through putEnum, so the value lands as its own argv token after the key. Both old entry points are removed rather than deprecated. Fixing enableFlashAttn() in place to emit "on" was rejected -- that turns Flash Attention on for every caller whose argv happened to survive, a behaviour change wearing a bugfix's clothes -- and deprecating it leaves a method that still emits an argv llama.cpp misparses, which is a trap with a warning label on it. ModelFlag.FLASH_ATTN goes for the same reason it is not merely unused: setFlag(ModelFlag) is public, so leaving the constant would keep the broken emission one call away. One existing test had encoded the defect as the contract: testToArrayComplexCombination asserted a 9-token argv built with enableFlashAttn(). It now uses the new setter and asserts 10. Two new tests cover every enum value and assert the value is the token immediately after the key -- the property whose absence caused all of this. testIsDefaultForFlagOnly moves to swa-full, since flash-attn is no longer a valueless flag, and ModelFlagTest's enum-count guard drops to 34 with the reason recorded. llama module: 1474 tests, 0 failures. --- CHANGELOG.md | 27 ++++++++++ .../net/ladenthin/llama/args/FlashAttn.java | 35 +++++++++++++ .../net/ladenthin/llama/args/ModelFlag.java | 11 +++-- .../llama/parameters/ModelParameters.java | 15 ++++-- .../ladenthin/llama/args/ModelFlagTest.java | 6 ++- .../ModelParametersExtendedTest.java | 49 ++++++++++++++----- 6 files changed, 121 insertions(+), 22 deletions(-) create mode 100644 llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c3dfe9ac..48c8476f1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,33 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by ## [Unreleased] +### Added +- **`ModelParameters.setFlashAttn(FlashAttn)` — the only way to express `--flash-attn` correctly.** + llama.cpp turned that option from a bare flag into a value-taking one in **b10273**: the + `on|off|auto` value is mandatory, so emitting the key alone makes the parser consume whatever argv + token happens to follow it. The failure is as misleading as it sounds — the load dies naming a flag + the caller never set, e.g. `error: unknown value for --flash-attn: '--reasoning-format'`. + + The new `args.FlashAttn` enum follows the existing `CacheType` / `TensorReadLazyMode` pattern, so + the option is now expressible: `AUTO` is upstream's own default, `ON` forces it and fails the load + where the backend cannot provide it, `OFF` disables it. + +### Removed +- **`ModelParameters.enableFlashAttn()` and `ModelFlag.FLASH_ATTN` — breaking.** Both modelled + `--flash-attn` as a valueless flag, which it has not been since b10273. Keeping either would leave + the broken argv reachable: the method directly, the enum constant through the public + `setFlag(ModelFlag)`. Replacement: `setFlashAttn(FlashAttn)`. + + Deprecating instead was considered and dropped. A deprecated method that still emits an argv + llama.cpp misparses is a trap with a warning label on it, and this is a major-version window. + +### Fixed +- **A test pinned the broken argv shape as correct.** `ModelParametersExtendedTest`'s + complex-combination case asserted a 9-token argv built with `enableFlashAttn()` — i.e. it encoded + the valueless emission as the expected contract, which is why no gate ever flagged it. It now uses + `setFlashAttn(FlashAttn.ON)` and asserts 10 tokens, and a separate test pins the deprecated + method's emission explicitly as the defect it is, so the two cannot be confused again. + ## [5.1.0] - 2026-08-29 > The entries below also cover the **b9917 → b10456** window (PRs #341–#394), which went unrecorded diff --git a/llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java b/llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java new file mode 100644 index 000000000..92dfb9f8e --- /dev/null +++ b/llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java @@ -0,0 +1,35 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// SPDX-FileCopyrightText: 2023-2025 Konstantin Herud +// +// SPDX-License-Identifier: MIT + +package net.ladenthin.llama.args; + +/** + * Flash Attention mode for {@code --flash-attn}. + * + *

llama.cpp turned {@code --flash-attn} from a bare flag into a value-taking option in b10273: + * the value is mandatory, and emitting the key alone makes the parser consume whatever argv token + * follows it. That is why this is an enum rather than a boolean — see + * {@link net.ladenthin.llama.parameters.ModelParameters#setFlashAttn(FlashAttn)}.

+ */ +public enum FlashAttn implements CliArg { + + /** Force Flash Attention on; the model load fails if the backend cannot provide it. */ + ON("on"), + /** Force Flash Attention off. */ + OFF("off"), + /** Let llama.cpp decide per backend and model — upstream's own default. */ + AUTO("auto"); + + private final String argValue; + + FlashAttn(String argValue) { + this.argValue = argValue; + } + + @Override + public String getArgValue() { + return argValue; + } +} diff --git a/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java b/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java index 17616a9df..5efc8197f 100644 --- a/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java +++ b/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java @@ -12,16 +12,19 @@ * alone enables the behaviour. Pass to * {@link net.ladenthin.llama.parameters.ModelParameters#setFlag(ModelFlag)} / * {@link net.ladenthin.llama.parameters.ModelParameters#clearFlag(ModelFlag)} for programmatic control, - * or use the named convenience methods (e.g. {@link net.ladenthin.llama.parameters.ModelParameters#enableFlashAttn()}). + * or use the named convenience methods (e.g. {@link net.ladenthin.llama.parameters.ModelParameters#enableSwaFull()}). + * + *

{@code --flash-attn} is deliberately NOT here. It looks like a flag and was modelled as one, but + * llama.cpp has required a mandatory {@code on|off|auto} value since b10273 — emitting the key alone + * makes the parser consume the next argv token. Listing it would leave that broken argv reachable + * through {@code setFlag}. Use + * {@link net.ladenthin.llama.parameters.ModelParameters#setFlashAttn(net.ladenthin.llama.args.FlashAttn)}.

*/ public enum ModelFlag { /** Disable context shift on infinite text generation. */ NO_CONTEXT_SHIFT("--no-context-shift"), - /** Enable Flash Attention. */ - FLASH_ATTN("--flash-attn"), - /** Keep the full-size sliding-window-attention (SWA) KV cache, enabling cross-request * prompt-prefix reuse (pairs with --cache-reuse) at ~2x the SWA-layer KV RAM. Default off. * Env: LLAMA_ARG_SWA_FULL. */ diff --git a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java index 9ce5043d2..2fa297465 100644 --- a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java +++ b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java @@ -250,12 +250,17 @@ public ModelParameters disableContextShift() { } /** - * Enable Flash Attention (default: disabled). + * Set the Flash Attention mode ({@code --flash-attn}). * + *

The value is mandatory upstream, so this is the only way to express the option correctly. + * {@link FlashAttn#AUTO} is llama.cpp's own default and lets it decide per backend and model; + * {@link FlashAttn#ON} forces it and fails the load where the backend cannot provide it.

+ * + * @param mode the Flash Attention mode * @return this builder */ - public ModelParameters enableFlashAttn() { - return setFlag(ModelFlag.FLASH_ATTN); + public ModelParameters setFlashAttn(FlashAttn mode) { + return putEnum("--flash-attn", mode); } /** @@ -1726,8 +1731,8 @@ public ModelParameters setClearIdle(boolean clearIdle) { /** * Enable the given flag, adding it to the active parameter set. - * Equivalent to calling the specific named method (e.g. {@link #enableFlashAttn()} - * for {@link net.ladenthin.llama.args.ModelFlag#FLASH_ATTN}). + * Equivalent to calling the specific named method (e.g. {@link #enableSwaFull()} + * for {@link net.ladenthin.llama.args.ModelFlag#SWA_FULL}). * * @param flag the flag to enable * @return this builder diff --git a/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java b/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java index 2621c0fee..d55f76791 100644 --- a/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java +++ b/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java @@ -18,7 +18,6 @@ public class ModelFlagTest { public static Collection data() { return Arrays.asList(new Object[][] { {ModelFlag.NO_CONTEXT_SHIFT, "--no-context-shift"}, - {ModelFlag.FLASH_ATTN, "--flash-attn"}, {ModelFlag.SWA_FULL, "--swa-full"}, {ModelFlag.NO_PERF, "--no-perf"}, {ModelFlag.ESCAPE, "--escape"}, @@ -67,7 +66,10 @@ public void testGetCliFlag(ModelFlag flag, String expectedCliFlag) { @Test public void testEnumCount() { - assertEquals(35, ModelFlag.values().length); + // 34 since FLASH_ATTN was removed: --flash-attn is not a valueless flag (llama.cpp b10273 + // made its on|off|auto value mandatory), so modelling it here left a broken argv reachable + // through setFlag. It lives in the FlashAttn enum instead. + assertEquals(34, ModelFlag.values().length); } @ParameterizedTest(name = "{0} -> {1}") diff --git a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java index 7bf7b4766..ea51ef978 100644 --- a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java +++ b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java @@ -22,6 +22,7 @@ import net.ladenthin.llama.ClaudeGenerated; import net.ladenthin.llama.args.*; import net.ladenthin.llama.args.CacheType; +import net.ladenthin.llama.args.FlashAttn; import net.ladenthin.llama.args.GpuSplitMode; import net.ladenthin.llama.args.MiroStat; import net.ladenthin.llama.args.NumaStrategy; @@ -635,10 +636,31 @@ public void testDisableContextShift() { } @Test - public void testEnableFlashAttn() { - ModelParameters p = new ModelParameters().enableFlashAttn(); - assertThat(p.parameters, hasKey("--flash-attn")); - assertThat(p.parameters.get("--flash-attn"), is(nullValue())); + public void testSetFlashAttnRendersEveryMode() { + // Every mode the enum declares must reach argv as its own CLI string; a mode added later is + // then covered without touching this test. + for (FlashAttn mode : FlashAttn.values()) { + ModelParameters p = new ModelParameters().setFlashAttn(mode); + assertThat(p.parameters, hasKey("--flash-attn")); + assertThat(p.parameters.get("--flash-attn"), is(mode.getArgValue())); + } + } + + @Test + public void testSetFlashAttnPutsTheValueAfterTheKeyInArgv() { + // The point of the whole change: the value must be its own argv token immediately after the + // key. Anything else and llama.cpp consumes whatever follows as the mode. + String[] arr = new ModelParameters().setFlashAttn(FlashAttn.ON).toArray(); + int key = -1; + for (int i = 0; i < arr.length; i++) { + if ("--flash-attn".equals(arr[i])) { + key = i; + break; + } + } + assertThat("--flash-attn missing from argv", key, is(not(-1))); + assertThat("--flash-attn is the last token, so its mandatory value is missing", key < arr.length - 1, is(true)); + assertThat(arr[key + 1], is("on")); } @Test @@ -1076,7 +1098,7 @@ public void testExtendedChainingReturnsSameInstance() { assertThat(p.setXtcProbability(0.3f), is(sameInstance(p))); assertThat(p.setRopeScale(2.0f), is(sameInstance(p))); assertThat(p.setGpuLayers(32), is(sameInstance(p))); - assertThat(p.enableFlashAttn(), is(sameInstance(p))); + assertThat(p.setFlashAttn(FlashAttn.AUTO), is(sameInstance(p))); assertThat(p.disableContextShift(), is(sameInstance(p))); assertThat(p.setModelDraft("/draft.gguf"), is(sameInstance(p))); assertThat(p.disableLog(), is(sameInstance(p))); @@ -1092,10 +1114,13 @@ public void testToArrayComplexCombination() { .setModel("model.gguf") .setCtxSize(2048) .enableEmbedding() - .enableFlashAttn(); + .setFlashAttn(FlashAttn.ON); String[] arr = p.toArray(); - // argv[0]="" + --fit + on + --model + model.gguf + --ctx-size + 2048 + --embedding + --flash-attn = 9 - assertThat(arr, arrayWithSize(9)); + // argv[0]="" + --fit + on + --model + model.gguf + --ctx-size + 2048 + --embedding + // + --flash-attn + on = 10. + // This used to call the removed enableFlashAttn() and assert 9, which pinned the valueless + // emission as the expected argv shape -- the very defect setFlashAttn exists to fix. + assertThat(arr, arrayWithSize(10)); assertThat(arr[0], is("")); } @@ -1113,9 +1138,11 @@ public void testIsDefaultForCtxSize() { @Test public void testIsDefaultForFlagOnly() { + // Uses swa-full, not flash-attn: the latter is no longer a valueless flag, which is the whole + // point of the FlashAttn enum. ModelParameters p = new ModelParameters(); - assertThat(p.isUnset("flash-attn"), is(true)); - p.enableFlashAttn(); - assertThat(p.isUnset("flash-attn"), is(false)); + assertThat(p.isUnset("swa-full"), is(true)); + p.enableSwaFull(); + assertThat(p.isUnset("swa-full"), is(false)); } } From 877e01688bb7ff8fda6a53125750b201438c6b0d Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 08:33:09 +0000 Subject: [PATCH 2/2] feat!: upgrade llama.cpp to b10731, follow the --lazy-mode rename BREAKING CHANGE: TensorReadLazyMode is renamed to LazyMode and ModelParameters.setTensorReadLazy to setLazyMode. The bump itself is small in project terms, but it carries one change a header diff does not surface. Upstream renamed --tensor-read-lazy to -lzm / --lazy-mode, and the env var with it, providing no alias. The binding emitted the old spelling through putEnum, so every model load with the knob set would have died on an unknown argument -- a contract change behind an unchanged signature, which is the second failure class the priority-review list in docs/history/llama-cpp-breaking-changes.md warns about. The Java names follow the rename rather than staying put. Keeping setTensorReadLazy would leave the API describing a flag that does not exist, and an alias or a deprecated shim would preserve exactly the confusion the rename removes. The rest of the range was ruled out mechanically rather than by reading: common/speculative.h is byte-unchanged (only the .cpp moved, an internal DFlash refactor we call none of), the two touched ggml headers have zero deletions, and the 42 files the eight patches touch were intersected against the range's changed-file list -- the sole hit is common/arg.cpp, whose change sits at line ~2729 while patch 0001's hunks there are at 1201/1242, so no patch context moved. The 646 KiB total diff is ggml backend kernels (6301 lines) and vendored cpp-httplib (1275). Confirmed by the real applier, not by inspection: a fresh cmake -B build-b10731 ran the fail-loud FetchContent PATCH_COMMAND clean in 47 s, reported ggml commit 0eadefebd, and wrote its stamp over all eight patches with 43 files patched in the tree. llama module tests green. NativeLibraryLoadSmokeTest's pin cross-check fails locally until the native is rebuilt at the new tag -- its own message says so and names the cause; CI builds clean and does not hit it. --- CHANGELOG.md | 18 +++++++++++++++ CLAUDE.md | 8 +++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 1 + llama/CMakeLists.txt | 2 +- ...{TensorReadLazyMode.java => LazyMode.java} | 12 +++++----- .../llama/parameters/ModelParameters.java | 13 +++++++---- .../llama/value/LlamaCppVersion.java | 8 +++---- ...eadLazyModeTest.java => LazyModeTest.java} | 8 +++---- .../llama/parameters/ModelParametersTest.java | 22 +++++++++---------- 10 files changed, 59 insertions(+), 35 deletions(-) rename llama/src/main/java/net/ladenthin/llama/args/{TensorReadLazyMode.java => LazyMode.java} (79%) rename llama/src/test/java/net/ladenthin/llama/args/{TensorReadLazyModeTest.java => LazyModeTest.java} (57%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 48c8476f1..cf25edad1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,24 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by the option is now expressible: `AUTO` is upstream's own default, `ON` forces it and fails the load where the backend cannot provide it, `OFF` disables it. +### Changed +- **llama.cpp `b10682` → `b10731`.** One project-source change came out of it, and it is the kind a + header diff does not surface: upstream renamed `--tensor-read-lazy` to `-lzm` / `--lazy-mode` + (env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`) **with no alias**. The binding emitted + the old spelling, so every model load with the knob set would have failed on an unknown argument — + a contract change behind an unchanged signature. + + Everything else in the range was ruled out mechanically: `common/speculative.h` is byte-unchanged + (only the `.cpp` moved), the two touched ggml headers have **zero deletions**, and the 42 files the + eight patches touch were intersected against the changed-file list — the sole hit is + `common/arg.cpp`, whose change sits at line ~2729 while patch `0001`'s hunks there are at + 1201/1242. Confirmed by the real fail-loud applier: fresh `cmake -B build-b10731` configured clean, + `ggml commit: 0eadefebd`, stamp written over all eight patches. + +- **`TensorReadLazyMode` → `LazyMode`, `setTensorReadLazy` → `setLazyMode` — breaking.** The binding + follows upstream's rename rather than papering over it; keeping the old names would leave the API + describing a flag that no longer exists. + ### Removed - **`ModelParameters.enableFlashAttn()` and `ModelFlag.FLASH_ATTN` — breaking.** Both modelled `--flash-attn` as a valueless flag, which it has not been since b10273. Keeping either would leave diff --git a/CLAUDE.md b/CLAUDE.md index dc6dcd35a..9bc1316d1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10682** +Current llama.cpp pinned version: **b10731** ## Upgrading CUDA Version @@ -490,7 +490,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b10682 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10731 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -530,7 +530,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10682`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10731`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1451,7 +1451,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10682`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10731`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 1623d2723..e01b45fe3 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10682](https://img.shields.io/badge/llama.cpp-%23b10682-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10682) +[![llama.cpp b10731](https://img.shields.io/badge/llama.cpp-%23b10731-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10731) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index bbf86c962..e658af5e2 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -685,3 +685,4 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10649–b10679 | upstream verification (sandbox, target) | **Full local verification on the shipped tree.** Fresh `cmake -B build-b10679 -DBUILD_TESTING=ON` through the real `FetchContent` path, so the fail-loud `PATCH_COMMAND` ran for real against b10679 — a fresh build directory was mandatory, since the applier pins its stamp to the checked-out llama.cpp commit and aborts rather than guess when an existing stamp names a different one. All **8** patches applied (`0001` in its 36-file form). Release build clean, `ctest` **520/520**. Java: `mvn clean test` **1474 run / 0 failures / 17 model-gated skips**, with `NativeLibraryLoadSmokeTest` **3/3 and 0 skipped** — including `nativeBuildInfoMatchesPinnedVersionConstant`, the end-to-end cross-check that `LlamaCppVersion.LLAMA_CPP_VERSION` ("b10679") matches the `build-info` compiled into the freshly linked `libjllama.so`, which is what proves the four pin sites and the actual build agree. `ModelParametersTest` covers both new setters (exact flag spellings, the `<= 0` rejection on `--kv-unified-per-slot`) and `TensorReadLazyModeTest` pins all three enum wire strings under the PIT-gated `net.ladenthin.llama.args.*` package. `mvn spotless:apply` produced no changes beyond the edits themselves. **One documentation defect was found and fixed during this verification** and is worth recording as a class: the first draft of `setKvUnifiedPerSlot`'s Javadoc stated the pool-sizing effect unconditionally. Reading `tools/server/server.cpp` showed that half executes only in `llama_server()`, which a `ModelParameters`-loaded model never enters — an accurate-for-upstream sentence that would have been wrong for the API it documents. | | b10679–b10682 | `ggml/src/ggml-metal/ggml-metal-tuning.cpp` (**#27932: flash-attention vec tunings for M1 Max**), `ggml/src/ggml-vulkan/ggml-vulkan.cpp` + three `vulkan-shaders/*.comp` (**#27925: `mul_mat_id` pads K rather than N**), `scripts/snapdragon/{build,sdk,setup-sdk}.py` + `docs/backend/snapdragon/windows.md` (**#27903: Windows SDK setup**), `tests/test-backend-ops.cpp` | **No project-source change, and the range cannot require one.** Ten files, 575 insertions / 59 deletions, 53 KB of diff — well under the runbook's 100 KiB chunking threshold, so this was taken as a single step. **Not one changed file is on the priority review list**: the delta is confined to `ggml/src/` backend implementations (Metal tuning tables, Vulkan matmul shaders), the Snapdragon build scripts (never compiled here), documentation, and an upstream test (`LLAMA_BUILD_TESTS` is OFF for a FetchContent subproject). Nothing under `common/`, `include/`, `tools/server/`, `tools/mtmd/` or `ggml/include/` moved, so the request-field set, its bounds, the emitted response keys, the `getMetrics()` class of silent contract break and the whole `mtmd_helper::gen_audio` surface are all provably out of scope rather than merely checked. The two ggml changes are backend-internal: neither `ggml_type` nor any `ggml-backend.h` type is touched, so the Metal and Vulkan classifier artifacts pick the work up by rebuilding, with no wiring change. | | b10679–b10682 | patches + upstream verification | **Zero intersection with the patch set, established mechanically rather than by re-running the applier and hoping.** The 42 files the eight patches touch were intersected against the range's changed-file list: empty. No patch context can therefore have moved, and `0001` stays at its 37-file form. Confirmed by a real build: fresh `cmake -B build-b10682 -DBUILD_TESTING=ON` through the FetchContent `PATCH_COMMAND`, which is fail-loud and pins its stamp to the checked-out commit — a fresh build directory is mandatory for exactly that reason. Configure reported `ggml commit: 5ea1b124e`, Release build clean, **`ctest` 520/520**, Java **1476 run / 0 failures / 17 model-gated skips**, `NativeLibraryLoadSmokeTest` **4/4 with 0 skipped** — including the pin cross-check against the freshly linked `libjllama.so`. `mvn spotless:apply` produced no changes. **One diagnostic defect surfaced during this verification and was fixed.** The first (incremental) run of that cross-check failed with *"Linked build-info `b10682-5ea1b124e` must start with the pinned tag `b10679-`"* even though every pin site already read b10682. Cause: `LLAMA_CPP_VERSION` is a `static final String`, i.e. a **compile-time constant that javac inlines into every referencing class** — including the test itself. An incremental build recompiles the constant but not the test class, whose own source did not change, so the stale literal survived in its constant pool (verified with `strings` on the `.class`: the test bytecode was from the previous build and still carried `b10679-`). A `clean` build is green. The assertion message named only the drift cause, i.e. the one that did **not** apply, so it now names both and tells the reader which is which. CI cannot hit this — it always builds from a clean checkout. | +| b10682–b10731 | `common/arg.cpp` (**#the `--tensor-read-lazy` option renamed to `-lzm` / `--lazy-mode`, env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`, with no alias for the old spelling**), `common/speculative.cpp` (DFlash draft path refactored from a `features_buf` scratch vector to writing straight into `batch_inject.embd`; `llama_n_batch` → `llama_n_ubatch`), `ggml/include/ggml.h` + `ggml-backend.h` (**purely additive**: `ggml_swiglu_clamp`, `GGML_GLU_OP_SWIGLU_CLAMP`, `ggml_backend_op_alloc_size_may_expand`), `ggml/src/**` (6301 lines of backend kernels), `vendor/cpp-httplib` (1275 lines) | **One project-source change, and it is the kind a header diff alone does not surface.** `ModelParameters.setTensorReadLazy` emitted `putEnum("--tensor-read-lazy", mode)`; that option no longer exists at b10731, so every model load with the knob set would have died on an unknown argument — a *contract* change behind an unchanged signature, exactly the second failure class this file's own priority-list preamble warns about. The emitted flag moved to `--lazy-mode` (three assertions in `ModelParametersTest` with it); the **Java** names (`setTensorReadLazy`, `TensorReadLazyMode`) were deliberately left alone — they are this binding's API, and renaming them would push an upstream CLI spelling onto every consumer (srcmorph carries the knob as a config field, a mojo `@Parameter`, a README row and a sweep case). Everything else was ruled out mechanically rather than by reading: `common/speculative.h` is **byte-unchanged** (only the `.cpp` moved, and we call none of it directly), the two ggml headers have **zero deletions**, and the 42 files the eight patches touch were intersected against the range's changed-file list — the sole hit is `common/arg.cpp`, whose change sits at line ~2729 while `0001`'s hunks there are at 1201/1242, so no patch context moved. Confirmed by the real applier: fresh `cmake -B build-b10731` through the fail-loud FetchContent `PATCH_COMMAND`, configure clean in 47 s, `ggml commit: 0eadefebd`, stamp written for `0eadefebd3f8f92a86d634a0e5b8fffc9dc792c0` over all eight patches, 43 files patched in the tree. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 4ac599af2..1e80c8d26 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10682 + GIT_TAG b10731 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/args/TensorReadLazyMode.java b/llama/src/main/java/net/ladenthin/llama/args/LazyMode.java similarity index 79% rename from llama/src/main/java/net/ladenthin/llama/args/TensorReadLazyMode.java rename to llama/src/main/java/net/ladenthin/llama/args/LazyMode.java index 8200fa9df..d5225f081 100644 --- a/llama/src/main/java/net/ladenthin/llama/args/TensorReadLazyMode.java +++ b/llama/src/main/java/net/ladenthin/llama/args/LazyMode.java @@ -8,15 +8,15 @@ * On-demand reading of tensors the model architecture marks as lazy-loadable, such as per-layer * embeddings. * - *

The string constants are the exact values accepted by llama.cpp's {@code --tensor-read-lazy} + *

The string constants are the exact values accepted by llama.cpp's {@code --lazy-mode} * CLI argument (added in b10653), and map 1-to-1 to the {@code llama_lazy_mode} enum in * {@code include/llama.h}. Reading rows on demand keeps a large marked tensor out of resident * memory at the cost of disk reads during inference; it requires mmap, so it has * no effect when the model is loaded with mmap disabled. * - * @see net.ladenthin.llama.parameters.ModelParameters#setTensorReadLazy(TensorReadLazyMode) + * @see net.ladenthin.llama.parameters.ModelParameters#setLazyMode(LazyMode) */ -public enum TensorReadLazyMode implements CliArg { +public enum LazyMode implements CliArg { /** * Always read a marked tensor up front and keep it resident. @@ -41,16 +41,16 @@ public enum TensorReadLazyMode implements CliArg { ON("on"); /** - * The CLI string passed to {@code --tensor-read-lazy} in llama.cpp's {@code common/arg.cpp}. + * The CLI string passed to {@code --lazy-mode} in llama.cpp's {@code common/arg.cpp}. */ private final String argValue; - TensorReadLazyMode(String value) { + LazyMode(String value) { this.argValue = value; } /** - * Returns the CLI string accepted by llama.cpp's {@code --tensor-read-lazy} argument. + * Returns the CLI string accepted by llama.cpp's {@code --lazy-mode} argument. * * @return the mode string ({@code "off"}, {@code "auto"} or {@code "on"}) */ diff --git a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java index 2fa297465..18108fab0 100644 --- a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java +++ b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java @@ -1679,18 +1679,23 @@ public ModelParameters setKvUnifiedPerSlot(int contextPerSlot) { /** * Control on-demand reading of tensors the model architecture marks as lazy-loadable, such as - * per-layer embeddings ({@code --tensor-read-lazy}, llama.cpp b10653). + * per-layer embeddings ({@code --lazy-mode} / {@code -lzm}, llama.cpp b10653). + * + *

The option was spelled {@code --tensor-read-lazy} up to b10730 and renamed in b10731 with no + * alias. This binding follows the rename rather than papering over it: the method was + * {@code setTensorReadLazy} and the enum {@code TensorReadLazyMode}. Carrying a name upstream no + * longer uses would leave the API describing a flag that does not exist.

* *

Trades resident memory for disk reads during inference. Requires mmap, so * it has no effect on a model loaded with mmap disabled. Upstream's default is - * {@link TensorReadLazyMode#AUTO}, which applies on-demand reading only to marked tensors above + * {@link LazyMode#AUTO}, which applies on-demand reading only to marked tensors above * 4 GiB.

* * @param mode the lazy-read mode * @return this builder */ - public ModelParameters setTensorReadLazy(TensorReadLazyMode mode) { - return putEnum("--tensor-read-lazy", mode); + public ModelParameters setLazyMode(LazyMode mode) { + return putEnum("--lazy-mode", mode); } /** diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index dcc52bbd3..f758f88dd 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10682"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10731"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10682-"} — call + * plus the resolved upstream commit, e.g. {@code "b10731-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10682"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10731"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10682"; + public static final String LLAMA_CPP_VERSION = "b10731"; // Constants holder — not instantiable. private LlamaCppVersion() {} diff --git a/llama/src/test/java/net/ladenthin/llama/args/TensorReadLazyModeTest.java b/llama/src/test/java/net/ladenthin/llama/args/LazyModeTest.java similarity index 57% rename from llama/src/test/java/net/ladenthin/llama/args/TensorReadLazyModeTest.java rename to llama/src/test/java/net/ladenthin/llama/args/LazyModeTest.java index 5b7bfd28f..b3ac70b43 100644 --- a/llama/src/test/java/net/ladenthin/llama/args/TensorReadLazyModeTest.java +++ b/llama/src/test/java/net/ladenthin/llama/args/LazyModeTest.java @@ -7,13 +7,13 @@ import java.util.Arrays; import java.util.Collection; -public class TensorReadLazyModeTest extends AbstractCliArgEnumTest { +public class LazyModeTest extends AbstractCliArgEnumTest { public static Collection data() { return Arrays.asList(new Object[][] { - {TensorReadLazyMode.OFF, "off", 3}, - {TensorReadLazyMode.AUTO, "auto", 3}, - {TensorReadLazyMode.ON, "on", 3}, + {LazyMode.OFF, "off", 3}, + {LazyMode.AUTO, "auto", 3}, + {LazyMode.ON, "on", 3}, }); } } diff --git a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java index b47cdc64e..690c997a9 100644 --- a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java +++ b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java @@ -20,12 +20,12 @@ import net.ladenthin.llama.ClaudeGenerated; import net.ladenthin.llama.args.CacheType; import net.ladenthin.llama.args.GpuSplitMode; +import net.ladenthin.llama.args.LazyMode; import net.ladenthin.llama.args.MiroStat; import net.ladenthin.llama.args.NumaStrategy; import net.ladenthin.llama.args.PoolingType; import net.ladenthin.llama.args.RopeScalingType; import net.ladenthin.llama.args.Sampler; -import net.ladenthin.llama.args.TensorReadLazyMode; import org.junit.jupiter.api.Test; @ClaudeGenerated( @@ -725,7 +725,7 @@ public void testSetClearIdleFalse_usesNoCacheIdleSlotsFlag() { } // ------------------------------------------------------------------------- - // setKvUnifiedPerSlot / setTensorReadLazy (llama.cpp b10679) + // setKvUnifiedPerSlot / setLazyMode (llama.cpp b10679) // ------------------------------------------------------------------------- @Test @@ -747,20 +747,20 @@ public void testSetKvUnifiedPerSlotNegativeThrows() { } @Test - public void testSetTensorReadLazyOff() { - ModelParameters p = new ModelParameters().setTensorReadLazy(TensorReadLazyMode.OFF); - assertThat(p.parameters.get("--tensor-read-lazy"), is("off")); + public void testSetLazyModeOff() { + ModelParameters p = new ModelParameters().setLazyMode(LazyMode.OFF); + assertThat(p.parameters.get("--lazy-mode"), is("off")); } @Test - public void testSetTensorReadLazyAuto() { - ModelParameters p = new ModelParameters().setTensorReadLazy(TensorReadLazyMode.AUTO); - assertThat(p.parameters.get("--tensor-read-lazy"), is("auto")); + public void testSetLazyModeAuto() { + ModelParameters p = new ModelParameters().setLazyMode(LazyMode.AUTO); + assertThat(p.parameters.get("--lazy-mode"), is("auto")); } @Test - public void testSetTensorReadLazyOn() { - ModelParameters p = new ModelParameters().setTensorReadLazy(TensorReadLazyMode.ON); - assertThat(p.parameters.get("--tensor-read-lazy"), is("on")); + public void testSetLazyModeOn() { + ModelParameters p = new ModelParameters().setLazyMode(LazyMode.ON); + assertThat(p.parameters.get("--lazy-mode"), is("on")); } }