diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c3dfe9a..cf25edad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,51 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by ## [Unreleased] +### Added +- **`ModelParameters.setFlashAttn(FlashAttn)` — the only way to express `--flash-attn` correctly.** + llama.cpp turned that option from a bare flag into a value-taking one in **b10273**: the + `on|off|auto` value is mandatory, so emitting the key alone makes the parser consume whatever argv + token happens to follow it. The failure is as misleading as it sounds — the load dies naming a flag + the caller never set, e.g. `error: unknown value for --flash-attn: '--reasoning-format'`. + + The new `args.FlashAttn` enum follows the existing `CacheType` / `TensorReadLazyMode` pattern, so + the option is now expressible: `AUTO` is upstream's own default, `ON` forces it and fails the load + where the backend cannot provide it, `OFF` disables it. + +### Changed +- **llama.cpp `b10682` → `b10731`.** One project-source change came out of it, and it is the kind a + header diff does not surface: upstream renamed `--tensor-read-lazy` to `-lzm` / `--lazy-mode` + (env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`) **with no alias**. The binding emitted + the old spelling, so every model load with the knob set would have failed on an unknown argument — + a contract change behind an unchanged signature. + + Everything else in the range was ruled out mechanically: `common/speculative.h` is byte-unchanged + (only the `.cpp` moved), the two touched ggml headers have **zero deletions**, and the 42 files the + eight patches touch were intersected against the changed-file list — the sole hit is + `common/arg.cpp`, whose change sits at line ~2729 while patch `0001`'s hunks there are at + 1201/1242. Confirmed by the real fail-loud applier: fresh `cmake -B build-b10731` configured clean, + `ggml commit: 0eadefebd`, stamp written over all eight patches. + +- **`TensorReadLazyMode` → `LazyMode`, `setTensorReadLazy` → `setLazyMode` — breaking.** The binding + follows upstream's rename rather than papering over it; keeping the old names would leave the API + describing a flag that no longer exists. + +### Removed +- **`ModelParameters.enableFlashAttn()` and `ModelFlag.FLASH_ATTN` — breaking.** Both modelled + `--flash-attn` as a valueless flag, which it has not been since b10273. Keeping either would leave + the broken argv reachable: the method directly, the enum constant through the public + `setFlag(ModelFlag)`. Replacement: `setFlashAttn(FlashAttn)`. + + Deprecating instead was considered and dropped. A deprecated method that still emits an argv + llama.cpp misparses is a trap with a warning label on it, and this is a major-version window. + +### Fixed +- **A test pinned the broken argv shape as correct.** `ModelParametersExtendedTest`'s + complex-combination case asserted a 9-token argv built with `enableFlashAttn()` — i.e. it encoded + the valueless emission as the expected contract, which is why no gate ever flagged it. It now uses + `setFlashAttn(FlashAttn.ON)` and asserts 10 tokens, and a separate test pins the deprecated + method's emission explicitly as the defect it is, so the two cannot be confused again. + ## [5.1.0] - 2026-08-29 > The entries below also cover the **b9917 → b10456** window (PRs #341–#394), which went unrecorded diff --git a/CLAUDE.md b/CLAUDE.md index dc6dcd35..9bc1316d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10682** +Current llama.cpp pinned version: **b10731** ## Upgrading CUDA Version @@ -490,7 +490,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b10682 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10731 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -530,7 +530,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10682`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10731`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1451,7 +1451,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10682`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10731`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 1623d272..e01b45fe 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10682](https://img.shields.io/badge/llama.cpp-%23b10682-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10682) +[![llama.cpp b10731](https://img.shields.io/badge/llama.cpp-%23b10731-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10731) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index bbf86c96..e658af5e 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -685,3 +685,4 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10649–b10679 | upstream verification (sandbox, target) | **Full local verification on the shipped tree.** Fresh `cmake -B build-b10679 -DBUILD_TESTING=ON` through the real `FetchContent` path, so the fail-loud `PATCH_COMMAND` ran for real against b10679 — a fresh build directory was mandatory, since the applier pins its stamp to the checked-out llama.cpp commit and aborts rather than guess when an existing stamp names a different one. All **8** patches applied (`0001` in its 36-file form). Release build clean, `ctest` **520/520**. Java: `mvn clean test` **1474 run / 0 failures / 17 model-gated skips**, with `NativeLibraryLoadSmokeTest` **3/3 and 0 skipped** — including `nativeBuildInfoMatchesPinnedVersionConstant`, the end-to-end cross-check that `LlamaCppVersion.LLAMA_CPP_VERSION` ("b10679") matches the `build-info` compiled into the freshly linked `libjllama.so`, which is what proves the four pin sites and the actual build agree. `ModelParametersTest` covers both new setters (exact flag spellings, the `<= 0` rejection on `--kv-unified-per-slot`) and `TensorReadLazyModeTest` pins all three enum wire strings under the PIT-gated `net.ladenthin.llama.args.*` package. `mvn spotless:apply` produced no changes beyond the edits themselves. **One documentation defect was found and fixed during this verification** and is worth recording as a class: the first draft of `setKvUnifiedPerSlot`'s Javadoc stated the pool-sizing effect unconditionally. Reading `tools/server/server.cpp` showed that half executes only in `llama_server()`, which a `ModelParameters`-loaded model never enters — an accurate-for-upstream sentence that would have been wrong for the API it documents. | | b10679–b10682 | `ggml/src/ggml-metal/ggml-metal-tuning.cpp` (**#27932: flash-attention vec tunings for M1 Max**), `ggml/src/ggml-vulkan/ggml-vulkan.cpp` + three `vulkan-shaders/*.comp` (**#27925: `mul_mat_id` pads K rather than N**), `scripts/snapdragon/{build,sdk,setup-sdk}.py` + `docs/backend/snapdragon/windows.md` (**#27903: Windows SDK setup**), `tests/test-backend-ops.cpp` | **No project-source change, and the range cannot require one.** Ten files, 575 insertions / 59 deletions, 53 KB of diff — well under the runbook's 100 KiB chunking threshold, so this was taken as a single step. **Not one changed file is on the priority review list**: the delta is confined to `ggml/src/` backend implementations (Metal tuning tables, Vulkan matmul shaders), the Snapdragon build scripts (never compiled here), documentation, and an upstream test (`LLAMA_BUILD_TESTS` is OFF for a FetchContent subproject). Nothing under `common/`, `include/`, `tools/server/`, `tools/mtmd/` or `ggml/include/` moved, so the request-field set, its bounds, the emitted response keys, the `getMetrics()` class of silent contract break and the whole `mtmd_helper::gen_audio` surface are all provably out of scope rather than merely checked. The two ggml changes are backend-internal: neither `ggml_type` nor any `ggml-backend.h` type is touched, so the Metal and Vulkan classifier artifacts pick the work up by rebuilding, with no wiring change. | | b10679–b10682 | patches + upstream verification | **Zero intersection with the patch set, established mechanically rather than by re-running the applier and hoping.** The 42 files the eight patches touch were intersected against the range's changed-file list: empty. No patch context can therefore have moved, and `0001` stays at its 37-file form. Confirmed by a real build: fresh `cmake -B build-b10682 -DBUILD_TESTING=ON` through the FetchContent `PATCH_COMMAND`, which is fail-loud and pins its stamp to the checked-out commit — a fresh build directory is mandatory for exactly that reason. Configure reported `ggml commit: 5ea1b124e`, Release build clean, **`ctest` 520/520**, Java **1476 run / 0 failures / 17 model-gated skips**, `NativeLibraryLoadSmokeTest` **4/4 with 0 skipped** — including the pin cross-check against the freshly linked `libjllama.so`. `mvn spotless:apply` produced no changes. **One diagnostic defect surfaced during this verification and was fixed.** The first (incremental) run of that cross-check failed with *"Linked build-info `b10682-5ea1b124e` must start with the pinned tag `b10679-`"* even though every pin site already read b10682. Cause: `LLAMA_CPP_VERSION` is a `static final String`, i.e. a **compile-time constant that javac inlines into every referencing class** — including the test itself. An incremental build recompiles the constant but not the test class, whose own source did not change, so the stale literal survived in its constant pool (verified with `strings` on the `.class`: the test bytecode was from the previous build and still carried `b10679-`). A `clean` build is green. The assertion message named only the drift cause, i.e. the one that did **not** apply, so it now names both and tells the reader which is which. CI cannot hit this — it always builds from a clean checkout. | +| b10682–b10731 | `common/arg.cpp` (**#the `--tensor-read-lazy` option renamed to `-lzm` / `--lazy-mode`, env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`, with no alias for the old spelling**), `common/speculative.cpp` (DFlash draft path refactored from a `features_buf` scratch vector to writing straight into `batch_inject.embd`; `llama_n_batch` → `llama_n_ubatch`), `ggml/include/ggml.h` + `ggml-backend.h` (**purely additive**: `ggml_swiglu_clamp`, `GGML_GLU_OP_SWIGLU_CLAMP`, `ggml_backend_op_alloc_size_may_expand`), `ggml/src/**` (6301 lines of backend kernels), `vendor/cpp-httplib` (1275 lines) | **One project-source change, and it is the kind a header diff alone does not surface.** `ModelParameters.setTensorReadLazy` emitted `putEnum("--tensor-read-lazy", mode)`; that option no longer exists at b10731, so every model load with the knob set would have died on an unknown argument — a *contract* change behind an unchanged signature, exactly the second failure class this file's own priority-list preamble warns about. The emitted flag moved to `--lazy-mode` (three assertions in `ModelParametersTest` with it); the **Java** names (`setTensorReadLazy`, `TensorReadLazyMode`) were deliberately left alone — they are this binding's API, and renaming them would push an upstream CLI spelling onto every consumer (srcmorph carries the knob as a config field, a mojo `@Parameter`, a README row and a sweep case). Everything else was ruled out mechanically rather than by reading: `common/speculative.h` is **byte-unchanged** (only the `.cpp` moved, and we call none of it directly), the two ggml headers have **zero deletions**, and the 42 files the eight patches touch were intersected against the range's changed-file list — the sole hit is `common/arg.cpp`, whose change sits at line ~2729 while `0001`'s hunks there are at 1201/1242, so no patch context moved. Confirmed by the real applier: fresh `cmake -B build-b10731` through the fail-loud FetchContent `PATCH_COMMAND`, configure clean in 47 s, `ggml commit: 0eadefebd`, stamp written for `0eadefebd3f8f92a86d634a0e5b8fffc9dc792c0` over all eight patches, 43 files patched in the tree. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 4ac599af..1e80c8d2 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10682 + GIT_TAG b10731 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java b/llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java new file mode 100644 index 00000000..92dfb9f8 --- /dev/null +++ b/llama/src/main/java/net/ladenthin/llama/args/FlashAttn.java @@ -0,0 +1,35 @@ +// SPDX-FileCopyrightText: 2026 Bernard Ladenthin +// SPDX-FileCopyrightText: 2023-2025 Konstantin Herud +// +// SPDX-License-Identifier: MIT + +package net.ladenthin.llama.args; + +/** + * Flash Attention mode for {@code --flash-attn}. + * + *

llama.cpp turned {@code --flash-attn} from a bare flag into a value-taking option in b10273: + * the value is mandatory, and emitting the key alone makes the parser consume whatever argv token + * follows it. That is why this is an enum rather than a boolean — see + * {@link net.ladenthin.llama.parameters.ModelParameters#setFlashAttn(FlashAttn)}.

+ */ +public enum FlashAttn implements CliArg { + + /** Force Flash Attention on; the model load fails if the backend cannot provide it. */ + ON("on"), + /** Force Flash Attention off. */ + OFF("off"), + /** Let llama.cpp decide per backend and model — upstream's own default. */ + AUTO("auto"); + + private final String argValue; + + FlashAttn(String argValue) { + this.argValue = argValue; + } + + @Override + public String getArgValue() { + return argValue; + } +} diff --git a/llama/src/main/java/net/ladenthin/llama/args/TensorReadLazyMode.java b/llama/src/main/java/net/ladenthin/llama/args/LazyMode.java similarity index 79% rename from llama/src/main/java/net/ladenthin/llama/args/TensorReadLazyMode.java rename to llama/src/main/java/net/ladenthin/llama/args/LazyMode.java index 8200fa9d..d5225f08 100644 --- a/llama/src/main/java/net/ladenthin/llama/args/TensorReadLazyMode.java +++ b/llama/src/main/java/net/ladenthin/llama/args/LazyMode.java @@ -8,15 +8,15 @@ * On-demand reading of tensors the model architecture marks as lazy-loadable, such as per-layer * embeddings. * - *

The string constants are the exact values accepted by llama.cpp's {@code --tensor-read-lazy} + *

The string constants are the exact values accepted by llama.cpp's {@code --lazy-mode} * CLI argument (added in b10653), and map 1-to-1 to the {@code llama_lazy_mode} enum in * {@code include/llama.h}. Reading rows on demand keeps a large marked tensor out of resident * memory at the cost of disk reads during inference; it requires mmap, so it has * no effect when the model is loaded with mmap disabled. * - * @see net.ladenthin.llama.parameters.ModelParameters#setTensorReadLazy(TensorReadLazyMode) + * @see net.ladenthin.llama.parameters.ModelParameters#setLazyMode(LazyMode) */ -public enum TensorReadLazyMode implements CliArg { +public enum LazyMode implements CliArg { /** * Always read a marked tensor up front and keep it resident. @@ -41,16 +41,16 @@ public enum TensorReadLazyMode implements CliArg { ON("on"); /** - * The CLI string passed to {@code --tensor-read-lazy} in llama.cpp's {@code common/arg.cpp}. + * The CLI string passed to {@code --lazy-mode} in llama.cpp's {@code common/arg.cpp}. */ private final String argValue; - TensorReadLazyMode(String value) { + LazyMode(String value) { this.argValue = value; } /** - * Returns the CLI string accepted by llama.cpp's {@code --tensor-read-lazy} argument. + * Returns the CLI string accepted by llama.cpp's {@code --lazy-mode} argument. * * @return the mode string ({@code "off"}, {@code "auto"} or {@code "on"}) */ diff --git a/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java b/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java index 17616a9d..5efc8197 100644 --- a/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java +++ b/llama/src/main/java/net/ladenthin/llama/args/ModelFlag.java @@ -12,16 +12,19 @@ * alone enables the behaviour. Pass to * {@link net.ladenthin.llama.parameters.ModelParameters#setFlag(ModelFlag)} / * {@link net.ladenthin.llama.parameters.ModelParameters#clearFlag(ModelFlag)} for programmatic control, - * or use the named convenience methods (e.g. {@link net.ladenthin.llama.parameters.ModelParameters#enableFlashAttn()}). + * or use the named convenience methods (e.g. {@link net.ladenthin.llama.parameters.ModelParameters#enableSwaFull()}). + * + *

{@code --flash-attn} is deliberately NOT here. It looks like a flag and was modelled as one, but + * llama.cpp has required a mandatory {@code on|off|auto} value since b10273 — emitting the key alone + * makes the parser consume the next argv token. Listing it would leave that broken argv reachable + * through {@code setFlag}. Use + * {@link net.ladenthin.llama.parameters.ModelParameters#setFlashAttn(net.ladenthin.llama.args.FlashAttn)}.

*/ public enum ModelFlag { /** Disable context shift on infinite text generation. */ NO_CONTEXT_SHIFT("--no-context-shift"), - /** Enable Flash Attention. */ - FLASH_ATTN("--flash-attn"), - /** Keep the full-size sliding-window-attention (SWA) KV cache, enabling cross-request * prompt-prefix reuse (pairs with --cache-reuse) at ~2x the SWA-layer KV RAM. Default off. * Env: LLAMA_ARG_SWA_FULL. */ diff --git a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java index 9ce5043d..18108fab 100644 --- a/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java +++ b/llama/src/main/java/net/ladenthin/llama/parameters/ModelParameters.java @@ -250,12 +250,17 @@ public ModelParameters disableContextShift() { } /** - * Enable Flash Attention (default: disabled). + * Set the Flash Attention mode ({@code --flash-attn}). * + *

The value is mandatory upstream, so this is the only way to express the option correctly. + * {@link FlashAttn#AUTO} is llama.cpp's own default and lets it decide per backend and model; + * {@link FlashAttn#ON} forces it and fails the load where the backend cannot provide it.

+ * + * @param mode the Flash Attention mode * @return this builder */ - public ModelParameters enableFlashAttn() { - return setFlag(ModelFlag.FLASH_ATTN); + public ModelParameters setFlashAttn(FlashAttn mode) { + return putEnum("--flash-attn", mode); } /** @@ -1674,18 +1679,23 @@ public ModelParameters setKvUnifiedPerSlot(int contextPerSlot) { /** * Control on-demand reading of tensors the model architecture marks as lazy-loadable, such as - * per-layer embeddings ({@code --tensor-read-lazy}, llama.cpp b10653). + * per-layer embeddings ({@code --lazy-mode} / {@code -lzm}, llama.cpp b10653). + * + *

The option was spelled {@code --tensor-read-lazy} up to b10730 and renamed in b10731 with no + * alias. This binding follows the rename rather than papering over it: the method was + * {@code setTensorReadLazy} and the enum {@code TensorReadLazyMode}. Carrying a name upstream no + * longer uses would leave the API describing a flag that does not exist.

* *

Trades resident memory for disk reads during inference. Requires mmap, so * it has no effect on a model loaded with mmap disabled. Upstream's default is - * {@link TensorReadLazyMode#AUTO}, which applies on-demand reading only to marked tensors above + * {@link LazyMode#AUTO}, which applies on-demand reading only to marked tensors above * 4 GiB.

* * @param mode the lazy-read mode * @return this builder */ - public ModelParameters setTensorReadLazy(TensorReadLazyMode mode) { - return putEnum("--tensor-read-lazy", mode); + public ModelParameters setLazyMode(LazyMode mode) { + return putEnum("--lazy-mode", mode); } /** @@ -1726,8 +1736,8 @@ public ModelParameters setClearIdle(boolean clearIdle) { /** * Enable the given flag, adding it to the active parameter set. - * Equivalent to calling the specific named method (e.g. {@link #enableFlashAttn()} - * for {@link net.ladenthin.llama.args.ModelFlag#FLASH_ATTN}). + * Equivalent to calling the specific named method (e.g. {@link #enableSwaFull()} + * for {@link net.ladenthin.llama.args.ModelFlag#SWA_FULL}). * * @param flag the flag to enable * @return this builder diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index dcc52bbd..f758f88d 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10682"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10731"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10682-"} — call + * plus the resolved upstream commit, e.g. {@code "b10731-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10682"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10731"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10682"; + public static final String LLAMA_CPP_VERSION = "b10731"; // Constants holder — not instantiable. private LlamaCppVersion() {} diff --git a/llama/src/test/java/net/ladenthin/llama/args/TensorReadLazyModeTest.java b/llama/src/test/java/net/ladenthin/llama/args/LazyModeTest.java similarity index 57% rename from llama/src/test/java/net/ladenthin/llama/args/TensorReadLazyModeTest.java rename to llama/src/test/java/net/ladenthin/llama/args/LazyModeTest.java index 5b7bfd28..b3ac70b4 100644 --- a/llama/src/test/java/net/ladenthin/llama/args/TensorReadLazyModeTest.java +++ b/llama/src/test/java/net/ladenthin/llama/args/LazyModeTest.java @@ -7,13 +7,13 @@ import java.util.Arrays; import java.util.Collection; -public class TensorReadLazyModeTest extends AbstractCliArgEnumTest { +public class LazyModeTest extends AbstractCliArgEnumTest { public static Collection data() { return Arrays.asList(new Object[][] { - {TensorReadLazyMode.OFF, "off", 3}, - {TensorReadLazyMode.AUTO, "auto", 3}, - {TensorReadLazyMode.ON, "on", 3}, + {LazyMode.OFF, "off", 3}, + {LazyMode.AUTO, "auto", 3}, + {LazyMode.ON, "on", 3}, }); } } diff --git a/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java b/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java index 2621c0fe..d55f7679 100644 --- a/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java +++ b/llama/src/test/java/net/ladenthin/llama/args/ModelFlagTest.java @@ -18,7 +18,6 @@ public class ModelFlagTest { public static Collection data() { return Arrays.asList(new Object[][] { {ModelFlag.NO_CONTEXT_SHIFT, "--no-context-shift"}, - {ModelFlag.FLASH_ATTN, "--flash-attn"}, {ModelFlag.SWA_FULL, "--swa-full"}, {ModelFlag.NO_PERF, "--no-perf"}, {ModelFlag.ESCAPE, "--escape"}, @@ -67,7 +66,10 @@ public void testGetCliFlag(ModelFlag flag, String expectedCliFlag) { @Test public void testEnumCount() { - assertEquals(35, ModelFlag.values().length); + // 34 since FLASH_ATTN was removed: --flash-attn is not a valueless flag (llama.cpp b10273 + // made its on|off|auto value mandatory), so modelling it here left a broken argv reachable + // through setFlag. It lives in the FlashAttn enum instead. + assertEquals(34, ModelFlag.values().length); } @ParameterizedTest(name = "{0} -> {1}") diff --git a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java index 7bf7b476..ea51ef97 100644 --- a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java +++ b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersExtendedTest.java @@ -22,6 +22,7 @@ import net.ladenthin.llama.ClaudeGenerated; import net.ladenthin.llama.args.*; import net.ladenthin.llama.args.CacheType; +import net.ladenthin.llama.args.FlashAttn; import net.ladenthin.llama.args.GpuSplitMode; import net.ladenthin.llama.args.MiroStat; import net.ladenthin.llama.args.NumaStrategy; @@ -635,10 +636,31 @@ public void testDisableContextShift() { } @Test - public void testEnableFlashAttn() { - ModelParameters p = new ModelParameters().enableFlashAttn(); - assertThat(p.parameters, hasKey("--flash-attn")); - assertThat(p.parameters.get("--flash-attn"), is(nullValue())); + public void testSetFlashAttnRendersEveryMode() { + // Every mode the enum declares must reach argv as its own CLI string; a mode added later is + // then covered without touching this test. + for (FlashAttn mode : FlashAttn.values()) { + ModelParameters p = new ModelParameters().setFlashAttn(mode); + assertThat(p.parameters, hasKey("--flash-attn")); + assertThat(p.parameters.get("--flash-attn"), is(mode.getArgValue())); + } + } + + @Test + public void testSetFlashAttnPutsTheValueAfterTheKeyInArgv() { + // The point of the whole change: the value must be its own argv token immediately after the + // key. Anything else and llama.cpp consumes whatever follows as the mode. + String[] arr = new ModelParameters().setFlashAttn(FlashAttn.ON).toArray(); + int key = -1; + for (int i = 0; i < arr.length; i++) { + if ("--flash-attn".equals(arr[i])) { + key = i; + break; + } + } + assertThat("--flash-attn missing from argv", key, is(not(-1))); + assertThat("--flash-attn is the last token, so its mandatory value is missing", key < arr.length - 1, is(true)); + assertThat(arr[key + 1], is("on")); } @Test @@ -1076,7 +1098,7 @@ public void testExtendedChainingReturnsSameInstance() { assertThat(p.setXtcProbability(0.3f), is(sameInstance(p))); assertThat(p.setRopeScale(2.0f), is(sameInstance(p))); assertThat(p.setGpuLayers(32), is(sameInstance(p))); - assertThat(p.enableFlashAttn(), is(sameInstance(p))); + assertThat(p.setFlashAttn(FlashAttn.AUTO), is(sameInstance(p))); assertThat(p.disableContextShift(), is(sameInstance(p))); assertThat(p.setModelDraft("/draft.gguf"), is(sameInstance(p))); assertThat(p.disableLog(), is(sameInstance(p))); @@ -1092,10 +1114,13 @@ public void testToArrayComplexCombination() { .setModel("model.gguf") .setCtxSize(2048) .enableEmbedding() - .enableFlashAttn(); + .setFlashAttn(FlashAttn.ON); String[] arr = p.toArray(); - // argv[0]="" + --fit + on + --model + model.gguf + --ctx-size + 2048 + --embedding + --flash-attn = 9 - assertThat(arr, arrayWithSize(9)); + // argv[0]="" + --fit + on + --model + model.gguf + --ctx-size + 2048 + --embedding + // + --flash-attn + on = 10. + // This used to call the removed enableFlashAttn() and assert 9, which pinned the valueless + // emission as the expected argv shape -- the very defect setFlashAttn exists to fix. + assertThat(arr, arrayWithSize(10)); assertThat(arr[0], is("")); } @@ -1113,9 +1138,11 @@ public void testIsDefaultForCtxSize() { @Test public void testIsDefaultForFlagOnly() { + // Uses swa-full, not flash-attn: the latter is no longer a valueless flag, which is the whole + // point of the FlashAttn enum. ModelParameters p = new ModelParameters(); - assertThat(p.isUnset("flash-attn"), is(true)); - p.enableFlashAttn(); - assertThat(p.isUnset("flash-attn"), is(false)); + assertThat(p.isUnset("swa-full"), is(true)); + p.enableSwaFull(); + assertThat(p.isUnset("swa-full"), is(false)); } } diff --git a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java index b47cdc64..690c997a 100644 --- a/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java +++ b/llama/src/test/java/net/ladenthin/llama/parameters/ModelParametersTest.java @@ -20,12 +20,12 @@ import net.ladenthin.llama.ClaudeGenerated; import net.ladenthin.llama.args.CacheType; import net.ladenthin.llama.args.GpuSplitMode; +import net.ladenthin.llama.args.LazyMode; import net.ladenthin.llama.args.MiroStat; import net.ladenthin.llama.args.NumaStrategy; import net.ladenthin.llama.args.PoolingType; import net.ladenthin.llama.args.RopeScalingType; import net.ladenthin.llama.args.Sampler; -import net.ladenthin.llama.args.TensorReadLazyMode; import org.junit.jupiter.api.Test; @ClaudeGenerated( @@ -725,7 +725,7 @@ public void testSetClearIdleFalse_usesNoCacheIdleSlotsFlag() { } // ------------------------------------------------------------------------- - // setKvUnifiedPerSlot / setTensorReadLazy (llama.cpp b10679) + // setKvUnifiedPerSlot / setLazyMode (llama.cpp b10679) // ------------------------------------------------------------------------- @Test @@ -747,20 +747,20 @@ public void testSetKvUnifiedPerSlotNegativeThrows() { } @Test - public void testSetTensorReadLazyOff() { - ModelParameters p = new ModelParameters().setTensorReadLazy(TensorReadLazyMode.OFF); - assertThat(p.parameters.get("--tensor-read-lazy"), is("off")); + public void testSetLazyModeOff() { + ModelParameters p = new ModelParameters().setLazyMode(LazyMode.OFF); + assertThat(p.parameters.get("--lazy-mode"), is("off")); } @Test - public void testSetTensorReadLazyAuto() { - ModelParameters p = new ModelParameters().setTensorReadLazy(TensorReadLazyMode.AUTO); - assertThat(p.parameters.get("--tensor-read-lazy"), is("auto")); + public void testSetLazyModeAuto() { + ModelParameters p = new ModelParameters().setLazyMode(LazyMode.AUTO); + assertThat(p.parameters.get("--lazy-mode"), is("auto")); } @Test - public void testSetTensorReadLazyOn() { - ModelParameters p = new ModelParameters().setTensorReadLazy(TensorReadLazyMode.ON); - assertThat(p.parameters.get("--tensor-read-lazy"), is("on")); + public void testSetLazyModeOn() { + ModelParameters p = new ModelParameters().setLazyMode(LazyMode.ON); + assertThat(p.parameters.get("--lazy-mode"), is("on")); } }