From fea2f0f6fd7af524504527e7d2c5cc622af5eaf2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 26 Jul 2026 16:32:56 +0000 Subject: [PATCH] Upgrade llama.cpp from b10092 to b10107 Bumps the pinned llama.cpp release (b10092 -> b10107). Updates the four pin sites (llama/CMakeLists.txt GIT_TAG + LLAMA_TAG, README badge/link, CLAUDE.md, LlamaCppVersion.LLAMA_CPP_VERSION) and appends the history rows. This range carries a real public-API break: upstream PR ggml-org/llama.cpp#20834 removed the use_mmap/use_mlock/use_direct_io booleans from common_params and llama_model_params, replacing them with a single `enum llama_load_mode load_mode`. train_engine.cpp is updated accordingly (params.use_mmap = false -> params.load_mode = LLAMA_LOAD_MODE_NONE, mirroring upstream finetune.cpp). The Java layer is unaffected (ModelFlag.MLOCK/NO_MMAP emit --mlock/--no-mmap, still accepted by common/arg.cpp, now deprecated aliases that set load_mode). All six patches (0001-0003, 0006-0008) apply unchanged; the OuteTTS generator anchors hold (tools/tts/tts.cpp unchanged). Verified locally: cmake configure + full native build green, 485/485 C++ ctests pass, NativeLibraryLoadSmokeTest (2/2) confirms the linked binary reports b10107 matching the version constant. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_017DYEzEhYiZb9M9jQQrfzQm --- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 4 ++-- llama/src/main/cpp/train_engine.cpp | 4 +++- .../java/net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 6 files changed, 16 insertions(+), 12 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 5ca68a949..9d7ee1928 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10092** +Current llama.cpp pinned version: **b10107** ## Upgrading CUDA Version @@ -429,7 +429,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b10092 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10107 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -469,7 +469,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10092`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10107`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1262,7 +1262,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10092`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10107`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 2931b236b..57bd8928d 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10092](https://img.shields.io/badge/llama.cpp-%23b10092-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10092) +[![llama.cpp b10107](https://img.shields.io/badge/llama.cpp-%23b10107-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10107) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index 02da78be5..9b2ff23a7 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -511,3 +511,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10075–b10076 | upstream verification (sandbox) | The `b10075...b10076` diff touches only `ggml/src/ggml-cuda/getrows.cu` — no overlap with `0001`'s `common/arg.{cpp,h}` region, the `tools/server/*` targets of `0002`/`0003`/`0006`/`0007`/`0008`, or the OuteTTS anchors (`tools/tts/tts.cpp` unchanged), so all **six** patches apply cleanly in filename order. Per-platform build + `ctest` confirmation by the CI pipeline. | | b10076–b10092 | `common/arg.cpp` + `common/{chat-auto-parser-generator.cpp,chat-auto-parser.h,chat-diff-analyzer.cpp}` + `ggml/src/{ggml-cpu/{kleidiai/kleidiai.cpp,llamafile/sgemm.cpp},ggml-cuda/{common.cuh,convert.cu,dequantize.cuh,getrows.cu,ggml-cuda.cu,topk-moe.{cu,cuh}},ggml-hexagon/ggml-hexagon.cpp,ggml-openvino/ggml-openvino.cpp,ggml-vulkan/ggml-vulkan.cpp,ggml-webgpu/**}` + `src/{llama-arch.{cpp,h},llama-model.cpp,llama-model-saver.cpp,llama-vocab.{cpp,h},models/{laguna.cpp,models.h}}` + `tools/{server/{server-context.cpp,server-stream.cpp},mtmd/models/qwen3vl.cpp}` + `tests/**` + `conversion/**` + `gguf-py/**` + `tools/ui/**` | **Additive/tuning-only, no public-API surface (48 files, ~4.8k insertions incl. the auto-followed `tools/ui` WebUI + python conversion scripts, 16 commits).** All **eight** priority-8 headers (`common/{common.h,chat.h,speculative.h,arg.h,download.h}`, `tools/mtmd/mtmd.h`, `include/{llama.h,llama-cpp.h}`) are **byte-identical**. Two patch-target files changed but only **outside** the patched regions: `common/arg.cpp` adds speculative **sidecar** draft-repo resolution (MTP/DFlash/Eagle3 discovery in `common_models_handler_apply` — a `LLAMA_EXAMPLE_DOWNLOAD`-only region, well away from patch `0001`'s `common_params_parse`/`common_params_parse_main` block, and `common/arg.h` is unchanged); `tools/server/server-context.cpp` adds a null-`ctx_tgt` guard in `load_model` (return 400/false instead of a 500 crash) — a different sub-region than patch `0002`'s `load_progress_callback` guard and patch `0003`'s slot-prompt-similarity getters. `tools/server/server-stream.cpp` (compiled into `jllama`, not a patch target) gains 7 additive lines in `server_res_spipe::on_complete()`. The bulk is ggml backend work (CUDA GET_ROWS quantized-type support + topk-MoE, Vulkan queue/mutex refactor, WebGPU CONV_2D_DW, Hexagon, OpenVINO, kleidiai), new **Laguna** (poolside) model arch + Qwen3-VL mtmd + chat auto-parser tuning inside the upstream-compiled `llama`/`llama-common` libs, and python `conversion/`/`gguf-py/` tooling (not built here). No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged). All **six** patches (`0001`–`0003`, `0006`–`0008`) apply unchanged. b10092 is the topmost release at bump time. | | b10076–b10092 | upstream verification (sandbox) | All **six** patches (`0001`–`0003`, `0006`–`0008`) re-verified against a clean b10092 checkout (ggml/llama.cpp commit `3ce7da2c8`): applied in filename order via `git apply`, all clean — the `b10076...b10092` diff's two patch-target files (`common/arg.cpp`, `tools/server/server-context.cpp`) changed only outside the patched regions (`0001`'s `common_params_parse*` block and `0002`/`0003`'s `load_model`/getter regions untouched), `0006`/`0007`'s `tools/server/server.cpp` and `0008`'s `tools/server/server-models.cpp` unchanged (`0006`'s standalone `git apply --check` fails only because its context is the post-`0001` tree, as documented). The OuteTTS generator ran clean against `tools/tts/tts.cpp @ b10092` (all anchors held, generated TU written). Per-platform build + `ctest` confirmation by the CI pipeline. | +| b10092–b10107 | `common/common.h` + `include/llama.h` (**public-API break**) + `common/{arg.cpp,chat.cpp,common.cpp,fit.cpp}` + `ggml/src/{ggml-cuda/{common.cuh,mmq.cu,mmq.cuh,quantize.cu,quantize.cuh},ggml-hexagon/**,ggml-metal/ggml-metal-device.m}` + `src/{llama.cpp,llama-model.cpp,llama-model-loader.{cpp,h},llama-quant.cpp}` + `tools/{llama-bench/llama-bench.cpp,mtmd/mtmd-helper.cpp}` + `examples/{training/finetune.cpp,diffusion/diffusion-cli.cpp}` + `tools/ui/**` | **First real public-API break in many ranges (80 files, ~39.5k insertions, 15 commits).** Upstream PR [#20834](https://github.com/ggml-org/llama.cpp/pull/20834) (commit `e6dd0e29`, "args: refactor mlock/mmap/directio into load-mode") **removed the `use_mmap` / `use_mlock` / `use_direct_io` booleans from both `common_params` (`common/common.h`) and `llama_model_params` (`include/llama.h`)** and replaced them with a single `enum llama_load_mode load_mode` (values `NONE`/`MMAP`/`MLOCK`/`DIRECT_IO`), plus new `llama_load_mode_name()` / `llama_load_mode_from_str()` API and a `load_mode` copy in `common_model_params_to_llama` (`common/common.cpp`). **Required one project source change:** `llama/src/main/cpp/train_engine.cpp` set `params.use_mmap = false` to force writable weights for training → now `params.load_mode = LLAMA_LOAD_MODE_NONE` (mirrors upstream `finetune.cpp`'s identical flip in the same range). The **Java layer is unaffected**: `ModelFlag.MLOCK` / `NO_MMAP` emit the `--mlock` / `--no-mmap` CLI flags, which `common/arg.cpp` still accepts (now deprecated, they set `load_mode`) — `common/arg.h` is byte-identical. The other **six** priority-8 headers (`common/{chat.h,speculative.h,arg.h,download.h}`, `tools/mtmd/mtmd.h`, `include/llama-cpp.h`) are **byte-identical**. No patch-target file changed (`common/arg.cpp` moved only in the deprecated-flag region, well away from `0001`'s parse block; `tools/server/{server.cpp,server-context.cpp,server-models.cpp}` all identical), so all **six** patches (`0001`–`0003`, `0006`–`0008`) apply unchanged. The bulk is ggml backend work (CUDA MMQ/quantize rework, Hexagon `act-ops` rewrite, Metal device tweak), an internal `llama-model-loader` refactor, a `llama-bench` cleanup, and the auto-followed `tools/ui` WebUI. No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged). b10107 is the topmost release at bump time. | +| b10092–b10107 | project source change (`train_engine.cpp`) + upstream verification (sandbox) | `train_engine.cpp` updated for the `use_mmap`→`load_mode` API break (see the row above); `LLAMA_LOAD_MODE_NONE` resolves via the already-included `llama.h`. All **six** patches (`0001`–`0003`, `0006`–`0008`) re-verified against a clean b10107 checkout (ggml/llama.cpp commit `c0bc8591`): applied in filename order via `git apply`, all clean — no patch-target file changed in the range (`0006`'s standalone `git apply --check` fails only because its context is the post-`0001` tree, as documented). The OuteTTS generator anchors hold (`tools/tts/tts.cpp` unchanged). Per-platform build + `ctest` confirmation by the CI pipeline. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 6c7cda6e1..c3922e737 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10092 + GIT_TAG b10107 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= @@ -196,7 +196,7 @@ execute_process( COMMAND ${CMAKE_COMMAND} -DTTS_SRC=${llama.cpp_SOURCE_DIR}/tools/tts/tts.cpp -DOUT_CPP=${JLLAMA_TTS_GEN_CPP} - -DLLAMA_TAG=b10092 + -DLLAMA_TAG=b10107 -P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/generate-tts-upstream.cmake RESULT_VARIABLE JLLAMA_TTS_GEN_RESULT ) diff --git a/llama/src/main/cpp/train_engine.cpp b/llama/src/main/cpp/train_engine.cpp index e4825d2f8..e06110fb7 100644 --- a/llama/src/main/cpp/train_engine.cpp +++ b/llama/src/main/cpp/train_engine.cpp @@ -59,7 +59,9 @@ bool finetune(const finetune_config &cfg, std::string &err) { // Training needs writable weights (mmap yields read-only pointers) and an f32 KV cache // (OUT_PROD has no f16 support) — same forced settings as upstream finetune.cpp. - params.use_mmap = false; + // b10107 replaced the use_mmap/use_mlock/use_direct_io booleans with a single load_mode + // enum; LLAMA_LOAD_MODE_NONE disables mmap so the weight pointers stay writable. + params.load_mode = LLAMA_LOAD_MODE_NONE; params.cache_type_k = GGML_TYPE_F32; params.cache_type_v = GGML_TYPE_F32; diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 32970b2f5..48bf993fd 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10092"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10107"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10092-"} — call + * plus the resolved upstream commit, e.g. {@code "b10107-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10092"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10107"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10092"; + public static final String LLAMA_CPP_VERSION = "b10107"; // Constants holder — not instantiable. private LlamaCppVersion() {}