diff --git a/CLAUDE.md b/CLAUDE.md index 6cbf976fd..33742d2c3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10173** +Current llama.cpp pinned version: **b10197** ## Upgrading CUDA Version @@ -429,7 +429,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b10173 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10197 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -469,7 +469,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10173`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10197`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1263,7 +1263,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10173`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10197`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index 8b2636059..5c9383115 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10173](https://img.shields.io/badge/llama.cpp-%23b10173-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10173) +[![llama.cpp b10197](https://img.shields.io/badge/llama.cpp-%23b10197-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10197) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index c0567b912..7a09a9b92 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -516,3 +516,11 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10107–b10154 | `common/{common.h,chat.h,chat.cpp,arg.cpp,arg.h}` + `tools/server/server.cpp` (**patch-target restructure — MCP-server support**) + new `common/{subproc.*,trie.*}` + `ggml/**` backends + `tools/ui/**` | **Two local patches refreshed; no project-source change (47 commits, ~100 files, ~5k line changes).** Upstream added **MCP-server support** ([#26170](https://github.com/ggml-org/llama.cpp/pull/26170) et al.): `common/arg.{cpp,h}` gain `common_print_available_devices()` + `--mcp-servers-config`/`--mcp-servers-json`; `common/{common.h,chat.h,chat.cpp}` change `thinking_end_tag` → `thinking_end_tags` (vector) and add `mcp_servers_*` fields (reasoning-budget internals — inside upstream-compiled `llama-common`/chat TUs, not referenced by project source). In `tools/server/server.cpp`, `llama_server()` gains a `server_mcp mcp_mgr` lifecycle (`mcp_mgr.start()`, **2-arg** `tools.setup(server_tools, mcp_mgr)` — was 1-arg — `mcp_mgr.shutdown()` in the cleanup lambdas) + a "MCP servers (experimental)" warn branch, and the resumable-streaming routes were **renamed** `/v1/stream/:conv_id` → `/v1/stream` and **moved** into the middle of the route/CORS/tools block. **Patch `0001` refreshed:** one `common/arg.h` hunk drifted (b10154 inserted `common_print_available_devices()` between `common_params_parse` and `common_params_to_map`); the other 36 files (incl. `common/arg.cpp`, the ~34 `common_params_parse_main` call sites, `tests/test-arg-parser.cpp`, `tools/server/server.cpp`, `tools/tts/tts.cpp`) applied unchanged. **Patch `0007` reworked** (the MCP-server restructure broke its contiguous extraction): the shared helper `llama_server_register_common_routes()` was **narrowed to the pure core route table** (`health` … `slots`) — `llama_server()` now keeps the streaming/GCP/CORS-proxy/MCP/tools blocks inline, **byte-identical to upstream b10154** (only the route-table block is factored out) — and `llama_server_attach()` registers the route table + non-router streaming on the new `/v1/stream` paths + GCP compat + 403 stubs for the experimental `/cors-proxy` + `/tools` endpoints it does not wire. `0002`/`0003`/`0006`/`0008` applied unchanged. b10154 is the topmost release at bump time. | | b10107–b10154 | patch refresh (`0001`,`0007`) + upstream verification (sandbox) | Both refreshed patches + `0002`/`0003`/`0006`/`0008` re-verified against a clean b10154 checkout (ggml/llama.cpp commit `0e4a03622`): applied in filename order via `git apply`, all clean, no rejects; the fail-loud `PATCH_COMMAND` (`apply-llama-patches.cmake`) and the OuteTTS generator (`tts.cpp @ b10154`, all anchors held) both passed on a fresh `cmake -B build -DBUILD_TESTING=ON`. **Full `jllama` shared-library build (compile + link) verified locally** (`libjllama.so`, `-O3`) — the reworked `server.cpp` (helper + `llama_server_attach`) and all project TUs compile and **link** against b10154 (no undefined references), confirming the MCP-server-era API surface. No project source needed changing (`train_engine.cpp`'s prior `load_mode` fix from b10092–b10107 still holds). Per-platform build + `ctest` confirmation by the CI pipeline. | | b10154–b10173 | `common/{common.h,common.cpp,chat.h,speculative.cpp}` (**`common_memory` refactor**) + `tools/server/server-context.cpp` + `src/**` model TUs + `tools/ui/**` | **No patch change, no project-source change (85 files, ~5k line changes; ~393 KiB full diff — the chunking runbook flags it, but the reviewed non-WebUI surface is all additive/internal).** The one notable API change: the free functions `common_context_seq_rm`/`seq_add`/`seq_cp` (`common/common.h`/`common.cpp`) were made **`static`** and wrapped in a new **`struct common_memory`** (`init` + `seq_rm`/`seq_add`/`seq_cp` methods) that `tools/server/server-context.cpp` now calls via a per-slot `common_memory mem` member — **entirely upstream-internal**; no project C++ (`jllama.cpp`, `json_helpers.hpp`, `jni_helpers.hpp`, tests) references the removed free functions, so nothing to change. Everything else is additive: new `common_chat_format` enum value (`COMMON_CHAT_FORMAT_PEG_MINIMAX_M3`), new speculative enum (`COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK`) + `common/speculative.cpp` DSpark support, `common/chat.cpp` PEG-parser additions, new `src/models/{dflash,eagle3}.cpp` model TUs, and trivial `server-task.{h,cpp}`/`server-schema.cpp` changes (param rename `ctx_main`→`ctx_tgt`, an added `SRV_TRC` trace, a blank line). New added files (`models/templates/MiniMax-M3.jinja`, `tools/mtmd/models/{mimo-audio,parakeet}.cpp`) are upstream-glob-managed (data/mtmd), not in our manually-listed `target_sources`; no server source we list was renamed or removed. All **7** patches (`0001`–`0003`, `0006`–`0009`) apply unchanged (only `0002`/`0003` touch `server-context.cpp`, and both hunks — line ~1152 load-progress-callback, line ~3965 slot-similarity getters — are clear of the memory-refactor hunks). **Full local verification:** fresh `cmake -B build -DBUILD_TESTING=ON` (fail-loud patch apply clean, OuteTTS generator regenerated from `tts.cpp @ b10173` with all anchors holding, ggml commit `e9fa0781f`) + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **485/485 passing**. b10173 is the topmost release at bump time. Per-platform build + `ctest` confirmation by the CI pipeline. | +| b10173–b10174 | `conversion/glm.py` + `src/llama-model.cpp` + `src/models/{glm-dsa.cpp,models.h}` | **No project-source change (4 files, single commit, ~22 KiB — first chunk toward the b10216 target).** Full diff `b10173...b10216` is ~538 KiB / 43 commits, over the 100 KiB chunk threshold, so the bump proceeds in reviewable steps per the chunking runbook. Upstream PR [#25980](https://github.com/ggml-org/llama.cpp/pull/25980) adds NextN/MTP speculative-decoding support for GLM_DSA (GLM-5.2): `llama_model::create_memory` (`src/llama-model.cpp`) gains a dedicated `LLM_ARCH_GLM_DSA` branch (a plain attention KV cache for the dense-MLA NextN/MTP draft head when `n_layer_nextn > 0`, else the existing DSA cache for the trunk layers) and the STEP35/HY_V3 MTP-context condition widens to include `LLM_ARCH_GLM_DSA`; `src/models/glm-dsa.cpp` gains the NextN graph-build code; `conversion/glm.py` adds the matching GGUF conversion metadata. Entirely upstream model-architecture internals — none of the priority-8 headers changed, and no patch-target file (`common/arg.{cpp,h}`, `tools/server/server*.cpp`, `vendor/sheredom/subprocess.h`) is touched. No OuteTTS generator anchor touched (`tools/tts/tts.cpp` unchanged). | +| b10173–b10174 | upstream verification (sandbox) | All **7** patches (`0001`–`0003`, `0006`–`0009`) re-verified against a clean b10174 checkout (ggml/llama.cpp commit `7be2c65dc`): applied in filename order via the fail-loud `PATCH_COMMAND` (`apply-llama-patches.cmake`) on a fresh `cmake -B build -DBUILD_TESTING=ON`, all clean — no patch-target file changed in the range. The OuteTTS generator anchors hold (`tts.cpp @ b10174`). **Full local verification:** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **485/485 passing**. Per-platform build + `ctest` confirmation deferred to the CI pipeline. | +| b10174–b10175 | `ggml/src/ggml-cuda/{mmq-config-rdna3-5.cuh,mmq-config-rdna3.cuh,mmq-config-rdna4.cuh,mmq.cuh}` | **No project-source change (4 files, single commit, ~94 KiB).** Upstream PR #26199 adds independently-tunable MMQ kernel configs for AMD RDNA3.5 and RDNA3 (new `mmq-config-rdna3-5.cuh`/`mmq-config-rdna3.cuh`, previously sharing RDNA4's table) and retunes the RDNA4 table. Entirely CUDA-backend kernel-tuning data — none of the priority-8 headers changed, no patch-target file touched. No OuteTTS generator anchor touched. | +| b10174–b10175 | upstream verification (sandbox) | All **7** patches re-verified against a clean b10175 checkout (ggml/llama.cpp commit `60bccc376`): fail-loud `PATCH_COMMAND` clean, no patch-target file changed in the range. OuteTTS generator anchors hold (`tts.cpp @ b10175`). **Full local verification:** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`) + `ctest` **485/485 passing**. Per-platform build + `ctest` confirmation deferred to the CI pipeline. | +| b10175–b10189 | `common/{common.h,sampling.cpp}` + `include/llama.h` + `tools/server/{server-context.cpp,server-task.cpp}` + `src/{llama-vocab.cpp,models/*}` + `ggml/**` backends + `tools/ui/**` | **No project-source change (14 commits, ~92 KiB full / ~63 KiB excluding WebUI).** `common/common.h` drops the unused inline `common_params_sampling::has_logit_bias()` (project never called it); `include/llama.h` gains a purely-additive `llama_vocab_get_suppress_tokens()` (upstream #26276, model-specific suppress-token support consumed internally by `common/sampling.cpp`'s `common_sampler_init`, upstream-compiled, no project source touches it). `tools/server/server-context.cpp` + `server-task.cpp` get cosmetic `sim_best`→`f_sim_best`/`sim_cur`→`f_sim_cur` renames plus new `SLT_TRC`/`SRV_TRC` trace lines in the LCP-similarity slot-selection code — well clear of the `0002`/`0003` patch hunks (lines ~1152/~3965); confirmed no overlap. Everything else is CUDA MMQ-shmem-guard/Metal-unwire/RPC-tensor-memset/SYCL-elementwise/vocab-suppress-token internals plus the auto-followed WebUI (IndexedDB fixes, greeting spacing). No patch-target file conflict, no OuteTTS generator anchor touched. | +| b10175–b10189 | upstream verification (sandbox) | All **7** patches re-verified against a clean b10189 checkout (ggml/llama.cpp commit `b2f221684`): fail-loud `PATCH_COMMAND` clean, no rejects despite the `server-context.cpp`/`server-task.cpp` churn in this range (confirmed no line overlap with the `0002`/`0003` hunks). OuteTTS generator anchors hold (`tts.cpp @ b10189`). **Full local verification:** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **485/485 passing**. Per-platform build + `ctest` confirmation deferred to the CI pipeline. | +| b10189–b10197 | `common/{common.h,common.cpp}` + `src/llama-context.cpp` + `ggml/**` (**ggml bumped to 0.18.0**) + `tools/ui/**` | **No project-source change (8 commits, ~92 KiB full / ~63 KiB excluding WebUI).** `common/common.h`/`common.cpp` gain a test-only `common_get_model_or_exit(int, char*[])` helper (consolidating the deleted `tests/get-model.{cpp,h}` — upstream test-infra only, not referenced by project source). `src/llama-context.cpp`'s destructor and `encode()`/`decode()` now call `synchronize()` before clearing `embd_seq` to avoid a race with in-flight async output copies — purely internal, no header signature changed. Everything else is ggml-CUDA Q2_0 support + the version bump to 0.18.0, a Vulkan `common.cuh` tweak, and the auto-followed WebUI README refresh. No patch-target file touched (`common/arg.{cpp,h}`, `tools/server/server*.cpp`, `vendor/sheredom/subprocess.h`, `include/llama.h` all unchanged), no OuteTTS generator anchor touched. | +| b10189–b10197 | upstream verification (sandbox) | All **7** patches re-verified against a clean b10197 checkout (ggml/llama.cpp commit `958d9c0be`): fail-loud `PATCH_COMMAND` clean, no patch-target file changed in the range. OuteTTS generator anchors hold (`tts.cpp @ b10197`). **Full local verification:** fresh configure + full `cmake --build` (`libjllama.so` + `jllama_test` compile and link, `-O3`, no undefined references) + `ctest` **485/485 passing**. Per-platform build + `ctest` confirmation deferred to the CI pipeline. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 9897696b7..0d4daaaf7 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10173 + GIT_TAG b10197 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= @@ -196,7 +196,7 @@ execute_process( COMMAND ${CMAKE_COMMAND} -DTTS_SRC=${llama.cpp_SOURCE_DIR}/tools/tts/tts.cpp -DOUT_CPP=${JLLAMA_TTS_GEN_CPP} - -DLLAMA_TAG=b10173 + -DLLAMA_TAG=b10197 -P ${CMAKE_CURRENT_SOURCE_DIR}/cmake/generate-tts-upstream.cmake RESULT_VARIABLE JLLAMA_TTS_GEN_RESULT ) diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 08d704b32..ec0a3f70f 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10173"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10197"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10173-"} — call + * plus the resolved upstream commit, e.g. {@code "b10197-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10173"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10197"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10173"; + public static final String LLAMA_CPP_VERSION = "b10197"; // Constants holder — not instantiable. private LlamaCppVersion() {}