bernardladenthin · bernardladenthin · Jul 27, 2026 · Jul 27, 2026
@@ -583,6 +583,7 @@ Current patches:
 | `0007-server-attach-http-frontend.patch` | **Adds `llama_server_attach(argc, argv, server_context&)`** so the `NativeServer` *attach mode* can serve an **already-loaded `LlamaModel`** over the upstream HTTP frontend — no second model load, no `start_loop()`; the LlamaModel's worker keeps driving the shared `server_context` and the HTTP routes post tasks to its queue (the queue is the synchronization point). Mechanically: (1) extracts the **pure core route table** (`health` … `slots`) out of `llama_server()` into `static void llama_server_register_common_routes(ctx_http, routes)` (shared, so the two entry points cannot drift on the core endpoint set). **Scope note (narrowed at the b10154 bump):** the helper deliberately carries **only** the stable, state-independent route table — **not** the resumable-streaming routes (their handlers differ between router / non-router), the GCP-compat shim, or the experimental **CORS-proxy / MCP-server / built-in-tools** wiring. b10154 (upstream MCP-server support) moved the streaming routes into the middle of that block and coupled tools/CORS to a per-call `server_mcp mcp_mgr` lifecycle, so the earlier contiguous "route-table + CORS-proxy + tools" extraction is no longer possible; `llama_server()` keeps all of that inline, **byte-identical to upstream b10154** (only the route-table block is factored out). (2) adds `llama_server_attach`, which parses only the HTTP-side argv via `common_params_parse`, starts the stream-session GC + `server_http_context`, registers the common route table, the **non-router** resumable-streaming handlers (upstream b10154 paths `/v1/stream` GET/DEL + `/v1/streams/lookup` POST), the GCP-compat shim, and **403 "disabled" stubs for `/cors-proxy` + `/tools`** (attach mode does not wire the experimental CORS-proxy / MCP / built-in-tools host — those belong to a full `llama-server`, not an embedded model), marks ready immediately (model already loaded), and blocks on the HTTP thread until `llama_server_request_shutdown()` — never calling `common_init()`, backend init, `ctx_server.terminate()` or `llama_backend_free()` (the embedding caller owns those). Applies after `0001`+`0006` (same file); closes the "NativeServer — reuse an already-loaded LlamaModel" TODO. Upstream-submittable ("server: let embedding callers attach the HTTP frontend to an existing server_context"). |
 | `0008-server-models-worker-cmd-override.patch` | **Makes router mode usable in-JVM.** The router (`server-models.cpp`) spawns each model worker by re-executing its own binary (`get_server_exec_path()` = `/proc/self/exe` & friends) — inside a JVM that binary is `java`, not a llama-server, so embedded router workers could never start. The patch adds env `LLAMA_SERVER_WORKER_CMD` (whitespace-split; read in `server_model_meta::update_args`) which replaces only the leading binary-path token of the rendered worker args, letting an embedding host relaunch workers through its own bootstrap — e.g. `java -cp app.jar net.ladenthin.llama.server.NativeServer` (each worker is then a fresh JVM running the classic single-model `NativeServer`). Exposed in Java as `NativeServer.setWorkerCommand(String...)` (JNI `setenv`); exercised by `RouterModeIntegrationTest` (Linux CI). Upstream-submittable (also useful for containerized/wrapped deployments). |
 | `0006-server-embed-native-server-jni.patch` | **Makes `server.cpp`'s `llama_server` embeddable in the JVM** so the `NativeServer` JNI bridge can run the full upstream HTTP server (WebUI included) inside `libjllama` — see "Two server modes" below. b9870 already exposes `int llama_server(int, char**)` (non-static; no `main` in the file), so the patch only adds embedded-mode support: (1) a `g_llama_server_embedded` flag + `llama_server_set_embedded()` / `llama_server_request_shutdown()` (declared in the committed `src/main/cpp/native_server_bridge.h`); (2) skips installing the process-wide SIGINT/SIGTERM handlers when embedded (they would hijack the JVM's); (3) in embedded mode parses the **forwarded** argv via `common_params_parse` instead of `common_params_parse_main` (whose `GetCommandLineW` recovery would pick up `java.exe`'s command line — the same Windows class of bug `0001` fixes). `llama_server_request_shutdown()` mirrors the SIGTERM path (invokes the installed `shutdown_handler` → `ctx_server.terminate()` unblocks `start_loop()`), giving JNI an out-of-band stop since `ctx_server` is loop-local. Applies **after `0001`** (which flips this call site to `common_params_parse_main`), so its context is the post-`0001` tree; regenerate against `0001`+source on a bump. Only touches `tools/server/server.cpp`. |
+| `0009-subprocess-guard-addchdir-np-old-glibc.patch` | **Fixes the b10154 cross-compile break on old glibc.** b10154 bumped the vendored `vendor/sheredom/subprocess.h` to a version that calls `posix_spawn_file_actions_addchdir_np` (a non-portable extension: glibc **≥ 2.29**, bionic API ≥ 34, macOS ≥ 10.15) to honor a spawn `process_cwd`, and added `common/subproc.cpp` (both pulled in via the new MCP-server support). Upstream guards that call **only for macOS**, so on **manylinux2014 (glibc 2.17)** the declaration is absent and `subprocess.h` (via `subproc.cpp` + `mtmd-helper.cpp` + `server-mcp.cpp`) fails to compile (`'posix_spawn_file_actions_addchdir_np' was not declared`). Android is unaffected — `__ANDROID_UNAVAILABLE_SYMBOLS_ARE_WEAK__` already makes the declaration visible (weak). The patch adds a `SUBPROCESS_HAVE_ADDCHDIR_NP` compile-time probe (`__GLIBC_PREREQ(2, 29)`, nested under `defined(__GLIBC__)` so non-glibc platforms never evaluate it) and, when unavailable, reports a requested cwd as `ENOSYS` instead of failing to compile — this build never spawns with a cwd. **Not reproducible on a modern-glibc dev box** (the `addchdir_np` branch is taken there); the manylinux CI job is the gate. Upstream-submittable (to sheredom/subprocess.h). Only touches `vendor/sheredom/subprocess.h`. |
 
 **`0005` was dropped at the b9981 bump.** Upstream's own `server-context.cpp` picked up an
 equivalent — and broader — fix for the same checkpoint-starvation problem: `create_checkpoint`
@@ -1036,7 +1037,7 @@ If the local check passes (`BUILD SUCCESS`), the `mvn package` job in
 - `json_helpers.hpp` — Pure JSON transformation helpers (no JNI, no llama state). Independently unit-testable.
 - `jni_helpers.hpp` — JNI bridge helpers (handle management + server orchestration). Includes `json_helpers.hpp`.
 - Uses `nlohmann/json` for JSON deserialization of parameters.
-- The upstream server library (`server-context.cpp`, `server-queue.cpp`, `server-task.cpp`, `server-schema.cpp`, `server-models.cpp`, and — since b9829 — `server-stream.cpp`) is compiled directly into `jllama` via CMake — there is no hand-ported `server.hpp` fork. **`server-stream.cpp` is mandatory, not optional:** it defines the resumable-streaming SSE replay buffer (`g_stream_sessions`, `stream_session_attach_pipe`, `stream_aware_should_stop`, `stream_conv_id_from_headers`, the `stream_pipe_*` types) that `server-context.cpp` / `server-http.cpp` / `server-models.cpp` now `#include "server-stream.h"` and call, so omitting it fails the link with undefined references. It is platform-neutral (threads + std mutex/condvar, no `subprocess.h`/`posix_spawn_*`), so it builds on Android too and sits outside the `server-models.cpp` Android guard. `jllama` wires its own JNI routes and never calls `g_stream_sessions.start_gc()` (only the excluded standalone `server.cpp` `main()` does), so its GC thread stays dormant. **Phase 2:** the upstream HTTP transport (`tools/server/server-http.cpp`) and its `cpp-httplib` backend (`vendor/cpp-httplib/httplib.cpp`) are now compiled into `jllama` too, so the OpenAI-compatible server can be driven natively from JNI *inside* `libjllama` — no separate `llama-server` executable (a JNI shared library loads anywhere a JVM runs, which a standalone binary does not). `server-http.cpp` does `#include "ui.h"` (the WebUI asset table that `tools/ui`/`llama-ui` normally generates); since the Svelte WebUI is not shipped, `src/main/cpp/webui_stub/ui.h` supplies the upstream **empty-asset** interface and leaves `LLAMA_UI_HAS_ASSETS` undefined (all static-asset-serving blocks compile out). `<cpp-httplib/httplib.h>` already resolves via `llama-common`'s `vendor/` include dir (same nlohmann/json 3.12.0 as the FetchContent copy). No SSL: `CPPHTTPLIB_OPENSSL_SUPPORT` is left undefined (plain-HTTP; bind localhost / front with a TLS proxy). **`server.cpp` is now compiled in too** (on non-Android — it and `server-tools.cpp` pull in `subprocess.h`/`posix_spawn_*`, so they share `server-models.cpp`'s Android guard): b9870 exposes its entry as `int llama_server(int, char**)` (no `main` in the file), and `patches/0006` makes it embeddable (no process signal handlers, forwarded-argv parse, out-of-band shutdown). The `NativeServer` JNI bridge (`src/main/cpp/native_server.cpp`) calls `llama_server` on a worker thread, so the **full** upstream server — WebUI and all — runs inside `libjllama`. See "Two server modes" below.
+- The upstream server library (`server-context.cpp`, `server-queue.cpp`, `server-task.cpp`, `server-schema.cpp`, `server-models.cpp`, and — since b9829 — `server-stream.cpp`) is compiled directly into `jllama` via CMake — there is no hand-ported `server.hpp` fork. **`server-stream.cpp` is mandatory, not optional:** it defines the resumable-streaming SSE replay buffer (`g_stream_sessions`, `stream_session_attach_pipe`, `stream_aware_should_stop`, `stream_conv_id_from_headers`, the `stream_pipe_*` types) that `server-context.cpp` / `server-http.cpp` / `server-models.cpp` now `#include "server-stream.h"` and call, so omitting it fails the link with undefined references. It is platform-neutral (threads + std mutex/condvar, no `subprocess.h`/`posix_spawn_*`), so it builds on Android too and sits outside the `server-models.cpp` Android guard. `jllama` wires its own JNI routes and never calls `g_stream_sessions.start_gc()` (only the excluded standalone `server.cpp` `main()` does), so its GC thread stays dormant. **Phase 2:** the upstream HTTP transport (`tools/server/server-http.cpp`) and its `cpp-httplib` backend (`vendor/cpp-httplib/httplib.cpp`) are now compiled into `jllama` too, so the OpenAI-compatible server can be driven natively from JNI *inside* `libjllama` — no separate `llama-server` executable (a JNI shared library loads anywhere a JVM runs, which a standalone binary does not). `server-http.cpp` does `#include "ui.h"` (the WebUI asset table that `tools/ui`/`llama-ui` normally generates); since the Svelte WebUI is not shipped, `src/main/cpp/webui_stub/ui.h` supplies the upstream **empty-asset** interface and leaves `LLAMA_UI_HAS_ASSETS` undefined (all static-asset-serving blocks compile out). `<cpp-httplib/httplib.h>` already resolves via `llama-common`'s `vendor/` include dir (same nlohmann/json 3.12.0 as the FetchContent copy). No SSL: `CPPHTTPLIB_OPENSSL_SUPPORT` is left undefined (plain-HTTP; bind localhost / front with a TLS proxy). **`server.cpp`, `server-tools.cpp` and `server-mcp.cpp` are now compiled in too** (on non-Android — they pull in `subprocess.h`/`posix_spawn_*`, so they share `server-models.cpp`'s Android guard): b9870 exposes `server.cpp`'s entry as `int llama_server(int, char**)` (no `main` in the file), and `patches/0006` makes it embeddable (no process signal handlers, forwarded-argv parse, out-of-band shutdown). **`server-mcp.cpp` is new in b10154** (upstream MCP-server support): both `server.cpp` (`llama_server`'s `mcp_mgr` lifecycle) and `server-tools.cpp` (`tools.setup(..., mcp_mgr)` / `server_mcp::call_tool`) reference `server_mcp`, so it **must** be in the `target_sources` list or the link fails with undefined `server_mcp::{start,shutdown,call_tool,list_tools,~server_mcp}` — **latent on Linux** (a shared object tolerates undefined symbols) but a **hard link error on macOS/ld64 and Windows/MSVC**. It is compiled only into `jllama`, not `jllama_test` (which links neither `server.cpp` nor `server-tools.cpp`). The `NativeServer` JNI bridge (`src/main/cpp/native_server.cpp`) calls `llama_server` on a worker thread, so the **full** upstream server — WebUI and all — runs inside `libjllama`. See "Two server modes" below.
 
 ### Two server modes (`OpenAiCompatServer` vs `NativeServer`)
 

@@ -429,19 +429,38 @@ endif()
 # entry point (server.cpp's `llama_server`, made embeddable by patches/0006) and its tools helper
 # (server-tools.cpp); jllama's JNI bridge (native_server.cpp) then calls llama_server on a worker
 # thread. This runs the *full* upstream HTTP server — WebUI included, every llama-server flag
-# forwarded — inside libjllama, with no separate llama-server executable. server.cpp and
-# server-tools.cpp both pull in vendor/sheredom/subprocess.h (posix_spawn_*), so they share the
-# non-Android guard used for server-models.cpp above; native_server.cpp links against llama_server
-# and is guarded too. On Android the NativeServer native methods are simply absent (its JNI calls
-# throw UnsatisfiedLinkError) — use OpenAiCompatServer there.
+# forwarded — inside libjllama, with no separate llama-server executable. server.cpp,
+# server-tools.cpp and server-mcp.cpp all pull in vendor/sheredom/subprocess.h (posix_spawn_*), so
+# they share the non-Android guard used for server-models.cpp above; native_server.cpp links against
+# llama_server and is guarded too. On Android the NativeServer native methods are simply absent (its
+# JNI calls throw UnsatisfiedLinkError) — use OpenAiCompatServer there.
+#
+# server-mcp.cpp is new in b10154 (upstream MCP-server support): both server.cpp (llama_server's
+# mcp_mgr lifecycle) and server-tools.cpp (tools.setup(..., mcp_mgr) / server_mcp::call_tool) now
+# reference server_mcp, so it MUST be compiled in or the link fails with undefined server_mcp
+# symbols. This is latent on Linux (a shared lib tolerates undefined symbols) but a hard link error
+# on macOS/ld64 and Windows. It is not compiled into jllama_test (that target links neither
+# server.cpp nor server-tools.cpp, so it never references server_mcp).
 if(NOT ANDROID_ABI AND NOT OS_NAME MATCHES "Android")
     target_sources(jllama PRIVATE
+        ${llama.cpp_SOURCE_DIR}/tools/server/server-mcp.cpp
         ${llama.cpp_SOURCE_DIR}/tools/server/server-tools.cpp
         ${llama.cpp_SOURCE_DIR}/tools/server/server.cpp
         ${CMAKE_SOURCE_DIR}/src/main/cpp/native_server.cpp
     )
 endif()
 
+# b10154's server-tools.cpp embeds a U+2192 (→) in a narrow string literal. MSVC (and clang-cl)
+# decode narrow literals in the system code page (1252 on the CI runners) and emit warning C4566
+# ("character ... cannot be represented in the current code page"), substituting the arrow at
+# runtime. /utf-8 makes the compiler read sources as UTF-8 and use a UTF-8 execution charset so the
+# arrow survives; it is the fix Microsoft documents for C4566. Scoped to jllama's own TUs and gated
+# on MSVC (true for both cl.exe and clang-cl; /utf-8 is invalid on gcc/clang), so non-Windows builds
+# are unaffected.
+if(MSVC)
+    target_compile_options(jllama PRIVATE /utf-8)
+endif()
+
 # Phase 2: also compile the upstream HTTP transport (server-http.cpp) and its
 # cpp-httplib backend directly into jllama, so the OpenAI-compatible server can be
 # driven natively from JNI — shipped inside libjllama, with no separate

@@ -0,0 +1,32 @@
+diff --git a/vendor/sheredom/subprocess.h b/vendor/sheredom/subprocess.h
+index 5e809023a..f980bf386 100644
+--- a/vendor/sheredom/subprocess.h
++++ b/vendor/sheredom/subprocess.h
+@@ -1203,10 +1203,27 @@ cleanup:
+   }
+   actions_created = 1;
+
++  /* [jllama] posix_spawn_file_actions_addchdir_np is a non-portable extension
++     (glibc >= 2.29, bionic API >= 34, macOS >= 10.15). On older glibc such as
++     manylinux2014 (glibc 2.17) the declaration is absent, which broke the
++     cross-compile build at b10154 (new vendored subprocess.h + common/subproc.cpp).
++     Detect it and fall back to reporting cwd-spawning as unsupported (ENOSYS)
++     instead of failing to compile. This build path never spawns with a cwd. */
++#if defined(__GLIBC__)
++#  if __GLIBC_PREREQ(2, 29)
++#    define SUBPROCESS_HAVE_ADDCHDIR_NP 1
++#  else
++#    define SUBPROCESS_HAVE_ADDCHDIR_NP 0
++#  endif
++#else
++#  define SUBPROCESS_HAVE_ADDCHDIR_NP 1
++#endif
+   // Set working directory
+   if (process_cwd) {
+ #if defined(__APPLE__) && MAC_OS_X_VERSION_MIN_REQUIRED >= 260000
+     posix_error = posix_spawn_file_actions_addchdir(&actions, process_cwd);
++#elif !SUBPROCESS_HAVE_ADDCHDIR_NP
++    posix_error = ENOSYS; /* [jllama] old glibc: addchdir_np unavailable */
+ #else
+ #if defined(__APPLE__) && defined(__clang__)
+ #pragma clang diagnostic push