llama.cpp : bump version to 0.4.0

[ggerganov]

Overview

llama.cpp 0.4.0 adds initial Qwen3.8-Flash-Next and Nemotron-3-Puzzle support, on-demand tensor reading, per-slot server context limits, video input options, and a ggml update to 0.23.0 with major sparse flash attention and RDMA work.

API changes

• Added llama_lazy_mode and lazy_mode (#27794).
• Added max_buf_size to quantize params (#27795).
• Bumped session/state versions for KV-cell token tracking (#27762).
• Added mtmd_input_part and mtmd_tokenize_from_parts() (#28250).
• Made mtmd_tokenize const-correct (#28310).
• Made mtmd_bitmap_init_lazy const-correct (#28307).
• Added mtmd helper video/init options (#24318).
• Server multimodal helpers accept mtmd_helper_init_opt (#24318).

New models

• Added initial Qwen3.8-Flash-Next (qwen4exp) architecture support; optimization improvements are still pending (#27742).
• Added NVIDIA Nemotron-3-Puzzle-75B-A9B support (#25444).
• Added DSpark support for Nemotron 3.5 (#27804).
• Added support for nanbeige4.2-3B (#27730).

Core changes

• Added lazy tensor reading (#27794).
• Renamed option to --lazy-mode (#27969).
• Added --n-cpu-ffn (#26622).
• Added per-layer expert routing/FFN (#28323).
• Added KV-cell token tracking (#27762).
• Added n-gram history lookup (#28040).
• Optimized KV cache restore (#27991).
• Stopped early sequence scans (#28011).
• Added quantizer RAM cap (#27795).
• Added quantizer row-slab streaming (#27830).
• Prevented RAM peaking during load (#27483).
• Autoscaled YaRN training context (#28030).
• Disabled non-fused GDN/LID (#27877).
• Simplified MiniMax-01 graph (#27790).
• Added DFlash2 support (#27816).
• Fused DFlash encoder into KV injection (#27310).
• Added sparse flash attention for DeepSeek-V4/GLM and Qwen4exp (#27970).
• Added Apple RDMA as an RPC transport (#26421).
• Added RPC event/async backend APIs (#18626).
• Fixed Qwen4exp graph issues (#27941).
• Added Qwen4exp recurrent rollback (#28123).
• Summed Qwen4exp indexer heads (#28023).
• Reduced Qwen4exp graph splits (#27880).
• Fixed DeepSeek-V4 input vision (#28154).
• Fixed Gemma-4 vision handling (#28335).
• Fixed Gemma-4 assistant (#28183).
• Added synthetic spec acceptance options (#27711).
• Made build info stream configurable (#28322).
• Fixed JSON-schema empty object grammar (#28279).

Multi-modality changes

• Added mtmd_tokenize_from_parts() (#28250).
• Added DeepSeek-V4-Flash-Vision-Exp (#28133).
• Added DeepSeek-V4 input vision handling (#28154).
• Fixed Gemma-4 vision handling (#28335).
• Fixed Idefics3 preprocessing (#28273).
• Fixed Qwen3-TTS (#28231).
• Added video parameters/--video-* (#24318).
• Made image preprocessors const-correct (#28310).
• Made audio preprocessors const-correct (#28307).

Server changes

• Added per-slot context limit (#24124).
• Accept data: URLs for media (#27735).
• Enabled preserve_reasoning by default (#28174).
• Reject prefilled assistant tool calls (#27626).
• Added synthetic spec acceptance options (#27711).
• Added pytest-xdist server tests (#28298).

UI changes

• Improved Chat Form Actions UI/UX (#27746).
• Replaced MCP overrides with tool policy (#27745).
• Moved Settings/MCP routes to dialogs (#27744).
• Updated dialog styling (#27743).
• Copy grouped agentic response text (#27832).
• Updated ESLint config (#27700).
• Updated package scripts (#28270).

ggml changes

• Bumped ggml from v0.22.0 to v0.23.0 (release).

Version 0.23.0 adds new operators and backend scheduling APIs for sparse attention, asynchronous execution, and allocation-dependency tracking. The most notable additions are sparse flash attention (ggml_flash_attn_ext_set_n_kv_max), RPC event/async APIs, and Apple RDMA transport support. It also includes broad correctness and performance work across CPU, GPU, and accelerator backends.