Feature/49 tool calling pipeline oom#51

Merged

michalharakal merged 9 commits intodevelopfrom

feature/49-tool-calling-pipeline-oom

Apr 11, 2026

Commits on Apr 11, 2026

fix: remove pre-transpose in LlamaRuntime to halve peak memory

michalharakal
and
claude
committed
perf: keep Q4_K weights quantized using native SIMD matmul kernel

michalharakal
and
claude
committed
fix: add Q4_K SIMD matmul and quantized-aware linear projection

michalharakal
and
claude
committed
fix: pre-transpose weights during loading to eliminate runtime OOM

michalharakal
and
claude
committed
fix: correct Q4_K/Q6_K pre-transpose for valid 8B model output

michalharakal
and
claude
committed
docs: add deep technical guide on weight quantization pipeline

michalharakal
and
claude
committed
feat: add instruct mode to smoke tests for chat-tuned models

michalharakal
and
claude
committed
fix: replace JVM-only Math.PI with kotlin.math.PI in VoxtralFlowMatching

michalharakal
and
claude
committed
fix: fix Antora Docker image and playbook for local builds

michalharakal
and
claude
committed