feat!: send enable_thinking only when it was actually set - #199
Conversation
main does not build at all -- not because of the unpublished 5.2.0 dependency, but because the POM is not well-formed XML: Non-parseable POM .../srcmorph/pom.xml: in comment after two dashes (--) next character must be > not f (position: END_TAG seen ... (--f... @54:73) XML forbids '--' inside a comment. The llama.version rationale I added in 5b4abeb spelled the CLI flags with their leading dashes (--flash-attn, --tensor-read-lazy, --lazy-mode) and used an em-dash-as-two-hyphens, four occurrences in one comment block. Maven cannot even read the project, so every goal fails before dependency resolution is reached -- which is why this hid behind the expected 5.2.0 failure. The flags are named without the leading dashes and the punctuation dash is replaced by a semicolon; no wording is lost. Verified: all four reactor POMs now have no '--' inside any comment (checked by regex over every <!-- --> block, not by eye), the file parses under a strict XML parser, and 'mvn validate' exits 0. My own miss: 5b4abeb edited a POM and I pushed it without running a single Maven command against it.
chatTemplateEnableThinking was a plain boolean defaulting to true, so every run put enable_thinking into the chat-template kwargs -- including runs whose chat template has never heard of the kwarg. llama.cpp's Jinja layer has been moving such unknown kwargs from "silently ignored" toward "warned about", and the only way to stop that noise was to set the knob to false, which means something else entirely. The obvious phrasing of the fix -- "send it only when it differs from the template's default" -- is not implementable: srcmorph would have to parse and evaluate the template to know that default, which is exactly the work it delegates to the binding. So the rule is "send it only when the user actually set it", and that needs a tri-state. AiGenerationConfig, AiModelDefinition and LlamaCppJniConfig (plus its builder) now carry @nullable Boolean; DEFAULT_CHAT_TEMPLATE_ENABLE_THINKING is null (unset) instead of true; the two isChatTemplateEnableThinking() getters become getChatTemplateEnableThinking() to match the wrapper type. true and false are both still forwarded verbatim -- unset is the only value that omits the kwarg. The kwarg assembly moves out of model() into a package-private buildChatTemplateKwargs(), which is what makes this testable at all: model() loads a GGUF, so nothing about the kwargs could be pinned without one. Five model-free guards now cover it, including the true case -- a later "send it only when false" shortcut would swallow a configured value, and now fails instead. reasoningEffort is deliberately left alone. It also defaults to a value that is sent on every run, but unlike enable_thinking it has an escape that means exactly "unset" (the empty string), so it is a documented choice rather than the same defect; the test javadoc records that rather than quietly widening the change. BREAKING CHANGE: chatTemplateEnableThinking is @nullable Boolean, not boolean, and its getters are renamed to getChatTemplateEnableThinking(). A configuration that never mentioned the knob keeps working and simply stops sending the kwarg; one that set it explicitly is unaffected.
19e5448 to
33a55f2
Compare
Review SummaryThis is a high-quality PR that correctly implements the tri-state for Strengths✅ Correct null-safety design: Using ✅ Sound implementation: The ✅ Comprehensive test coverage: All five test cases cover critical boundaries:
✅ Type safety: @nullable annotations with NullAway enforcement ensure proper null handling throughout ✅ Documentation: Excellent javadocs explaining design rationale and tri-state semantics ✅ POM fix: XML comment issue (double-dash inside comment) correctly resolved Notes
No bugs or security issues identified. Ready to merge. |
Merge #200 first
This branch is rebased onto #200, which makes
srcmorph/pom.xmlparseable again — without it Maven cannot read the project at all, so nothing here can be built or tested. Once #200 lands, its commit drops out of this diff.CI will still be red afterwards:
mainpinsnet.ladenthin:llama:5.2.0, which is not published yet. That is a known blocker on every srcmorph PR and unrelated to this change.Summary
chatTemplateEnableThinkingbecomes a tri-state; theenable_thinkingkwarg is sent only when it was actually set. It was a plainbooleandefaulting totrue, so every run handed the kwarg to the chat template — including templates that have never heard of it. llama.cpp's Jinja layer has been moving such unknown kwargs from "silently ignored" toward "warned about", and the only way to stop that noise was to set the knob tofalse, which means something else entirely.@Nullable Booleanrather than a clevererboolean.model()into a package-privatebuildChatTemplateKwargs(). This is what makes the behaviour testable at all:model()loads a GGUF, so nothing about the kwargs could be pinned without one.What changed, concretely
AiGenerationConfig,AiModelDefinitionandLlamaCppJniConfig(plus its builder) carry@Nullable Boolean;DEFAULT_CHAT_TEMPLATE_ENABLE_THINKINGisnull(unset) instead oftrue; bothisChatTemplateEnableThinking()getters are renamedgetChatTemplateEnableThinking()to match the wrapper type.trueandfalseare both still forwarded verbatim — unset is the only value that omits the kwarg.Two things found while resolving the rebase, both kept
[Unreleased]entry said the pin went to5.2.0-SNAPSHOT, while the POM has said5.2.0since5b4abeb. Corrected here rather than left as a knowingly false line.reasoningEffortdefaults to"low"and is sent on every run. Same shape of problem, but unlikeenable_thinkingit has an escape that means exactly "unset" (the empty string), so a non-gpt-oss user can switch it off without picking a value that says something else. Documented choice, not a second defect; the test javadoc records that instead of quietly widening this PR.Test plan
mvn teston the rebased branch:BUILD SUCCESS, 0 failures, 0 skipped in all three modules (srcmorph-cli39,srcmorph-maven-plugin32). Run with-Dllama.version=5.2.0-SNAPSHOT, the locally installed binding built from the state #408 merged — API-identical to 5.2.0, since #409 changed onlyspotbugs-exclude.xml.mutationThreshold100 on all three modules (pre-rebase run):srcmorph777/777 — up from 775, the two new accessors generate two mutants and both are killed —srcmorph-cli16/16,srcmorph-maven-plugin62/62buildChatTemplateKwargs(), including thetruecase — a later "send it only when false" shortcut would swallow a configured value, and now fails instead of passing silentlytensorReadLazy/TensorReadLazyModeleft anywhere in the tree.Related issues / PRs
Depends on #200. Closes the
enable_thinkingentry inTODO.md, removed here — announced during the 1.2.0 audit cycle and never landed; this is that.Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.mdfeat!+BREAKING CHANGEtrailer)