Skip to content

feat!: send enable_thinking only when it was actually set - #199

Merged
bernardladenthin merged 2 commits into
mainfrom
claude/enable-thinking-tristate
Sep 1, 2026
Merged

feat!: send enable_thinking only when it was actually set#199
bernardladenthin merged 2 commits into
mainfrom
claude/enable-thinking-tristate

Conversation

@bernardladenthin

@bernardladenthin bernardladenthin commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Merge #200 first

This branch is rebased onto #200, which makes srcmorph/pom.xml parseable again — without it Maven cannot read the project at all, so nothing here can be built or tested. Once #200 lands, its commit drops out of this diff.

CI will still be red afterwards: main pins net.ladenthin:llama:5.2.0, which is not published yet. That is a known blocker on every srcmorph PR and unrelated to this change.

Summary

  • chatTemplateEnableThinking becomes a tri-state; the enable_thinking kwarg is sent only when it was actually set. It was a plain boolean defaulting to true, so every run handed the kwarg to the chat template — including templates that have never heard of it. llama.cpp's Jinja layer has been moving such unknown kwargs from "silently ignored" toward "warned about", and the only way to stop that noise was to set the knob to false, which means something else entirely.
  • The obvious phrasing of the fix is not implementable, and that shapes the design. "Send it only when it differs from the template's default" would require srcmorph to parse and evaluate the template — exactly the work it delegates to the binding. So the rule is "send it only when the user actually set it", which needs @Nullable Boolean rather than a cleverer boolean.
  • The kwarg assembly moves out of model() into a package-private buildChatTemplateKwargs(). This is what makes the behaviour testable at all: model() loads a GGUF, so nothing about the kwargs could be pinned without one.

What changed, concretely

AiGenerationConfig, AiModelDefinition and LlamaCppJniConfig (plus its builder) carry @Nullable Boolean; DEFAULT_CHAT_TEMPLATE_ENABLE_THINKING is null (unset) instead of true; both isChatTemplateEnableThinking() getters are renamed getChatTemplateEnableThinking() to match the wrapper type. true and false are both still forwarded verbatim — unset is the only value that omits the kwarg.

Two things found while resolving the rebase, both kept

  • A stale CHANGELOG claim from feat!: wire flashAttn up for real and follow the lazyMode rename #198. The [Unreleased] entry said the pin went to 5.2.0-SNAPSHOT, while the POM has said 5.2.0 since 5b4abeb. Corrected here rather than left as a knowingly false line.
  • My first assertion was wrong and the test failed. "A default config sends nothing" — no: reasoningEffort defaults to "low" and is sent on every run. Same shape of problem, but unlike enable_thinking it has an escape that means exactly "unset" (the empty string), so a non-gpt-oss user can switch it off without picking a value that says something else. Documented choice, not a second defect; the test javadoc records that instead of quietly widening this PR.

Test plan

  • Full reactor mvn test on the rebased branch: BUILD SUCCESS, 0 failures, 0 skipped in all three modules (srcmorph-cli 39, srcmorph-maven-plugin 32). Run with -Dllama.version=5.2.0-SNAPSHOT, the locally installed binding built from the state #408 merged — API-identical to 5.2.0, since #409 changed only spotbugs-exclude.xml.
  • PIT at mutationThreshold 100 on all three modules (pre-rebase run): srcmorph 777/777 — up from 775, the two new accessors generate two mutants and both are killed — srcmorph-cli 16/16, srcmorph-maven-plugin 62/62
  • Five model-free guards over buildChatTemplateKwargs(), including the true case — a later "send it only when false" shortcut would swallow a configured value, and now fails instead of passing silently
  • Rebase conflicts were both in docs; every Java file auto-merged. Verified afterwards that feat!: wire flashAttn up for real and follow the lazyMode rename #198's rename came through: no tensorReadLazy / TensorReadLazyMode left anywhere in the tree.
  • CI is green on this branch — no, see the banner

Related issues / PRs

Depends on #200. Closes the enable_thinking entry in TODO.md, removed here — announced during the 1.2.0 audit cycle and never landed; this is that.

Checklist

  • I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • My commits follow Conventional Commits (feat! + BREAKING CHANGE trailer)
  • No security-sensitive changes

main does not build at all -- not because of the unpublished 5.2.0 dependency,
but because the POM is not well-formed XML:

  Non-parseable POM .../srcmorph/pom.xml: in comment after two dashes (--)
  next character must be > not f (position: END_TAG seen ... (--f... @54:73)

XML forbids '--' inside a comment. The llama.version rationale I added in
5b4abeb spelled the CLI flags with their leading dashes (--flash-attn,
--tensor-read-lazy, --lazy-mode) and used an em-dash-as-two-hyphens, four
occurrences in one comment block. Maven cannot even read the project, so every
goal fails before dependency resolution is reached -- which is why this hid
behind the expected 5.2.0 failure.

The flags are named without the leading dashes and the punctuation dash is
replaced by a semicolon; no wording is lost.

Verified: all four reactor POMs now have no '--' inside any comment (checked by
regex over every <!-- --> block, not by eye), the file parses under a strict XML
parser, and 'mvn validate' exits 0.

My own miss: 5b4abeb edited a POM and I pushed it without running a single
Maven command against it.
chatTemplateEnableThinking was a plain boolean defaulting to true, so every
run put enable_thinking into the chat-template kwargs -- including runs whose
chat template has never heard of the kwarg. llama.cpp's Jinja layer has been
moving such unknown kwargs from "silently ignored" toward "warned about", and
the only way to stop that noise was to set the knob to false, which means
something else entirely.

The obvious phrasing of the fix -- "send it only when it differs from the
template's default" -- is not implementable: srcmorph would have to parse and
evaluate the template to know that default, which is exactly the work it
delegates to the binding. So the rule is "send it only when the user actually
set it", and that needs a tri-state.

AiGenerationConfig, AiModelDefinition and LlamaCppJniConfig (plus its builder)
now carry @nullable Boolean; DEFAULT_CHAT_TEMPLATE_ENABLE_THINKING is null
(unset) instead of true; the two isChatTemplateEnableThinking() getters become
getChatTemplateEnableThinking() to match the wrapper type. true and false are
both still forwarded verbatim -- unset is the only value that omits the kwarg.

The kwarg assembly moves out of model() into a package-private
buildChatTemplateKwargs(), which is what makes this testable at all: model()
loads a GGUF, so nothing about the kwargs could be pinned without one. Five
model-free guards now cover it, including the true case -- a later "send it
only when false" shortcut would swallow a configured value, and now fails
instead.

reasoningEffort is deliberately left alone. It also defaults to a value that
is sent on every run, but unlike enable_thinking it has an escape that means
exactly "unset" (the empty string), so it is a documented choice rather than
the same defect; the test javadoc records that rather than quietly widening
the change.

BREAKING CHANGE: chatTemplateEnableThinking is @nullable Boolean, not boolean,
and its getters are renamed to getChatTemplateEnableThinking(). A configuration
that never mentioned the knob keeps working and simply stops sending the kwarg;
one that set it explicitly is unaffected.
@claude

claude Bot commented Sep 1, 2026

Copy link
Copy Markdown

Review Summary

This is a high-quality PR that correctly implements the tri-state for chatTemplateEnableThinking. All changes are well-designed and thoroughly tested.

Strengths

Correct null-safety design: Using @Nullable Boolean properly represents the three states (unset/true/false)

Sound implementation: The buildChatTemplateKwargs() method correctly checks for null before dereferencing and converts the Boolean to String properly

Comprehensive test coverage: All five test cases cover critical boundaries:

  • Unset (null) → kwarg omitted
  • True → "true" sent
  • False → "false" sent
  • Blank reasoning effort → also omitted
  • Both kwargs together → both sent when configured

Type safety: @nullable annotations with NullAway enforcement ensure proper null handling throughout

Documentation: Excellent javadocs explaining design rationale and tri-state semantics

POM fix: XML comment issue (double-dash inside comment) correctly resolved

Notes

  • Breaking change (getter/setter rename from is* to get*) properly marked with feat! and documented
  • Backward compatibility: existing configs without this knob see the new behavior (no kwarg sent)
  • Factory pattern works correctly through the builder chain

No bugs or security issues identified. Ready to merge.

@bernardladenthin
bernardladenthin merged commit 31bdb70 into main Sep 1, 2026
19 of 27 checks passed
@bernardladenthin
bernardladenthin deleted the claude/enable-thinking-tristate branch September 1, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants