fix(config): hold the concise response cap to the shared-window reservation - #520
Conversation
…vation Post-#507 the summary/verifier/reply calls send REVIEW_CONCISE_MAX_OUTPUT_TOKENS as max_tokens against the same shared window the budgeter packed to budget - buffer, but the only boot check on that value was >= 1 — so a concise cap above the effective buffer booted cleanly into exactly the state the active-model rule refuses, and on providers that validate prompt + max_tokens <= context the summary call fails deterministically after the multi-batch review was already paid for. validateConciseResponseCap now applies the same shared-window reservation rule as the active model's own response cap: with token budgeting on and no separate-output-budget declared, a concise cap above reservedOutputTokens is refused at boot. The refusal names REVIEW_CONCISE_MAX_OUTPUT_TOKENS and every way out (lower the cap, raise REVIEW_OUTPUT_BUFFER_TOKENS, or set separate-output-budget=true) in the existing rule's style. Skipped when budgeting is off or the active model has a separate output budget, so the shipped deepseek-v4-flash shape keeps booting; an empty cap stays allowed. A guard test walks the whole shipped model table from application.properties with each entry active, so a shipped combination the rule would refuse can never ship unseen. Fixes #517
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
🤖 ThrillhouseBot PR SummaryWhat this PR doesAdds a boot-time validation in StartupConfigValidator that refuses a REVIEW_CONCISE_MAX_OUTPUT_TOKENS above the active shared-window model's reserved output buffer, mirroring the existing active-model response-cap rule, and skips the check when token budgeting is off or the model declares a separate output budget. Extends StartupConfigValidatorTest with five new tests, including a walk of all shipped model defaults with each entry active, and adjusts two existing tests whose per-model buffer override now falls below the default concise cap. Control-Flow Diagram🔀 Show diagramflowchart TD
A["StartupConfigValidator boot"] --> B["validateConciseResponseCap"]
B --> C{"activeModel.maxInputTokens greater than 0?"}
C -- "no - budgeting off" --> OK["boot proceeds without concise reservation check"]
C -- "yes" --> D{"activeModel.separateOutputBudget is true?"}
D -- "yes - nothing reserved" --> OK
D -- "no - shared window" --> E{"concise cap greater than reservedOutputTokens?"}
E -- "no - cap fits buffer" --> OK
E -- "yes - overrun" --> F["add refusal naming concise env var, buffer key, escape hatch"]
F --> G["problems non-empty - ConfigValidationException"]
Changes Overview
Changed Files
Risk Assessment
No new issues found in this PR, but the review cannot be approved until CI is confirmed green.
|
| Check | Type | Status | Detail |
|---|---|---|---|
| frontend | check-run | ⏳ Pending | - |
| changes | check-run | ⏳ Pending | - |
| format | check-run | ⏳ Pending | - |
| test | check-run | ⏳ Pending | - |
| trivy | check-run | ⏳ Pending | - |
| actionlint | check-run | ⏳ Pending | - |
| dependency-review | check-run | ⏳ Pending | - |
Automated review by ThrillhouseBot. Reply with /review to re-run.



What type of PR is this?
Description
Post-#507, the summary/verifier/reply calls send
REVIEW_CONCISE_MAX_OUTPUT_TOKENSasmax_tokens, but the only boot check on that value was>= 1— while the summary prompt is still packed against the same shared window with only the output buffer reserved. A concise cap above the effective buffer on a shared-window model booted cleanly into exactly the state the active-model rule exists to refuse: on providers that validateprompt + max_tokens <= context, the summary call 400s deterministically, retriesmaxAiRetriestimes, and the fully paid multi-batch review is discarded (audit report AUDIT3-B, finding 1).validateConciseResponseCapnow applies the same shared-window reservation rule as the active model's own response cap (validateEffectiveBudget): when token budgeting is on and the active model does not declareseparate-output-budget, a concise cap abovereservedOutputTokensis refused at boot. The refusal message namesREVIEW_CONCISE_MAX_OUTPUT_TOKENSand every fix option in the existing rule's style: lower the cap, raiseREVIEW_OUTPUT_BUFFER_TOKENSto cover it, or setthrillhousebot.ai.models."<model>".separate-output-budget=truewhen the model's response allowance is independent of its input window. The rule is skipped when budgeting is off (no packed prompt to overrun) or on a separate-output-budget model (nothing is reserved) — so the shipped deepseek-v4-flash shape keeps booting, and an empty cap (provider default) stays allowed, consistent with the uncapped-active-model policy.Shipped defaults are unaffected: concise 8192 == buffer 8192 and the rule is strict-greater. A guard test walks the entire shipped model table from
application.propertieswith each entry active (the #502 lesson: the rule only fires for the active model, so a bad shipped combination would pass every fixed-model test and refuse boot only for deployments naming that model).Scope:
StartupConfigValidator.validateConciseResponseCapandStartupConfigValidatorTestonly. Two existing tests that override the active model's output buffer below the default concise cap (bootsWhenAModelSettingsEntryIsValid,bootsWhenAPerModelOverrideRepairsABrokenGlobalCombination) now align their concise cap with the shrunken reservation — that combination is precisely the overrun state the audit calls out ("or lowers the buffer below the concise default 8192"), and those tests pin per-model resolution, not the concise rule.Related Issues
Fixes #517
How Has This Been Tested?
Red/green proof —
failsFastWhenTheConciseCapExceedsTheBufferOnASharedWindow(derived from the audit probeAudit3BConciseCapGuardProbeTest.theSameValueOnTheConciseCapBootsWithoutObjection: the identical 384000 that is refused on the active cap boots without objection on the concise cap) fails on unfixed5c04600in exactly the claimed way:With the fix it passes, refusing boot with:
New coverage, mirroring the active rule's pinning tests:
failsFastWhenTheConciseCapExceedsTheBufferOnASharedWindow— refusal, message names the env var, the buffer key, and the separate-output-budget escape hatch.allowsAConciseCapAboveTheBufferWhenTheOutputBudgetIsSeparate— the deepseek-v4-flash shape (384000 out, 8192 buffer, separate budget) keeps booting.allowsAConciseCapAboveTheBufferWhenTokenBudgetingIsDisabled— budgeting off skips the rule, same as the active rule.holdsTheConciseCapToTheActiveModelsEffectiveBuffer— the reservation compared against is the active model's resolved buffer (per-model override respected).ShippedDefaults.everyShippedModelBootsUnderTheShippedConciseCap— walks all 18 shipped model entries fromapplication.properties(no env source, shipped${VAR:default}values) with each one active; every one must boot under the shipped concise cap.Gates:
spotless:applyclean,clean compile spotbugs:check spotless:checkgreen (BugInstance size 0), fullclean testsuite green (Tests run: 2571, Failures: 0, Errors: 0, Skipped: 0). JaCoCo ∩ diff: zero missed lines and zero missed branches in the changed main code.Checklist
Screenshots / Logs
See the red/green proof above.
Additional Notes
Latent (misconfiguration-triggered) per the audit: no correct configuration behaves worse than before #507, and default deployments were never exposed. No behavior change for separate-output-budget models, disabled budgeting, or an unset cap.