Skip to content

fix(config): hold the concise response cap to the shared-window reservation - #520

Merged
devops-thiago merged 1 commit into
release/v0.6.0from
fix/517-concise-cap-guard
Aug 11, 2026
Merged

fix(config): hold the concise response cap to the shared-window reservation#520
devops-thiago merged 1 commit into
release/v0.6.0from
fix/517-concise-cap-guard

Conversation

@devops-thiago

@devops-thiago devops-thiago commented Aug 10, 2026

Copy link
Copy Markdown
Owner

What type of PR is this?

  • 🐛 Bug fix
  • ✨ Feature
  • 📝 Documentation
  • 🔧 Refactor
  • 🚀 Performance
  • ✅ Test
  • 🔒 Security
  • 📦 Dependency update
  • 🏗️ CI/CD

Description

Post-#507, the summary/verifier/reply calls send REVIEW_CONCISE_MAX_OUTPUT_TOKENS as max_tokens, but the only boot check on that value was >= 1 — while the summary prompt is still packed against the same shared window with only the output buffer reserved. A concise cap above the effective buffer on a shared-window model booted cleanly into exactly the state the active-model rule exists to refuse: on providers that validate prompt + max_tokens <= context, the summary call 400s deterministically, retries maxAiRetries times, and the fully paid multi-batch review is discarded (audit report AUDIT3-B, finding 1).

validateConciseResponseCap now applies the same shared-window reservation rule as the active model's own response cap (validateEffectiveBudget): when token budgeting is on and the active model does not declare separate-output-budget, a concise cap above reservedOutputTokens is refused at boot. The refusal message names REVIEW_CONCISE_MAX_OUTPUT_TOKENS and every fix option in the existing rule's style: lower the cap, raise REVIEW_OUTPUT_BUFFER_TOKENS to cover it, or set thrillhousebot.ai.models."<model>".separate-output-budget=true when the model's response allowance is independent of its input window. The rule is skipped when budgeting is off (no packed prompt to overrun) or on a separate-output-budget model (nothing is reserved) — so the shipped deepseek-v4-flash shape keeps booting, and an empty cap (provider default) stays allowed, consistent with the uncapped-active-model policy.

Shipped defaults are unaffected: concise 8192 == buffer 8192 and the rule is strict-greater. A guard test walks the entire shipped model table from application.properties with each entry active (the #502 lesson: the rule only fires for the active model, so a bad shipped combination would pass every fixed-model test and refuse boot only for deployments naming that model).

Scope: StartupConfigValidator.validateConciseResponseCap and StartupConfigValidatorTest only. Two existing tests that override the active model's output buffer below the default concise cap (bootsWhenAModelSettingsEntryIsValid, bootsWhenAPerModelOverrideRepairsABrokenGlobalCombination) now align their concise cap with the shrunken reservation — that combination is precisely the overrun state the audit calls out ("or lowers the buffer below the concise default 8192"), and those tests pin per-model resolution, not the concise rule.

Related Issues

Fixes #517

How Has This Been Tested?

  • Unit tests
  • Integration tests
  • Manual testing

Red/green proof — failsFastWhenTheConciseCapExceedsTheBufferOnASharedWindow (derived from the audit probe Audit3BConciseCapGuardProbeTest.theSameValueOnTheConciseCapBootsWithoutObjection: the identical 384000 that is refused on the active cap boots without objection on the concise cap) fails on unfixed 5c04600 in exactly the claimed way:

[ERROR] dev.thiagogonzaga.thrillhousebot.config.StartupConfigValidatorTest.failsFastWhenTheConciseCapExceedsTheBufferOnASharedWindow -- Time elapsed: 0.030 s <<< FAILURE!
org.opentest4j.AssertionFailedError: Expected dev.thiagogonzaga.thrillhousebot.config.ConfigValidationException to be thrown, but nothing was thrown.
	at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3234)
	at dev.thiagogonzaga.thrillhousebot.config.StartupConfigValidatorTest.assertFailsValidation(StartupConfigValidatorTest.java:232)
	at dev.thiagogonzaga.thrillhousebot.config.StartupConfigValidatorTest.failsFastWhenTheConciseCapExceedsTheBufferOnASharedWindow(StartupConfigValidatorTest.java:356)

With the fix it passes, refusing boot with:

- the effective output buffer (8192) must be >= REVIEW_CONCISE_MAX_OUTPUT_TOKENS (384000, quarkus.langchain4j.openai.concise.chat-model.max-tokens) for model 'deepseek-chat' so the token budget reserves the response cap the summary/verifier/reply calls send. Lower REVIEW_CONCISE_MAX_OUTPUT_TOKENS, raise REVIEW_OUTPUT_BUFFER_TOKENS to cover it, or set thrillhousebot.ai.models."deepseek-chat".separate-output-budget=true if this model's response allowance is independent of its input window.

New coverage, mirroring the active rule's pinning tests:

  • failsFastWhenTheConciseCapExceedsTheBufferOnASharedWindow — refusal, message names the env var, the buffer key, and the separate-output-budget escape hatch.
  • allowsAConciseCapAboveTheBufferWhenTheOutputBudgetIsSeparate — the deepseek-v4-flash shape (384000 out, 8192 buffer, separate budget) keeps booting.
  • allowsAConciseCapAboveTheBufferWhenTokenBudgetingIsDisabled — budgeting off skips the rule, same as the active rule.
  • holdsTheConciseCapToTheActiveModelsEffectiveBuffer — the reservation compared against is the active model's resolved buffer (per-model override respected).
  • ShippedDefaults.everyShippedModelBootsUnderTheShippedConciseCap — walks all 18 shipped model entries from application.properties (no env source, shipped ${VAR:default} values) with each one active; every one must boot under the shipped concise cap.

Gates: spotless:apply clean, clean compile spotbugs:check spotless:check green (BugInstance size 0), full clean test suite green (Tests run: 2571, Failures: 0, Errors: 0, Skipped: 0). JaCoCo ∩ diff: zero missed lines and zero missed branches in the changed main code.

Checklist

  • My code follows the project's coding standards
  • I have performed a self-review of my own code
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the documentation accordingly
  • My changes generate no new warnings or errors

Screenshots / Logs

See the red/green proof above.

Additional Notes

Latent (misconfiguration-triggered) per the audit: no correct configuration behaves worse than before #507, and default deployments were never exposed. No behavior change for separate-output-budget models, disabled budgeting, or an unset cap.

…vation

Post-#507 the summary/verifier/reply calls send
REVIEW_CONCISE_MAX_OUTPUT_TOKENS as max_tokens against the same shared
window the budgeter packed to budget - buffer, but the only boot check on
that value was >= 1 — so a concise cap above the effective buffer booted
cleanly into exactly the state the active-model rule refuses, and on
providers that validate prompt + max_tokens <= context the summary call
fails deterministically after the multi-batch review was already paid for.

validateConciseResponseCap now applies the same shared-window reservation
rule as the active model's own response cap: with token budgeting on and no
separate-output-budget declared, a concise cap above reservedOutputTokens
is refused at boot. The refusal names REVIEW_CONCISE_MAX_OUTPUT_TOKENS and
every way out (lower the cap, raise REVIEW_OUTPUT_BUFFER_TOKENS, or set
separate-output-budget=true) in the existing rule's style. Skipped when
budgeting is off or the active model has a separate output budget, so the
shipped deepseek-v4-flash shape keeps booting; an empty cap stays allowed.

A guard test walks the whole shipped model table from application.properties
with each entry active, so a shipped combination the rule would refuse can
never ship unseen.

Fixes #517
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@sonarqubecloud

Copy link
Copy Markdown

@thrillhousebot

Copy link
Copy Markdown

🤖 ThrillhouseBot PR Summary

What this PR does

Adds a boot-time validation in StartupConfigValidator that refuses a REVIEW_CONCISE_MAX_OUTPUT_TOKENS above the active shared-window model's reserved output buffer, mirroring the existing active-model response-cap rule, and skips the check when token budgeting is off or the model declares a separate output budget. Extends StartupConfigValidatorTest with five new tests, including a walk of all shipped model defaults with each entry active, and adjusts two existing tests whose per-model buffer override now falls below the default concise cap.

Control-Flow Diagram

🔀 Show diagram
flowchart TD
  A["StartupConfigValidator boot"] --> B["validateConciseResponseCap"]
  B --> C{"activeModel.maxInputTokens greater than 0?"}
  C -- "no - budgeting off" --> OK["boot proceeds without concise reservation check"]
  C -- "yes" --> D{"activeModel.separateOutputBudget is true?"}
  D -- "yes - nothing reserved" --> OK
  D -- "no - shared window" --> E{"concise cap greater than reservedOutputTokens?"}
  E -- "no - cap fits buffer" --> OK
  E -- "yes - overrun" --> F["add refusal naming concise env var, buffer key, escape hatch"]
  F --> G["problems non-empty - ConfigValidationException"]
Loading

Changes Overview

  • Files changed: 2
  • Lines added: +194
  • Lines removed: -1

Changed Files

File Change Summary
src/main/java/dev/thiagogonzaga/thrillhousebot/config/StartupConfigValidator.java Modified Adds shared-window reservation check for the concise response cap, mirroring validateEffectiveBudget, with separate-budget and budgeting-off escapes.
src/test/java/dev/thiagogonzaga/thrillhousebot/config/StartupConfigValidatorTest.java Modified Adds five concise-cap rule tests including a shipped-defaults walk; aligns two existing tests with the new rule.

Risk Assessment

Risk Count
🔴 Critical 0
🟠 High 0
🟡 Medium 0
🔵 Low 0

No new issues found in this PR, but the review cannot be approved until CI is confirmed green.

⚠️ CI Checks Status

Some checks are still pending or have failed:

Check Type Status Detail
frontend check-run ⏳ Pending -
changes check-run ⏳ Pending -
format check-run ⏳ Pending -
test check-run ⏳ Pending -
trivy check-run ⏳ Pending -
actionlint check-run ⏳ Pending -
dependency-review check-run ⏳ Pending -

Automated review by ThrillhouseBot. Reply with /review to re-run.

@thrillhousebot thrillhousebot Bot added bug Something isn't working java Pull requests that update java code testing Test coverage and test quality labels Aug 10, 2026
@devops-thiago
devops-thiago merged commit 93e9ebc into release/v0.6.0 Aug 11, 2026
14 checks passed
@devops-thiago
devops-thiago deleted the fix/517-concise-cap-guard branch August 11, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working java Pull requests that update java code testing Test coverage and test quality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant