Skip to content

fix: work-on-task-assistant states findings, not conclusions - #299

Merged
bborbe merged 1 commit into
masterfrom
fix/agent-output-findings-not-conclusions
Oct 17, 2026
Merged

bborbe merged 1 commit into
masterfrom
fix/agent-output-findings-not-conclusions

Conversation

@bborbe

@bborbe bborbe commented Oct 17, 2026

Copy link
Copy Markdown
Owner

Problem

vault-cli:work-on-task-assistant is dispatched to find a task and its guides, but it also runs its own corroboration analysis and states conclusions from evidence it merely grepped. On 2026-10-17 it added an unsanctioned 🔬 Independent read-only corroboration block asserting:

  • the router log held "exactly one request" in a window that held 16 [req] lines (five PIDs across four builds)
  • "the absence of any worker-attributable line means the request did not arrive"

The second is the opposite of the truth for that router: it logs non-200s unconditionally but sampler-gates successful requests to at most one per 10 s, so absence is the signature of success. Taken at face value, the finding would have inverted the diagnosis the owning session was producing — and the session had to re-derive both claims before it could proceed.

Change

Two rules added immediately above <output_format> in agents/work-on-task-assistant.md:

  1. The block list below is closed — render the bracketed blocks that apply and nothing else; hand back command output inside the block it belongs to, attributed to the command that produced it.
  2. Findings, not conclusions — state what a command printed, never what it means; mark any inference (inference — unverified). A log the agent did not write may be sampled, truncated, or shared by concurrent writers, and its silence rarely means absence.

The dated precedent is recorded inline, matching the file's existing convention.

Notes

  • Markdown / agent-definition change → Direct per this repo's CLAUDE.md § Choosing a Flow, on a feature worktree as that section requires.
  • make precommit passes (exit 0).
  • No hand-made version bump: this repo is autoRelease: true, so the change adds an ## Unreleased bullet and the releaser owns the cut.

@bborbe
bborbe marked this pull request as ready for review October 17, 2026 10:18

@ben-s-pull-request-reviewer ben-s-pull-request-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review: fix: work-on-task-assistant states findings, not conclusions


Must Fix (Critical)

None.

Should Fix (Important)

None.

Nice to Have (Optional)

None.


Selector Mode Traceability

  • Candidates: 0 rules matched by Step 4b-i glob filter
  • Applicable: 0 rules selected by Step 4c-sel
  • Mechanicals: 0 findings (ast-grep run over CHANGELOG.md, agents/work-on-task-assistant.md)
  • Result: selector clean — no adjudication needed

Notes:

  • Changed files are CHANGELOG.md (+4) and agents/work-on-task-assistant.md (+4) — both markdown, no Go source.
  • The two additions to the agent prompt are correct and sufficient:
    1. Block list closure (line 431): "The block list below is closed. Render the bracketed blocks that apply and nothing else — never invent a section."
    2. Findings not conclusions (line 433): "State what a command printed; never what it means." with explicit prohibition on characterizing unsampled logs and required (inference — unverified) markup for any inference stated.
  • Both rules directly address the documented failure mode (unsanctioned 🔬 Independent read-only corroboration block asserting facts opposite to what the log showed).
  • CHANGELOG.md entry is thorough and accurately describes the bug and its fix.
  • No unsanctioned corroboration block or inference-marking violations exist in the changed code.

{
  "verdict": "approve",
  "summary": "PR adds two rules to the work-on-task-assistant agent prompt: a closed block list declaration and a findings-not-conclusions directive that directly address the reported failure. Changes are minimal, targeted, and correct. No mechanical findings, no applicable judgment rules.",
  "comments": [],
  "concerns_addressed": [
    {
      "concern": "correctness: agent states conclusions rather than findings — unsanctioned corroboration block asserts router log held 'exactly one request' when it held 16, and that absence of worker lines meant request did not arrive (opposite of truth)",
      "disposition": "addressed",
      "detail": "agents/work-on-task-assistant.md:431 adds closed block list declaration; :433 adds findings-not-conclusions rule requiring evidence to be stated as findings only, with (inference — unverified) markup for any characterization"
    },
    {
      "concern": "correctness: agent may invent report sections not sanctioned by the prompt",
      "disposition": "addressed",
      "detail": "agents/work-on-task-assistant.md:431 declares the block list closed: 'never invent a section'"
    }
  ]
}

@bborbe
bborbe merged commit a1e0196 into master Oct 17, 2026
5 checks passed
@bborbe
bborbe deleted the fix/agent-output-findings-not-conclusions branch October 17, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant