Repository navigation
P0: Reproduce and prevent verified Goal completion without an artifact-backed closure receipt (#279) #281
Description
Source / impact
A real multi-agent user reported on field report #279 that on 2026-10-18 a Goal displayed completed although the scaffold was cut and target files were untouched. This is user-reported and not yet independently reproduced; do not assume every completed Goal lacks evidence.
Investigation
- Trace V1 and native V2 routes from the assistant control tool and configured checks through
completeGoal,mergeAuditEvaluation,settleCurrentProgress, persistence and the rendered status. - Distinguish status "completed", claimed semantic evidence, successful host checks, and actual project artifacts; compare requirement-to-proof linking, active revision, verifier session provenance and snapshot freshness after concurrent writers.
- Do not accept a model statement "done", a TODO checkbox change or a provider response as artifact proof on its own.
Acceptance / failing regression first
- Deterministic fixture: zero actual target-file changes, claims of complete, and outdated semantic/progress evidence -> never show verified completion; capture actionable reason and unchanged active/paused state.
- Positive fixture: required files/content and command outputs actually meet the declared contracts, independent verifier agrees and no newer steering/revision exists -> completed with durable closure receipt.
- Closure receipt includes Goal ID/revision, requirement-to-evidence mapping, commands with exit code/timestamp and hashes/paths for validated artifacts (safely redacted), and clearly separates inferred semantic verdict from host observations.
- Check V1 legacy, native V2, replay/restart, moving OpenCode 2.1.4 and dev, concurrent write/revision race, and zero-config objectives. Receipt must be read-only, not new completion authority.
- Link to #275 / PR #276 and PR #280, but do not combine an unverified wide concurrency refactor with completion proof changes.
Out of scope
Do not automatically mark previously completed Goals as fraudulent; capture a real reporter reproduction before making general claims.
Activity
ByBrawe commentedon Oct 27, 2026
Source audit note:
src/verification/audit.tscurrently rejects required requirements lacking current trusted evidence, andsrc/verification/semantic.tsrejects aprovenverdict with zero file/host evidence references; Goal already includes abenchmarks/oracles/false-complete-trap.mjsfixture. These safeguards do not prove the 2026-10-18 field report impossible: the reporter specifically describes a completed state with untouched targets. Need run a field-shaped fixture and inspect verifier evidence corroboration, file hashes/quotes and Goal revision/status/history for mismatch or stale-persisted state. Do not claim reproduced or resolved; propose a durable evidence-linked closure receipt and an adversarial verifier case.fix(goal): reject runtime counters as proof of generic deliverables (#…
test(goal): reject synthetic runtime-only completion; preserve corrob…
test(goal): foreground verifier transport cannot turn generic Goal co…
test(goal): full semantic audit refuses synthetic done without projec…
fix(goal): require host file/command evidence to actually belong to t…
test(goal): unrelated host check cannot certify a generic Goal requir…
ByBrawe commentedon Oct 27, 2026
2026-10-27 source-level root-cause candidate and first fix: PR #286 demonstrates a concrete over-acceptance path.
semanticVerifierHostEvidence()supplies host-passing runtime turn/progress IDs even for zero deliverables; before the patchguardSemanticProcessResults()returnedprovenunchanged for every generic non-N-turn semantic objective. A verifier citing ONLYgoal-runtime-turns-r1could therefore satisfyapplySemanticVerifierResultsandcompleteGoal, without a single file quote or command check. An existing verifier transport test previously expected genericprovenwith only this counter. The new fail-closed guard requires actual host-corroborated file excerpts or a passing current, requirement-linked host command/file check; N-turn process obligations retain runtime counting. Added regression showing a generic Goal with zero artifacts cannot persistcompleted. This is source-proven risk, not yet proof it caused the field reporter's 2026-10-18 incident. Full Windows/Linux CI is pending. Still need artifact-hash closure receipts and field replay before closing #281.fix(goal): allow valid cross-requirement host file proof, still rejec…
test(goal): preserve cross-requirement host proof, reject uncited che…
test(goal): use real host README file-contract evidence, never runtim…
fix(goal): distinguish explicit deliverables, file targets and deleti…
fix(goal): host-verify deleted path absence and recheck after verifie…
fix(goal): include independent read-only file absence checks in verif…
fix(goal): require target-linked proof for mixed process goals, unrel…
test(goal): stop mixed-cadence and unrelated-check false completions;…
fix(goal): align V1/V2 semantic verifier absence proof paths and regr…
fix(goal): align V1/V2 semantic verifier absence proof paths and regr…
fix(goal): align V1/V2 semantic verifier absence proof paths and regr…
fix(goal): keep Linux paths case-sensitive in requirement-scoped proo…
test(goal-v2): real native verifier rejects deleted file recreated du…
fix(goal): require concrete deliverable proof and verify deletion bef…
ByBrawe commentedon Oct 27, 2026
2026-10-27: source-level false-completion fix merged in PR #286, commit
3473741. All 11 GitHub Actions workflows green, including Windows/Linux Node 20+24 strict release-readiness package smoke, real V2 semantic verifier, pinned stable & experimental OpenCode 2 hosts, restart and Loop coexistence. A generic deliverable no longer accepts runtime turn/progress counters or unrelated passed checks; mixed turn+artifact goals need both; deletion-only named paths can be verified with symlink-safe host absence proof and post-verifier recheck to reject concurrent recreation. Three P1 review issues were fixed and resolved. Keep #281 open for evidence-linked durable closure receipts (goal/revision/file-hash, requirement mapping), broader adversarial corpus and replay of the exact multi-agent reporter incident. Merged tomain, not yet published npm, and do not claim this proves every prior false-complete case solved.release(goal): stage 1.3.50 verified V1/V2 concurrency and proof gate (…
release(goal): 1.3.50 maintenance fixes and native OpenCode compatibi…