Skip to content

fix(goal): reject runtime-only proof for ordinary completion claims (#281) - #286

Merged
ByBrawe merged 19 commits into
mainfrom
fix/semantic-runtime-only-false-completion-20261027
Oct 27, 2026
Merged

ByBrawe merged 19 commits into
mainfrom
fix/semantic-runtime-only-false-completion-20261027

Conversation

@ByBrawe

@ByBrawe ByBrawe commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Confirmed source-level false-completion loophole

semanticVerifierHostEvidence() injects goal-runtime-turns-rN and goal-runtime-progress-rN as trust=host, passed=true. The verifier may cite those valid IDs alone and say proven even for a generic work product objective such as implement src/slug.js. Prior guardSemanticProcessResults() returned all non-N-turn cases unchanged; applySemanticVerifierResults thus marked a semantic requirement proven, making completeGoal potentially accept runtime counters as deliverable proof despite no artifact or check evidence. Existing transport test actually expected exactly this behavior. This is a real code path, though it has not yet been matched to the reporter's Oct 18 incident (#279).

Fix

  • For non-process semantic requirements, downgrade model proven to unknown unless it includes a host-corroborated file quote or explicitly referenced current passing host file/command evidence. Mere Goal-owned turns/workspace-mutation counts never suffice.
  • Explicit exactly/at least N Goal turns process requirements retain their host-runtime proof path and existing cadence guards.
  • Add regressions for no files/zero changes/many irrelevant mutations, stale or failing host evidence, legitimate file quote/host command and pure temporal counter proof. Correct the old positive verifier transport test to show accepted tool submission is not a completion claim.

Merge criteria

Full Goal Windows/Linux tests, pinned OpenCode 2.1.4 native host (PR #285), restart/Goal+Loop coexistence and existing semantic process fixtures are green. No docs that claim all prior false-complete reports reproduced or fixed. Keep #281 open for durable artifact-hash closure receipts and a real field-shaped replay. Do not merge external #276 simultaneously without rebasing its verification results onto this gate.

References: #281, #279, #275 / PR #276.

Copilot AI balanced review requested due to automatic review settings October 27, 2026 00:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T00:20:30.151219Z b782039 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b7820396ff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/verification/process.ts Resolved
Comment thread src/verification/process.ts Resolved
Comment thread src/verification/process.ts Resolved
… verify deletion absence and recreation race (#281)
@ByBrawe
ByBrawe merged commit 3473741 into main Oct 27, 2026
43 checks passed
@github-actions
github-actions Bot deleted the fix/semantic-runtime-only-false-completion-20261027 branch October 27, 2026 10:42
ByBrawe added a commit that referenced this pull request Oct 27, 2026
…current main

This maintenance mirror preserves exact contributor source/test blob hashes from muhammad-luay/opencode-goal@6e988e5, against Goal main including #286 and #287, without importing fork CI workflows. This is a *test branch*, not a PR merge or published release.

Co-authored-by: muhammad-luay <muhammad-luay@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants