Skip to content

fix(evals): bind compound handoff claims to native proof - #186

Merged
vriesd merged 19 commits into
mainfrom
fix/platform-pass
Oct 9, 2026
Merged

vriesd merged 19 commits into
mainfrom
fix/platform-pass

Conversation

@vriesd

@vriesd vriesd commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Why

Qualification rejected a truthful handoff saying that node scripts/verify.mjs passed on Linux and independent review passed. The evaluator treated the platform and review clauses as an unsupported command suffix.

What changed

  • Keep command, review, and qualifier facts separate. Preserve quoted metadata and ordinary prose.
  • Bind platforms from the canonical host registry to their own native evidence, including matching deferred captures. Passing claims retain zero-exit and non-observe requirements.
  • Add live-handoff regression evidence, proof corruption controls, and multi-capture order tests.

Scope

Only the delivery evaluator and its regression evidence change. The packed 9.6.0 plugin, policy, catalog, case instructions, models, sampling, and release plan remain identical.

Tradeoffs

Command-local typed facts preserve claim ownership. A global text rewrite or broad integrity detector would reject unrelated implementation prose. Unsupported status grammar remains a failure.

Blast Radius

Maintainers can grade the observed compound handoff while retaining its native proof requirements. Wrong platform claims on passing checks and non-pass observations now fail against their own evidence.

Verification

  • RED regression commits precede corrections. All eight actual saved campaign inputs pass unchanged with the repair.
  • 422 focused tests pass. Independent controls cover native proof corruption, three genuine deferred captures, metadata, ordinary prose, and contradictory continuations.
  • Full push preflight passes with 3,038 passing tests, 20 existing skips, and zero failures. Independent exact-head review and maintained packing confirm the approved package, policy, catalog, and plan are unchanged.

Original stopped campaigns remain failed. Offline replay supplies repair evidence, not live qualification.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-09T10:33:14.612174Z 10421ad PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10421adc97

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread evals/delivery-presentation.ts Outdated
Comment thread evals/delivery-scenario-checks.ts
@vriesd
vriesd merged commit 2fc6f6f into main Oct 9, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants