Skip to content

Contract v10: machine-readable verify_required on the boundary result#254

Merged
pengfei-threemoonslab merged 2 commits into
mainfrom
check-verify-required
Jul 7, 2026
Merged

Contract v10: machine-readable verify_required on the boundary result#254
pengfei-threemoonslab merged 2 commits into
mainfrom
check-verify-required

Conversation

@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor

Closes the check/verify semantic gap's last prose dependency (review problem #3; roadmap item 4). shipgate check already warn-and-routes when a diff touches a tool surface it cannot gate (#237) — but the deferral lived in warning prose and diagnostics, exactly what agents lose on long sessions. The boundary result now carries top-level verify_required: bool (true when undeclared_gap or coverage_gap fires), and the field joins agent_result_control_fields. decision="allow" + verify_required=true = "boundary clean, capability not yet gated — run verify before completion." Deterministic projection of the existing deferral; no second verdict; IE thresholds untouched.

Contract 9 → 10 (additive; kit prompts say ">= 9 or newer" and stay true, so bundled prompts + render hashes are deliberately untouched). Sweep: .well-known (contract_version + control fields), boundary JSON schema regen, 8 goldens regenerated (only the new field appears; false across the whole corpus), switch-on enumerations in README/AGENTS.md/llms.txt/quickstart/docs/agents/*, keystone contract doc (v10 note + field semantics + Runtime contract: 10 stamp), docs/architecture.md stamp, CHANGELOG Unreleased, llms-full regen. True-path assertions added to both deferral tests; false-path to clean-allow.

Full CI-style suite green locally (-n auto -m "not perf"); shipgate check on this diff: allow.

Note for the site: threemoonslab.com quotes the old README banner + switch lists; it vendors from release tags, so the daily sync PR at the next release carries this — flagged so prose review happens then.

🤖 Generated with Claude Code

shipgate check already escalated to warn and routed to verify when a
diff touched a tool surface it cannot gate (#237), but the deferral was
only prose + diagnostics — exactly what agents lose on long sessions.
The boundary result now carries a top-level `verify_required: bool`,
true whenever undeclared_gap or coverage_gap fires, and the field joins
AGENT_RESULT_CONTROL_FIELDS. decision="allow" with verify_required=true
reads "boundary clean, capability not yet gated: run verify before
completion". Deterministic projection of the existing deferral; no
second verdict; IE thresholds untouched.

Contract 9 -> 10 (additive; ">= 9" guidance in kits stays true, so the
bundled prompts are deliberately untouched). Swept: .well-known
contract_version + agent_result_control_fields, boundary JSON schema
regen, 8 boundary goldens regenerated (only the new field changes,
false on the full corpus), the switch-on enumerations across README /
AGENTS.md / llms.txt / quickstart / docs/agents/*, the keystone
contract doc (v10 note + field semantics), CHANGELOG Unreleased, and
llms-full regen. True-path assertions added to the declared-surface and
undeclared-surface deferral tests; false-path to the clean-allow test.

Full CI-style suite green (-n auto, not perf).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

CI failure root-caused and fixed (amended into the commit): my local suite run predated the final amend, and the contract bump had four more pinned layers than the initial sweep found — (1) hardcoded expected control-field lists in test_local_contract.py + test_agent_instructions_renderers.py (×2); (2) contract_version: "9" literal in the codex-skill recipes.md (three byte-identical copies, bumped together); (3) that recipe edit changes the kit render hash → EXPECTED_CODEX_SKILL_RENDER_SHA256 updated and the outgoing hash appended to the kit metadata's prior_render_sha256 per the documented kit-change protocol; (4) the downstream-instruction renderers (agents_md/claude_md/cursor) enumerate the switch fields in generated text → extended, which cascades to the committed .cursor/rules/agents-shipgate.mdc and the docs/target-repo-agent-snippets.md copyable block, both regenerated from the renderer (not hand-edited) so the matches-renderer tests pin them. Full CI-style suite + static-only lint + ruff green on the final amended state, run in the foreground this time. #255 rebased on the amended commit; both branches force-pushed.

🤖 Generated with Claude Code

…s to AgentResultV1

Two P2s from review:

- Docs described an impossible control state: the evaluator escalates a
  clean allow to decision="warn" whenever the deferral fires, so
  allow+verify_required=true cannot be observed. agent-contract-current,
  CHANGELOG, and the field/tuple comments now state the real pair
  (warn+true) and that a plain allow always has verify_required=false.
- agent_result_control_fields advertised verify_required while the
  legacy docs/agent-result-schema.v1.json lacked the field (it lived
  only on the boundary subclass). Moved to the shared AgentResultV1
  base with a default of false and regenerated the legacy schema, so
  the advertised control-field list validates against both schemas.

Full CI-style suite + static-only lint + ruff green on this state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

All three findings addressed:

  • P1 (CI red / stale expectations) — this was fixed before the review landed: the cited assertions were repaired in the amended commit (force-pushed), and the run on the current head is green (test 5m1s pass, run 28825768061). The review checkout at /private/tmp/shipgate-review-pr254 predates the force-push. On the open design question: the generated skill minimum moves to v10 — the regenerated kits now emit verify_required guidance in instructions, so requiring contract_version: "10" or newer is the coherent floor.
  • P2 (impossible allow+verify_required=true) — correct: the evaluator escalates to warn whenever the deferral fires, so that pair cannot be observed. Docs now state the real contract: the observable pair is decision="warn" + verify_required=true, and a plain allow always has verify_required=false (agent-contract-current.md, CHANGELOG, field/tuple comments).
  • P2 (control field not in the legacy schema)verify_required moved to the shared AgentResultV1 base (default false) and docs/agent-result-schema.v1.json regenerated, so agent_result_control_fields validates against both the boundary and legacy schemas.

Full CI-style suite + static-only lint + ruff green on the new commit; #255 rebased on top.

🤖 Generated with Claude Code

@pengfei-threemoonslab pengfei-threemoonslab merged commit b97b667 into main Jul 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant