Skip to content

eval: integrate terminal DeltaWire v7 learnings - #119

Merged
bigboateng merged 3 commits into
mainfrom
codex/deltawire-v7-integration
Jul 22, 2026
Merged

eval: integrate terminal DeltaWire v7 learnings#119
bigboateng merged 3 commits into
mainfrom
codex/deltawire-v7-integration

Conversation

@bigboateng

Copy link
Copy Markdown
Contributor

Summary

Integrate the reusable learnings from terminal DeltaWire v7 experiment PR #118 without merging raw experiment evidence.

V7 established a precise split: assignment, activation, pinned runtime, validate/render/check, and semantic records 1–500 passed; canonical plan and contract-receipt retention failed. The experiment attempt is immutable, invalid as a causal pair, and cannot be rerun.

Changes

  • Replace Gemini-specific pre-activation discovery ceremony with a host-neutral DeltaWire skill.
  • Preserve the proven low-freedom contract that v7 failed to follow:
    • read /task-contract.json;
    • create exactly one canonical .deltawire/plans/<task>.dw.json plan;
    • invoke the full retained-receipt contract checker command;
    • retain plan, config, schema, state, output, and contract receipt.
  • Add parser-grounded plan-v1 reference documentation and generic validated range, matrix, rows, and variants examples.
  • Add credential-free actual-binary conformance and negative controls.
  • Add assignment-grounded treatment receipt tests and B0 contamination tests.
  • Add a Git-object-backed lock over 64 files at terminal experiment commit c340db338a2758fbb33efd84209d43e4e2fd6be3.
  • Add only compact derived terminal receipts/reports, normalized to repository-relative identifiers.

Root cause and value

The v7 model activated the correct frozen bundle and read the matching reference/example, but wrote plan.json at task root and failed to retain the contract receipt. The reusable skill now makes those fragile paths explicit while keeping the detailed parser protocol progressively disclosed in a reference.

This converts one failed treatment-fidelity attempt into a deterministic, testable integration boundary without claiming a causal comparison or authorizing another paid run.

Evidence policy

Included:

  • reusable skill/reference/examples and validators;
  • compact derived receipts, readiness, security record, and terminal summary;
  • Git-object evidence lock.

Excluded:

  • raw Harbor jobs;
  • Gemini trajectories and transcripts;
  • logs;
  • expanded 500-record output;
  • approval records and credentials;
  • VM-local paths.

Raw evidence remains only on draft experiment PR #118 at commit c340db33.

Validation

  • full labs/20-deltawire/scripts/validate.sh passed;
  • Go unit, race, repeat, vet, runtime-boundary, and determinism checks passed;
  • 12 canonical task contracts and 96 mutation failures passed;
  • v1–v7 Git-object evidence locks passed;
  • skill frontmatter and host-neutral/static rejection tests passed;
  • actual-binary range/matrix/rows/variants conformance passed locally;
  • malformed JSON and undocumented fields were rejected;
  • production cleanliness passed;
  • no CI path performs a live model call.
{
  "READY_FOR_72": false,
  "pair_valid": false,
  "rerun_authorized": false,
  "six_run_canary_started": false,
  "full_run_started": false
}

@bigboateng
bigboateng marked this pull request as ready for review July 22, 2026 20:49
@bigboateng
bigboateng merged commit 5cff4e9 into main Jul 22, 2026
3 checks passed
@bigboateng
bigboateng deleted the codex/deltawire-v7-integration branch July 22, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant