Skip to content

eval(deltawire): integrate treatment-fidelity preflight - #106

Merged
bigboateng merged 4 commits into
mainfrom
codex/deltawire-v4-integration
Jul 22, 2026
Merged

eval(deltawire): integrate treatment-fidelity preflight#106
bigboateng merged 4 commits into
mainfrom
codex/deltawire-v4-integration

Conversation

@bigboateng

Copy link
Copy Markdown
Contributor

Why

DeltaWire's benchmark preflight needed deterministic task contracts, binary parity, independent semantic verification, and fail-closed treatment attribution before any larger experiment could be considered.

This clean integration PR projects the reusable implementation from the conflicted experiment lineage onto current main. PR #99 remains the provenance record for raw experimental evidence.

What changed

  • Adds the DeltaWire runtime hardening, deterministic plan/schema handling, contract-driven task projections and oracles, 12 canonical fixtures, and 96 required mutation failures.
  • Adds frozen manifests and matrix validation, Harbor 0.20.0 gates, identical Linux/amd64 binary staging, structured Gemini shell/lifecycle parsing, independent plan-contract checks, and fail-closed B0 contamination checks.
  • Adds the versioned v2-v4 probe infrastructure and the v4 environment/treatment/end-to-end receipt boundaries.
  • Adds compact readiness records, receipts, reports, and an explicit integration-projection record bound to evidence commit c200dc0eb3c4dff0e4732cc4b2daa1cd535703e3.
  • Excludes raw trajectories, Harbor logs, expanded generated output, and other large run evidence from this branch; those remain on eval/deltawire-preflight-v4 and PR eval(deltawire): prepare 72-run preflight and semantic oracles #99.

Observed v4 outcome

The one authorized D1-only v4 probe passed semantic exact match and treatment-use fidelity. Harbor's verifier generated a passing environment receipt, but the receipt was not retained from the main task container. The fail-closed end-to-end status is therefore fail; no probe-release.v2 was created and the range-large/r1 pair was not launched.

READY_FOR_72=false. No savings, cost, or benchmark claim is made.

Validation

  • bash labs/20-deltawire/scripts/validate.sh
  • Go tests, race tests, 20x repeat tests, vet, runtime-boundary and determinism checks
  • 12 canonical task contracts and 96 mutation failures
  • Plan-contract acceptance/rejection, structured fidelity, rerun/release/full-mode, matrix, binary-parity, readiness, and production-cleanliness checks
  • No live model calls in CI

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

Pitot coding-agent E2E

Intelligence Flow is the verification source; Pitot's public README carries the latest main status.

Agent Ubuntu macOS Windows Result Evidence
Claude ✅ Pass · real CLI ✅ Pass · real CLI ✅ Pass · real CLI ✅ Passing All platforms are required
Cursor ✅ Pass · hook verified ✅ Pass · hook verified ✅ Pass · hook verified ✅ Passing All platforms are required
Codex ✅ Pass · hook verified ✅ Pass · hook verified ✅ Pass · hook verified ✅ Passing All platforms are required
Gemini ✅ Pass · hook verified ✅ Pass · hook verified ✅ Pass · hook verified ✅ Passing All platforms are required
OpenCode ✅ Pass · hook verified ✅ Pass · hook verified ✅ Pass · hook verified ✅ Passing All platforms are required

Source commit: 5add6e4c5946

@bigboateng
bigboateng marked this pull request as ready for review July 22, 2026 14:21
@bigboateng
bigboateng merged commit e3e085d into main Jul 22, 2026
22 checks passed
@bigboateng
bigboateng deleted the codex/deltawire-v4-integration branch July 22, 2026 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant