Skip to content

eval: DeltaWire v7 discoverable treatment pair - #118

Merged
bigboateng merged 4 commits into
mainfrom
eval/deltawire-v7
Jul 22, 2026
Merged

eval: DeltaWire v7 discoverable treatment pair#118
bigboateng merged 4 commits into
mainfrom
eval/deltawire-v7

Conversation

@bigboateng

@bigboateng bigboateng commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Purpose

Preserve the terminal v7 experiment attempt for one frozen range-large/v7-r1 D1→conditional-B0 pair. This PR changes treatment discoverability and its measurement contract without changing the DeltaWire binary, runtime image, Harbor version, Gemini CLI, model, task semantics, verifier, timeout, or credential boundary.

PR #117 cleanly integrated the reusable v6 runtime infrastructure. Closed draft PR #115 remains the unmerged raw v6 provenance branch.

ZCA slices

  1. Deterministic treatment boundary: host-neutral skill bundle, parser-grounded reference, generic validated examples, assignment-grounded receipts, and a fail-closed one-shot runner.
  2. Causal execution: one authorized D1 attempt ran. It passed semantic verification but failed treatment fidelity, so B0 was withheld and no causal comparison exists.

The value is a precise boundary result: assignment, activation, runtime, lifecycle, setup, environment transport, and semantic generation worked; canonical plan/contract-receipt retention did not.

Frozen identities

  • Infrastructure commit: a8d2d8260ceed56baf0d1cf8fcc2f40776e9f7b6
  • Manifest-bound commit: 369f3c7c0d751da72fa658a2ec810557d6669e39
  • Manifest SHA-256: 67789b85c08802a87f5d14bb72aa2d36068c43a45aa27187ddab5d66dba3eca4
  • Terminal evidence commit: c340db33
  • Linux/amd64 binary SHA-256: e5198d15000e093a2a28e57ad4a093dde8f66bcab21462549ddf9114064c25f4
  • Image ID: sha256:589fce58d7ddf09910876afe0198bcee0086a840af6de7c08515bbce3e8367a7
  • Harbor: 0.20.0
  • Gemini CLI: 0.51.0
  • Agent: preinstalled-gemini-cli-v1
  • Model: google/gemini-3.1-pro-preview

Terminal outcome

D1 trial db50c949-e6b2-4fcf-8ba0-3d0f56b0109e completed with Harbor exit 0, no exception, semantic reward 1.0, and exact ordered records 1–500.

Passed:

  • frozen treatment assignment and bundle digest;
  • structured skill activation;
  • pinned agent/model/image/binary identities;
  • setup, environment, artifact transport, schema integrity;
  • one validate, render, and check lifecycle;
  • matching range example and reference reads;
  • no brute-force loop or binary introspection;
  • semantic exactness.

Failed:

  • Gemini wrote plan.json rather than .deltawire/plans/range-large.dw.json;
  • no canonical retained plan;
  • no retained in-container contract receipt;
  • independent contract recheck found zero canonical plans;
  • strict public-contract access evidence was not proven.

Therefore D1 treatment fidelity and end-to-end status are red. B0 was not launched, the pair is invalid, and v7 cannot be rerun.

Usage diagnostics

  • Input tokens: 170281
  • Output tokens: 3010
  • Input + output: 173291
  • Cache tokens: 112782
  • Harbor-reported cost: 0.1736744
  • Wall time: 64.163724 seconds

These are D1-only diagnostics. No cost, savings, benchmark, or causal claim is made.

Security and evidence

The VM evidence was preserved byte-for-byte. The final scan covered 33 files / 152,579 bytes with zero exact credential matches, zero generic-secret matches, zero closure mismatches, zero temporary credential files, and zero B0 trials.

{
  "READY_FOR_72": false,
  "pair_complete": false,
  "pair_valid": false,
  "rerun_authorized": false,
  "six_run_canary_started": false,
  "full_run_started": false
}

Any further paid attempt requires a separately designed and explicitly authorized v8. This PR remains draft and unmerged experiment provenance.

@bigboateng
bigboateng force-pushed the eval/deltawire-v7 branch 3 times, most recently from b908d33 to c820f62 Compare July 22, 2026 19:45
@bigboateng

Copy link
Copy Markdown
Contributor Author

Pre-live gate reached; no paid model call was made.

  • Manifest-bound head: 369f3c7c0d751da72fa658a2ec810557d6669e39
  • Manifest SHA-256: 67789b85c08802a87f5d14bb72aa2d36068c43a45aa27187ddab5d66dba3eca4
  • Pre-live evidence commit: c4935e9b
  • Linux skill conformance: pass for range, matrix, rows, and variants with repeated-byte determinism; malformed/unsupported controls rejected; zero model calls
  • Exact manifest-head CI: all four checks passed
  • VM dry-run: D1 then conditional B0; only D1 receives the frozen skill bundle
  • Pair root: absent/empty after dry-run
  • Pair authorized: false
  • READY_FOR_72=false; canary/full not started

The live pair remains blocked until the user says exactly:

approve the frozen range-large/v7-r1 pair

@bigboateng

Copy link
Copy Markdown
Contributor Author

Evidence-head CI is green on c4935e9b8c1056255370b67c568f2983b1aac883:

No pair was launched. The result root remains empty and the exact authorization boundary is unchanged.

@bigboateng

Copy link
Copy Markdown
Contributor Author

Terminal v7 result published at evidence commit c340db33.

Outcome: D1 semantic pass, D1 treatment-fidelity fail, B0 withheld, pair invalid, rerun forbidden. The failure is canonical plan/contract-receipt retention: Gemini used plan.json at task root despite activating the correct bundle and completing validate/render/check with exact 1–500 output.

Evidence closure: 95dc20298e27fb2b295ac694688f7d8b14a78aaaa16c854e6642158e15f45240.
Approval receipt: c6b493851951e72ead0d15e38231f47d036331f94ded94d9d3dceb38db7c3154.

No rerun or B0 execution is authorized. A future attempt requires an explicit v8 plan.

@bigboateng
bigboateng marked this pull request as ready for review July 22, 2026 20:41
@bigboateng
bigboateng merged commit b8f9384 into main Jul 22, 2026
4 checks passed
@bigboateng
bigboateng deleted the eval/deltawire-v7 branch July 22, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant