feat(bench): harden PowerPlan and final benchmark gate

[Punisheroot]

Summary

Separate benchmark power planning from final holdout evaluation and make both stages deterministic, digest-bound, and fail-closed.

Calibration observations now produce an immutable PowerPlan before the final schedule is frozen. The final gate accepts only the exact scheduled holdout identities and suppresses economic statistics whenever the campaign, plan, schedule, observations, or quality contract is invalid.

Closes #7.

Implementation

• Add an offline experiment power-plan entry point that:

  • accepts calibration observations only;
  • requires complete FrontierDirect/NeedleMiss pairs;
  • rejects missing, duplicate, mismatched, failed, non-positive, contaminated, non-beneficial, or zero-variance inputs;
  • emits a deterministic needle.power-plan/2 artifact with canonical input and artifact digests.

• Bind the PowerPlan to the frozen corpus and campaign commitment, and bind the final schedule to the exact serialized plan and required pair count for each route.

• Harden experiment final-report so that it:

  • loads the frozen campaign, schedule, and PowerPlan using bounded reads;
  • verifies their raw digests and semantic commitments;
  • requires exactly one observation for every scheduled task, route, split, repetition, pair seed, and arm;
  • preserves infrastructure, quality, staleness, worker, discovery, and recomputation failures instead of filtering them;
  • calculates the paired cost ratio and deterministic two-sided 95% BCa interval only after the complete contract passes.

• Add deterministic synthetic calibration and final-observation fixtures for both built-in routes, plus adversarial coverage for incomplete pairs, duplicated identities, stale artifacts, campaign mismatches, schedule count changes, calibration leakage, invalid costs, and failed evidence.

• Update the public synthetic campaign, schedule, manifest, PowerPlan, benchmark documentation, and project status.

This changes the PowerPlan schema from v1 to v2 and the machine-readable final-gate report to v3. Existing checked-in fixtures were migrated accordingly.

Validation

cargo fmt --all -- --check
passed

cargo test --locked -p needle-bench final_gate
8 passed

cargo test --locked -p needle-bench --test publication_surface
3 passed

cargo test --locked -p needle-bench
53 passed

cargo test --locked -p needle-app
84 passed

cargo clippy --locked --workspace --all-targets -- -D warnings
passed with no findings

npm ci
completed; reported 3 high-severity audit findings

npm run build
passed

cargo run --locked -p needle-app -- experiment power-plan benchmarks/corpus/router-cache/synthetic-calibration-observations.jsonl --corpus benchmarks/corpus/router-cache/manifest.json --campaign benchmarks/corpus/router-cache/campaign.json
emitted a deterministic synthetic PowerPlan with no planning failures and 3 required pairs per route

cargo run --locked -p needle-app -- experiment final-report benchmarks/corpus/router-cache/synthetic-final-observations.jsonl --corpus benchmarks/corpus/router-cache/manifest.json --bootstrap-resamples 10000 --seed 42
verified 3/3 pairs per route and failed closed, as expected, because the public PowerPlan is synthetic

cargo test --locked --workspace
one needle-runtime cache-hit test failed during the aggregate run; the same test passed immediately when rerun in isolation

No provider calls or paid benchmark observations were executed.

Risks/limitations

The checked-in PowerPlan and observations are intentionally synthetic and remain ineligible for an economic claim. This change validates the benchmark protocol but does not establish real provider-backed savings.

A production claim still requires private calibration and holdout material, a production-sealed PowerPlan, an approved provider budget, and a complete provider run.

The aggregate workspace test run observed one isolated cache-hit failure outside the changed benchmark files. Its focused rerun passed, but the aggregate run was not fully green.

Bootstrap work and observation inputs are bounded, and statistical work begins only after the complete frozen contract validates.

Documentation/evidence

Updated:

• docs/BENCHMARKING.md
• benchmarks/README.md
• PROJECT_STATUS.md
• public router-cache manifest, campaign, schedule, and PowerPlan
• deterministic synthetic calibration and final-observation fixtures

The synthetic fixtures are protocol-validation evidence only and are not published economic evidence.

AI assistance and human verification

AI assistance: code, tests, and documentation — implemented the digest-bound PowerPlan and final-gate changes, generated the synthetic fixtures, and exercised the required deterministic and adversarial cases.
Human verification: reviewed the complete 18-file diff, checked artifact and campaign bindings, inspected the fail-closed statistical path, verified the reported commands and outputs, and confirmed that no provider calls, credentials, unrelated edits, or live benchmark artifacts were included.

Ownership checklist

• I read and understand the complete diff and can explain every material change.
• I personally verified the reported tests and technical or performance claims.
• I finalized and personally published the commits and this pull request.
• I checked for unrelated edits, credentials, generated files, and live artifacts.
• Public documentation and PROJECT_STATUS.md are updated when applicable.