Harden CMA tuning and add structured progress telemetry

[CharlesFauman]

Summary

• allow explicit, validated fitness matchup sets while reserving holdout matchups
• add versioned machine-readable CMA config and per-candidate progress records
• use candidate-independent SplitMix64 common-random-number schedules with paired seat seeds
• keep every scheduled attempt in the fitness denominator and attribute draws, setup failures, turn limits, stalls, and panics separately
• bind tuning manifests to exact weight bytes and fail closed on invalid holdout execution
• validate dimensions, overflow, nonzero seeds, balanced holdout seats, and CLI values
• enable the existing ai-tune binary test harness

Verification

• cargo fmt --all -- --check
• cargo test --locked --profile server-release -p phase-ai --features tune --bin ai-tune --jobs 4 (19 passed)
• cargo clippy --locked --profile server-release -p phase-ai --features tune --bin ai-tune --jobs 4 -- -D warnings

The structured records intentionally use phase-ai-cma-config-v2 and phase-ai-cma-progress-v2. This draft is source-reviewed and compiled/tested; it is opened as draft for maintainer feedback before any upstream merge.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d962ce9c-4b0b-4039-a69b-353efa431dfa

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Parse changes introduced by this PR

✓ No card-parse changes detected.

[matthewevans]

Changes requested — the tuning artifact admission path fails open, and the central paired-run path lacks a discriminating runtime test.

🔴 Blocker

tools/ai_tune.rs:296-300 maps a missing or unrecognized artifact group to TuneGroup::Eval. The manifest validation at tools/ai_tune.rs:364-522 checks kind, hash, and schedule but does not read or cross-check the producer-manifest group written at tools/ai_tune.rs:2069-2084. A malformed or substituted artifact can therefore enter the evaluation group instead of being rejected.

Please make artifact-group admission fail closed: require an exact known artifact group, require it to agree with the producer manifest's group, and add rejection coverage for missing, unknown, and mismatched groups.

🟡 Non-blocking

The paired-seed loops at tools/ai_tune.rs:1347-1393 and their runner/progress wiring at tools/ai_tune.rs:1896-1991 do not yet have a seam-driven runtime test. Current coverage exercises helper, record, and preflight paths, but does not demonstrate that paired calls receive the same seed and that every outcome contributes to both the denominator and progress accounting. Please add a central-path test that does.

The current parse-diff artifact reports: “✓ No card-parse changes detected.” Paired-seed and performance gates are pending; those are CI evidence only and are not the basis for this request.

Recommendation: address the fail-closed admission validation and add the central runtime test, then request re-review.