[Task] GPQA contamination checks and the format-effect contrast #18

The original leakage worry — calibrating on items drawn from the benchmark that is then evaluated — is now answered by the design itself: arm 1 calibrates on simplescaling/s1K-1.1, which shares no question with GPQA, and arm 2's 64 GPQA items are drawn from outside Diamond. What remains is to verify that separation and to report the contrast it produces.

Verification

• Confirm by question ID that GPQA-Diamond is a subset of GPQA main and extended, which is what makes the exclusion rule necessary.
• Confirm the 64 calibration items contain no Diamond question.
• Confirm the 64 items are disjoint from arm 2's evaluation set.
• Confirm no s1K-1.1 calibration sample reproduces a GPQA question.
• Record the result of each check next to the committed ID list.

Report

On the shared Diamond questions, put arm 1 (s1K-1.1-calibrated) and arm 2 (GPQA-calibrated) side by side for every backbone and row. Their difference isolates the effect of calibrating in the evaluation's own format, with no contamination confound left to explain it. Use paired comparisons on the shared questions.

Depends on: #7, #15

Done when

• All four checks pass and are recorded.
• The Diamond-slice comparison between the two calibration formats is posted.

Part of #2