Conversation
|
Reviewed at eabfad9. The harness design holds up where it matters. One thing in the diff does not belong to it. Verified rather than taken on trustThe anti-circularity property is real, which is the whole load-bearing claim. The discovery floor moved with the suite. 103 to 138, with 138 tests pass. Worth saying how I got there: they failed once locally, on
|
The okf/ notes are local project-knowledge tooling, not NOOP documentation; the parity harness does not depend on them.
|
Dropped |
Add shared parity cases with independent expected-result validation
PR1 (#1534) protects the static Swift/Kotlin inventory and parity-debt baseline;
it does not execute both platforms. PR2 adds the portable behavioral-test
contract: cases with shared inputs and literal expected results, deterministic
expansion, an authoritative shard/operation registry, exact split/remerge, and
input-bound result artifacts.
A platform can validate its own artifact against the shared expectations without
building the other platform or waiting for its output. A later comparison takes
both result files and checks their agreement without rebuilding either platform.
Both sides are also checked against the expected results, so two implementations
returning the same wrong answer cannot pass merely by agreeing. Expected values
are checked-in literals, not results generated from the Python reference runner.
Every case hash binds its inputs, identity, function, shard, expected result and
explicit suite version. Output metadata must match those cases and the full Git
source revision explicitly required by the caller. A stale or mixed revision,
changed case, malformed output or missing result fails closed. Revision metadata
is a consistency contract, not cryptographic proof of which code ran; later CI
must establish trustworthy build/artifact provenance. Source revisions are kept
out of persistent case definitions so cases remain reusable across code changes.
Relationship to native tests and the remaining series
The goal is to maintain platform-independent behavioral cases once and execute
them on each platform. UI, permissions, OS integration and platform-internal
checks remain native. PR2 preserves all existing native tests and contains only
a small Python clamp demonstration plus mutant controls on both sides. Python
agreement is not evidence of Swift/Kotlin product parity.
PR3 adds native runners and migrates the first small real algorithm's existing
Swift/Kotlin cases into the shared format, preserving their expected assertions.
Redundant native cases should only be removed after their former checks are
proven to run through both adapters. Further migrations follow incrementally.
PR4 adds product-source CI enforcement. PR2 extends the governance tool self-tests
and case-spec triggers without introducing coupled native builds or enforcement.
See
Tools/PARITY_HARNESS.mdfor exact metadata fields, standalone and artifact-onlyCLI examples, canonical JSON rules and exit codes.
Validation
All 138 governance tests pass, including 35 portable-harness tests. Broad Tools
self-test discovery passes all 247 tests (the same 138 governance tests plus
109 adjacent tests). Coverage includes standalone CLI success without a peer
artifact, agreeing-but-wrong failures, exact oracle types, stale/mixed revisions,
version/hash binding, required expectations, and both mutant controls.
Independent subprocess checks and read-only review passed.
git diff --checkpasses; the draft OKF record has no conformance or security errors.