Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ All notable user-facing changes are tracked here. Public statements in release n
- Strict typed Python surface: `py.typed`, `MatchStatus`, `Literal` policies, `TypedDict` summaries, a typed `TargetRow`, and mypy strict and pyright gates (`make typecheck`).
- Example and snapshot testing flow: executable doctests, Rust `expect-test`-style snapshots, static API contract tests, and a typed quickstart (`make examples-test`).
- The whole `dotmatch`/`quickdna` package now passes the strict typed gate, with Hypothesis invariants and a Python 3.9 runtime compatibility check (`make compat39`).
- Scientific reliability gate: scalar oracle equivalence across scan/indexed/exact/Hamming policies, stream count conservation, lossless integer count round-trips above float precision, gzip/plain FASTQ equivalence, seeded panel determinism, FASTQ/count/AssaySpec fuzzing, byte-deterministic assignment/panel/feature-matrix artifacts, indexed-vs-scan agreement, panel safety-vs-distance checks, unique-only feature-matrix counts, and mutation-sensitivity cases (`make scientific-tests`). See `docs/scientific-reliability.md`.
- Scientific reliability gate: scalar oracle equivalence across scan/indexed/exact/Hamming policies, stream count conservation, lossless integer count round-trips above float precision, gzip/plain FASTQ equivalence, seeded panel determinism, FASTQ/count/AssaySpec fuzzing, byte-deterministic assignment/panel/feature-matrix artifacts, indexed-vs-scan agreement, panel safety-vs-distance checks, unique-only feature-matrix counts, sensitivity artifact determinism, and mutation-sensitivity cases (`make scientific-tests`). See `docs/scientific-reliability.md`.

### Changed
- Assignment result records (`MatchResult`, `AssignmentStats`, `PosteriorAssignment`, `FastqRecord`, `StreamAssignment`) are immutable `NamedTuple`s. Field access and keyword construction are unchanged. They are now tuple-compatible, so equality with a plain tuple is true; `dataclasses.asdict`, `fields`, and `replace` no longer apply.
Expand Down
3 changes: 2 additions & 1 deletion docs/scientific-reliability.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ protects that contract; it does not by itself prove biological correctness.
agreement, repeated-identifier agreement, gzip/plain equivalence, and target
ID uniqueness are checked before assignment.
- **Determinism.** Seeded panel design and feature-matrix artifacts are
reproducible for the same inputs; assignment TSVs are byte-identical.
reproducible for the same inputs; assignment TSVs are byte-identical, and
sensitivity artifacts are byte-identical apart from the recorded wall clock.
- **Boundary fuzzing.** Generated valid and corrupted FASTQ, count TSV, and
AssaySpec TOML inputs must either parse into typed records or raise a
documented input error, never an unexpected exception.
Expand Down
37 changes: 37 additions & 0 deletions python/tests/test_scientific_invariants.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
design_panel,
hamming_distance,
)
from dotmatch.sensitivity import run_sensitivity
from hypothesis import given, settings
from hypothesis import strategies as st

Expand Down Expand Up @@ -322,3 +323,39 @@ def test_feature_matrix_counts_only_unique_assignments(
expected[key] = expected.get(key, 0) + 1
observed = {(row["cell_barcode"], row["target_id"]): int(row["count"]) for row in rows}
assert observed == expected


def test_sensitivity_artifacts_are_deterministic_except_wall_clock(tmp_path: Path) -> None:
targets = tmp_path / "targets.tsv"
targets.write_text("target_id\ttarget_seq\nA\tACGTACGT\nB\tTTTTTTTT\n", encoding="utf-8")
reads = tmp_path / "reads.fastq"
reads.write_text(
"@r1\nACGTACGT\n+\nIIIIIIII\n"
"@r2\nACGTACGA\n+\nIIIIIIII\n"
"@r3\nTTTTTTTT\n+\nIIIIIIII\n",
encoding="ascii",
)
first = run_sensitivity(
targets=targets,
reads=reads,
target_start=0,
target_length=8,
out_dir=tmp_path / "first",
batch_size=2,
)
second = run_sensitivity(
targets=targets,
reads=reads,
target_start=0,
target_length=8,
out_dir=tmp_path / "second",
batch_size=2,
)

first.pop("execution", {}).pop("seconds", None)
second.pop("execution", {}).pop("seconds", None)
assert first == second
for name in sorted(path.name for path in (tmp_path / "first").iterdir()):
Comment thread
dnncha marked this conversation as resolved.
Resolved
if name == "summary.json":
continue
assert (tmp_path / "first" / name).read_bytes() == (tmp_path / "second" / name).read_bytes(), name
Loading