Skip to content

A ledger rule's explained set is printed but never checked, so a diff can shrink underneath it and every run stays green #452

Description

@derek73

compare.py computes by_issue -- the exact set of names each rule explained -- prints it (tools/differential/compare.py:868), and then passes only set(by_issue) to dormant_rules. Membership is checked; the counts never are. The only property any run asserts about a rule's reach is "it explained at least one name."

The shape

classify() takes the first rule whose declared fields are a superset of the observed diff. A rule therefore keeps matching when the diff beneath it shrinks -- and shrinking is the common direction, since most parser fixes move fewer fields, not more.

Freiherr von Richthofen V is the case that already happened. #410 narrowed its diff from {given, family, suffix} to {family, suffix}; the fix(#424) a title-led chain before the numeral is the one name piece rule declared all three, so it kept claiming the name and no run named the movement. docs/design/decisions.md#H1 records it:

a shrinking diff kept matching and no run ever named it. A real movement behind a green gate

That rule's fields are ["family", "suffix"] today, narrowed by hand during #410, and it explains exactly 1 name. The instance is closed. The mechanism that let it hide is not.

Why no existing guard sees it

  • dormant (Make a ledger rule that explains nothing say why (#372) #373) covers one point on the scale: zero. A rule that explained 8 names and now explains 3 is not dormant, and nothing reports it.
  • _CORPUS_CLAIMS measures what a rule's name_regex reaches, not what it explains -- the two differ by the fields test and by rule order. It is a unit test with no baseline worker, so it cannot measure explanation even in principle: the diff set only exists inside a compare.py run.
  • _CROSS_RULE_WINNERS pins which rule wins for a hand-picked list of contested names. A wall around known arguments, not a census.

This is the gap #451 reports from the other side. #451's rule hides growth because it has no name_regex; fix(#424) hid a shrink despite having one. Both are "a rule broader than the diff it explains", which decisions.md#H1 already calls "the lesson worth keeping rather than either fix."

Sketch

A per-rule declared reach in the ledger, checked by compare.py against the run's own by_issue:

[[change]]
issue = "fix(#424) a title-led chain before the numeral is the one name piece"
name_regex = "(?i)^freiherr\\s+von\\s+richthofen\\s+v$"
fields = ["family", "suffix"]
explains = 1

Open in the design, not decided here:

Relationship to #451

Independent and deliberately unbundled. #451's bundle retires the fields-only rule and lands without this; this closes the half of the shape a name_regex does not protect against. Bundling them would make #451's commits unbisectable.

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions