fix(extract): a marker-led clause reads maiden, whatever pair encloses it (#335) - #446
Merged
Conversation
…ncloses it "Jane Smith née Jones" gives maiden 'Jones'; "Jane Smith (née Jones)" gave nickname 'née Jones', because the parens are a nickname pair by default and nothing looked inside them. The clause says "maiden" out loud, so the caller should not have to say it in Policy. extract already emits Role.MAIDEN regions and group already drops the marker inside one (#329), so this is a role swap at the existing emit site plus one predicate beside _suffix_shaped -- in the branch S1 has already declined, which keeps S1's precedence by construction. Extracting the region rather than masking the delimiters is what keeps "Jane (née Jones) Smith" reading family 'Smith': M2's take runs to the end of the name, and the closing paren says where the maiden name stops. A marker with no word after it stays a nickname, so M1's (Nee) boundary is not contradicted on the unconfigured path. M1's boundary example was this very input and moves to M1's own stated exception, the suffix-shaped clause S1 takes first. Two existing tests move with it. A quoted clause that is itself marker-led is now maiden too -- M3 is keyed on content, not on which pair matched -- so the #329 row that contrasted the two roles has no contrast left; it keeps its place recording that, and the role filter it pinned (_group.py:841) is pinned instead by a marker glued to punctuation, which M3's whitespace-split test declines and the tokenizer's token-level one still tags. And test_parser's corpus filter stripped only a trailing period, so a bracketed '(geb.' slipped past a filter whose comment says "no marker already present"; those names now carry a maiden clause of their own, and appending a second one is no longer a single variable. Closes #335 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six corpus names change reading under rules.md#M3 against 2.1.0 and
2.0.0, five against 1.4.0. One rule per ledger, keyed on the names by
literal: the shape is "a bracketed clause whose first word is a
marker", which is a question about the parse and not about the string,
so a fields-only rule standing in for it would pre-excuse every future
nickname/maiden confusion in the corpus.
`fields` is exactly what each baseline's diff moves. Against 2.1.0 all
six move {nickname, maiden}. Against 2.0.0 the five Latin names move
that pair while the CJK name also flips given/family, so that ledger
gets two rules rather than one declaring the union -- a union would
have let a given/family regression on any Latin name pass unreported,
which is what PR #444 found behind fix(#424) and had to narrow.
Against 1.4.0 the CJK name is absent, and that is the interesting
half. Its diff there is now {given, middle, family, maiden}: 1.4 read
the fullwidth brackets as name text, so no nickname changes hands, and
fix(cjk-maiden-marker) already describes exactly that diff and takes
the name. fix(cjk-fullwidth-paren-nickname), whose only name it was,
is declared dormant rather than deleted -- the reasoning it carries
about its `fields` boundary is recorded nowhere else, and the
declaration is load bearing in reverse.
_carries() could not see the markers in question: it strips a token's
abbreviating period but not its bracket, so 'Anna Müller (geb.
Schmidt)' read as carrying no maiden vocabulary and the presence guard
refused a rule that was right. Stripping delimiter characters only
ever finds markers that are really there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
decisions.md gains an M3 section: why the clause is EXTRACTED as a region rather than having its delimiters masked for M2 to consume (the two agree on a trailing clause and diverge on an interior one, because M2's take runs to the end of the name), why the condition is a word after the marker, the real division of labour -- one word versus a marker plus a word, not markerless versus marker-led -- the suffix-word divergence as an accepted consequence, and why implemented: names _extract.py alone. The durable finding is recorded there too: _maiden_marked asks a whitespace-split question and tokenize asks a token-level one, and that divergence is the only thing keeping _group.py:841's role filter reachable. Anyone unifying the two tests would orphan it silently. M1's entry records that its opt-in narrowed, and corrects its own 2026-08-05 premise: #329 was never gate-visible. Under the default policy no corpus name reached the drop pass, so the CJK name's 2.0.0-to-2.1.0 movement was the order flip. It is not gate-visible now either -- a ledger rule narrows by which fields move, never by what they hold, so reverting the drop leaves the same six names moving the same pair and all three gates green (measured by mutation). The README's conclusion under its maiden table says the same beside the table it belongs to, in place of a claim that the gate had gained the coverage. maiden_markers.py said the bracketed Japanese form needs Policy(maiden_delimiters=...). It does not, since M3. Also recorded, not resolved: M3 puts the markers to work in a position they never acted on by default -- the first word of any nickname clause -- and the vocabulary-collision criterion was never applied to it. 'Jane "Roz Smith" Jones' read nickname 'Roz Smith' through 2.1 and now reads maiden 'Smith' with no nickname. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Rosalind Roz Smith" read maiden 'Smith' and no family name at all, and "Rosalind Roz Jones Smith" read maiden 'Jones Smith'. Roz is the ordinary English diminutive of Rosalind; marker matching is whole-token, case-folded and period-insensitive, so it was the same string as the Czech abbreviation roz., and a maiden marker takes every word after it. The defect is as old as the marker vocabulary and lives on M2's bare path -- it is not #335's, and predates the 2.2 work entirely. It was found reviewing #335, which is a different thing. Grounded in decisions.md#vocabulary-collisions C-i: a word belongs in its set's ambiguous subset iff it is borne as an ordinary name in the position the vocabulary claim acts on, and Roz is borne exactly there, the word after a given name. MAIDEN_MARKERS has no ambiguous subset and no code path consults one, so ship and do-not-ship are the only two expressions of C-i this set has -- which is why the remedy is removal rather than a marking. The full participle rozená stays and is unambiguous. What stops working is the abbreviation, for Czech and Slovak callers: "Anna Nováková roz. Svobodová" now reads middle 'Nováková roz.', family 'Svobodová', which is how 1.4.0 read it. Both moving names return to their 1.4.0 readings. A caller who needs the abbreviation adds it to their own Lexicon. The fix(#274) ledger rule hand-copies this vocabulary, so its roz alternative goes too, and the guard that requires every alternative to match a shipped entry is what caught it. No corpus name contains the string in any case or position: the alternative claimed nothing, the rule's recorded corpus claim is unchanged, and all three gates report the same counts as before the removal. Four hard-recorded counts moved with the entry and were recounted rather than decremented: the set ships 16, fix(#274)'s members reach 3 of them, the corpora hold 4 markers (nee arrived with #414) and the rule covers 3. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e role test does Four things the mutation pass turned up, none of them behavior. S1's precedence over M3 was stated in rules.md and executed by nothing. It is now an M3 example line, "Jane Smith (née Jr.)", which reads family 'née' and suffix 'Jr.' identically on 1.4.0, 2.1.0 and here, so the corpus row it adds diffs against nothing. Be exact about which mutation that pins, because the obvious one is inert: hoisting the role swap above the _suffix_shaped call changes no reading at all, since the S1 branch never reads `role` -- it masks the two delimiter spans and hands the content to the token stream, and the role it would have carried is never consulted. What inverts the precedence is giving M3 the clause first, an early append of a Role.MAIDEN region ahead of the S1 test: that reads the example as family 'Smith', maiden 'Jr.', and with the line in place it fails test_example[M3-2]. The Accepted block gained a sentence, because the two suffix-word examples now sitting in one rule look like a contradiction until you know S1 asks about the WHOLE clause: "née Jr." is suffix-shaped by its trailing period, "née V" is not, and only the second reaches M3. The `role is Role.NICKNAME` test cannot change the outcome -- `order` holds exactly two roles, so a role that is not NICKNAME is already MAIDEN and the assignment is a no-op either way -- and no test can catch its removal. Kept, for the day `order` gains a third bucket, and now commented rather than left to look load-bearing. decisions.md#M3 said loosening the second-word test causes "three failures and no more". True of pytest and the weaker half: it also turns the 2.1.0 gate red with three unexplained names, and one of them splits its two bracket pairs across the maiden and nickname fields, contradicting M1 outright. That is the signal a future reader should check first. Three bare _group.py:841 citations become _group.group plus the quoted predicate, per the dotted-symbol convention the pipeline comments already use: nothing recomputes a line number, and one edit above it invalidated all three at once. The README's marker count gains the recompute command decisions.md ships beside its own counts, and says which counting convention it uses -- whole tokens, where the neighbouring _carries guard deliberately asks a wider question and gets 5 rather than 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Thirteen factual corrections, no behavior. Each was re-measured here
before landing; two reviewer figures conflicted and both are settled
below by measurement.
The ledger's marker arithmetic lost a carve-out when I recounted it:
thirteen entries go unreached, but 旧姓 is one of them and DOES appear
as a corpus token, which is why it has a rule of its own. Twelve is
the number that means "no diff to classify", as it was before.
Counts that were stale or never right: the isascii note said 12 of 17
(the set ships 16) and claimed one corpus name depends on the
substring branch -- it is two, both 旧姓, and it was seven before this
branch's delimiter strip moved the parenthesized née names onto the
token branch. The alternation roster still described a roz fragment
this branch deleted. Lexicon.maiden_markers' public docstring gave
"roz." as an example of the defaults. test_parser's filter comment
said three names; it turns away nine, five carrying a maiden clause
under M3 and four -- three one-word clauses and the S1-diverted
"(née Jr.)" -- that M3 declines and that would have been safe.
The release note said six corpus names are every name with the shape.
Seven have it; the seventh is "Jane Smith (née Jr.)", which S1 takes.
Five of the six movers are rule or issue examples, not four. Its roz
bullet pointed at "the bracketed-clause change above", which is below
it.
The Excluded block is keyed by vocabulary set, not by rule, and two
cross-references called it "#M1's". rules.md called roz "deliberately
not shipped" when it shipped through 2.1 and was removed here -- the
distinction that block draws on purpose, and a Czech caller reading
rules.md would have concluded nothing changed for them.
Two claims were too strong. The roz defect predates M3, but M3 would
have widened it: with the entry restored, "Jane Smith (Roz Jones)"
reads maiden 'Jones' where 2.1.0 read nickname 'Roz Jones'. And the
{nickname, maiden} field pair is the 2.1.0 ledger's alone -- at 2.0.0
the CJK name declares four fields, at 1.4.0 it is not a fix(#335) name
-- which is the premise for keeping those ledgers split.
The role-test comment said the test "cannot be false today". It is
false whenever a maiden pair matched; what it cannot do is change the
outcome.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six coverage gaps, each found by a mutation that left the suite and
the gates green.
The OPENS-WITH half of the predicate was pinned by nothing: widening
it from the first word to any word passed everything. "Jane Smith
(Jones née)" closes it, and is parity -- 1.4.0 and 2.1.0 read the
nickname too.
M3 reaching pairs other than the parenthesis was pinned by nothing
either. Gating the swap to '(' and the double quote left the whole
suite green AND the 2.1.0 gate green, its fix(#335) rule quietly
falling from six names to five, with only 1.4.0 going red. The
fullwidth pair is the one the docstring and the release note both
advertise as newly working without configuration, so it gets a
default-policy row of its own, beside a quote-pair row for the
commonest way a nickname is actually written.
markerless_parenthesized_clause_stays_a_nickname executed without
pinning anything: its clause is one word, so the length condition
refused it before the vocabulary condition was consulted, duplicating
the boundary above it. Two words now, so the vocabulary test is the
one under load; the corpus spelling stays named in the note.
"Smith (née Jones)" is N3's shape meeting M3, and N3 was not amended
when M3 landed. A marker-led clause is not a nickname clause, so N3
does not reach it and the one name word keeps the given reading its
bare spelling has had since 1.4.0. Recorded as an N3 Accepted line,
pinned as a row, and classified in all three ledgers -- its own rule
rather than a sixth alternative in the fix(#335) one, since it moves
four fields where those move two.
S1-over-M3 rested on a doc example asserting one field. The full
reading is now a row, because the surprising part is not the suffix:
it is that the marker word becomes the family name.
_clause_free_latin_corpus_names had no non-emptiness assertion, and
this branch widened its filter. An empty parametrization passes as a
skip.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The contributor docs recorded M3 the day it landed; the guide a caller actually reads did not, and CI doctests the examples rather than the prose around them, so nothing noticed. customize.rst said "a delimiter pair routes to exactly one field, and maiden_delimiters states the more specific intent". Since M3 the pair is asked second: the content decides first, and only a clause that says nothing about itself is left for the pair to route. Its maiden_delimiters table row used "Jane Smith (née Jones)" as the worked example, which is now the canonical case for NOT needing the knob; it uses the markerless parenthesized surname instead, which is what only a caller can declare. usage.rst's "Delimited content is not always a nickname" section knew about the suffix reading and not the maiden one. It now opens with M3 and its two boundaries, with doctests, and says the suffix reading is taken first. The release note said "bracketed" nine times over parenthesis examples alone. M3 reaches all eleven shipped pairs, and quotes are how a nickname is most often written, so a caller had no way to anticipate that a quoted clause moves too. rules.md's M Background and M1 amendment stated the two-word rule without S1's carve-out, which both now carry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
N3's Accepted block records that a marker-led clause is not a nickname clause, so the rule does not reach `Smith (née Jones)` and the family comes back empty. Whether that empty family is right is a separate question: 1.4.0 kept it, the emptying arrived with #274's marker consumption in 2.0, and the same shape has been repaired twice since -- at H1 (#410) and at P5's reserve (#411). #445 carries the third. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #446 +/- ##
=======================================
Coverage 98.60% 98.61%
=======================================
Files 45 45
Lines 3090 3095 +5
=======================================
+ Hits 3047 3052 +5
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
One cause under most of this: 7a611a0 corrected a batch of corpus counts and 7355de6 then added a corpus row, falsifying six of them two commits later. A corpus row is a global variable and only the machine-checked constants fail loudly, so every count here is recounted and the three that quantify over the corpus now carry the one-liner that recomputes them, the way tools/differential/README.md does. The filter turns away TEN names, not nine, six of them carrying a maiden clause; it went from 638 names to 628, not 637; eight names depended on the substring branch before the delimiter strip, not seven. Three claims about mutations did not hold. The quote-pair row said gating the swap to '(' and the double quote leaves the suite green -- its own sibling 270 lines below, added in the same commit, fails. It said nine of eleven pairs have no row; three pairs are exercised and eight are not. The fullwidth row said only 1.4.0 reddens; 2.0.0 reddens too, on the four-field rule it has for that name. And the markerless row said dropping the vocabulary test reads maiden 'Johnson' -- it reads 'Mary Johnson', because #329's drop is gated on the first token carrying vocab:maiden-marker and Mary does not. Smaller: the S1 row is the other three fields, not four; 1.4.0 read the bare one-word shape with middle 'née', which is why the sibling rule declares middle; the 2.1.0 ledger's rule carries six alternatives where its siblings carry five; nee appears in two corpus names; and an exclusion comment naming test_ledger_guards as its authority said 46 where that test pins 51 -- self-falsifying rather than merely stale. customize.rst's maiden_delimiters row made the exact error decisions.md records having made once already: markerless versus marker-led, when the line is one word versus a marker plus a word. A caller with '(Nee)' data would have concluded they need no configuration; measured, the default gives nickname there. Both customize.rst paragraphs now state the second-word condition and S1's carve-out, and neither says "name word" -- M3 does not test for one. The three new ledger blocks gain a revisit marker: they declare `family`, and `family` is what #445 would change, so that fix would land inside a rule written to explain something else. The corpus spelling 'Cherice J. (Johnson) Williams' gets its row back beside the two-word one that fences the vocabulary condition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…entry Three places a reader looks, none of which said this. rules.md's Not in scope gains the goal: no shipped wordlist parses every name and none is meant to -- the promise is a mechanism for every TYPE of vocabulary, with callers configuring the members their data needs. mechanisms.md gains the reviewable criterion under Verification shapes. cases.py's docstring said what a row must carry and never what earns one. Written because a review round reported "11 of 16 maiden markers have no behavioral pin" as a coverage gap and proposed a roster to close it. It is not a gap: født, fødd and född are one class, and every fork the code actually takes was already pinned. A count over a wordlist reads as a finding because it carries a number, and it inflates with every entry added -- which is the tell that it measures the data rather than the behavior. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…asured The VOCABULARY-EXERCISES-FORKS entry named four forks for maiden_markers. Measured, the set has none: it is read at exactly two sites, both plain whole-token membership after _normalize, and all sixteen members collapse to one structural class across bare, bracketed and lone-bracketed shapes with the marker text normalised out. Two of the four named forks were not parser behavior at all -- the substring split is _carries, a test guard that deliberately asks wider, and edge-period normalization is _normalize, which discriminates spellings across every vocabulary rather than members of this one. That makes the entry's argument stronger, and it now says so: rows on this set earn their place on clause SHAPE, never on which member appeared. The unit is the INPUT, matching cases.py's wording rather than the entry's old "one member and not another"; a row plainly earns its place with no member fork behind it. Three repairs beyond that. The delimiter enumeration claimed exhaustiveness and missed WORD_INTERNAL_DELIMITERS, which splits the same-character class in two -- 'Jane Smith"' reports UNBALANCED_DELIMITER and "Jane Smith'" reports nothing -- while listing ASCII against fullwidth, which is not a fork: swapping the pair across all 45 corpus names holding a paren changes no field and no ambiguity kind. The entry blurred the line Derek drew, so it states it: a delimiter pair the library SHIPS is caller-visible behavior and deleting one reddens three tests, while vocabulary a caller configures is data. And a row may earn its place a second way, with no fork behind it, by pinning a name a release note advertises by name. Its counts are gone rather than corrected. They were present-tense prose over a drifting wordlist, which root AGENTS.md forbids, this PR moved 17 to 16 by removing roz, and the descriptive figure was wrong anyway -- five members carry rows, so eleven have none. Two merge blockers. A cases.py note carried "leaves the entire suite green" beside the correcting sentence that says the suite catches it; the previous commit appended the truth and left the falsehood standing. And neither recompute one-liner ran as pasted: both bodies were indented inside the string handed to python -c, so both raised IndentationError. I had run them flush-left in my own shell, which is the inert-measurement shape they exist to prevent. Both are dedented, carry a note saying why, and were re-run by reconstructing them from the source and pasting. The #445 marker went on the bracketed rules only. The bare spelling is claimed at 1.4.0 by fix(#274), which also declares `family`, so a "keep the family" fix would land inside it with the gate green -- the same absorption on the same field at the same baseline. Marked there too, with the names to re-measure listed. Also: three ledger regexes gained (?i) in the previous commit without being mentioned, a real widening (the class now matches NEE and NÉE) bounded by _Claim's digest; customize.rst said suffix-shaped content "is read as a suffix" when S1 drops the brackets and parses the content as if written bare; _policy.py's autodoc'd docstring never got that clarification; and a 63/9 count is replaced by its recompute command, since 9 does not reproduce under its own definition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #335.
parse("Jane Smith née Jones").maidengives'Jones';parse("Jane Smith (née Jones)").maidengave'', withnée Joneslanding innickname— the parenthesis pair is a nickname delimiter by default and nothing looked inside the clause. The marker is right there; the parse had enough information.A clause whose content opens with a recognized maiden marker and carries a word after it is now emitted as a
Role.MAIDENregion whichever bucket its pair sits in. One predicate beside_suffix_shaped, one role swap at the existing emit site.This is not the mechanism the issue proposes
#335 argues for masking the two delimiter spans and letting #274's bare-marker rule consume the content. Measured, the two agree on every trailing clause and diverge on an interior one:
Jane Smith (née Jones)Smith, maidenJonesJane (née Jones) SmithSmith, maidenJones'', maidenJones SmithM2's take runs to the end of the name; the closing delimiter says where the maiden name stops, and masking discards it. Extraction is also the smaller change — extract already emits
Role.MAIDENregions and group already drops the marker inside one (#329).It reaches every shipped pair, not just parentheses
Quotes are how nicknames are most often written, and
Jane Smith "née Jones"andJane Smith 'née Jones'move exactly as the parenthesized spelling does. The Japanese山田 花子（旧姓 佐藤）now works withoutPolicy(maiden_delimiters=...).What does not move
A lone marker in brackets stays a nickname (
Jane Smith (née)), which keeps M1's(Nee)boundary true on the unconfigured path — loosening that condition turns the 2.1.0 gate red on three corpus names, one of which splits its clauses acrossmaidenandnickname. A markerless clause stays a nickname (Cherice J. (Johnson) Williams), which is whatmaiden_delimitersremains for. And a suffix-shaped clause is S1's (Jane Smith (née Jr.)→ suffixJr.).One shape to know about
A name that is only a marker clause and one name word now reads that word as the given name rather than the family:
Smith (née Jones)→ givenSmith, maidenJones, where every earlier version read familySmith, nicknamenée Jones. That is the bracketed form agreeing with the bare one. The empty family it leaves is the bare form's and is not new here — but it is not 1.4.0's either, and #445 tracks whether it is right. It has its own rule in all three ledgers, since its diff moves four fields where the other marker-led names move two.Separately,
rozleaves the default vocabularyThe Czech abbreviation of rozená, and also the English diminutive Roz; matching is case- and period-insensitive. The defect is M2's bare path and predates this work —
Rosalind Roz Smithreads family'', maidenSmithon master, and removal restores 1.4.0 parity.rozenástays; Czech callers add the abbreviation to a Lexicon. M3 would have widened the collision, which is why it surfaced here.Also: what earns a case row
A review round reported "11 of 16 maiden markers have no behavioral pin" as a coverage gap. It is not one — measured, all 16 members collapse to a single structural class, so the set has no member fork at all and rows on it earn their place on clause shape. That reasoning is now written down where it can be cited:
rules.md's Not-in-scope gains Vocabulary completeness (no shipped wordlist parses every name, and none is meant to — the promise is a mechanism per type, with callers configuring members for their data),mechanisms.mdgains VOCABULARY-EXERCISES-FORKS, andcases.py's docstring says what earns a row beside what a row must carry. Each ships the command that recomputes its claims rather than a count that drifts.Verification
Suite 5923;
compare.pyexits 0 at 1.4.0, 2.0.0 and 2.1.0 with zero unexplained; mypy, ruff and the Sphinx doctest build clean. Seven of 1,080 corpus names move at 2.1.0/2.0.0 and six at 1.4.0; eight have the shape, and the eighth is the one S1 takes.Four review rounds. No parser defects found by any of them — every finding was prose or coverage, and each was measured before being acted on. Mutation-tested: the second-word condition, the vocabulary condition, the marker-position condition, the delimiter-pair reach, S1's precedence, the
rozremoval, and the_grouprole filter each now fail something when broken.🤖 Generated with Claude Code