Say what #320 actually narrowed, and why

Six prose corrections from the review pass; no behavior change.

_types.py's STABLE_TAGS comment is the hand-maintained twin of
docs/modules.rst's block, which the branch updated and this one did
not -- it still promised the "initial" tag for any initial-SHAPED
word, which since #320 is false for CJK. Nothing pins the two copies
against each other, so the comment now says so.

is_initial's docstring named the wrong mechanism: measured, '씨.'
carried vocab:suffix and vocab:suffix-word before the fix as after,
and is_suffix_lenient took it either way -- the veto rejected
is_suffix_strict alone, via _is_suffix_strict_n. cases.py already had
this right, so the docstring was contradicting the row it explains.

_NO_INITIALS' rationale argued from two things that do not hold. It
justified membership phonologically ("a syllable, not a letter"), but
Devanagari is an abugida and Arabic an abjad and the branch's own
tests assert both keep their initials -- so the stated criterion would
have put Devanagari IN. The operative question is orthographic: does
the tradition abbreviate a given name to one character plus a period?
That matters concretely for Thai (#317), an abugida the old wording
would misfile. And the enum does NOT admit only scripts that determine
a convention -- KATAKANA is in it so the classifier can name what it
declines, and neither DEFAULT_SCRIPT_ORDERS nor segment_scripts'
default mentions it. The conclusion (do not derive _NO_INITIALS from
the enum) survives on the better ground that membership is granted on
assorted grounds.

The release note claimed the fix for "a period", true only of U+002E:
_normalize strips the ASCII period alone, so 씨． (U+FF0E), 씨。
(U+3002) and 씨｡ (U+FF61) miss the vocabulary lookup before the veto is
reached, and all three parse identically before and after this branch.
Scoped, with the still-open _normalize gap named.

Also: test_classify's fixture comment and its docstring made opposite
claims about whether й shipping is load-bearing (the docstring was
right -- _LEX is local); that docstring named four scripts where the
sibling test asserts six; cases.py said "fields" where only the field
ASSIGNMENT matches; test_vocab called a digit "the" edge of \w when _
is equally inside it; and AGENTS.md's suffix_not_acronyms bullet now
says which of the two predicates the tension is about.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: derek73

AGENTS.md

@@ -176,7 +176,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_
 
 **Cyrillic suffix regexes need `re.I` even when the pattern is suffix-only** — a Latin title-cased word (`Ivanovich`) keeps its suffix lowercase, so `re.I` seemed skippable; but an irregular Cyrillic suffix can be nearly the whole word (`ильич`), so title-casing capitalizes into the suffix itself (`Ильич`). `east_slavic_patronymic_cyrillic` shipped without `re.I` on the Latin reasoning and silently failed on capitalized irregular forms — don't assume Latin's title-case safety transfers to Cyrillic. (#185)
 
-**`suffix_not_acronyms` vs `is_an_initial` tension** — single-letter roman numeral suffixes (`i`, `v`) are in `suffix_not_acronyms` but also match the `is_an_initial` regex (single uppercase letter), so `is_suffix()` rejects them. The lenient test lives in `is_suffix_lenient()`, which accepts `suffix_not_acronyms` members unconditionally and is only safe in unambiguous positions: (1) suffix-comma detection uses it via `are_suffixes_after_comma()`; (2) lastname-comma post-comma parsing uses it inline, only when `nxt is None` and `len(parts)==2` (no `parts[2]` suffix segment). See issues #136, #144. **The tension is Latin-only since #320** — the veto is now scoped to scripts that HAVE initials — `_vocab.is_initial` ANDs the shape test with "not in `_policy._NO_INITIALS`", that constant listing the four scripts which do NOT (Han, Hangul, Hiragana, Katakana) — so the seven single-character CJK members of `suffix_not_acronyms` (씨, 様, 氏, 군, 양, 님, 殿) no longer collide with it when written with a period: `"김민준 씨."` gives suffix `씨.` where the veto had made it the family name, and `"김민준, 씨."` where it had made it the given name. Latin is untouched — `V.`/`I.` still lose to the veto and still need `is_suffix_lenient()`.
+**`suffix_not_acronyms` vs `is_an_initial` tension** — single-letter roman numeral suffixes (`i`, `v`) are in `suffix_not_acronyms` but also match the `is_an_initial` regex (single uppercase letter), so `is_suffix()` rejects them. The lenient test lives in `is_suffix_lenient()`, which accepts `suffix_not_acronyms` members unconditionally and is only safe in unambiguous positions: (1) suffix-comma detection uses it via `are_suffixes_after_comma()`; (2) lastname-comma post-comma parsing uses it inline, only when `nxt is None` and `len(parts)==2` (no `parts[2]` suffix segment). See issues #136, #144. **The tension is Latin-only since #320** — the veto is now scoped to scripts that HAVE initials, and v1's single `is_an_initial` is now two predicates, of which the tension is about `_vocab.is_initial` (the one `is_suffix()`/`is_suffix_strict` vetoes with; `is_initial_shaped` is the bare v1 shape test and vetoes nothing) — `_vocab.is_initial` ANDs that shape test with "not in `_policy._NO_INITIALS`", that constant listing the four scripts which do NOT (Han, Hangul, Hiragana, Katakana) — so the seven single-character CJK members of `suffix_not_acronyms` (씨, 様, 氏, 군, 양, 님, 殿) no longer collide with it when written with a period: `"김민준 씨."` gives suffix `씨.` where the veto had made it the family name, and `"김민준, 씨."` where it had made it the given name. Latin is untouched — `V.`/`I.` still lose to the veto and still need `is_suffix_lenient()`.
 
 **Comparing against v1 needs `PYTHONSAFEPATH=1` AND a directory outside the worktree** — `uv run --isolated --no-project --with 'nameparser==1.4.0'` still puts the checkout's `nameparser/` ahead of the pinned wheel on `sys.path`, so the "v1" side silently imports the branch and every comparison reports parity. Run it as `cd <scratch dir> && PYTHONSAFEPATH=1 uv run --isolated --no-project --with 'nameparser==1.4.0' python -c "..."`. This produces false confidence rather than an error, so it invalidates results without ever looking wrong.
 

docs/release_log.rst

@@ -29,7 +29,7 @@ Release Log
     - Fix spaced CJK postnominal honorifics parsing as name parts: 씨, 박사, 선생님, 교수님, 군, 양 (Korean — standardly written as their own token), 先生, 女士, 小姐, 博士, 教授 (Chinese, with 先生/博士/教授 shared with Japanese), and 様, 氏 (Japanese) now route to ``suffix``, so ``王小明 先生`` reads family ``王小明`` where the family-first default had confidently made 先生 the given name. Whole-token matching, which also reaches a glued surname+honorific token, since segmentation splits off the surname first (``김씨`` reads family 김, suffix 씨). The glued forms whole-token matching cannot reach are handled by the peel described below (closes #307)
     - Fix glued CJK honorifics parsing as part of the name: an honorific written against the name — ``田中さん``, ``山田太郎様``, ``김민준씨``, ``김민준님``, ``王小明先生`` — is now split off the end of the last name token and routed to ``suffix``, where it had been swallowed by the name (the whole of ``田中さん`` was the family name; ``김민준씨`` gave given 민준씨). The peeled name then goes through the ordinary machinery, so the Korean split still happens (``김민준씨`` → family 김, given 민준, suffix 씨) and the ``田中さん`` case stops misreading as a kana-licensed composite. Only entries that can never END a name peel — 씨, 님, 박사, 박사님, 선생님, 교수님, さん, さま, くん, ちゃん, 様, 先生, 教授, 女士, 小姐 — while 양, 군, 氏, 博士 and 殿 are recognized in their spaced form only, because 김지양 is a given name, 田中博士 is Tanaka Hiroshi as readily as Doctor Tanaka, and some ninety Japanese surnames end in 殿 (鵜殿, 真殿); 君 is recognized in neither form, since 王君 is a complete Chinese name, though its kana spelling くん peels. Seven entries are new vocabulary in this change (さん, さま, くん, ちゃん, 殿, 님, 박사님), so their spaced forms route to ``suffix`` too (``田中 さん``, ``田中 殿``, ``김민준 님``, ``김민준 박사님``). 박사님 closes a gap in the shipped set rather than opening new ground: 선생님 and 교수님 shipped in 2.1 without it, so ``김민준박사님`` stranded 박사 in the given name and the spaced ``김민준 박사님`` came back as two suffixes for one honorific. Exactly one honorific peels off a token, and every entry is a whole honorific rather than a part of one. A token that is nothing but an honorific is no longer taken apart either — ``선생님`` and ``박사`` now stay whole where hangul segmentation had split them 선 + 생님 and 박 + 사, since 선 and 박 are listed surnames — which is **default-on** in its own right, hangul segmentation being a default. A configured segmenter benefits twice over: it is handed the name without the honorific, and the honorific the peel just cut off does not then look to it like a boundary the writer drew — so ``parser_for(locales.JA, segmenter=ja_segmenter())`` reads ``山田太郎様`` as family 山田, given 太郎, suffix 様. Worth knowing before you upgrade: that exemption is what makes a GLUED honorific stop protecting a name from division, so a family name written alone with one — ``田中さん`` — now divides the way bare ``田中`` already did (family 田, given 中, suffix さん). It is exactly and only the peeled tail that is exempt. A SPACED honorific is a token boundary its writer typed, and anything standing beside a name calls the segmenter off, so the name is left as written: ``田中 さん`` and ``佐藤 氏`` keep family 田中 and 佐藤 under the pack — the division the pack gives them without this change, whichever field the honorific itself lands in. That is the conservative reading rather than a claim about intent: a spaced honorific cannot be told apart from a spaced given name by position, and counting it as one keeps four real surnames whole (``佐藤 氏``, ``田中 様``, ``鈴木 先生``, ``中村 教授``) at the price of the one division it then declines to make (``山田太郎 様``). Writing the honorific spaced is therefore an opt-out in its own right on the SEGMENTER path, alongside declining the pack or the segmenter. It is no lever where the VOCABULARY divides the name, the two spellings agreeing exactly there — ``김민준 씨`` and ``김민준씨`` both give family 김, given 민준, suffix 씨, as do ``王小明 先生`` and ``王小明先生`` under the Chinese pack — and Korean data has no pack to decline either, hangul segmentation being on by default. **Default-on: changes parse output for glued CJK honorific forms**, through ``HumanName`` as well as the 2.0 API (closes #308)
     - Fix a comma or a 间隔号 stopping the glued-honorific peel: an honorific written against the name is now split off and routed to ``suffix``, so ``김, 민준씨`` gives family 김, given 민준, suffix 씨 — the same as the spaced ``김 민준씨`` — and likewise ``田中, 太郎さん``, which also matches its spaced form. ``田中さん, PhD`` and ``威廉·莎士比亚さん`` peel too; the first of those does *not* otherwise match its spaced form, since ``田中さん PhD`` leaves PhD in ``suffix`` beside さん while after a comma it reads as a ``title`` — where the credential lands is the comma's business, not the peel's. Previously each of these left the honorific inside the name. A comma no longer switches the peel off; what it does now is say which runs of the name to look in, and those are the two around a family comma, an honorific being as often glued to the given name as to the family. Anything past those two runs is out of reach, which is the one limit worth knowing: ``김, 민준 지훈씨`` peels, while ``김, 민준, 지훈씨`` (a third run) and ``김,, 민준씨`` (a doubled comma, which puts the name in a later run) do not. The reach also rests on the second run being name text, and a one-word part before the comma reads as a family comma even when the part after it is entirely suffix-shaped — so ``田中さん, V.`` keeps さん in the family name where ``田中さん, PhD`` gives it up. Same credential, opposite answer, and unchanged from 1.4.0 in that spelling. The 间隔号 does not switch the peel off either. Both marks say where a name divides into surname and given, and an honorific is not part of the name in either reading. The surname split still stands down for both, unchanged — a comma still means the writer said where the family name ends, and the 间隔号 still marks a transcription. ``田中さん, 太郎`` is unaffected, because the honorific there is not at the end of the name: 太郎 is. **Default-on**, and it reaches ``HumanName`` too (closes #312)
-    - Fix a period after a CJK honorific stopping it being recognized: ``씨.``, ``様.``, ``氏.``, ``님.``, ``군.``, ``양.`` and ``殿.`` now route to ``suffix`` like their periodless spellings, where the trailing period had left them inside the name — the family name in ``"김민준 씨."``, the given name in ``"김민준, 씨."``. The cause was v1's initial regex, ``^(\w\.|[A-Z])?$`` (``REGEXES["initial"]``, still public v1 API), whose ``\w`` is Unicode-aware and so matched a hangul syllable or a Han ideograph as readily as a letter; the strict suffix test applies that as a veto (``V.`` in ``"John V. Smith"`` is a middle initial, not roman five), and a veto written for Latin was being asked of scripts it was never about. The cost ran past the honorific itself: because the vetoed token read as name text, the glued-honorific peel's scan back for its site stopped at it instead of stepping over it, took it as the site, found no honorific at the end of it and gave up — so ``"田中さん, 様."`` kept ``さん`` inside the family name while ``"田中さん, 様"`` peeled it. Measured against 1.4.0, ``"김민준, 씨."`` and ``"田中さん, 様."`` were returning exactly what 1.x returns, so the honorific work earlier in this release had a hole in it wherever the honorific was written with a period. An initial is a single LETTER standing in for a name, and Han ideographs, hangul syllables and kana are morphemes and syllables rather than letters, so the veto now asks its question only of the scripts where it means something. Alphabets keep their initials untouched — ``"А. С. Пушкин"``, ``"م. الفارسي"`` and ``"Ա. Խաչատրյան"`` are unaffected, and so is the Ukrainian conjunction entry below, where a punctuated ``Й.`` still outranks the conjunction ``й``. The public ``initial`` tag follows the same line: ``씨.`` no longer carries it. **Default-on**, and it reaches ``HumanName`` too (#320)
+    - Fix an ASCII period after a CJK honorific stopping it being recognized: ``씨.``, ``様.``, ``氏.``, ``님.``, ``군.``, ``양.`` and ``殿.`` now route to ``suffix`` like their periodless spellings, where the trailing period had left them inside the name — the family name in ``"김민준 씨."``, the given name in ``"김민준, 씨."``. The cause was v1's initial regex, ``^(\w\.|[A-Z])?$`` (``REGEXES["initial"]``, still public v1 API), whose ``\w`` is Unicode-aware and so matched a hangul syllable or a Han ideograph as readily as a letter; the strict suffix test applies that as a veto (``V.`` in ``"John V. Smith"`` is a middle initial, not roman five), and a veto written for Latin was being asked of scripts it was never about. The cost ran past the honorific itself: because the vetoed token read as name text, the glued-honorific peel's scan back for its site stopped at it instead of stepping over it, took it as the site, found no honorific at the end of it and gave up — so ``"田中さん, 様."`` kept ``さん`` inside the family name while ``"田中さん, 様"`` peeled it. Measured against 1.4.0, ``"김민준, 씨."`` and ``"田中さん, 様."`` were returning exactly what 1.x returns, so the honorific work earlier in this release had a hole in it wherever the honorific was written with a period. An initial is a single LETTER standing in for a name, and Han ideographs, hangul syllables and kana are morphemes and syllables rather than letters, so the veto now asks its question only of the scripts where it means something. Alphabets keep their initials untouched — ``"А. С. Пушкин"``, ``"م. الفارسي"`` and ``"Ա. Խաչատրյան"`` are unaffected, and so is the Ukrainian conjunction entry below, where a punctuated ``Й.`` still outranks the conjunction ``й``. The public ``initial`` tag follows the same line: ``씨.`` no longer carries it. Read ``period`` strictly here: the fix is scoped to the ASCII full stop U+002E, because that is the only period ``_normalize`` strips. The fullwidth U+FF0E and the ideographic U+3002 (with its halfwidth twin U+FF61) — the stops a CJK writer is likelier to type — leave the honorific unmatchable by the vocabulary lookup, which runs before the veto is ever consulted, so ``"김민준 씨．"`` still reads the honorific as the family name. That is a separate, still-open gap in ``_normalize`` rather than in the veto: those spellings parse identically before and after this change, and widening the strip is follow-up work. **Default-on**, and it reaches ``HumanName`` too (#320)
     - Fix NFD-decomposed input missing the East Asian defaults entirely: script classification now normalizes to NFC before deciding, so a Korean or Japanese name typed on macOS — where decomposed text is routine — gets the same order rule as its composed twin, which it silently did not before. Segmentation MATCHING deliberately stays raw, so an unspaced NFD hangul name is ordered correctly but not split, rather than being split in the wrong place. One gotcha worth stating plainly: parse output preserves the encoding it was given, so for NFD input ``name.family == "김"`` is ``False`` even though it is the same name — compare NFC-normalized text when comparing across encodings (#272)
     - Fix the Ukrainian conjunction ``й`` not joining the pieces around it: it is the euphonic alternate of ``і``, the two chosen by the surrounding vowel and consonant rather than by meaning (``"Олесь і Олена"`` but ``"Марія й Петро"``), so real Ukrainian data carries both spellings and shipping only ``і`` recognized just one of them. ``"Олесь й Олена Коваленки"`` now gives given ``"Олесь й Олена"`` where the ``й`` previously landed in ``middle``. Same treatment as the ``и``/``і`` entries added in 2.0.0, single-letter carve-out included: the conjunction joins only once the name has enough pieces, and a punctuated initial still wins, so ``"Й. Сліпий"`` is unaffected. Raised in a comment on #267
 

nameparser/_pipeline/_vocab.py

@@ -89,10 +89,12 @@ def is_initial_shaped(text: str) -> bool:
 def is_initial(text: str) -> bool:
     """'A.' / 'j.' / bare capital -- v1's is_an_initial, narrowed to
     scripts that HAVE initials (#320). v1's \\w is Unicode-aware and
-    matched CJK too, which vetoed period-written CJK honorifics ('씨.')
-    out of the suffix vocabulary -- and, downstream of that, left the
-    glued honorific in a name carrying such a token unpeeled
-    ('田中さん 様.')."""
+    matched CJK too, which made period-written CJK honorifics ('씨.')
+    fail is_suffix_strict -- the veto in _is_suffix_strict_n, NOT the
+    vocabulary: suffix_as_written has no veto, so classify tagged '씨.'
+    'vocab:suffix' either way, and is_suffix_lenient took it either way
+    too. Downstream of that one strict-test No, the glued honorific in
+    a name carrying such a token went unpeeled ('田中さん 様.')."""
     return is_initial_shaped(text) and not _in_initialless_script(text)
 
 

nameparser/_policy.py

@@ -142,20 +142,27 @@ class Script(StrEnum):
 #: quantify over this one union (HANGUL simply omitted).
 _JA_SCRIPTS = (Script.HAN, Script.HIRAGANA, Script.KATAKANA)
 
-#: Scripts whose characters cannot BE an initial. A Han ideograph, a
-#: hangul syllable and a kana are each a morpheme or a syllable rather
-#: than a letter, so a single one never stands in for a name the way
-#: "J." stands in for "John". Membership is not "non-Latin": should a
-#: Script.CYRILLIC or Script.DEVANAGARI ever be added, it does NOT
-#: belong here -- those are alphabets, they have letters, and their
-#: initials are real ("А. С. Пушкин", "م. الفارسي").
+#: Scripts whose characters cannot BE an initial. The criterion is
+#: orthographic CONVENTION, not what a character is: does the writing
+#: tradition abbreviate a given name to ONE character plus a period,
+#: the way "J." stands in for "John"? Han, hangul and kana have no
+#: such convention, so a lone punctuated 씨/様/김 is not a shortened
+#: name and the veto has nothing to veto there. Do not restate that
+#: phonologically ("letters, not syllables") -- Devanagari is an
+#: abugida and Arabic an abjad, neither has letters in that sense, and
+#: both abbreviate, so should Script.CYRILLIC or Script.DEVANAGARI
+#: ever be added neither belongs here; their initials are real and
+#: pinned as such ("А. С. Пушкин", "م. الفارسي").
 #:
-#: Enumerated rather than spelled *_SCRIPT_RANGES: the table admits a
-#: script that DETERMINES A CONVENTION (see Script), which is a
-#: different question from whether that script has initials. The four
-#: members coinciding today is what has been implemented, not a
-#: property of the enum -- a Thai entry (#317) must not inherit this
-#: answer without someone deciding it.
+#: Enumerated rather than derived from _SCRIPT_RANGES' keys: the
+#: Script enum admits a member so that SOME behavior may key on it
+#: (see Script), on assorted grounds -- KATAKANA is in it so the
+#: classifier can name what it deliberately declines, and neither
+#: DEFAULT_SCRIPT_ORDERS nor segment_scripts' default mentions it.
+#: Membership therefore settles nothing about abbreviation: the four
+#: coinciding today is what has been implemented, not a property of
+#: the enum -- and Thai (#317) is an abugida too, so it must not
+#: inherit this answer without someone deciding it.
 _NO_INITIALS = (Script.HAN, Script.HANGUL, Script.HIRAGANA,
                 Script.KATAKANA)