**Design philosophy — positional and language-agnostic.** The parser assigns parts by *position* plus small sets of words that join to neighbors; it never detects language. A name's language can't be reliably inferred from Latin-script transliteration ("Ali" is Arabic or Italian; "Van"/"Della"/"Bin" are first names in some cultures, particles in others), so language-specific rules belong in opt-in `Constants` config, never global defaults. Many "wrong for language X" reports (#133, #150, #130, #85, #103, #146, #83) are irreducible ambiguities — e.g. `de Mesnil` (want last name) vs `Van Johnson` (want first name) are the same `[prefix][word]` shape. Before adding a rule, confirm it doesn't break the opposite case (run the full suite — Portuguese and "Van Johnson" tests are the usual canaries). **The one scoped exception (2.1, #271/#272): script-conditional behavior is permitted exactly where the SCRIPT ITSELF — not statistics about it — determines the convention.** The never-detect-language rule above is about Latin *transliteration*, where the signal genuinely is destroyed; native script is a different question, and it is answered per behavior rather than per script. Five defaults fall out of it, plus a sixth that applies the same not-a-guess standard to specific WORDS rather than to a script (#308's honorific peel, below). Wholly-Han, wholly-Hangul and kana-licensed names read family-first (`Policy.script_orders`) — no language detection needed, because zh and ja both write family-first in native script, so order cannot be misread even though the language is unknowable. Unspaced hangul splits into surname + given name (`config/surnames.py` ships the Korean census list as DEFAULT vocabulary) — nothing but Korean is written in hangul and the surnames are a closed census set, and the vocabulary is self-selecting besides: a hangul entry can only ever match hangul text. Hiragana licenses Japanese (#272) — a name whose characters stay inside Han∪kana while carrying at least one kana cannot be Chinese (the kana rules it out) and is not a transcription (foreign names are transcribed in katakana ALONE, マイケル has no kanji), so 高橋みなみ and 山田 エミ read family-first too; mechanically they resolve to the HIRAGANA entry, the license's carrier key. PURE katakana is excluded and keeps the positional default: マイケル・ジャクソン is a transcribed foreign name in its source order. And the 间隔号 U+00B7 (#298) is the transcription marker for scripts that HAVE no transcription script: a name it divides (威廉·莎士比亚 — flanked by classified characters on both sides, so Catalan's Gal·la is untouched) keeps its source order and never segments — the orthography names the convention, exactly as pure katakana does, with the divider carrying the signal instead of the script. And a listed CJK honorific glued to the END of a name token is split off it (#308) — 田中さん is 田中 plus さん — on the same orthography-settles-it test, narrowed for the glued position: an entry peels only where it can never end a name, so 씨/님/さん/様/先生 peel while 양/군/氏/博士/殿 stay spaced-only (김지양 and 田中博士 are names, and ~90 Japanese surnames end in 殿) and 君 is in NEITHER set (王君 is a complete Chinese name), though its kana spelling くん peels. Like the nakaguro's tokenize-level separation described next, it is reached by neither policy opt-out — but for its own reason: the vocabulary carries the license itself rather than borrowing the script's, so `segment_scripts` has nothing to say about it. Since #312 it also crosses the FAMILY comma and the 间隔号, which is the more surprising fact, since both of those still stop the surname split standing right beside it: each answers where a name DIVIDES into surname and given, and the peel never asks that question — so `김, 민준씨` reads exactly as the spaced `김 민준씨` does (family 김, given 민준, suffix 씨) while the split stands down as before. Its site is accordingly the name-bearing segment runs — `segments[:2]` under a family comma, and `segments[0]` as before otherwise, the family comma being the one structure that splits the name itself across two runs, with the honorific as often glued to the given side as to the family. That is the whole reach and nothing past it (`김, 민준 지훈씨` peels; `김, 민준, 지훈씨` and `김,, 민준씨` do not, both landing in a third run), and whether `segments[1]` is name text at all is now ASKED rather than inferred from the structure — `segment` does not guarantee it, since a one-word part before the comma reads as FAMILY_COMMA even when the part after it is entirely suffix-shaped, and the peel walking into such a run took `V.` for its site, found no listed tail and abandoned (#319). The question is `segment`'s own suffix-comma predicate, lifted into `_vocab.is_wholly_suffix` so the two stages cannot drift: a wholly suffix-shaped second run is declined and the scan stays in `segments[0]`, so `田中さん, V.` and `田中さん, Ph. D.` give さん up as `田中さん, PhD` always did. The test is necessary but NOT sufficient, and the second condition is not decoration: every honorific tail is also a suffix word, so a glued honorific is itself part of what makes its run read as suffix-shaped, and declining a run that holds the ONLY site loses the peel outright. `segments[0]` must therefore offer a peel site of its own before the second run is declined — `이, J.씨` and `선생님, J.씨` pass the suffix test and are scanned anyway, keeping the pre-#319 reading, while `김민준씨, J.씨` has a site on both sides and peels the person's own 씨 rather than the junk one behind the comma. Uniform in the PEEL, that is — where the credential itself lands is `assign`'s question and still differs by spelling (`PhD` → `title`, `V.` → `given`, `Ph. D.` → `suffix`). Not `_is_post_nominal` pluralized: the run predicate says yes both to what the token predicate vetoes (`V.`, `V`, `I` — the class the defect was reported as) and to what the token predicate never sees at all, since `period_joined_vocab` and the delimiter routes are the run predicate's alone (`Msc.Ed.` and `J.씨` reach it that way, and `田中さん, Msc.Ed.` moves with the rest). `Policy(lenient_comma_suffixes=False)` drops this call to the strict token test too — so those three read as name text again and keep the pre-#319 answer, while `Ph. D.` peels under the knob regardless, its merged `phd` passing the strict test. `田中さん, 太郎` is unchanged, and not because of its comma — the honorific there is not at the end of the name, 太郎 is. The nakaguro belongs to the same doctrine but is decided a layer down: U+30FB and its halfwidth twin U+FF65 separate tokens like whitespace, unconditionally and in tokenize, so neither policy opt-out (`script_orders={}`, `segment_scripts=()`) reaches it — the codepoints are CJK-only and appear in no other script's names, which is what licenses a tokenize-level rule where U+00B7 (also the Catalan punt volat, interior to Gal·la) needs the flanked-by-classified-script guard `_tokenize_region` gives it (#298). Han segmentation stays OPT-IN (`locales.ZH` for Chinese, `locales.JA` for Japanese) — a zh surname list corrupts Japanese kanji names, since 高 is a common Chinese surname and 高橋一郎 would split 高+橋一郎 where the correct reading is 高橋+一郎; no surname list divides a kanji name at all, so `locales.JA` activates the stage and a pluggable `Parser(segmenter=...)` does the dividing. Latin-script input is never touched by any of this: "Kim Min-jun" is genuinely order-ambiguous and stays governed by `name_order` and opt-in packs. Before adding a script-conditional rule, work out which of the three it is — certain, certain for this one behavior only, or a statistical guess wearing a script's clothes.
0 commit comments