Skip to content

Glued CJK honorifics are absorbed into the name (山田太郎様 → family 山田太郎様) #308

Description

@derek73

The common way to write CJK honorifics is glued to the name: 田中さん, 王先生, and the standard mail-addressing form 山田太郎様. Whole-token vocabulary can never match these, so today the honorific is absorbed whole — and a kana honorific even changes classification (田中さん is Han+hiragana → kana-licensed → family=田中さん), and would mislead a configured ja segmenter.

The fix is a mechanism, not constants: a list-driven tail-peel on the last name token, running before classification/order/segmentation — the honorific-stripping analogue of script_segment. Design questions this issue exists to settle:

  • Safety: 山田さん must peel while 山田さくら must not — tail-matching against a closed honorific list is probably sufficient (no Japanese given name is さん/様/先生), but the boundary needs the same treatment the segmenter contract got (what may peel, what must decline).
  • Default-on or opt-in: glued 様/さん at token tail looks unambiguous (the doctrine's "orthography settles it" test), but that claim needs adversarial review against real given names.
  • Interaction order: the peel must precede effective_script, the family-first license, and the segmentation stage, and the peeled honorific needs a field (suffix) and span bookkeeping consistent with the anti-Strange parsing of name w lastname prefix and title before and after #100 invariant (spans index the original — a peel is a token split, not a rewrite).

Depends on the spaced-honorific vocabulary issue for the list itself.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions