Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,10 @@ Each named attribute (`title`, `first`, etc.) is a `@property` that joins its co
3. Add `self.x = x if x is not None else self.C.x` in body — use `is not None`, not `or`, to allow falsy values like `""`
4. conftest auto-restores scalar CONSTANTS between tests, but tests that *set* CONSTANTS mid-run still need their own try/finally

**Adding a new mutable/collection `Constants` attribute** (a `SetManager`/`TupleManager`-backed group, e.g. `extra_nickname_delimiters`): add it to `_COLLECTION_CONFIG_ATTRS` in `tests/conftest.py`, or tests that mutate the global `CONSTANTS` copy will leak state into later tests. Contents must be deep-copyable (the snapshot uses `copy.deepcopy`) — already true for the existing manager types.

Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_deepcopy_roundtrip`/`test_extra_nickname_delimiters_deepcopy_roundtrip` in `tests/test_constants.py`), not just reliance on conftest's autouse snapshot/restore exercising it incidentally. `TupleManager`/`RegexTupleManager.__getattr__` answer *any* unknown attribute lookup — including dunder probes like `__deepcopy__` — so a new manager subtype or a `__getattr__` tweak can silently break `copy.deepcopy` (this bit `RegexTupleManager` before the dunder-lookup guard was added). A direct test on the new attribute's own manager instance catches that where the conftest fixture, which never asserts on the copy, would not.

**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PREFIXES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PREFIXES` ∩ `first_name_prefixes` (position-dependent: leading token → first-name join, mid-name → last-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `first_name_prefixes`).

**Adding a flag-gated post-parse transform** (reorder/adjust, e.g. `patronymic_name_order`) — add a `Constants` boolean (default `False`), implement a `handle_*()` method, and call it in `post_process()` after `handle_firstnames()` and before `handle_capitalization()`, gated on the flag. Default-off keeps existing parses byte-for-byte unchanged. (#85; extension point for #185 Turkic.)
Expand Down
25 changes: 25 additions & 0 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,31 @@ Each set of constants comes with :py:func:`~nameparser.config.SetManager.add` an
the constants for your project. These methods automatically lower case and
remove punctuation to normalize them for comparison.

Adding Custom Nickname Delimiters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:py:meth:`~nameparser.parser.HumanName.parse_nicknames` recognizes three
built-in delimiters -- ``quoted_word``, ``double_quotes`` and
``parenthesis`` -- read from :py:attr:`~nameparser.config.Constants.regexes`,
so overriding e.g. ``CONSTANTS.regexes.parenthesis`` still works exactly as
before. To recognize an *additional* delimiter without overriding one of the
built-ins, add a pattern to
:py:obj:`~nameparser.config.Constants.extra_nickname_delimiters` (empty by
default) under any key, then re-run
:py:meth:`~nameparser.parser.HumanName.parse_full_name` to pick it up:

.. doctest::

>>> import re
>>> from nameparser import HumanName
>>> hn = HumanName("Benjamin {Ben} Franklin", constants=None)
>>> hn.nickname
''
>>> hn.C.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
>>> hn.parse_full_name()
>>> hn.nickname
'Ben'

Other editable attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
8 changes: 8 additions & 0 deletions nameparser/config/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,7 @@ class Constants:
suffix_acronyms_ambiguous: SetManager
capitalization_exceptions: TupleManager[str]
regexes: RegexTupleManager
extra_nickname_delimiters: TupleManager[re.Pattern[str]]
_pst: Set[str] | None

string_format = "{title} {first} {middle} {last} {suffix} ({nickname})"
Expand Down Expand Up @@ -414,6 +415,13 @@ def __init__(self,
self.suffix_acronyms_ambiguous = SetManager(suffix_acronyms_ambiguous)
self.capitalization_exceptions = TupleManager(capitalization_exceptions)
self.regexes = RegexTupleManager(regexes)
# Named, appendable group of *additional* delimiter patterns that
# parse_nicknames() iterates after its three built-in delimiters
# (quoted_word/double_quotes/parenthesis, read live from self.regexes
# so overriding those keeps working as before). Empty by default; add
# a pattern here (and re-parse) to recognize a new delimiter without
# needing to override parse_nicknames() itself. See issue #112.
self.extra_nickname_delimiters = TupleManager()
self.patronymic_name_order = patronymic_name_order

def _invalidate_pst(self) -> None:
Expand Down
27 changes: 19 additions & 8 deletions nameparser/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -784,14 +784,14 @@ def parse_nicknames(self) -> None:
white space to allow for quotes in names like O'Connor and Kawai'ae'a.
Double quotes and parenthesis can span white space.

Loops through 3 :py:data:`~nameparser.config.regexes.REGEXES`;
`quoted_word`, `double_quotes` and `parenthesis`.
Loops through the built-in `quoted_word`, `double_quotes` and
`parenthesis` patterns in :py:attr:`~nameparser.config.Constants.regexes`,
followed by any patterns added to
:py:attr:`~nameparser.config.Constants.extra_nickname_delimiters` --
see the "Adding Custom Nickname Delimiters" section of the
customization docs.
"""

re_quoted_word = self.C.regexes.quoted_word
re_double_quotes = self.C.regexes.double_quotes
re_parenthesis = self.C.regexes.parenthesis

def handle_match(m: 're.Match[str]') -> str:
# Fall back to the whole match when the regex has no capturing
# group (e.g. a custom override regex without one, like
Expand Down Expand Up @@ -826,10 +826,21 @@ def handle_match(m: 're.Match[str]') -> str:
self.nickname_list.append(content)
return ''

# Same handle_match for all three delimiters: suffix-shaped content
# Same handle_match for every delimiter: suffix-shaped content
# is rare in quotes but not impossible, and the logic is delimiter-
# agnostic, so there's no reason to special-case parenthesis here.
for _re in (re_quoted_word, re_double_quotes, re_parenthesis):
# The three built-ins are read live from self.C.regexes (not copied),
# so overriding e.g. self.C.regexes.parenthesis keeps working as
# before; extra_nickname_delimiters is iterated afterward so callers
# can add new delimiter patterns at runtime without needing to
# override parse_nicknames() itself -- see issue #112.
delimiters = (
self.C.regexes.quoted_word,
self.C.regexes.double_quotes,
self.C.regexes.parenthesis,
*self.C.extra_nickname_delimiters.values(),
)
for _re in delimiters:
self._full_name = _re.sub(handle_match, self._full_name)

def squash_emoji(self) -> None:
Expand Down
6 changes: 6 additions & 0 deletions tests/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,9 @@ def assertIs(self, first: object, second: object, msg: object = None) -> None:

def assertIsNot(self, first: object, second: object, msg: object = None) -> None:
assert first is not second, msg or f"{first!r} is {second!r}"

def assertIsNone(self, expr: object, msg: object = None) -> None:
assert expr is None, msg or f"{expr!r} is not None"

def assertIsNotNone(self, expr: object, msg: object = None) -> None:
assert expr is not None, msg or "unexpectedly None"
1 change: 1 addition & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
"first_name_prefixes",
"capitalization_exceptions",
"regexes",
"extra_nickname_delimiters",
)


Expand Down
34 changes: 34 additions & 0 deletions tests/test_constants.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import copy
import pickle
import re
import timeit

from nameparser import HumanName
Expand Down Expand Up @@ -60,6 +61,17 @@ def test_can_change_global_constants(self) -> None:
# No manual cleanup needed: the autouse fixture in conftest.py snapshots
# and restores the global CONSTANTS collections around every test.

def test_can_add_global_extra_nickname_delimiter(self) -> None:
# https://github.com/derek73/python-nameparser/issues/112
hn = HumanName("")
hn.C.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
hn2 = HumanName("Benjamin {Ben} Franklin")
self.assertEqual(hn2.has_own_config, False)
self.m(hn2.nickname, "Ben", hn2)
# No manual cleanup needed: the autouse fixture in conftest.py snapshots
# and restores the global CONSTANTS collections (including
# extra_nickname_delimiters) around every test.

def test_remove_multiple_arguments(self) -> None:
hn = HumanName("Ms Hon Solo", constants=None)
hn.C.titles.remove('hon', 'ms')
Expand Down Expand Up @@ -129,6 +141,7 @@ def test_pickle_roundtrip_preserves_customizations(self) -> None:
c.titles.add('customtitle')
c.prefixes.add('customprefix')
c.titles.remove('hon')
c.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)

# Safe: round-tripping a Constants the test just built, not untrusted data.
restored = pickle.loads(pickle.dumps(c))
Expand All @@ -142,6 +155,8 @@ def test_pickle_roundtrip_preserves_customizations(self) -> None:
# The collections must also keep their manager type, not just contents.
self.assertEqual(type(restored.titles), SetManager)
self.assertEqual(type(restored.prefixes), SetManager)
self.assertIn('curly_braces', restored.extra_nickname_delimiters)
self.assertEqual(type(restored.extra_nickname_delimiters), TupleManager)

def test_pickle_roundtrip_preserves_instance_scalar_override(self) -> None:
"""An instance-level scalar override must survive a pickle round-trip."""
Expand Down Expand Up @@ -185,6 +200,25 @@ def test_regexes_deepcopy_roundtrip(self) -> None:
# The EMPTY_REGEX default still applies to genuinely unknown keys.
self.assertEqual(dup.does_not_exist, EMPTY_REGEX)

def test_extra_nickname_delimiters_deepcopy_roundtrip(self) -> None:
"""copy.deepcopy of extra_nickname_delimiters must round-trip.

Mirrors test_regexes_deepcopy_roundtrip: extra_nickname_delimiters is a
plain TupleManager (not RegexTupleManager), but shares the same
__getattr__/__reduce__ machinery, so it's exercised here directly
rather than only incidentally via conftest's autouse snapshot/restore.
"""
c = Constants()
c.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)

dup = copy.deepcopy(c.extra_nickname_delimiters)

self.assertEqual(type(dup), TupleManager)
self.assertEqual(dict(dup), dict(c.extra_nickname_delimiters))
# Plain TupleManager has no EMPTY_REGEX fallback: unknown keys are None.
self.assertIsNone(dup.does_not_exist)
self.assertIsNotNone(dup.curly_braces)

def test_regextuplemanager_ignores_dunder_lookups(self) -> None:
"""Unknown dunder names report as absent, not as the EMPTY_REGEX default.

Expand Down
56 changes: 56 additions & 0 deletions tests/test_nicknames.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import re

import pytest

from nameparser import HumanName
Expand All @@ -14,6 +16,49 @@ def test_nickname_in_parenthesis(self) -> None:
self.m(hn.last, "Franklin", hn)
self.m(hn.nickname, "Ben", hn)

# https://github.com/derek73/python-nameparser/issues/112
def test_add_custom_nickname_delimiter(self) -> None:
hn = HumanName("Benjamin {Ben} Franklin", constants=None)
# curly braces aren't a recognized delimiter by default
self.m(hn.nickname, "", hn)
hn.C.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
hn.parse_full_name()
self.m(hn.first, "Benjamin", hn)
self.m(hn.last, "Franklin", hn)
self.m(hn.nickname, "Ben", hn)

def test_remove_custom_nickname_delimiter(self) -> None:
hn = HumanName("Benjamin {Ben} Franklin", constants=None)
hn.C.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
hn.parse_full_name()
self.m(hn.nickname, "Ben", hn)
del hn.C.extra_nickname_delimiters['curly_braces']
hn.parse_full_name()
self.m(hn.nickname, "", hn)

def test_multiple_custom_nickname_delimiters_together(self) -> None:
# Two extras registered at once must both be recognized in a single
# parse, independent of insertion order.
hn = HumanName("Benjamin {Ben} <Benny> Franklin", constants=None)
hn.C.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
hn.C.extra_nickname_delimiters['angle_brackets'] = re.compile(r'<(.*?)>', re.U)
hn.parse_full_name()
self.m(hn.first, "Benjamin", hn)
self.m(hn.last, "Franklin", hn)
self.m(hn.nickname, "Ben Benny", hn)

def test_overriding_builtin_regex_still_affects_nickname_parsing(self) -> None:
# The pre-existing customization path (overriding self.C.regexes
# directly, documented since before #112) must keep working now that
# parse_nicknames() also consults extra_nickname_delimiters.
hn = HumanName("Benjamin [Ben] Franklin", constants=None)
self.m(hn.nickname, "", hn)
hn.C.regexes['parenthesis'] = re.compile(r'\[(.*?)\]', re.U)
hn.parse_full_name()
self.m(hn.first, "Benjamin", hn)
self.m(hn.last, "Franklin", hn)
self.m(hn.nickname, "Ben", hn)

def test_two_word_nickname_in_parenthesis(self) -> None:
hn = HumanName("Benjamin (Big Ben) Franklin")
self.m(hn.first, "Benjamin", hn)
Expand Down Expand Up @@ -142,6 +187,17 @@ def test_ambiguous_suffix_acronym_in_parenthesis_stays_nickname(self) -> None:
self.m(hn.nickname, "JD", hn)
self.m(hn.suffix, "", hn)

def test_ambiguous_suffix_acronym_in_extra_delimiter_stays_nickname(self) -> None:
# Same suffix-vs-nickname disambiguation as above, but through a
# custom delimiter added via extra_nickname_delimiters -- confirms
# handle_match() is applied uniformly regardless of which delimiter
# matched, not just the three built-ins.
hn = HumanName("JEFFREY {JD} BRICKEN", constants=None)
hn.C.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
hn.parse_full_name()
self.m(hn.nickname, "JD", hn)
self.m(hn.suffix, "", hn)


# class MaidenNameTestCase(HumanNameTestBase):
#
Expand Down