Skip to content

gh-156444: Fix a negated character set with IGNORECASE and LOCALE - #156445

Merged
serhiy-storchaka merged 1 commit into
python:mainfrom
serhiy-storchaka:gh-156444-charset-loc-ignore
Aug 27, 2026
Merged

gh-156444: Fix a negated character set with IGNORECASE and LOCALE#156445
serhiy-storchaka merged 1 commit into
python:mainfrom
serhiy-storchaka:gh-156444-charset-loc-ignore

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Aug 27, 2026

Copy link
Copy Markdown
Member

charset_loc_ignore() tested the whole set once per locale case and returned true if either test matched. That complements a set before closing it under case rather than after, so [bc] matched b'B' and [^bc] matched it too.

It now walks the set itself and tries both cases for every member, which puts the closure on the members and leaves NEGATE complementing the closed set. SRE(charset) is untouched, and an uncased character short-circuits straight into it.

BIGCHARSET and RANGE_UNI_IGNORE are not handled: neither can occur in the set of a bytes pattern, and re.LOCALE rejects str patterns.

This also fixes gh-155985. The difference fusion is safe once the engine handles an embedded NEGATE correctly, so the workaround in GH-155993 is no longer needed:

>>> re.fullmatch(rb'\w(?<!b)', b'B', re.I | re.L)
>>> re.fullmatch(rb'[A-z--[0-m]]', b'N', re.I | re.L)

charset_loc_ignore() tested the whole set once per locale case and took
the disjunction, which complements a set before closing it under case
instead of after: [bc] matched b'B', but so did [^bc].  Match both cases
of the character against every set member instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@serhiy-storchaka serhiy-storchaka added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Aug 27, 2026
@serhiy-storchaka
serhiy-storchaka merged commit 6671d27 into python:main Aug 27, 2026
108 of 110 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

@serhiy-storchaka
serhiy-storchaka deleted the gh-156444-charset-loc-ignore branch August 27, 2026 11:11
@bedevere-app

bedevere-app Bot commented Aug 27, 2026

Copy link
Copy Markdown

GH-156456 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Aug 27, 2026
@bedevere-app

bedevere-app Bot commented Aug 27, 2026

Copy link
Copy Markdown

GH-156457 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Aug 27, 2026
@bedevere-app

bedevere-app Bot commented Aug 27, 2026

Copy link
Copy Markdown

GH-156458 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Aug 27, 2026
serhiy-storchaka added a commit that referenced this pull request Aug 27, 2026
…ALE (GH-156445) (GH-156457)

charset_loc_ignore() tested the whole set once per locale case and took
the disjunction, which complements a set before closing it under case
instead of after: [bc] matched b'B', but so did [^bc].  Match both cases
of the character against every set member instead.
(cherry picked from commit 6671d27)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
serhiy-storchaka added a commit that referenced this pull request Aug 27, 2026
…ALE (GH-156445) (GH-156458)

charset_loc_ignore() tested the whole set once per locale case and took
the disjunction, which complements a set before closing it under case
instead of after: [bc] matched b'B', but so did [^bc].  Match both cases
of the character against every set member instead.
(cherry picked from commit 6671d27)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Negative lookbehind ignored under re.IGNORECASE | re.LOCALE

1 participant