Skip to content

Fix path separator mismatch in open_mfdatatree on Windows - #87

Open
ClaudioStrumia wants to merge 1 commit into
mainfrom
fix/pathlib-openmfdatatree
Open

Fix path separator mismatch in open_mfdatatree on Windows#87
ClaudioStrumia wants to merge 1 commit into
mainfrom
fix/pathlib-openmfdatatree

Conversation

@ClaudioStrumia

Copy link
Copy Markdown
Collaborator

Problem

On Windows, glob.glob() returns file paths using backslashes (\) as
separators, while the path descriptor passed to open_mfdatatree (and the
regex built from it) assumes forward slashes (/).

This mismatch causes two issues:

  • The regex built from paths fails to match the backslash-separated
    filenames returned by glob, since regex.match(fname) returns None
    and calling .group(field) on it raises an AttributeError.
  • Even before that, backslash sequences coming from typical Windows paths
    (e.g. C:\Users\...) can be interpreted by re.compile as invalid
    regex escape sequences (e.g. \U is parsed as the start of an
    8-digit unicode escape), raising a re.PatternError: incomplete escape.

Fix

Both the path descriptor (paths) and the paths returned by glob
(fnames) are now normalized to use forward slashes via
Path(...).as_posix(). This ensures the regex is always built and matched
against POSIX-style paths, regardless of the OS.

On POSIX systems (Linux/macOS) this is a no-op, since paths already use
forward slashes there.

Testing

Ran the full test suite on Windows before and after the fix. The following
three tests, which were failing before the fix with
re.PatternError: incomplete escape \U at position 2, now pass:

  • tests/test_routines.py::TestOpen::test_open_multiple_file_tree
  • tests/test_routines.py::TestOpenMFDataTree::test_one_level_depth
  • tests/test_routines.py::TestOpenMFDataTree::test_two_level_depth

No new test failures were introduced (remaining failures are pre-existing
and unrelated to this change, e.g. os.sysconf not available on Windows,
multiprocessing/shared-memory handling, and : not being a valid character
in Windows filenames).

@atrabattoni atrabattoni self-assigned this Sep 7, 2026
@atrabattoni atrabattoni added the bug Something isn't working label Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants