Skip to content

CI: pytest-matrix py3.13 segfaults during garbage collection (pre-existing, blocks unrelated PRs) #363

Description

@ywatanabe1989

pytest-matrix-on-ubuntu-py3.13 has been failing with a hard Segmentation fault (core dumped). It is not a test failure — the interpreter dies — and the workflow's three retries all crash. py3.11 and py3.12 pass on the same commits.

It is pre-existing, not caused by any current PR

Two independent controls:

  1. Two unrelated branches fail the identical check with the identical crash: PR chore(sqlite-out): eradicate SQLite vocabulary from scitex-python (58 -> 0) #362 (feat/eradicate-sqlite-vocabulary, job 99189287240, fail at 4m10s) and PR chore: purge every trace of the SQLite engine #361 (chore/purge-sqlite, job 99159485168, fail at 4m13s). Both are documentation/vocabulary sweeps that touch no runtime code path involved here. Two unrelated changesets cannot both introduce the same segfault.
  2. develop's own last tests workflow run (2026-08-10) has conclusion failure. The branch was red before either PR existed.

The crash is in the garbage collector, not in any one call site

This matters for whoever picks it up, because the three crashes have three different Python stacks but the same top frame:

Fatal Python error: Segmentation fault
Current thread 0x00007f2292456b80 (most recent call first):
  Garbage-collecting
  File ".../python3.13/typing.py", line 1031 in __init__
  File ".../python3.13/typing.py", line 169 in _type_convert
  File ".../python3.13/typing.py", line 191 in _type_check
  File ".../site-packages/typing_extensions.py", line 1233 in __new__
  File ".../site-packages/openai/types/responses/response_compaction_item_param_param.py", line 11 in <module>
  ... importlib frames ...
  File ".../src/scitex/helpers/_optional_deps.py", line 217 in check_optional_deps
  File ".../src/scitex/helpers/_install_guide.py", line 81 in check_module_deps

The second and third crashes in the same job land somewhere else entirely — _pytest/nodes.py:286 in iter_parentspytest_timeout.py:392 in _get_item_settings — with no scitex frame at all.

So check_optional_deps appears in one of three crashes, and only as the caller at the bottom of the stack (it does an importlib.import_module of openai); Garbage-collecting is the crash site in all three. Reading the first traceback alone would send someone to "fix" check_optional_deps, which is not where the fault is. The GC simply happens to fire during whatever allocation-heavy work is in flight.

Extension modules: ... (total: 199) — including numpy, pandas, pyarrow, h5py, scipy, lxml, greenlet, sqlalchemy.cyextension, psycopg2. A GC crash with 199 C extensions loaded under CPython 3.13 points at a C-extension that mis-implements tp_traverse/tp_clear (or an incompatibility with 3.13's GC changes), not at SciTeX Python code.

Suggested next steps

  • Reproduce locally on 3.13 with PYTHONMALLOC=debug and faulthandler to get a consistent stack.
  • Bisect by extension: -p no:randomly, then selectively skip the heaviest importers, to find which extension's presence is required for the crash.
  • gc.freeze() after imports, or gc.disable() during collection, as a diagnostic (not a fix) to confirm the GC is the trigger.
  • If it reproduces without SciTeX in the picture, it belongs upstream.

Until then, py3.13 in the matrix is a known-red check and should not block review of unrelated PRs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions