Skip to content

✅ test: gate GraalPy gaps on probed capabilities#682

Merged
gaborbernat merged 1 commit into
fix-graalpy-enotsupfrom
graalpy-test-compat
Jul 20, 2026
Merged

✅ test: gate GraalPy gaps on probed capabilities#682
gaborbernat merged 1 commit into
fix-graalpy-enotsupfrom
graalpy-test-compat

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

Stacked on #681, whose ENOTSUP fix filelock needs before it will import on GraalPy at all. Review that one first;
the base here is its branch, so this diff is the test-only commit on top.

GraalPy 24.2.2 (macos-aarch64, uv python install graalpy-3.11.0) ran the suite with 55 failures. GraalPy is not in
CI, so CI cannot validate any of this. I wrote every gate to hold on Linux, macOS, Windows, Termux/Android and
pypy3.11 too, and checked pypy3.11 by running it.

Where the 55 went

Category Count Outcome
asyncio cancellation 33 xfail, non-strict (GraalPy contextlib bug, characterized)
audit events 7 skip on audit-events
deferred finalization 6 skip on prompt-finalization/collected-finalization
os.mkfifo missing 4 skip on the existing fifo capability
dir_fd fault injection 2 skip on link-dir-fd
class collection 2 skip on class-collection
O_NOFOLLOW ignored 1 skip on o-nofollow, now probed
harness assumptions 3 fixed, run everywhere

Final GraalPy run: 1105 passed, 153 skipped, 28 xfailed, 5 xpassed, 0 failed.

Genuinely fixed (3)

These three encoded an assumption of the harness, so they now run on every runtime instead of skipping:

  • test_write_non_starvation counted reader releases from the moment it spawned the writer process. GraalPy needs
    seconds to boot an interpreter, so the reader chain drained before the writer ever reached the lock and a slow
    start read as starvation. It now snapshots the count when the writer signals it has reached the lock and asserts
    on the delta. On CPython the snapshot is 0, so the assertion is unchanged.
  • Two heartbeat tests compared a utime round-trip for bit equality. GraalPy returns the timestamp one nanosecond
    off, which made them fail about one run in three. They compare with a tolerance now, which also covers filesystems
    with coarse timestamp granularity. The property under test spans 1000 seconds, so a millisecond tolerance weakens nothing.
  • test_singleton_instance_tracking_is_unique_per_subclass carried a hasattr(sys, "pypy_version_info") skip it
    never needed, since it touches no finalizer. I ran it on PyPy 3.11 without the skip and it passes.

Guarded on a probed capability (16)

These live in tasks/coverage_pragmas.py alongside the existing probes, keeping a skip and a coverage exclusion
from drifting apart. Each one probes the behavior instead of reading sys.implementation.name:

  • prompt-finalization holds where __del__ runs as the last reference drops, and replaces the three
    hasattr(sys, "pypy_version_info") gates. I measured CPython True, PyPy False, GraalPy False, so PyPy keeps
    skipping exactly what it skipped and CPython keeps running it.
  • collected-finalization holds where gc.collect() runs a pending __del__: CPython True, PyPy True,
    GraalPy False. GraalPy hands finalizers to the host collector and never gets them back.
  • class-collection holds where gc.collect() reclaims a dynamically built class. GraalPy never does, even
    after 30 collections.
  • audit-events holds where sys.audit reaches a hook from sys.addaudithook. GraalPy accepts the hook and
    calls it for nothing, so seven tests that drive filelock through sqlite3.connect events see an empty stream.
  • link-dir-fd narrows the existing dir-fd, which probes os.open. GraalPy has
    os.open in os.supports_dir_fd but not os.link, so _link_no_replace skips its dir_fd branch and never opens
    the directory that two fault-injection tests count their faults against (7 directory closes vs CPython's 8).
  • o-nofollow used to read hasattr(os, "O_NOFOLLOW"), and GraalPy defines the flag then follows the link
    anyway. It now opens a real symlink and asks for the refusal, matching how link-follow-symlinks already handles
    PyPy. It falls back to the constant where symlinks cannot be created, leaving Windows unaffected.

xfail (33 instances across 20 tests)

GraalPy raises RuntimeError: generator didn't stop after athrow() from its own
contextlib._GeneratorContextManagerBase.__aexit__. Reduced to ten lines with no filelock involved:

@asynccontextmanager
async def gate(aw):
    await aw
    yield

async def second():
    async with gate(asyncio.sleep(10)):
        pass

Cancelling second while it is suspended inside __aenter__ raises RuntimeError instead of CancelledError on
GraalPy 24.2.2, and CancelledError on CPython 3.11 and PyPy 3.11. filelock's @asynccontextmanager use is
idiomatic and the raising frame belongs to the interpreter stdlib, so these are xfail rather than skip and should
start passing once GraalPy fixes it. The gate drives a coroutine by hand, needing no event loop, and is strict=False because the
deviation depends on where the cancellation lands (5 of the 33 still pass).

The same defect clears __context__ when an exception is thrown into a suspended frame, which is what
generator-exception-context covers for test_finish_connection_chains_rollback_then_close_error and
test_coroutine_on_acquired_error_preserves_context.

CPython is unaffected

  • tox r -e 3.14 on darwin: 100% line and branch coverage, 12710/12710 lines, diff-cover 100%.
  • src/filelock is untouched.
  • No pre-existing capability value changes on CPython/darwin (o-nofollow stays True), and no new pragma appears
    in src/, so CPython measures an identical line set. The one new # pragma: needs link-dir-fd sits on a test
    decorator and excludes nothing where the capability is present.
  • No GraalPy-specific coverage pragma turned out to be necessary. Coverage runs only on CPython, where the guarded
    tests never provide the sole cover for a src/ line.
  • pypy3.11: 1226 passed, 65 skipped, 0 failed.
  • tox r -e type and tox r -e fix clean.

No changelog entry: #679 made the same kind of test-and-coverage-infrastructure change without one.

GraalPy 24.2.2 ran the suite with 55 failures. Each test now gates on a probe of
the behavior it needs rather than on an interpreter name.

Three tests needed a fix, not a gate:

- test_write_non_starvation counted reader releases from the moment it started
  the writer process, so a slow interpreter start read as starvation. It now
  counts from the moment the writer reaches the lock.
- Two heartbeat tests compared a utime round-trip for bit equality, and GraalPy
  returns the timestamp one nanosecond off. They now compare with a tolerance,
  which also covers filesystems with coarse timestamps.
- test_singleton_instance_tracking_is_unique_per_subclass carried a PyPy skip it
  never needed. It touches no finalizer, so it now runs everywhere.

The new capabilities live in tasks/coverage_pragmas.py beside the existing
probes, keeping a skip and a coverage exclusion from disagreeing:

- prompt-finalization replaces three hasattr(sys, "pypy_version_info") gates. It
  holds only on a refcounting collector, so PyPy keeps skipping those tests and
  CPython keeps running them.
- collected-finalization and class-collection cover a GraalPy that runs no
  __del__ and reclaims no class, even after a forced gc.collect().
- audit-events holds where sys.audit reaches an installed hook. GraalPy accepts
  the hook and never calls it, so seven tests that drive filelock through
  sqlite3.connect events observe nothing.
- link-dir-fd is narrower than dir-fd. GraalPy takes os.open relative to a
  directory descriptor but not os.link, so the backend never opens the directory
  that two fault-injection tests count their faults against.
- o-nofollow now probes the refusal instead of the constant, because GraalPy
  defines O_NOFOLLOW and follows the link anyway.

Twenty cancellation tests are xfail rather than skipped. GraalPy answers
athrow() with "RuntimeError: generator didn't stop after athrow()" raised from
its own contextlib, so a cancellation crossing an async context manager arrives
as the wrong exception. The gate drives a coroutine by hand to observe that, and
stays non-strict because the deviation depends on where the cancellation lands.

src/filelock is untouched and no pre-existing capability value changes, so
CPython measures the lines it measured before.
@gaborbernat gaborbernat added the skip news Internal change; exempt from the news fragment check label Jul 20, 2026
@gaborbernat gaborbernat reopened this Jul 20, 2026
@gaborbernat
gaborbernat merged commit d360b0f into fix-graalpy-enotsup Jul 20, 2026
112 of 114 checks passed
@gaborbernat
gaborbernat deleted the graalpy-test-compat branch July 20, 2026 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip news Internal change; exempt from the news fragment check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant