Skip to content

ci(examples): run example scripts once in their own job - #430

Closed
d-v-b wants to merge 3 commits into
mainfrom
ci/run-examples-once
Closed

d-v-b wants to merge 3 commits into
mainfrom
ci/run-examples-once

Conversation

@d-v-b

@d-v-b d-v-b commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

🤖 AI text below 🤖

Example scripts now run once, in a new Example scripts CI job, instead of in every pytest matrix job. tests/test_examples.py uses uv add --script to point each script's PEP 723 header at the local checkout, and runs the script with uv run.

Why

  • Each example runs in an environment built from its own header, so every matrix job ran the same thing.
  • The old test rewrote headers with a regex, tomlkit and packaging, which run in the test environment. In zarr-python PR 4544 this broke the min_deps job: with packaging==22.* on Python 3.12.3, the URL zarr @ file:////home/... (four slashes) is rejected as invalid.
  • uv add resolves the header. That showed that sharding_coalescing.py and codec_pipeline_performance.py declared requires-python = ">=3.11", although zarr requires 3.12. Both headers now say >=3.12.

Changes

  • tests/test_examples.py: one test per example, marked examples. The test copies the script, runs uv add --script <copy> <checkout>, then runs uv run --reinstall-package zarr <copy>. The regex, tomlkit and packaging code is gone. The checkout path now comes from __file__ instead of the working directory. The Windows skip is removed, because no Windows job runs these tests.
  • tests/conftest.py: new --run-examples option. Tests marked examples are skipped without it, in the same way that slow_hypothesis tests are skipped without --run-slow-hypothesis. The matrix jobs therefore skip them without any change to their commands.
  • pyproject.toml: registers the examples marker, and removes tomlkit from the test dependency group, which had no other users. uv stays, because tests/test_regression/test_v2_dtype_regression.py uses it.
  • Justfile: new test-examples recipe, which runs pytest --run-examples tests/test_examples.py.
  • .github/workflows/test.yml: new examples job, which runs just test-examples in the test.py3.13-minimal hatch environment. It is added to test-complete.
  • examples/README.md: says that an example needs a PEP 723 header, and how examples are run.

Testing

  • HATCH_ENV=test.py3.13-minimal just test-examples passes locally: the path check and all three examples.
  • Without --run-examples, the three example tests are skipped. slow_hypothesis tests are still skipped without --run-slow-hypothesis.
  • With two examples broken at once (one exits non-zero, one has a >=3.11 header), both tests fail with their own message, and the third example still runs.
  • prek run --all-files passes.
Notes
  • The first commit ran the examples from a bash loop in the Justfile. The second commit replaces it with pytest, because the loop stopped at the first failing example and hid the others.
  • No changelog fragment: the change affects CI and contributors, not library users.
  • ci/big-endian/Dockerfile still installs python3-tomlkit. The big-endian job does not run this test, so the package is unused there. The agent left the image unchanged because it could not build the s390x image locally.

🤖 Generated with Claude Code

Each example's PEP 723 header defines its environment, so running
tests/test_examples.py in every pytest matrix job repeated the same
work. Its header-rewriting helper also depended on the test
environment's packaging and tomlkit, which broke the min_deps job on
Python 3.12.3. A new `just examples` recipe uses `uv add --script` to
point each header at the local checkout, and one CI job runs it.

`uv add` resolves the header, and that showed two examples declaring
requires-python >=3.11 although zarr requires 3.12. Those headers now
say >=3.12. tomlkit was only used by the removed test, so it is
dropped from the test dependency group.

Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b added 2 commits October 31, 2026 21:22
The previous commit ran the examples from a bash loop, which stops at
the first failing script, so one broken example hid the others. The
examples now run from tests/test_examples.py again, one test per
script. The tests are marked `examples` and skipped unless pytest gets
--run-examples, as slow_hypothesis tests are with
--run-slow-hypothesis, so the matrix jobs skip them. `just examples`
passes the option, and the CI job runs it in the test.py3.13-minimal
hatch environment.

Assisted-by: ClaudeCode:claude-opus-5-5
`just examples` could mean listing or building the examples.
`test-examples` says what the recipe does, and follows the
`<group>-<variant>` naming of recipes like `coverage-html` and
`docs-build`.

Assisted-by: ClaudeCode:claude-opus-5-5
@d-v-b

d-v-b commented Nov 2, 2026

Copy link
Copy Markdown
Owner Author

🤖 AI text below 🤖

Superseded by zarr-developers#4586, which merged this branch upstream.

@d-v-b d-v-b closed this Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant