Skip to content

ci: split pytest suites and schedule environment smoke tests - #95

Merged
wlgys8 merged 1 commit into
mainfrom
94-optimize-pytest-ci
Oct 2, 2026
Merged

wlgys8 merged 1 commit into
mainfrom
94-optimize-pytest-ci

Conversation

@wlgys8

@wlgys8 wlgys8 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Closes #94

Summary

  • split pytest CI into parallel fast and numba suites for PRs and pushes to main/stable
  • schedule the full suite, including all-environment subprocess smoke tests, nightly and on manual dispatch
  • register and apply slow, numba, and integration pytest markers
  • keep the complete WBT Numba module together so first-use kernel compilation is paid by the dedicated suite rather than the fast suite
  • upload JUnit test results and report the slowest test phases

Validation

  • prek run --files .github/workflows/README.md .github/workflows/tests.yml pyproject.toml motrix_envs/tests/test_wbt_numba.py test/test_all_envs.py
  • fast suite: 1007 passed, 2 skipped in 78.33s
  • Numba suite: 37 passed in 59.25s
  • pytest --collect-only --strict-markers for fast, Numba, and integration selections
  • git diff --check

The full local pytest command remains unchanged and continues to run all tests.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Renaming the single pytest check into pytest (fast)/(numba)/(full) changes merge-gating status checks and requires a human to update branch-protection settings to keep required checks enforced.

Review effort: Balanced
Findings: None

What changed in this PR

This PR restructures the tests.yml CI workflow to reduce day-to-day feedback latency while preserving Numba-runtime and full-environment coverage (closes #94). It introduces pytest markers to tier tests, splits PR/push runs into parallel fast and numba suites, and defers the expensive all-environment subprocess smoke test (test_all_demos) to a nightly/manual-dispatch full suite. Documentation in the workflow README explains the tiers, local commands, and guidance for future Numba environments.

Changes:

  • Register slow/numba/integration markers in pyproject.toml and apply them (test_all_demos → slow+integration; whole test_wbt_numba.py module → numba, two long tests → slow).
  • Replace the single CI job with an event-driven matrix: fast/numba on PR/push, full on schedule/dispatch; add JUnit upload and --durations=20 with --strict-markers.
  • Add a nightly cron, per-event concurrency group, per-suite Numba cache keys, and expanded README documentation.
File Description
test/​test_all_envs.py Marks the all-environment smoke test as slow+integration so it only runs in the full/nightly suite.
pyproject.toml Registers the slow, numba, and integration pytest markers used for CI tiering.
motrix_envs/​tests/​test_wbt_numba.py Module-level numba marker keeps WBT kernels in one suite; adds slow to the two expensive tests.
.github/​workflows/​tests.yml Event-driven suite matrix, nightly schedule, per-suite caches, JUnit artifacts, and --strict-markers.
.github/​workflows/​README.md Documents the new suite tiers, local commands, marker semantics, and future-Numba guidance.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@wlgys8
wlgys8 force-pushed the 94-optimize-pytest-ci branch from 3749fdd to 665bc7c Compare October 2, 2026 11:09
@wlgys8
wlgys8 requested a balanced review from Copilot October 2, 2026 11:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The CI/marker changes are low-risk, but the accompanying rewrites to core Numba kernel cloning, fingerprinting, and compilation-cache logic are subtle and correctness-critical, warranting final human review.

Review effort: Balanced
Findings: None

@wlgys8
wlgys8 force-pushed the 94-optimize-pytest-ci branch from 665bc7c to 1a1c588 Compare October 2, 2026 11:45
@wlgys8
wlgys8 merged commit a99ee24 into main Oct 2, 2026
6 checks passed
@wlgys8
wlgys8 deleted the 94-optimize-pytest-ci branch October 2, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: 分层 pytest 测试并将全环境 smoke test 移至 nightly

2 participants