Run tests in CI, and gate releases on them - #10
Conversation
Nothing ran the tests: .github/workflows held only publish.yml and release.yml, both firing on a tag, so a red commit could reach the ComfyUI Registry — where it lands on real installs. test.yml runs pytest and the node check on pull_request and on pushes to main, and exposes workflow_call so release.yml and publish.yml can each depend on it. A tag whose tests fail now ships nothing, to the Registry or to GitHub Releases. Python pinned to 3.10, the floor in requires-python; the node check names its file because node --test on a bare directory fails when it discovers nothing.
There was a problem hiding this comment.
Pull request overview
Adds a dedicated CI test workflow and wires release/publish workflows to depend on it, preventing tags from publishing or releasing when tests fail.
Changes:
- Introduces
.github/workflows/test.ymlto run Python (pytest) and Node (node --test) test suites on PRs and pushes tomain, and as a reusable workflow. - Updates
.github/workflows/release.ymlto run the test workflow first and gate the release job on it. - Updates
.github/workflows/publish.ymlto run the test workflow first and gate the publish job on it.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| .github/workflows/test.yml | Adds a reusable CI workflow that runs backend (pytest) and frontend (node test runner) tests. |
| .github/workflows/release.yml | Adds a test job and gates the release job on it. |
| .github/workflows/publish.yml | Adds a test job and gates the publish job on it. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| - name: Backend tests | ||
| run: | | ||
| pip install -e ".[test]" |
There was a problem hiding this comment.
Done — python -m pip and python -m pytest, so both target the interpreter setup-python provisioned.
| test: | ||
| uses: ./.github/workflows/test.yml |
There was a problem hiding this comment.
Fixed, in both directions.
test.yml now declares permissions: contents: read at the workflow level, so every caller gets the reduced scope rather than each one remembering to pass it — this job runs repo code via pip install -e . and never writes anything back.
release.yml also had contents: write at the workflow level; that has moved onto the release job, which is the only thing that needs it to create the release.
| test: | ||
| uses: ./.github/workflows/test.yml | ||
|
|
||
| release: | ||
| needs: test |
There was a problem hiding this comment.
Correct, and it is a pre-existing bug rather than something this PR introduced — release.yml has always checked out the dispatch ref. publish.yml already pinned ref: ${{ github.event.inputs.tag || github.ref_name }}; release.yml now matches it.
Outside this PR's original scope, but it is two lines, the fix already existed in the sibling file, and leaving a known-wrong release-notes path in a file I am editing anyway is the worse call. Flagged in the PR description.
Note it does not extend to the reusable test job: a workflow_call job runs at the caller's ref, so a manual backfill still tests main. That corner is called out in the description as accepted.
…on manual release Review follow-ups on #10. test.yml declares contents: read at workflow level so every caller gets it — release.yml granted contents: write workflow-wide, which the reusable job inherited despite only running repo code. That write scope now sits on the release job alone. Also fixes a pre-existing bug the review surfaced: release.yml checked out the dispatch ref, so re-releasing an old tag read main's CHANGELOG rather than the tag's. publish.yml already had the ref pin; release.yml now matches.
Nothing has ever run the tests in CI.
.github/workflows/held onlypublish.ymlandrelease.yml, both triggered by a tag — so a red commit could reach the ComfyUI Registry, where it lands on real installs. The 293 backend tests only ever ran on a maintainer machine.What this does
test.yml—pytest -qplusnode --test tests/test_upload.mjs, onpull_requestand pushes tomain. Also exposesworkflow_call.release.yml/publish.yml— each gains atestjob (uses: ./.github/workflows/test.yml) andneeds: test. A red tag now ships nothing, to the Registry or to GitHub Releases.CLAUDE.md— the commands block still said "there is no JS test harness"; there is one as of Folder upload: drag a folder in, or pick one (#5) #9.Choices
requires-python. One version, no matrix — add one when a version-specific bug actually bites.node --test <dir>treats a directory it finds no tests in as a missing module and fails outright.Known corner, accepted
The manual
workflow_dispatchchangelog-backfill path tests the dispatch ref (main) rather than the old tag being backfilled. Costs a minute, and can block a backfill while main is red — safe-fail, and not worth conditional gymnastics to avoid.Verification
Both suites pass on this branch (293 passed / 1 skipped; 3 JS tests), and all three workflow files parse with the expected job graphs —
test,test + publish,test + release. The gating itself only proves out on the next tag push.