Skip to content

Migrate off setuppy#245

Merged
JayBazuzi merged 9 commits intomainfrom
migrate-off-setuppy
Mar 1, 2026
Merged

Migrate off setuppy#245
JayBazuzi merged 9 commits intomainfrom
migrate-off-setuppy

Conversation

@JayBazuzi
Copy link
Contributor

@JayBazuzi JayBazuzi commented Mar 1, 2026

Summary by Sourcery

Switch packaging and publishing workflows from direct setup.py invocation to PEP 517/518 build tooling and adjust tests and config accordingly.

Enhancements:

  • Update internal build scripts to construct wheels via python -m build instead of calling setup.py directly.
  • Make mypy acceptance test build and install wheels from the dist directory generically rather than constructing wheel filenames manually.
  • Ensure temporary setup.py files are cleaned up robustly with a retrying unlink helper.

Build:

  • Introduce a minimal pyproject.toml declaring setuptools as the build backend.
  • Change publish shell scripts to copy the appropriate setup.*.py to setup.py and invoke python -m build before uploading with twine.
  • Update setup.publish.py and setup.minimal.py to adjust sys.path so shared setup utilities can be imported from a dedicated setup directory.
  • Remove the legacy top-level setup/setup.py file and migrate off direct setup.py usage.

CI:

  • Update the publish_to_pypi GitHub workflow to install the build package in addition to setuptools, wheel, and twine.

Tests:

  • Adjust the mypy integration test to build wheels with python -m build, detect the generated wheel file dynamically, and reinstall it for type-checking.

Chores:

  • Update tidy_code.sh to run git add --renormalize . after code tidying.

JayBazuzi and others added 9 commits March 1, 2026 19:00
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
Co-Authored-By: Llewellyn Falco <llewellyn.falco@gmail.com>
@sourcery-ai
Copy link

sourcery-ai bot commented Mar 1, 2026

Reviewer's Guide

Migrates the project from directly invoking setup.py to using PEP 517 python -m build for wheels/sdists, while updating test, publish scripts, and CI to work with a temporary setup.py shim and adding a minimal pyproject.toml build-system declaration.

Sequence diagram for the new publish script build process

sequenceDiagram
    actor Maintainer
    participant Shell
    participant Publish_script as publish_approvaltests_sh
    participant Setup_py_shim as setup_py
    participant Build_module as python_m_build
    participant Setuptools_backend as setuptools_build_meta
    participant Dist_dir as dist_directory
    participant Twine

    Maintainer->>Shell: run publish_approvaltests_sh
    Shell->>Publish_script: execute script
    Publish_script->>Publish_script: copy setup_setup_publish_py to setup_py
    Publish_script->>Build_module: python -m build .
    Build_module->>Setuptools_backend: build_sdist_and_wheel
    Setuptools_backend->>Setup_py_shim: import setup_utils and call do_the_setup
    Setup_py_shim->>Setuptools_backend: provide package_metadata_and_config
    Setuptools_backend->>Dist_dir: write wheel_and_sdist_files
    Publish_script->>Publish_script: remove setup_py
    Publish_script->>Twine: twine upload dist_files
    Twine->>Dist_dir: read built_artifacts
Loading

File-Level Changes

Change Details Files
Switch test wheel-building flow to use python -m build with a temporary setup.py and generic wheel discovery.
  • Replace direct execution of per-package setup scripts with python -m build --wheel . after copying the relevant setup script to setup.py.
  • Ensure any pre-existing dist directory is removed before each build and validate exactly one wheel is produced using globbing.
  • Install the built wheel using the discovered filename instead of constructing it from version and build number.
  • Add a quiet mode to the helper that runs Python subprocesses, suppressing stdout/stderr when requested.
  • Introduce a retrying unlink helper to reliably remove the temporary setup.py file.
test__mypy_accepts_our_packages.py
Update publish scripts and setup entry points to work with PEP 517 builds via a temporary setup.py and an importable setup_utils module.
  • Modify publish shell scripts to copy the appropriate setup.* script to setup.py, invoke python -m build ., then remove setup.py and upload with twine.
  • Adjust setup.publish.py and setup.minimal.py to locate and prepend a setup subdirectory to sys.path so they can import do_the_setup and requirement helpers from setup_utils.
  • Remove the legacy setup/setup.py in favor of per-target setup scripts that are used as shims for building.
setup/publish_approvaltests.sh
setup/publish_minimal.sh
setup/publish_approval_utilities.sh
setup/setup.publish.py
setup/setup.minimal.py
setup/setup.approval_utilities.py
setup/setup.py
Align tooling and configuration with the new build process and repository hygiene.
  • Add a minimal pyproject.toml declaring setuptools/wheel as the build-system requirements and setuptools.build_meta as the backend.
  • Update the GitHub Actions publish workflow to install the build package in addition to setuptools, wheel, and twine.
  • Exclude the top-level setup.py shim from mypy checks since it is now generated transiently.
  • Extend the tidy_code.sh script to run git add --renormalize . after tidying, likely to normalize line endings or attributes.
  • Leave .gitignore, requirements.dev.txt, and some setup scripts touched (possibly formatting or normalization) without semantic changes.
pyproject.toml
.github/workflows/publish_to_pypi.yml
mypy.ini
tidy_code.sh
.gitignore
requirements.dev.txt

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@JayBazuzi JayBazuzi merged commit f69fb5c into main Mar 1, 2026
86 checks passed
@JayBazuzi JayBazuzi deleted the migrate-off-setuppy branch March 1, 2026 19:56
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues, and left some high level feedback:

  • In the shell publish scripts, consider using trap to ensure setup.py is removed even if python -m build . or twine upload fails, so you don’t leave a stray setup.py in the repo on error.
  • The _setup_dir logic in setup.setup.publish.py and setup.setup.minimal.py uses Path(__file__).resolve().parent / "setup", which for files already in setup/ points to setup/setup; if setup_utils.py lives directly in setup/, you can simplify this to Path(__file__).resolve().parent (or drop the sys.path manipulation entirely) to avoid confusion and potential path issues.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In the shell publish scripts, consider using `trap` to ensure `setup.py` is removed even if `python -m build .` or `twine upload` fails, so you don’t leave a stray `setup.py` in the repo on error.
- The `_setup_dir` logic in `setup.setup.publish.py` and `setup.setup.minimal.py` uses `Path(__file__).resolve().parent / "setup"`, which for files already in `setup/` points to `setup/setup`; if `setup_utils.py` lives directly in `setup/`, you can simplify this to `Path(__file__).resolve().parent` (or drop the `sys.path` manipulation entirely) to avoid confusion and potential path issues.

## Individual Comments

### Comment 1
<location path="tidy_code.sh" line_range="5" />
<code_context>
 set -euo pipefail

 mise task --quiet run tidy_code
+git add --renormalize .
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Be cautious about staging all files from within a tidy script.

`git add --renormalize .` will stage *all* tracked files, not just those affected by this tidy run, which can lead to accidentally committing unrelated changes (e.g., widespread line-ending updates). If you only want to normalize a subset of paths, consider limiting the command to specific directories or omitting `git add` here and leaving staging to the caller.
</issue_to_address>

### Comment 2
<location path="test__mypy_accepts_our_packages.py" line_range="27-28" />
<code_context>
+        finally:
+            _unlink_with_retry(pathlib.Path("setup.py"))
+
+        wheel_files = glob.glob("dist/*.whl")
+        assert len(wheel_files) == 1, f"Expected 1 wheel, found {wheel_files}"
         _run_python_checked(
             [
</code_context>
<issue_to_address>
**suggestion (testing):** Also assert that the wheel belongs to the expected package, not just that there is exactly one wheel.

As written, this will pass as long as there’s exactly one wheel in `dist/`, even if it’s for a different package. To make the test safer, also assert that the wheel filename matches the expected package name, e.g. `Path(wheel_files[0]).name.startswith(f"{package_name}-")`, so mismatches between the built wheel and the package under test are caught.
</issue_to_address>

### Comment 3
<location path="test__mypy_accepts_our_packages.py" line_range="52-61" />
<code_context>


 def _run_python_checked(
-    args: typing.List[str], cwd: typing.Optional[pathlib.Path] = None
+    args: typing.List[str],
+    cwd: typing.Optional[pathlib.Path] = None,
+    quiet: bool = False,
 ) -> None:
     subprocess.run(
         [sys.executable, *args],
         check=True,
         cwd=cwd,
+        stdout=subprocess.DEVNULL if quiet else None,
+        stderr=subprocess.DEVNULL if quiet else None,
     )

</code_context>
<issue_to_address>
**suggestion:** Quiet mode hides build output, which can make diagnosing failing packaging tests difficult.

Using `quiet=True` here suppresses both stdout and stderr, so if the build fails in CI you lose the error details needed to debug packaging issues. Consider keeping stderr visible (or redirecting it to a log file) when `quiet` is enabled, or only applying `quiet` to the subsequent `pip install` step so failures remain diagnosable.

Suggested implementation:

```python
        # Suppress only stdout in quiet mode so build / packaging errors remain visible
        stdout=subprocess.DEVNULL if quiet else None,
        stderr=None,

```

If any callers relied on stderr being suppressed by `quiet=True`, they may need to be updated to handle visible error output now (for example, by explicitly redirecting or capturing stderr at the call site if necessary).
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

set -euo pipefail

mise task --quiet run tidy_code
git add --renormalize .
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Be cautious about staging all files from within a tidy script.

git add --renormalize . will stage all tracked files, not just those affected by this tidy run, which can lead to accidentally committing unrelated changes (e.g., widespread line-ending updates). If you only want to normalize a subset of paths, consider limiting the command to specific directories or omitting git add here and leaving staging to the caller.

Comment on lines +27 to +28
wheel_files = glob.glob("dist/*.whl")
assert len(wheel_files) == 1, f"Expected 1 wheel, found {wheel_files}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Also assert that the wheel belongs to the expected package, not just that there is exactly one wheel.

As written, this will pass as long as there’s exactly one wheel in dist/, even if it’s for a different package. To make the test safer, also assert that the wheel filename matches the expected package name, e.g. Path(wheel_files[0]).name.startswith(f"{package_name}-"), so mismatches between the built wheel and the package under test are caught.

Comment on lines 52 to +61
def _run_python_checked(
args: typing.List[str], cwd: typing.Optional[pathlib.Path] = None
args: typing.List[str],
cwd: typing.Optional[pathlib.Path] = None,
quiet: bool = False,
) -> None:
subprocess.run(
[sys.executable, *args],
check=True,
cwd=cwd,
stdout=subprocess.DEVNULL if quiet else None,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Quiet mode hides build output, which can make diagnosing failing packaging tests difficult.

Using quiet=True here suppresses both stdout and stderr, so if the build fails in CI you lose the error details needed to debug packaging issues. Consider keeping stderr visible (or redirecting it to a log file) when quiet is enabled, or only applying quiet to the subsequent pip install step so failures remain diagnosable.

Suggested implementation:

        # Suppress only stdout in quiet mode so build / packaging errors remain visible
        stdout=subprocess.DEVNULL if quiet else None,
        stderr=None,

If any callers relied on stderr being suppressed by quiet=True, they may need to be updated to handle visible error output now (for example, by explicitly redirecting or capturing stderr at the call site if necessary).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant