Skip to content

docs(deps): simplify the managing-dependencies skill #1972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
WilliamBergamin merged 1 commit into from
Sep 25, 2026
+48 −88
Merged

docs(deps): simplify the managing-dependencies skill #1972

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 48 additions & 88 deletions .claude/skills/managing-dependencies/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,121 +1,81 @@
---
name: managing-dependencies
description: >-
Use when adding, updating, pinning, or reviewing any dependency in this repo's requirements/*.txt files, including taking a Dependabot bump or writing a requirement line by hand. Also use when a pip install fails on the older Python jobs while newer ones pass, or when CI errors with "Could not find a version that satisfies", "No matching distribution found", "Requires-Python >=3.x", or "ResolutionImpossible". Triggers include: "add <package> to requirements", "bump/update <package>", "pin <package>", "fix this dependabot PR", "CI can't install <package> on Python 3.7/3.8/3.9", "requires-python error in the install step". This skill defines the layout and version-constraint conventions to follow; reach for it before hand-editing any requirements/*.txt file.
Use when adding, bumping, pinning, or reviewing a dependency in requirements.txt or requirements/*.txt, including taking or fixing a Dependabot PR. Also use when pip install fails on some Python versions in CI but not others ("Could not find a version that satisfies the requirement", "No matching distribution found", "Requires-Python", "ResolutionImpossible"), including PyPy-only failures, and when dropping support for an old Python version.
---

# Managing dependencies

## Overview
This repo supports a range of Python versions, and one set of requirements files has to install on all of them. Dependabot proposes the newest release, so its bumps often raise a lower bound past what the oldest Pythons can install, and CI fails at the install step on those jobs only. The fix is to split the requirement with PEP 508 `python_version` markers so each Python gets the newest release it can install. Dropping an old Python to turn CI green is not the fix.

Every `requirements/*.txt` file in this repo follows **one layout convention** and **one version-constraint pattern**. This keeps a single set of pins working across the whole Python matrix and every diff readable. The mechanism for making one line behave differently per interpreter is [PEP 508](https://peps.python.org/pep-0508/) environment markers. That spec is the authority for the marker syntax used throughout this skill.
## Where the facts live

Two facts about this repo drive everything below:
Read these instead of trusting memory; they change often.

- It supports **Python `>=3.7`** (`requires-python` and the classifiers in `pyproject.toml`). Its CI matrix in `.github/workflows/ci-build.yml` tests every version from **3.7 through 3.14 plus PyPy** (`pypy3.10`, `pypy3.11`).
- Many popular packages keep raising their minimum Python. A bump that raises a dependency's **lower bound** to a release requiring a newer Python makes pip's resolver find nothing installable on the old interpreters. The install step then fails there before tests even run.
- **Supported Pythons:** `requires-python` and the `Programming Language :: Python :: 3.X` classifiers in `pyproject.toml`, and the CI matrix.
- **Who installs a file, on which Python:** `grep -rn requirements .github/workflows scripts requirements pyproject.toml`. This also catches `-r` includes. For a hit in a script, find the CI job that runs it and read its `python-version`.
- **Whether a file ships to users:** a file that `pyproject.toml` reads into package metadata (`dependencies` or `optional-dependencies` under `[tool.setuptools.dynamic]`) ends up in the published wheel.

The convention resolves this without dropping old-Python support. Pin each interpreter to the newest release it can actually install, using PEP 508 `python_version` markers.
## Workflow

## File-layout convention
- [ ] 1. Find which Pythons install the file. A file installed only on the newest Python takes the bump as-is.
- [ ] 2. Find the newest release each affected Python can install (see "Finding the ceiling").
- [ ] 3. If every Python the line covers can install the new lower bound, take the bump as one line. Done.
- [ ] 4. Otherwise split the line (see "Splitting a line").
- [ ] 5. Re-read the section: no Python matches two lines, and the newest line is open-ended.

Each file lists **one dependency per section**: Name of the dependency, an optional rationale note (starting with `Note:`, explaining why a version is pinned or split), then the requirement line(s), separated from the next section by a blank line. This makes every pin self-documenting. Keep the note to the *why*; do not restate the per-interpreter ceilings or floors that are already readable on the requirement lines below, and omit the note entirely when the split is self-evident (as the `websockets` entry does).
Change only the section you are fixing; don't reformat other sections in the same change.

```
# pytest
pytest>=7.0.1,<9

# pytest-cov
# Note: pytest-cov 7.1+ requires Python >=3.9; cap older interpreters below it.
pytest-cov>=4,<7.1.0; python_version < "3.9"
pytest-cov>=7.1.0,<8; python_version >= "3.9"
```

Keep this layout when adding or editing dependencies. Never leave an empty trailing `;` (a fossil of a collapsed split; delete it).

## Which files need Python-version markers

A marker split is only needed for requirements files installed across the **full** Python matrix. Which file you are editing decides this. To see where a file is installed, read `.github/workflows/ci-build.yml`. It is the source of truth for which Python versions install which requirements files.
## File layout

| File | Installed on | Needs markers? |
| -------------------------------- | --------------------------------------------------- | ----------------------------------- |
| `requirements/testing.txt` | full matrix (3.7 → 3.14, pypy) | **Yes, if a bump raises the floor** |
| `requirements/optional.txt` | full matrix (also packaged as the `optional` extra) | **Yes, if a bump raises the floor** |
| `requirements/tools.txt` | latest supported Python only | No, just take the bump |
| `requirements/databases.txt` | latest supported Python only (databases job) | No, just take the bump |
| `requirements/documentation.txt` | not installed in CI (manual docs script only) | No, just take the bump |
Each file starts with a `# pip install -r <path>` header, then one section per dependency: a `# <name>` line, optional `# Note:` lines, the requirement line(s), and a blank line.

If the bump lands in a latest-Python-only file, take it as-is. No markers, no ceiling, just the layout convention above.

**`optional.txt` is special:** it is read into wheel metadata via `[tool.setuptools.dynamic]` in `pyproject.toml` (`optional-dependencies.optional`). Use **full-line comments only** there, never trailing inline comments, so nothing leaks into the packaged metadata.

## The version-constraint pattern

When a dependency's floor rises to a release that requires a newer Python, **do not** just take the bump, and **do not** drop old-Python support to make CI pass. Instead, split the requirement into `python_version`-marked lines that **partition the whole matrix**. Every interpreter matches exactly one line. Old interpreters keep the last compatible release (with an explicit ceiling), and the newest line is open-ended so future Pythons stay covered.

```
# pytest
# Note: pytest 9 requires Python >=3.10; cap older interpreters below it.
pytest>=7.0.1,<9; python_version < "3.10"
pytest>=9.1.1,<10; python_version >= "3.10"
```
# pip install -r requirements/testing.txt

## Canonical marker style
# pkg
pkg>=1.4,<2; python_version < "3.9"
pkg>=2.0,<2.3; python_version == "3.9"
pkg>=2.5.1,<3; python_version >= "3.10"

Consistency matters because these lines are read and edited often, and a stray style makes diffs noisy. Standardize on this:

- Spaces around every operator: `python_version >= "3.10"`, never `python_version>="3.10"`.
- Double-quoted `major.minor` string: `"3.10"`. (`packaging` compares these version-aware, so
`"3.10" > "3.9"` is correctly true, no lexicographic surprise.)
- Use `>=` / `<` for the Python boundary; avoid `>` / `<=` so the boundary version lands on exactly one side.
- One space after the `;`, none before: `pkg>=1,<2; python_version >= "3.10"`.
- The old-side line always carries an explicit upper bound (the floor-jump version).
- The marker set must be **exhaustive and mutually exclusive** across the matrix. The newest line ends open-ended (`>= "X.Y"`), never a bare `==` that leaves future Pythons unmatched.

## Deriving the versions to pin
# other-pkg
# Note: other-pkg 4.x breaks on Python 3.8 but declares no requires_python, so pip can't filter it.
other-pkg>=3,<4
```

You need two numbers: the **floor** (which Python the new release requires) and the old-side **ceiling** (the first release that raised that floor).
**Notes:** write one only for what the lines can't say: a cap that isn't obvious, a coupling with another package, or a release with wrong metadata. Don't restate versions or which Python gets which release; a routine split needs no note. Existing notes that only restate their lines are legacy: drop one when you edit its section, and don't copy it.

1. **Floor.** Read the metadata for the _exact target version_ at `https://pypi.org/pypi/<package>/<target-version>/json`. The `info.requires_python` field gives the new minimum (e.g. `">=3.10"`). A `null` there means the release declares no floor.
## Finding the ceiling

2. **Ceiling.** Walk the release history at `https://pypi.org/pypi/<package>/json` and find the **first version that raised the floor above the oldest matrix Python**. The old-side ceiling is `< <that version>`. For example, if `aiodns` jumped to `>=3.10` at version **4.0.0**, the old-side cap is `<4` even if the target is `4.0.4` (pinning `<4.0.4` would wrongly admit 4.0.0–4.0.3, which are also 3.10-only).
An old Python's ceiling is the first release that dropped it, not the release Dependabot proposes. If `pkg` dropped 3.9 at 2.3 and the target is 2.5.1, the 3.9 line is `<2.3`, since 2.3 through 2.5.0 are 3.10-only too.

Why an explicit ceiling instead of trusting pip to filter by `Requires-Python`? Because that filtering only holds if every future release keeps its metadata correct; a single mis-tagged release would silently float onto an untested interpreter. An explicit ceiling makes the intent self-documenting and robust.
- **From a red CI job:** the failing install log is ground truth. The lowest version on its `Ignored the following versions that require a different python version` line is that Python's ceiling.
- **For any Python:** `pip index versions <pkg> --python-version 3.9` lists what 3.9 can install, newest first. Run it again for the next Python up; the first release newer than 3.9's newest is 3.9's ceiling.
- **The pre-bump lower bound** is in the Dependabot PR title (`update pkg requirement from <3,>=2.0 to >=2.5.1,<3`) or `git diff`.

**If you arrived here from a red CI job:** the failing _install_ log is ground truth. It names the interpreter that failed and the versions pip was actually offered, e.g.:
## Splitting a line

```
ERROR: Ignored the following versions that require a different python version: 4.0.4 Requires-Python >=3.10
ERROR: Could not find a version that satisfies the requirement aiodns>=4.0.4 (from versions: ..., 3.6.0, 3.6.1)
```
Split only the Pythons that the bumped line covers and the new lower bound excludes. Leave existing bands for older Pythons alone, even when they are coarser than they could be.

Cross-check the PyPI value against that log so you are never guessing. (If instead the failure is a real test failure, or hits _every_ Python version, this pattern does not apply, so investigate the bump normally.)
- Lines run oldest Python to newest. The newest line takes the bump and stays open-ended (`>= "X.Y"`).
- Each band that splits off keeps the line's pre-bump lower bound and gets its ceiling from "Finding the ceiling". Pythons that share a ceiling share a band.
- Marker style: `pkg>=1,<2; python_version == "3.9"`. Spaces around operators, double-quoted `"X.Y"`, one space after `;`. A one-Python band is `== "X.Y"`; a band spanning several is `>= "X.Y" and < "X.Z"`, or `< "X.Z"` when it is the oldest. Boundaries use `>=` and `<` so each Python lands on exactly one side. `python_version` is major.minor only (3.9.5 reports `"3.9"`), so `== "3.9"` covers every 3.9 patch; don't use `python_full_version` here, where `== "3.9"` matches only 3.9.0.

## Two harder shapes
A section with a single gated line (installed only on the newest Python, or only on PyPy) is intentional; don't fill in the other Pythons.

**Different old Pythons need different ceilings.** When several old interpreters each cap at a different release, emit one line per divergent interpreter with `== "X.Y"`, then a final open-ended line. This is the `aiohttp` pattern already in `testing.txt` / `optional.txt`:
## Gotchas

```
# aiohttp
# Note: aiohttp's minimum Python rose in stages; cap each old interpreter at its last compatible release.
aiohttp>=3.7.3,<3.9; python_version == "3.7"
aiohttp>=3.7.3,<3.11; python_version == "3.8"
aiohttp>=3.13.5,<4; python_version >= "3.9"
```

**Only PyPy breaks.** Sometimes a bump drops PyPy _wheels_ while CPython at the same version is fine (the failing jobs are `pypy*`, not `3.7`/`3.8`/`3.9`). A `python_version` split would wrongly downgrade CPython too. Gate on the implementation instead, as in the `cryptography` pattern in `testing.txt`. Note the exact marker name and casing: **`implementation_name == "pypy"`** (lowercase `pypy`), not `platform_python_implementation`. A single gated line is enough. You do **not** need a second negated line for the other interpreters, because an unmarked line already applies everywhere the marked one is skipped.

```
# cryptography
# Note: cryptography 46+ dropped PyPy 3.10 wheels; pin to <46 for PyPy 3.10 only.
cryptography<46; implementation_name == "pypy" and python_version == "3.10"
```
- **Dependabot rewrites only the last line** of a split section, the open-ended one. When it bumps a split package, check which Pythons that line covers; the new lower bound may exclude the oldest of them, which then splits off.
- **Releases with no `requires_python`** can't be filtered by pip, so they install on Pythons they don't support and fail at import or test time rather than at install. `pip index` lists them too. Pin an explicit ceiling with a note saying why. When the culprit comes in loosely through another package, pin it in that package's section so the coupling stays visible.
- **Only PyPy jobs fail:** the release usually dropped PyPy wheels. Gate on the implementation instead of downgrading CPython: `pkg<46; implementation_name == "pypy" and python_version == "3.10"`. Lowercase `pypy`, and `implementation_name`, not `platform_python_implementation`. No unmarked companion line is needed.
- **Files that ship:** full-line comments only; an inline comment breaks the package build. Their lower bounds are the minimum users must install, so raise one only for a reason.
- **Not this pattern:** a real test failure, or an install failure on every Python. Investigate the bump normally.

## Collapse when a Python is dropped
## Dropping a Python version

Marker splits are maintenance cost, so remove them when they stop earning their keep. When a Python version is dropped from the CI matrix (and from `requires-python` / the classifiers), collapse any split whose only reason was that version back into a single unmarked line, and delete the trailing `;`. A leaner file is easier for both humans and Dependabot to reason about.
When a Python leaves the classifiers and the CI matrix, merge any band that only served it into its neighbor. If a section is left with one line, remove its marker and the `;`.

## What to leave alone
## Leave alone

- **Do not touch `requires-python` or the CI matrix.** Keeping 3.7 working on old dependency versions is the entire point; changing the floor is a separate, deliberate decision.
- **Prefer markers over a Dependabot `ignore`.** An `ignore` rule freezes newer Pythons on the old version too, and hides the version knowledge in config. Reserve `ignore` for the rare dep that must stay pinned everywhere for reproducible output.
- `requires-python`, the classifiers, and the CI matrix. Changing which Pythons are supported is its own decision.
- Packages under `ignore` in `.github/dependabot.yml`: maintainers bump those by hand on purpose. Prefer a marker split over adding a new `ignore`, which freezes the package on every Python.
Loading