build(selenium-devtools-py): make the adapter fit to publish, and keep it that way

.github/actions/python-package/action.yml

@@ -0,0 +1,46 @@
+name: Python package checks
+description: Install, test and build the Python adapter — the checks CI and the release share.
+
+# One definition because both callers must agree. Running the release guard
+# weaker than CI is the failure this replaces: it used to test without
+# installing, and the tests that pin selenium's BiDi surface skip when selenium
+# is absent, so a bump that moved those internals passed the guard and degraded
+# at runtime.
+
+inputs:
+  working-directory:
+    description: The package root.
+    required: false
+    default: packages/selenium-devtools-py
+
+runs:
+  using: composite
+  steps:
+    # Every step names the directory: a composite action does not inherit the
+    # caller's `defaults.run.working-directory`.
+    - name: Install the adapter and its runtime dependency
+      shell: bash
+      working-directory: ${{ inputs.working-directory }}
+      run: pip install -e '.[test]'
+
+    - name: Contract is in sync with shared
+      shell: bash
+      working-directory: ${{ inputs.working-directory }}
+      run: |
+        python scripts/gen_contract.py
+        git diff --exit-code src/selenium_devtools/_contract.py
+
+    - name: 🧪 Unit tests
+      shell: bash
+      working-directory: ${{ inputs.working-directory }}
+      run: PYTHONPATH=src python -m unittest discover -s tests
+
+    # Packaging used to be exercised for the first time at release, where a
+    # broken `pyproject.toml` surfaces with the publish button already pressed.
+    - name: 📦 Build sdist + wheel
+      shell: bash
+      working-directory: ${{ inputs.working-directory }}
+      run: |
+        python -m pip install --upgrade build twine
+        python -m build
+        twine check --strict dist/*

.github/workflows/python-release.yml

@@ -2,7 +2,7 @@ name: Manual PyPI Publish
 
 # Mirrors release.yml's manual, button-triggered shape for the Python adapter.
 # Unlike npm (NPM_TOKEN), PyPI uses trusted publishing (OIDC) — no secret.
-# Bump the version in packages/selenium-devtools-py/pyproject.toml before running.
+# Bump `__version__` in src/selenium_devtools/__init__.py before running.
 
 on:
   workflow_dispatch:
@@ -20,25 +20,72 @@ defaults:
   run:
     working-directory: packages/selenium-devtools-py
 
+# A version uploads once, so an overlapping run fails on the winner's upload.
+concurrency:
+  group: python-release-${{ inputs.target }}
+  cancel-in-progress: false
+
 jobs:
   release:
     runs-on: ubuntu-latest
     environment: ${{ inputs.target }}
     permissions:
       id-token: write # PyPI trusted publishing (OIDC) — no token/secret needed
+      contents: write # commits the version bump and tags the published tree
     steps:
+      # main, and with history: the release DECIDES the version by consuming
+      # `changes/`, then commits and tags the result. A tag cannot be the input
+      # to that, because it would have to name a version nothing has computed
+      # yet — so `py-v<version>` is an output, pointing at exactly the tree that
+      # was published.
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
         with:
           ref: 'main'
+          fetch-depth: 0
       - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
         with:
           python-version: '3.12'
-      - name: 🧪 Unit tests (release guard)
-        run: PYTHONPATH=src python -m unittest discover -s tests
-      - name: 📦 Build sdist + wheel
+      # Consumes changes/ — bumps `__version__` by the strongest level pending
+      # and writes the changelog section. On testpypi it stays in the working
+      # tree so a dry run builds the real next artifact without spending the
+      # fragments; only a pypi release commits it below.
+      - name: Apply pending change fragments
+        run: python scripts/changes.py apply
+      # Before the build, because the alternative is finding out from twine's
+      # 400 once the checks have already run. Also what refuses a release with
+      # no fragments and nothing new to say: the version would be unchanged and
+      # therefore already published.
+      - name: Version is not already on the index
+        env:
+          INDEX_HOST: ${{ inputs.target == 'testpypi' && 'test.pypi.org' || 'pypi.org' }}
         run: |
-          python -m pip install --upgrade build
-          python -m build
+          VERSION=$(sed -n 's/^__version__ = "\(.*\)"$/\1/p' src/selenium_devtools/__init__.py)
+          if [ -z "$VERSION" ]; then
+            echo "::error::could not read __version__ from src/selenium_devtools/__init__.py"
+            exit 1
+          fi
+          # Only a 404 is evidence the version is free. Treating "not 200" as
+          # free makes a 5xx or a DNS failure read as availability, which is the
+          # opposite of what this step is for: the preflight would pass on no
+          # information and the duplicate would surface as an upload error.
+          CODE=$(curl -sS --retry 3 --retry-delay 2 --retry-all-errors \
+            -o /dev/null -w '%{http_code}' \
+            "https://$INDEX_HOST/pypi/selenium-devtools-py/$VERSION/json" || echo 000)
+          case "$CODE" in
+            404) echo "$VERSION is free on $INDEX_HOST" ;;
+            200)
+              echo "::error::selenium-devtools-py $VERSION is already on $INDEX_HOST — a version uploads once, so add a change fragment or bump __version__"
+              exit 1 ;;
+            *)
+              echo "::error::$INDEX_HOST answered HTTP $CODE; cannot establish whether $VERSION is free"
+              exit 1 ;;
+          esac
+      - uses: ./.github/actions/python-package
+      # The pinned backend is the one a `pip install` user runs, and an older
+      # one serves none of the adapter's routes and scopes while leaving the run
+      # green, so this refuses the publish rather than the merge.
+      - name: Pinned backend serves this adapter's contract
+        run: python scripts/check_backend_pin.py
       - name: 🚀 Publish to PyPI
         if: ${{ inputs.target == 'pypi' }}
         uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
@@ -50,3 +97,54 @@ jobs:
         with:
           packages-dir: packages/selenium-devtools-py/dist
           repository-url: https://test.pypi.org/legacy/
+
+      # After the publish, so main never claims a release that did not happen.
+      # A dry run reaches none of this: its bump stays in the runner's tree and
+      # the fragments live on for the real release to spend.
+      - name: Record the release
+        if: ${{ inputs.target == 'pypi' }}
+        run: |
+          VERSION=$(sed -n 's/^__version__ = "\(.*\)"$/\1/p' src/selenium_devtools/__init__.py)
+          git config user.email "bot@webdriver.io"
+          git config user.name "WebdriverIO Release Bot"
+          git add src/selenium_devtools/__init__.py CHANGELOG.md changes/
+
+          # The tag hangs off the PUBLISH, not off whether files changed. A
+          # first release has no fragments to consume, so nothing is staged and
+          # the older form returned here — publishing a version that then
+          # carried no tag, which is the one case the tag exists to cover.
+          if git diff --cached --quiet; then
+            echo "nothing to commit; tagging the published tree"
+          else
+            git commit -m "chore(selenium-devtools-py): release $VERSION"
+            # PyPI has already accepted the upload, so failing to land this
+            # leaves a published version unrecorded and a rerun refused by the
+            # index preflight. Concurrency only serialises this workflow, and
+            # anyone may push to main meanwhile — so rebase onto whatever
+            # arrived and try again rather than giving up on the first reject.
+            pushed=false
+            for attempt in 1 2 3 4 5; do
+              if git push origin HEAD:main; then
+                pushed=true
+                break
+              fi
+              echo "main moved; rebasing onto it (attempt $attempt)"
+              git fetch origin main
+              git rebase origin/main || {
+                git rebase --abort || true
+                echo "::error::could not rebase the release commit onto main"
+                break
+              }
+            done
+            if [ "$pushed" != true ]; then
+              echo "::error::selenium-devtools-py $VERSION is PUBLISHED but its release commit is not on main. Land __version__, CHANGELOG.md and the changes/ deletions by hand before the next release."
+              exit 1
+            fi
+          fi
+
+          if git rev-parse -q --verify "refs/tags/py-v$VERSION" >/dev/null; then
+            echo "py-v$VERSION already exists"
+          else
+            git tag "py-v$VERSION"
+            git push origin "py-v$VERSION"
+          fi

.github/workflows/python.yml

@@ -20,19 +20,19 @@ on:
       - packages/selenium-devtools-py/**
       - packages/shared/src/**
       - .github/workflows/python.yml
+      - .github/workflows/python-release.yml
+      - .github/actions/python-package/**
   pull_request:
     paths:
       - packages/selenium-devtools-py/**
       - packages/shared/src/**
       - .github/workflows/python.yml
+      - .github/workflows/python-release.yml
+      - .github/actions/python-package/**
 
 permissions:
   contents: read
 
-defaults:
-  run:
-    working-directory: packages/selenium-devtools-py
-
 jobs:
   test:
     runs-on: ubuntu-latest
@@ -42,20 +42,27 @@ jobs:
         # 4.44 requires it, and network capture requires 4.44+.
         python-version: ['3.10', '3.13']
     steps:
+      # Full history: the change-fragment gate diffs this branch against the
+      # base, which a depth-1 checkout cannot see.
       - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
       - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
         with:
           python-version: ${{ matrix.python-version }}
-      # selenium is the adapter's only runtime dependency, and the tests that
-      # pin its BiDi surface (tests/test_selenium_surface.py) skip without it —
-      # so a bump that moved those internals used to pass CI green and degrade
-      # at runtime. Installing the package is what makes those guards real.
-      - name: Install the adapter and its runtime dependency
-        run: pip install -e '.[test]'
-
-      - name: Contract is in sync with shared
-        run: |
-          python scripts/gen_contract.py
-          git diff --exit-code src/selenium_devtools/_contract.py
-      - name: 🧪 Unit tests
-        run: PYTHONPATH=src python -m unittest discover -s tests
+      # A source change with no fragment is a release that cannot describe
+      # itself, and the version it would ship is decided by these files.
+      - name: Source changes carry a change fragment
+        if: github.event_name == 'pull_request'
+        working-directory: packages/selenium-devtools-py
+        env:
+          BASE_REF: ${{ github.base_ref }}
+        run: python scripts/changes.py check --base "origin/$BASE_REF"
+      - uses: ./.github/actions/python-package
+      # The pin is the backend a `pip install` user actually runs, and the drift
+      # check above cannot see it: that compares `_contract.py` against shared,
+      # this compares it against the tarball npm serves. Red here means the next
+      # PyPI release would ship features the pinned backend cannot answer.
+      - name: Pinned backend serves this adapter's contract
+        working-directory: packages/selenium-devtools-py
+        run: python scripts/check_backend_pin.py

CLAUDE.md

@@ -28,6 +28,7 @@ Run from repo root unless noted.
 | `pnpm lint` | Lint all packages in parallel. Includes `eslint-plugin-security` for a subset of CodeQL findings; deeper taint-flow checks surface on the PR's CodeQL scan. |
 | `pnpm demo:wdio` / `pnpm demo:nightwatch` / `pnpm demo:selenium` | Run the per-framework example projects. Useful for manual verification of UI or runtime changes. |
 | `pnpm dev` | Run all packages in parallel dev mode. |
+| `python3 packages/selenium-devtools-py/scripts/changes.py next-version` | The version a Python-adapter release would publish, from the fragments pending in `changes/`. `check --base <ref>` is the CI gate; `apply` is what the release runs. |
 
 `selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:mocha:allure` / `:jest` / `:cucumber`.
 
@@ -223,9 +224,21 @@ When the right place is ambiguous (something between `shared` and `core`, or bet
 ### Before pushing
 
 - `pnpm build`, `pnpm test`, `pnpm lint`. Don't push red.
+- A changeset for a published npm package, or a `changes/` fragment for the Python adapter — see § Releasing a change.
 - For UI or runtime changes: verify in `examples/<framework>/`.
 - Deeper security findings (taint flow, polynomial-redos with adjacent quantifiers) surface on the PR's CodeQL scan; review and fix those before merge.
 
+### Releasing a change
+
+Two mechanisms, and the Python one exists because the npm one cannot reach it. Changesets discovers packages through the pnpm workspace and identifies them by `package.json`; `packages/selenium-devtools-py` is in neither, so a changeset naming `selenium-devtools-py` does not degrade — it raises "not in the workspace", fails `changeset version`, and takes the npm release for every other package down with it.
+
+- **Published npm package changed** → `pnpm changeset`, committed as `.changeset/*.md`.
+- **`packages/selenium-devtools-py/src/` changed** → a fragment under `packages/selenium-devtools-py/changes/`, frontmatter carrying the bump level alone (`patch`/`minor`/`major`). `python.yml` refuses a branch that changes `src/` and documents nothing. A direct `CHANGELOG.md` edit satisfies it **only until the first release** — before one there is nothing to bump from and the pending entry IS the changelog section; after one (detected by a `py-v*` tag existing) it documents the change but bumps nothing, so the release would find no fragment and republish a version the index already holds.
+
+Neither is hand-versioned: both assemble the version and the changelog at release. The Python release additionally consumes its fragments, bumps `__version__` (the single source — `pyproject.toml` reads it via `dynamic = ["version"]`), and tags `py-v<version>` **after** a successful publish, so the tag is an output pointing at the published tree rather than an input naming a version nothing has computed yet.
+
+`BACKEND_NPM_VERSION` is the backend a `pip install` user actually runs, so the npm release goes first; `release.yml` opens the pin bump as a PR, and `scripts/check_backend_pin.py` refuses a PyPI publish whose pinned backend cannot serve the contract.
+
 ### Commits
 
 - Small, focused. Don't bundle unrelated changes.

CONTRIBUTING.md

@@ -82,11 +82,29 @@ Then pick:
 
 Commit the generated `.changeset/*.md` with your change. You don't edit `CHANGELOG.md` or version numbers — the release generates those from your changeset. Publishing itself is a **manual step a maintainer runs** (the "Manual NPM Publish" GitHub Action), so your job ends at landing the changeset.
 
+### The Python adapter has its own
+
+**If your change touches `packages/selenium-devtools-py/src/`, add a change fragment** — a file under `packages/selenium-devtools-py/changes/`:
+
+```md
+---
+minor
+---
+
+Serve the page collector from the backend, so DOM replay works from a published install.
+```
+
+The frontmatter is the bump level alone (`patch`, `minor`, `major`); the body is what a user reading the changelog needs to know. CI refuses a branch that changes `src/` and documents nothing. As with changesets you don't edit the version or `CHANGELOG.md` — the release consumes the fragments, takes the strongest level pending, bumps `__version__`, writes the changelog section and tags `py-v<version>`.
+
+```bash
+python3 packages/selenium-devtools-py/scripts/changes.py next-version   # what a release would publish
+```
+
 ## Before you push
 
 - `pnpm build`, `pnpm test`, and `pnpm lint` all green — don't push red.
 - UI / runtime changes verified in `examples/<framework>/`.
-- A changeset added if a published package changed (`pnpm changeset`).
+- A changeset added if a published package changed (`pnpm changeset`), or a change fragment under `packages/selenium-devtools-py/changes/` if the Python adapter's `src/` changed.
 - User-facing changes (a new option, CLI, flag, output, or workflow) update the relevant README **and** are mirrored to the [WebdriverIO devtools webpage](https://webdriver.io/docs/devtools) in the same change.
 
 ## Pull requests

packages/selenium-devtools-py/CHANGELOG.md

@@ -0,0 +1,33 @@
+# selenium-devtools-py
+
+Assembled at release from the fragments in `changes/` — see that directory's
+README. Changesets generates the npm packages' changelogs from the pnpm
+workspace, which this package is not a member of, so it has its own mechanism
+of the same shape.
+
+## 0.1.0
+
+First release.
+
+Python Selenium adapter for the WebdriverIO DevTools dashboard, feeding the same
+backend and UI as the JavaScript adapters over the language-neutral
+`{scope, data}` WebSocket contract.
+
+- **Live mode** — command capture and the test tree, browser console and network
+  over BiDi, assertion rows, per-command screenshots and selectors, DOM replay,
+  and a pushed CDP screencast.
+- **Trace mode** — the same portable `trace.zip` the JavaScript adapters write,
+  with action snapshots, sources and a transcript. Built by the backend on the
+  adapter's behalf, since this package ships no Node.
+- **Run controls** — Run, Rerun, Run-all and Preserve & Rerun, with reruns
+  selected by pytest nodeid and spawned in pytest's own rootdir.
+- **pytest plugin** — auto-discovered, opt-in per run (`--devtools`,
+  `--devtools-trace`), per project (`[tool.pytest.ini_options]`) or per shell
+  (`DEVTOOLS_ENABLE`). Installing it never changes how an existing suite behaves.
+- Ships `py.typed`, so the annotations already on the public API reach a
+  consumer's type checker instead of resolving to `Any`.
+- Requires Python 3.10+, `selenium>=4.44`, and Node.js 18+ on PATH for the
+  backend.
+
+Known gaps against the JavaScript adapters are listed under Roadmap in the
+README.

packages/selenium-devtools-py/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 WebdriverIO
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.