Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/actions/use-locked-versions/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES.
# All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Holds every later `pip install` in the job, and every isolated build
# environment pip sets up, to the versions uv.lock records: the packages'
# dependencies, and through the root `ci` group the tools and build backends
# CI installs. A pull request's checks then change when this repository does
# and not when something upstream is released. `--locked` also refuses a lock
# that no longer matches the metadata, so a dependency cannot change without
# it. pip reads its build constraint separately from the install one.
#
# The scheduled run skips this and resolves the newest releases, which is what
# a fresh install of the published packages gets.
name: Use the locked versions
description: Constrain pip to the versions uv.lock records.
runs:
using: composite
steps:
- if: github.event_name != 'schedule'
shell: bash
run: |
python -m pip install --quiet 'uv==0.11.20'
uv export --locked --all-packages --all-extras --all-groups \
--no-emit-workspace --no-hashes --no-header --no-annotate --quiet \
--output-file "$RUNNER_TEMP/constraints.txt"
echo "PIP_CONSTRAINT=$RUNNER_TEMP/constraints.txt" >> "$GITHUB_ENV"
echo "PIP_BUILD_CONSTRAINT=$RUNNER_TEMP/constraints.txt" >> "$GITHUB_ENV"
14 changes: 12 additions & 2 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,17 @@ updates:
directory: /
schedule:
interval: weekly
- package-ecosystem: pip
directory: /packages/structured-data-models-client
# The required checks install what uv.lock records, so a new release reaches
# them only through one of these; the nightly CI run resolves the newest
# releases on its own. `lockfile-only` moves the lock and never a package's
# own bounds, which say what a user may install and are changed by hand.
# Minor and patch releases come as one pull request a week; a major one
# alone.
- package-ecosystem: uv
directory: /
schedule:
interval: weekly
versioning-strategy: lockfile-only
groups:
minor-and-patch:
update-types: [minor, patch]
30 changes: 25 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,27 @@
# from where it is maintained, which a fork cannot reach. The generated
# bindings are pinned by SHA-256 in PROVENANCE.json either way, and a drifted
# copy fails the build it ships in.
#
# Every job that installs anything holds it to the versions uv.lock records,
# through `.github/actions/use-locked-versions`, except the engine wheels,
# which build and are smoke-tested in cibuildwheel's own environments. The
# nightly run resolves the newest releases instead, so an upstream release
# that breaks a fresh install shows up within a day without failing anyone's
# pull request.
name: CI

on:
pull_request:
push:
branches: [main]
schedule:
- cron: '41 5 * * *'

permissions:
contents: read

concurrency:
group: ci-${{ github.ref }}
group: ci-${{ github.event_name }}-${{ github.ref }}
cancel-in-progress: true

jobs:
Expand All @@ -30,6 +39,7 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- uses: ./.github/actions/use-locked-versions
- run: pip install --upgrade pip 'ruff==0.16.*'
- run: ruff check .
- run: ruff format --check .
Expand All @@ -50,6 +60,7 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- uses: ./.github/actions/use-locked-versions
# The engine needs its runtime dependencies present to be checked at all,
# and a toolchain to install. It ships py.typed exactly like the other
# four, so leaving it out would make the gate narrower than the promise.
Expand Down Expand Up @@ -100,6 +111,7 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
- uses: ./.github/actions/use-locked-versions
# databricks, snowflake and s3 are here for their drivers, not for a
# server: those tests mock the connection. Without the driver, importing
# the backend raises and every test covering it skips. snowflake was
Expand Down Expand Up @@ -139,6 +151,7 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- uses: ./.github/actions/use-locked-versions
- run: sudo apt-get update -qq && sudo apt-get install -y -qq build-essential cmake
# ruff is here for the root suite, not for linting: the test that
# checks the engine ignore-list note shells out to it and skips
Expand All @@ -162,10 +175,12 @@ jobs:
python -m pytest tests -q -m 'not live_nim and not live_warehouse'
# The client's suite again, with an engine present and the models able to
# run in process. The test job installs neither, so every client test
# that needs one skips there and runs nowhere. PyTorch comes from its CPU
# index, since the default one resolves to a far larger CUDA build.
# that needs one skips there and runs nowhere. PyTorch's CPU index is
# added beside the default one, which a far larger CUDA build would come
# from otherwise: `+cpu` outranks the same version without it, and the
# locked versions of PyTorch's own requirements are on the default one.
- run: |
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install torch --extra-index-url https://download.pytorch.org/whl/cpu
pip install -e './packages/structured-data-models-client[test,in-process]'
python -m pytest packages/structured-data-models-client/tests -q \
-m 'not live_nim and not live_warehouse'
Expand All @@ -181,6 +196,7 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- uses: ./.github/actions/use-locked-versions
- run: sudo apt-get update -qq && sudo apt-get install -y -qq build-essential cmake
- run: pip install --upgrade pip build
# Each distribution builds in an isolated environment holding only the
Expand Down Expand Up @@ -229,8 +245,10 @@ jobs:

# The engine's wheels, from the same `[tool.cibuildwheel]` table the release
# reads. One runner per platform and architecture, so every wheel is
# smoke-tested on the machine it targets.
# smoke-tested on the machine it targets. They resolve the newest releases
# on every pull request already, so the nightly run leaves them out.
engine-wheels:
if: github.event_name != 'schedule'
needs: build
strategy:
fail-fast: false
Expand Down Expand Up @@ -324,6 +342,7 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- uses: ./.github/actions/use-locked-versions
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: wheels
Expand Down Expand Up @@ -381,6 +400,7 @@ jobs:
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- uses: ./.github/actions/use-locked-versions
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: wheels
Expand Down
20 changes: 20 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,26 @@ Two more gates run in CI and are quick to run first:
.venv/bin/python scripts/mermaid_notices.py --check
```

The required checks on a pull request install the versions `uv.lock` records
rather than the newest ones, so an upstream release changes nothing there
until a dependabot pull request moves the lock. That covers the tools they
run and the build backends as well, through the `ci` group in the root
`pyproject.toml`. To install the same versions, export them with the uv those
checks use, point pip at the file, and bring pip itself to the locked version
before the installs above:

```bash
.venv/bin/pip install 'uv==0.11.20'
.venv/bin/uv export --locked --all-packages --all-extras --all-groups \
--no-emit-workspace --no-hashes -o .venv/constraints.txt
export PIP_CONSTRAINT=.venv/constraints.txt PIP_BUILD_CONSTRAINT=.venv/constraints.txt
.venv/bin/pip install --upgrade pip
```

A change to a package's dependencies, or to what the required checks
install, runs `uv lock` in the same commit. They refuse a lock that no longer
matches them.

Changing anything a caller can observe, such as an exported name, signature,
or wire request shape, requires recording a new snapshot in the same commit.
This makes the change visible in review:
Expand Down
26 changes: 24 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@
# Not a distribution. This exists so that `uv lock` has a workspace root to
# resolve from, which is what gives nSpect's open source scan something to
# inventory: a Python project with no lock file is reported as having zero
# open source components. Nothing is built or published from here, and
# nothing installs from the lock.
# open source components. Nothing is built or published from here. The
# required checks on a pull request install the versions the lock records,
# through `.github/actions/use-locked-versions`.
[project]
name = "workspace"
version = "0"
Expand All @@ -29,6 +30,27 @@ requires-python = ">=3.10"
[tool.uv.workspace]
members = ["packages/*"]

# What the required checks install besides the packages, so the lock holds it
# as well and dependabot moves it: the tools they run, and the build backends
# pip fetches into an isolated environment for each package. The engine's two
# bounds repeat its [build-system] table, which a locked version has to meet.
[dependency-groups]
ci = [
"build",
"cyclonedx-bom",
"mypy",
"pip",
"pybind11>=2.13.6",
"pyspark-client",
"pyyaml",
"ruff==0.16.*",
"scikit-build-core<0.12.0",
"setuptools>=77",
"setuptools-scm",
"snowflake-snowpark-python",
"twine",
]

[tool.uv.sources]
structured-data-models-api = { workspace = true }
structured-data-models-pql = { workspace = true }
Expand Down
Loading
Loading