From f14527a7f5a30d245cd291521c9175195bca4803 Mon Sep 17 00:00:00 2001 From: Simone Carolini Date: Fri, 2 Oct 2026 13:31:59 +0200 Subject: [PATCH 1/2] feat(template): release through continuo's public release API The template release workflow submits to /api/v1/releases with the job's GitHub Actions OIDC token (a fresh one per call) and polls GET /api/v1/releases/{id} to a terminal status, bounded to about 15 minutes. RELEASE_ENDPOINT keeps its name and is now continuo's base URL; the workflow fails before building when it is empty or malformed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01QtrP3QHzy3ubJfRXaHB57m Signed-off-by: Simone Carolini --- CHANGELOG.md | 11 +++ README.md | 9 ++- docs/boundary-contract.md | 26 +++--- template/.github/workflows/release.yml | 108 +++++++++++++++++++++++-- template/README.md | 35 +++++--- tests/test_template.py | 67 +++++++++++++++ 6 files changed, 226 insertions(+), 30 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index eafc508..738f316 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,17 @@ follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### Changed + +- The `template/` release workflow calls continuo's public release API: it + submits to `/api/v1/releases` with the job's GitHub Actions + OIDC token (a fresh one per call) and polls `GET /api/v1/releases/{id}` for up + to about 15 minutes, failing on `rejected` and `superseded`. `RELEASE_ENDPOINT` + keeps its name and is now continuo's base URL (`scheme://host[:port]`, no + path); the workflow fails before building when it is empty or carries a path. + The repository must be bound in continuo's `ciAuth.bindings`, and a service's + first release is an operator bootstrap. + ## [0.8.0] - 2026-10-01 Packages in this release: `continuo-python-runtime` 0.8.0 (no runtime code diff --git a/README.md b/README.md index c3aca8c..4d27dd8 100644 --- a/README.md +++ b/README.md @@ -101,9 +101,12 @@ the Go parser has not been taught is a production outage, not a refactor. service name (one service name per domain repo). 3. Configure repository variables in GitHub (Settings → Secrets and variables → Actions): `REGISTRY` (your Docker registry), `BUCKET` (your - S3 bucket for contract artifacts), `RELEASE_ENDPOINT` (the release - webhook endpoint). `RELEASE_ENDPOINT` is the **base URL** of the Continuo - API (no `/releases` suffix) — the workflow appends `/releases` itself. + S3 bucket for contract artifacts), `RELEASE_ENDPOINT` (the base + URL of your continuo install, `scheme://host[:port]` with no path). The + workflow calls `/api/v1/releases` with its GitHub Actions + OIDC token, so the repository must be bound to your service in continuo's + `ciAuth.bindings` (see [Releasing from CI](https://github.com/carolsimone/continuo/blob/main/deploy/README.md#releasing-from-ci-github-actions)), + and an operator bootstraps the service's first release. 4. Configure repository secrets: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for the S3 upload. The template workflow pushes the built image to GHCR using the workflow's own `GITHUB_TOKEN` (granted diff --git a/docs/boundary-contract.md b/docs/boundary-contract.md index 13f2cb7..b7f0898 100644 --- a/docs/boundary-contract.md +++ b/docs/boundary-contract.md @@ -99,7 +99,7 @@ s3://///contract.yaml - `output_columns` types come from the supported set: `BIGINT`, `INT`/`INTEGER`, `DOUBLE PRECISION`, `NUMERIC(p,s)`/`DECIMAL(p,s)`, `VARCHAR(n)`/`CHAR(n)`/`TEXT`, `TIMESTAMP`, `DATE`, `BOOLEAN`. -- Ordering is a hard rule: **upload completes before `POST /releases`** — +- Ordering is a hard rule: **upload completes before `POST /api/v1/releases`** — Continuo does no existence check (D3); a POST racing its own upload fails at the parsing stage. @@ -285,21 +285,29 @@ own `import` statements, resolved by static AST analysis ## 13.3 Surface 3 — the release call ``` -POST /releases +POST /api/v1/releases +Authorization: Bearer { "service": "marketing-py", # one service name per domain repo "release_id": "", "image_tag": "/:", # the image the executor will run - "repo": "owner/name", # where the source lives (remediation) - "commit_sha": "", # must contain scripts + contracts "kind": "python" } ``` -202 Accepted `{"release_id": …, "status": "received"}`; 400 on any missing -field. Idempotent on `release_id` — safe to retry. `repo` + `commit_sha` -must point at the actual source of the scripts and contract files, because -the remediation agent fetches them from GitHub to propose fix PRs. +The token's audience is the origin of the continuo install, and the +repository must be bound to the service in continuo's `ciAuth.bindings` +([Releasing from CI](https://github.com/carolsimone/continuo/blob/main/deploy/README.md#releasing-from-ci-github-actions)). A CI +token supplies `repo` and `commit_sha` (the repository and commit the workflow +runs in), so the body leaves them out; they must point at the actual source of +the scripts and contract files, because the remediation agent fetches them from +GitHub to propose fix PRs. 202 Accepted `{"release_id": …, "status": …}`; 400 +on a missing or unknown field. Idempotent on `release_id` with the same body — +safe to retry. The workflow then polls `GET /api/v1/releases/{release_id}` +with a fresh token until `terminal` is true: `promoted` is success, `rejected` +and `superseded` are failures. The first release of a service is an operator +bootstrap (`"bootstrap": true`), which a CI binding may send only with +`allowBootstrap`. ## 13.4 Surface 4 — the runtime image @@ -399,7 +407,7 @@ intentionally differ from the dbt job env (`SCHEMA`/`DBT_TARGET_SCHEMA`/dbt 2. merge contract files → contract.yaml; compute the per-node hash fields (§13.2) 3. build + push the image (scripts + contracts + harness baked in) 4. upload contract.yaml → s3://///contract.yaml -5. POST /releases {…, kind: "python"} # only after 3 and 4 succeed +5. POST /api/v1/releases {…, kind: "python"} # only after 3 and 4 succeed; then poll to a terminal status ``` Provisioning Continuo hands each domain repo, once: S3 write credentials diff --git a/template/.github/workflows/release.yml b/template/.github/workflows/release.yml index 51bf8b4..8237496 100644 --- a/template/.github/workflows/release.yml +++ b/template/.github/workflows/release.yml @@ -6,11 +6,30 @@ concurrency: cancel-in-progress: true env: SERVICE: your-service-name # one service name per domain repo + # Repository variable RELEASE_ENDPOINT: the base URL of continuo's ui, as + # scheme://host[:port] with no path (e.g. https://continuo.example.com). + # This repository must be bound to $SERVICE in continuo's `ciAuth.bindings`. + RELEASE_ENDPOINT: ${{ vars.RELEASE_ENDPOINT }} jobs: release: runs-on: ubuntu-latest + timeout-minutes: 45 permissions: { contents: read, packages: write, id-token: write } steps: + # Fail before building anything when the endpoint is missing or malformed. + # CONTINUO_ORIGIN (the endpoint without a trailing slash) is both the API + # base and the OIDC audience. + - name: Check release endpoint + run: | + if [ -z "$RELEASE_ENDPOINT" ]; then + echo "::error::repository variable RELEASE_ENDPOINT is not set; set it to continuo's base URL (scheme://host)" + exit 1 + fi + if ! printf '%s' "$RELEASE_ENDPOINT" | grep -Eq '^https?://[^/?#]+/?$'; then + echo "::error::RELEASE_ENDPOINT must be continuo's base URL as scheme://host[:port] with no path (got '$RELEASE_ENDPOINT')" + exit 1 + fi + echo "CONTINUO_ORIGIN=${RELEASE_ENDPOINT%/}" >> "$GITHUB_ENV" - uses: actions/checkout@v4 - uses: astral-sh/setup-uv@v5 # PRECONDITION: continuo-python-runtime must be published to PyPI (this repo's release pipeline). @@ -53,14 +72,87 @@ jobs: env: AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} - - name: POST /releases # strictly after build+push and upload + # Strictly after build+push and upload: continuo does no existence check, so + # a release that races its own artifacts fails at the parsing stage. + # + # Submits the release to continuo's public API and waits for its verdict. + # Each call carries a GitHub Actions OIDC token whose audience is the origin + # of RELEASE_ENDPOINT. The tokens expire within minutes, so every call asks + # for a fresh one; no token is ever printed. The token carries `repo` and + # `commit_sha`, so the body leaves them out. Promoted is success; rejected + # and superseded fail the job. + - name: Submit release to continuo run: | - body=$(jq -n \ - --arg service "$SERVICE" \ + resp="$(mktemp)" + trap 'rm -f "$resp"' EXIT + + oidc_token() { + local aud + aud="$(jq -rn --arg a "$CONTINUO_ORIGIN" '$a | @uri')" + printf 'Authorization: bearer %s\n' "$ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ + | curl -sS --fail -H @- "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=${aud}" \ + | jq -er '.value' + } + + # call METHOD PATH [BODY]: prints the HTTP status (000 when curl or the + # token request failed); the response body is left in $resp. + call() { + local method="$1" path="$2" body="${3:-}" token + token="$(oidc_token)" || { echo "could not request a GitHub Actions OIDC token" >&2; echo 000; return 0; } + local -a args=(-sS -X "$method" -o "$resp" -w '%{http_code}' -H @- -H 'Accept: application/json') + [ -z "$body" ] || args+=(-H 'Content-Type: application/json' -d "$body") + local out + out="$(printf 'Authorization: Bearer %s\n' "$token" | curl "${args[@]}" "${CONTINUO_ORIGIN}${path}" || true)" + echo "${out:-000}" + } + + fail_with_response() { + echo "::error::$1: HTTP $2 $(jq -r '"code=\(.code // "unknown") error=\(.error // "none")"' "$resp" 2>/dev/null || true)" + exit 1 + } + + body="$(jq -n \ --arg release_id "$RELEASE_ID" \ + --arg service "$SERVICE" \ --arg image_tag "$IMAGE_TAG" \ - --arg repo "${{ github.repository }}" \ - --arg commit_sha "$GITHUB_SHA" \ - '{service: $service, release_id: $release_id, image_tag: $image_tag, repo: $repo, commit_sha: $commit_sha, kind: "python"}') - curl --fail-with-body -X POST "${{ vars.RELEASE_ENDPOINT }}/releases" \ - -H 'Content-Type: application/json' -d "$body" + '{release_id: $release_id, service: $service, image_tag: $image_tag, kind: "python"}')" + + # A submit is idempotent on release_id, so transient failures are retried. + for attempt in 1 2 3; do + status="$(call POST /api/v1/releases "$body")" + case "$status" in 000 | 429 | 502 | 503 | 504) sleep $((attempt * 5)) ;; *) break ;; esac + done + [ "$status" = 202 ] || fail_with_response "submitting release $RELEASE_ID" "$status" + echo "submitted release $RELEASE_ID (service=$SERVICE) to $CONTINUO_ORIGIN" + + # Poll to a terminal status: 90 polls x 10 s = 15 minutes. + last="" + for _ in $(seq 1 90); do + status="$(call GET "/api/v1/releases/$RELEASE_ID")" + case "$status" in + 200) ;; + 000 | 429 | 502 | 503 | 504) sleep 10; continue ;; + *) fail_with_response "reading release $RELEASE_ID" "$status" ;; + esac + state="$(jq -r '.status // empty' "$resp")" + if [ "$state" != "$last" ]; then echo "release $RELEASE_ID status: ${state:-unknown}"; last="$state"; fi + if [ "$(jq -r '.terminal // false' "$resp")" = true ]; then + ui_url="$(jq -r '.ui_url // empty' "$resp")" + [ -z "$ui_url" ] || echo "details: $ui_url" + case "$state" in + promoted) exit 0 ;; + rejected) + echo "::error::release $RELEASE_ID rejected: $(jq -r '"\(.reject_reason // "") \(.reject_detail // "")"' "$resp")" + exit 1 ;; + superseded) + echo "::error::release $RELEASE_ID was superseded by a newer release before it could be promoted" + exit 1 ;; + *) + echo "::error::release $RELEASE_ID ended with unexpected status '$state'" + exit 1 ;; + esac + fi + sleep 10 + done + echo "::error::timed out waiting for release $RELEASE_ID to reach a terminal status (last: ${last:-unknown})" + exit 1 diff --git a/template/README.md b/template/README.md index 1151b9e..8641aab 100644 --- a/template/README.md +++ b/template/README.md @@ -1,6 +1,6 @@ -# Continuo Python Domain Repository Template +# continuo Python Domain Repository Template -This is a copy-ready template for implementing a [Continuo Python domain repo](https://github.com/carolsimone/continuo-python-runtime). +This is a copy-ready template for implementing a [continuo Python domain repo](https://github.com/carolsimone/continuo-python-runtime). ## Quick Start @@ -9,16 +9,28 @@ This is a copy-ready template for implementing a [Continuo Python domain repo](h 3. **Configure repository variables** in GitHub (Settings → Secrets and variables → Actions): - `REGISTRY`: Your Docker registry (e.g., `ghcr.io/org`) - `BUCKET`: Your S3 bucket for contract artifacts - - `RELEASE_ENDPOINT`: Your release webhook endpoint. This is the **base - URL** of the Continuo API (no `/releases` suffix) — the workflow - appends `/releases` itself. + - `RELEASE_ENDPOINT`: the base URL of your continuo install, as + `scheme://host[:port]` with no path (for example + `https://continuo.example.com`). The workflow calls + `/api/v1/releases` and fails before building anything + when the variable is empty or carries a path. 4. **Configure repository secrets**: - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for S3 uploads - `release.yml` already logs in to `ghcr.io` with the built-in `GITHUB_TOKEN` (no extra secret needed) — only add your own login step if `REGISTRY` points at a registry other than `ghcr.io` -5. **Write your contracts** in `contracts/` and **implement scripts** in `scripts/` -6. **Push to main** to trigger the release pipeline +5. **Bind the repository in continuo.** The workflow authenticates with its + GitHub Actions OIDC token (`id-token: write`, already set), so no secret is + stored. The operator who runs the install lists this repository under + `ciAuth.bindings` for your service name; see + [Releasing from CI](https://github.com/carolsimone/continuo/blob/main/deploy/README.md#releasing-from-ci-github-actions). Until the + repository is bound, every release call is refused. +6. **Bootstrap the service once.** The first release of a service has no + production version to validate against, so an operator promotes it with + `"bootstrap": true`; the workflow never sends that flag. Releases from the + workflow work once the service has been bootstrapped. +7. **Write your contracts** in `contracts/` and **implement scripts** in `scripts/` +8. **Push to main** to trigger the release pipeline ## Choosing a base @@ -29,7 +41,7 @@ need one Dockerfile in your repo. **Shape 1 — `Dockerfile`, `FROM` the engine image (simplest).** Builds `FROM ghcr.io/carolsimone/continuo-python-runtime-:vX.Y.Z`, an image -that already has the Continuo runtime and one engine adapter installed and +that already has the continuo runtime and one engine adapter installed and pinned by the publisher. You only add your `contracts/` and `scripts/` (and any extra dependency your script needs). Pin by tag or digest (`:vX.Y.Z@sha256:`) for reproducibility. Use this unless you have a @@ -74,9 +86,12 @@ The CI/CD pipeline (`release.yml`) performs the six-step orchestration: 3. **Run domain tests** (optional, if `tests/` exists) 4. **Merge** contracts into a single artifact 5. **Build and push** Docker image -6. **Upload contract** and **POST release notification** +6. **Upload contract**, then **submit the release** to continuo's + `POST /api/v1/releases` and poll `GET /api/v1/releases/{id}` for up to about + 15 minutes. The job succeeds when the release is `promoted` and fails when it + is `rejected` or `superseded` ## Resources -- [Continuo Python Runtime Documentation](https://github.com/carolsimone/continuo-python-runtime) +- [continuo Python Runtime Documentation](https://github.com/carolsimone/continuo-python-runtime) - [Boundary Contract (design §13)](https://github.com/carolsimone/continuo-python-runtime/blob/main/docs/boundary-contract.md) diff --git a/tests/test_template.py b/tests/test_template.py index 32aaec8..5d7ba0c 100644 --- a/tests/test_template.py +++ b/tests/test_template.py @@ -1,7 +1,10 @@ """Test the domain-repo template.""" +import os +import subprocess from pathlib import Path +import pytest import yaml from continuo_python_runtime.cli import main @@ -61,6 +64,70 @@ def test_release_workflow_cancels_superseded_main_runs(): } +def _release_job(): + workflow = yaml.safe_load( + (TEMPLATE / ".github" / "workflows" / "release.yml").read_text() + ) + return workflow, workflow["jobs"]["release"] + + +def _step(job, name): + return next(step for step in job["steps"] if step.get("name") == name) + + +def test_release_workflow_uses_the_public_release_api(): + """The release call is the authenticated public API, bounded and token-safe.""" + workflow, job = _release_job() + names = [step.get("name") for step in job["steps"]] + submit = _step(job, "Submit release to continuo")["run"] + + assert job["permissions"]["id-token"] == "write" + assert isinstance(job["timeout-minutes"], int) + assert workflow["env"]["RELEASE_ENDPOINT"] == "${{ vars.RELEASE_ENDPOINT }}" + # Submit comes after the image build and the contract upload. + assert names.index("Submit release to continuo") > names.index("Upload contract artifact") + assert "/api/v1/releases" in submit + assert "audience=" in submit + assert "terminal" in submit + assert "seq 1 90" in submit # a bounded poll + # The bare, unauthenticated `${{ vars.RELEASE_ENDPOINT }}/releases` call is gone. + assert "/releases\"" not in (TEMPLATE / ".github" / "workflows" / "release.yml").read_text() + # The body is built with jq, and the token is only ever sent as a header. + assert "jq -n" in submit + assert "echo \"$token" not in submit and "ACTIONS_ID_TOKEN_REQUEST_TOKEN\" >&2" not in submit + + +@pytest.mark.parametrize( + ("endpoint", "ok"), + [ + ("", False), + ("continuo.example.com", False), + ("https://continuo.example.com/api", False), + ("https://continuo.example.com?x=1", False), + ("https://continuo.example.com", True), + ("https://continuo.example.com/", True), + ("http://localhost:8090", True), + ], +) +def test_release_workflow_fails_closed_on_a_bad_endpoint(tmp_path, endpoint, ok): + """The first step rejects an unset or malformed RELEASE_ENDPOINT before any build.""" + _, job = _release_job() + assert job["steps"][0]["name"] == "Check release endpoint" + github_env = tmp_path / "github_env" + github_env.touch() + result = subprocess.run( + ["bash", "-eo", "pipefail", "-c", job["steps"][0]["run"]], + env={"PATH": os.environ["PATH"], "RELEASE_ENDPOINT": endpoint, "GITHUB_ENV": str(github_env)}, + capture_output=True, + text=True, + ) + assert (result.returncode == 0) is ok, result.stdout + result.stderr + if ok: + assert github_env.read_text() == f"CONTINUO_ORIGIN={endpoint.rstrip('/')}\n" + else: + assert "RELEASE_ENDPOINT" in result.stdout + + def test_readme_and_template_name_images_the_publisher_emits(): """Engine-selection examples must name images the publisher actually pushes. From 51df3156f88ccc81d2ad157824854e70c0592be1 Mon Sep 17 00:00:00 2001 From: Simone Carolini Date: Fri, 2 Oct 2026 13:50:05 +0200 Subject: [PATCH 2/2] fix(template): harden the release workflow's endpoint check and add a preflight - Match RELEASE_ENDPOINT as a whole string so a multi-line value cannot inject lines into $GITHUB_ENV; reject userinfo and whitespace; never echo the value. - Add a preflight read of /api/v1/current-prod before the build, naming ciAuth.bindings and the audience on a 401/403. - Poll for about 50 minutes with timeout-minutes 60; do not cancel a run that is waiting for its release; do not sleep after the last retry. - Document the exact-origin requirement and mark the template change Breaking for newly copied templates in the changelog. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01QtrP3QHzy3ubJfRXaHB57m Signed-off-by: Simone Carolini --- CHANGELOG.md | 26 +++-- README.md | 3 +- template/.github/workflows/release.yml | 112 ++++++++++++------- template/README.md | 16 +-- tests/test_template.py | 143 +++++++++++++++++++++---- 5 files changed, 222 insertions(+), 78 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 738f316..3c21ffa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,14 +7,24 @@ follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Changed -- The `template/` release workflow calls continuo's public release API: it - submits to `/api/v1/releases` with the job's GitHub Actions - OIDC token (a fresh one per call) and polls `GET /api/v1/releases/{id}` for up - to about 15 minutes, failing on `rejected` and `superseded`. `RELEASE_ENDPOINT` - keeps its name and is now continuo's base URL (`scheme://host[:port]`, no - path); the workflow fails before building when it is empty or carries a path. - The repository must be bound in continuo's `ciAuth.bindings`, and a service's - first release is an operator bootstrap. +- **Breaking for newly copied templates.** The `template/` release workflow + calls continuo's public release API instead of an unauthenticated webhook: + - `RELEASE_ENDPOINT` keeps its name but is now the origin of continuo's + `auth.publicUrl` (`scheme://host[:port]`, no path), which is also the OIDC + audience. + - The repository must be bound to the service in continuo's + `ciAuth.bindings`. + - The first release of a service is an operator bootstrap; the workflow never + sends `bootstrap`. + + The workflow checks the endpoint and does an authenticated read of + `/api/v1/current-prod` before building, so a wrong URL, audience or missing + binding fails early. It then submits to `/api/v1/releases` + with a fresh GitHub Actions OIDC token per call and polls + `GET /api/v1/releases/{id}` for up to about 50 minutes (`timeout-minutes: 60`), + failing on `rejected` and `superseded`. A newer push no longer cancels a run + that is waiting for its release (`cancel-in-progress: false`). Workflows + already copied from the template are untouched. ## [0.8.0] - 2026-10-01 diff --git a/README.md b/README.md index 4d27dd8..5b8ddd1 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,8 @@ the Go parser has not been taught is a production outage, not a refactor. 3. Configure repository variables in GitHub (Settings → Secrets and variables → Actions): `REGISTRY` (your Docker registry), `BUCKET` (your S3 bucket for contract artifacts), `RELEASE_ENDPOINT` (the base - URL of your continuo install, `scheme://host[:port]` with no path). The + URL of your continuo install, `scheme://host[:port]` with no path, the origin of continuo's + `auth.publicUrl`). The workflow calls `/api/v1/releases` with its GitHub Actions OIDC token, so the repository must be bound to your service in continuo's `ciAuth.bindings` (see [Releasing from CI](https://github.com/carolsimone/continuo/blob/main/deploy/README.md#releasing-from-ci-github-actions)), diff --git a/template/.github/workflows/release.yml b/template/.github/workflows/release.yml index 8237496..e4308aa 100644 --- a/template/.github/workflows/release.yml +++ b/template/.github/workflows/release.yml @@ -1,35 +1,90 @@ name: release on: push: { branches: [main] } +# A run that is waiting for continuo's verdict must not be cancelled by a newer +# push: the release it submitted keeps going in continuo regardless, and the job +# would stop reporting its outcome. Runs queue instead, and continuo supersedes +# an older release when a newer one is promoted. concurrency: group: release - cancel-in-progress: true + cancel-in-progress: false env: SERVICE: your-service-name # one service name per domain repo - # Repository variable RELEASE_ENDPOINT: the base URL of continuo's ui, as - # scheme://host[:port] with no path (e.g. https://continuo.example.com). + # Repository variable RELEASE_ENDPOINT: the origin of continuo's `auth.publicUrl`, + # exactly: lowercase host, no default port, no path (e.g. https://continuo.example.com). + # It is also the OIDC audience, which assumes continuo's default `ciAuth.audience`. # This repository must be bound to $SERVICE in continuo's `ciAuth.bindings`. RELEASE_ENDPOINT: ${{ vars.RELEASE_ENDPOINT }} jobs: release: runs-on: ubuntu-latest - timeout-minutes: 45 + timeout-minutes: 60 permissions: { contents: read, packages: write, id-token: write } steps: # Fail before building anything when the endpoint is missing or malformed. - # CONTINUO_ORIGIN (the endpoint without a trailing slash) is both the API - # base and the OIDC audience. + # The whole value must match: one line, no whitespace, no credentials, no + # path. CONTINUO_ORIGIN (the endpoint without a trailing slash) is both the + # API base and the OIDC audience. The value is never echoed. - name: Check release endpoint run: | if [ -z "$RELEASE_ENDPOINT" ]; then echo "::error::repository variable RELEASE_ENDPOINT is not set; set it to continuo's base URL (scheme://host)" exit 1 fi - if ! printf '%s' "$RELEASE_ENDPOINT" | grep -Eq '^https?://[^/?#]+/?$'; then - echo "::error::RELEASE_ENDPOINT must be continuo's base URL as scheme://host[:port] with no path (got '$RELEASE_ENDPOINT')" + origin_re='^https?://[^/?#@[:space:]]+/?$' + if ! [[ $RELEASE_ENDPOINT =~ $origin_re ]]; then + echo "::error::RELEASE_ENDPOINT must be continuo's base URL as scheme://host[:port]: one line, no path, no credentials" exit 1 fi echo "CONTINUO_ORIGIN=${RELEASE_ENDPOINT%/}" >> "$GITHUB_ENV" + # Fail before building anything when the URL, the token audience or the + # repository binding is wrong: an authenticated read of /api/v1/current-prod + # is refused (401 or 403) unless the repository is bound in `ciAuth.bindings`. + # Also writes the helpers the submit step reuses. Every call asks for a fresh + # OIDC token (they expire within minutes) and passes it to curl only as a + # header read from stdin, so it never reaches a log or a process listing. + - name: Preflight continuo API + run: | + cat > "$RUNNER_TEMP/continuo-api.sh" <<'HELPERS' + oidc_token() { + local aud + aud="$(jq -rn --arg a "$CONTINUO_ORIGIN" '$a | @uri')" + printf 'Authorization: bearer %s\n' "$ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ + | curl -sS --fail -H @- "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=${aud}" \ + | jq -er '.value' + } + + # call METHOD PATH [BODY]: prints the HTTP status (000 when curl or the + # token request failed); the response body is left in $resp. + call() { + local method="$1" path="$2" body="${3:-}" token out + token="$(oidc_token)" || { echo "could not request a GitHub Actions OIDC token" >&2; echo 000; return 0; } + local -a args=(-sS -X "$method" -o "$resp" -w '%{http_code}' -H @- -H 'Accept: application/json') + [ -z "$body" ] || args+=(-H 'Content-Type: application/json' -d "$body") + out="$(printf 'Authorization: Bearer %s\n' "$token" | curl "${args[@]}" "${CONTINUO_ORIGIN}${path}" || true)" + echo "${out:-000}" + } + + fail_with_response() { + echo "::error::$1: HTTP $2 $(jq -r '"code=\(.code // "unknown") error=\(.error // "none")"' "$resp" 2>/dev/null || true)" + exit 1 + } + HELPERS + + resp="$(mktemp)" + trap 'rm -f "$resp"' EXIT + . "$RUNNER_TEMP/continuo-api.sh" + for attempt in 1 2 3; do + status="$(call GET /api/v1/current-prod)" + case "$status" in 000 | 429 | 502 | 503 | 504) [ "$attempt" -eq 3 ] || sleep $((attempt * 5)) ;; *) break ;; esac + done + case "$status" in + 200) echo "continuo API reachable and this repository is bound" ;; + 401 | 403) + echo "::error::continuo refused this repository (HTTP $status). Check that RELEASE_ENDPOINT is exactly the origin of continuo's auth.publicUrl (the OIDC audience), and that this repository's id is listed in continuo's ciAuth.bindings for the service." + exit 1 ;; + *) fail_with_response "reading $CONTINUO_ORIGIN/api/v1/current-prod" "$status" ;; + esac - uses: actions/checkout@v4 - uses: astral-sh/setup-uv@v5 # PRECONDITION: continuo-python-runtime must be published to PyPI (this repo's release pipeline). @@ -76,40 +131,15 @@ jobs: # a release that races its own artifacts fails at the parsing stage. # # Submits the release to continuo's public API and waits for its verdict. - # Each call carries a GitHub Actions OIDC token whose audience is the origin - # of RELEASE_ENDPOINT. The tokens expire within minutes, so every call asks - # for a fresh one; no token is ever printed. The token carries `repo` and - # `commit_sha`, so the body leaves them out. Promoted is success; rejected - # and superseded fail the job. + # Each call carries a fresh GitHub Actions OIDC token (see the preflight + # step) whose audience is the origin of RELEASE_ENDPOINT. The token carries + # `repo` and `commit_sha`, so the body leaves them out. Promoted is success; + # rejected and superseded fail the job. - name: Submit release to continuo run: | resp="$(mktemp)" trap 'rm -f "$resp"' EXIT - - oidc_token() { - local aud - aud="$(jq -rn --arg a "$CONTINUO_ORIGIN" '$a | @uri')" - printf 'Authorization: bearer %s\n' "$ACTIONS_ID_TOKEN_REQUEST_TOKEN" \ - | curl -sS --fail -H @- "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=${aud}" \ - | jq -er '.value' - } - - # call METHOD PATH [BODY]: prints the HTTP status (000 when curl or the - # token request failed); the response body is left in $resp. - call() { - local method="$1" path="$2" body="${3:-}" token - token="$(oidc_token)" || { echo "could not request a GitHub Actions OIDC token" >&2; echo 000; return 0; } - local -a args=(-sS -X "$method" -o "$resp" -w '%{http_code}' -H @- -H 'Accept: application/json') - [ -z "$body" ] || args+=(-H 'Content-Type: application/json' -d "$body") - local out - out="$(printf 'Authorization: Bearer %s\n' "$token" | curl "${args[@]}" "${CONTINUO_ORIGIN}${path}" || true)" - echo "${out:-000}" - } - - fail_with_response() { - echo "::error::$1: HTTP $2 $(jq -r '"code=\(.code // "unknown") error=\(.error // "none")"' "$resp" 2>/dev/null || true)" - exit 1 - } + . "$RUNNER_TEMP/continuo-api.sh" body="$(jq -n \ --arg release_id "$RELEASE_ID" \ @@ -120,14 +150,14 @@ jobs: # A submit is idempotent on release_id, so transient failures are retried. for attempt in 1 2 3; do status="$(call POST /api/v1/releases "$body")" - case "$status" in 000 | 429 | 502 | 503 | 504) sleep $((attempt * 5)) ;; *) break ;; esac + case "$status" in 000 | 429 | 502 | 503 | 504) [ "$attempt" -eq 3 ] || sleep $((attempt * 5)) ;; *) break ;; esac done [ "$status" = 202 ] || fail_with_response "submitting release $RELEASE_ID" "$status" echo "submitted release $RELEASE_ID (service=$SERVICE) to $CONTINUO_ORIGIN" - # Poll to a terminal status: 90 polls x 10 s = 15 minutes. + # Poll to a terminal status: 300 polls x 10 s = 50 minutes. last="" - for _ in $(seq 1 90); do + for _ in $(seq 1 300); do status="$(call GET "/api/v1/releases/$RELEASE_ID")" case "$status" in 200) ;; diff --git a/template/README.md b/template/README.md index 8641aab..d6e4a9e 100644 --- a/template/README.md +++ b/template/README.md @@ -9,11 +9,15 @@ This is a copy-ready template for implementing a [continuo Python domain repo](h 3. **Configure repository variables** in GitHub (Settings → Secrets and variables → Actions): - `REGISTRY`: Your Docker registry (e.g., `ghcr.io/org`) - `BUCKET`: Your S3 bucket for contract artifacts - - `RELEASE_ENDPOINT`: the base URL of your continuo install, as - `scheme://host[:port]` with no path (for example - `https://continuo.example.com`). The workflow calls - `/api/v1/releases` and fails before building anything - when the variable is empty or carries a path. + - `RELEASE_ENDPOINT`: the base URL of your continuo install. It must be + exactly the origin of continuo's `auth.publicUrl`: lowercase host, no + default port, no path (for example `https://continuo.example.com`). The + workflow also uses it as the OIDC token audience, which assumes continuo's + default `ciAuth.audience`; if the install sets a different one, change the + `audience` the workflow requests. It calls + `/api/v1/releases`, and fails before building anything + when the variable is empty or malformed, or when continuo refuses this + repository (a preflight read of `/api/v1/current-prod`). 4. **Configure repository secrets**: - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` for S3 uploads - `release.yml` already logs in to `ghcr.io` with the built-in `GITHUB_TOKEN` @@ -88,7 +92,7 @@ The CI/CD pipeline (`release.yml`) performs the six-step orchestration: 5. **Build and push** Docker image 6. **Upload contract**, then **submit the release** to continuo's `POST /api/v1/releases` and poll `GET /api/v1/releases/{id}` for up to about - 15 minutes. The job succeeds when the release is `promoted` and fails when it + 50 minutes (the job's `timeout-minutes` is 60). The job succeeds when the release is `promoted` and fails when it is `rejected` or `superseded` ## Resources diff --git a/tests/test_template.py b/tests/test_template.py index 5d7ba0c..6698645 100644 --- a/tests/test_template.py +++ b/tests/test_template.py @@ -1,7 +1,11 @@ """Test the domain-repo template.""" +import json import os +import shutil import subprocess +import threading +from http.server import BaseHTTPRequestHandler, HTTPServer from pathlib import Path import pytest @@ -52,18 +56,6 @@ def test_template_demonstrates_multiple_named_reads(): assert f'ctx.read("{name}")' in script, f"declared read {name!r} unused by the script" -def test_release_workflow_cancels_superseded_main_runs(): - """Only the newest main-branch release may finish publishing.""" - workflow = yaml.safe_load( - (TEMPLATE / ".github" / "workflows" / "release.yml").read_text() - ) - - assert workflow["concurrency"] == { - "group": "release", - "cancel-in-progress": True, - } - - def _release_job(): workflow = yaml.safe_load( (TEMPLATE / ".github" / "workflows" / "release.yml").read_text() @@ -75,26 +67,66 @@ def _step(job, name): return next(step for step in job["steps"] if step.get("name") == name) +def _logical_lines(script): + """The script's lines with backslash continuations joined.""" + return script.replace("\\\n", " ").splitlines() + + +def test_release_workflow_queues_runs_instead_of_cancelling_them(): + """A run waiting for continuo's verdict must not be cancelled by a newer push.""" + workflow, _ = _release_job() + + assert workflow["concurrency"] == { + "group": "release", + "cancel-in-progress": False, + } + + def test_release_workflow_uses_the_public_release_api(): - """The release call is the authenticated public API, bounded and token-safe.""" + """The release call is the authenticated public API, preflighted and bounded.""" workflow, job = _release_job() names = [step.get("name") for step in job["steps"]] + preflight = _step(job, "Preflight continuo API")["run"] submit = _step(job, "Submit release to continuo")["run"] assert job["permissions"]["id-token"] == "write" - assert isinstance(job["timeout-minutes"], int) + assert job["timeout-minutes"] == 60 assert workflow["env"]["RELEASE_ENDPOINT"] == "${{ vars.RELEASE_ENDPOINT }}" - # Submit comes after the image build and the contract upload. + # The endpoint check and the preflight run before anything is built. + assert names[:2] == ["Check release endpoint", "Preflight continuo API"] + assert names.index("Preflight continuo API") < names.index("Build and push image") assert names.index("Submit release to continuo") > names.index("Upload contract artifact") + assert "/api/v1/current-prod" in preflight + assert "ciAuth.bindings" in preflight + assert "audience=" in preflight assert "/api/v1/releases" in submit - assert "audience=" in submit assert "terminal" in submit - assert "seq 1 90" in submit # a bounded poll - # The bare, unauthenticated `${{ vars.RELEASE_ENDPOINT }}/releases` call is gone. - assert "/releases\"" not in (TEMPLATE / ".github" / "workflows" / "release.yml").read_text() - # The body is built with jq, and the token is only ever sent as a header. + assert "seq 1 300" in submit # a poll bounded to 50 minutes assert "jq -n" in submit - assert "echo \"$token" not in submit and "ACTIONS_ID_TOKEN_REQUEST_TOKEN\" >&2" not in submit + # Retries sleep between attempts, never after the last one. + for script in (preflight, submit): + assert 'for attempt in 1 2 3' in script + assert '[ "$attempt" -eq 3 ] || sleep' in script + + +def test_release_workflow_never_exposes_the_token(): + """No token value is echoed, and tokens reach curl only as a header read from stdin.""" + _, job = _release_job() + scripts = [step["run"] for step in job["steps"] if "run" in step] + lines = [line.strip() for script in scripts for line in _logical_lines(script)] + + assert not any(line.startswith("set -x") or " set -x" in line for line in lines) + # Both token-carrying curl calls read their header from stdin. + assert "\n".join(lines).count("-H @-") == 2 + token_refs = ("$token", "${token}", "ACTIONS_ID_TOKEN_REQUEST_TOKEN") + for line in lines: + if any(ref in line for ref in token_refs): + # The only uses: a header printed into a pipe that feeds curl. + assert "printf 'Authorization:" in line and "| curl" in line, line + if line.startswith(("echo", "printf")) and "Authorization:" not in line: + assert not any(ref in line for ref in token_refs), line + # No curl command line carries an Authorization header literally. + assert "-H 'Authorization" not in line and '-H "Authorization' not in line, line @pytest.mark.parametrize( @@ -104,13 +136,24 @@ def test_release_workflow_uses_the_public_release_api(): ("continuo.example.com", False), ("https://continuo.example.com/api", False), ("https://continuo.example.com?x=1", False), + ("https://continuo.example.com#frag", False), + ("https://continuo.example.com\n", False), + ("https://continuo.example.com\nINJECTED=1", False), + ("https://continuo.example.com\nhttps://other.example.com", False), + ("https://continuo.example.com/\nINJECTED=1", False), + ("https://user:secret@continuo.example.com", False), + ("https://continuo.example.com ", False), ("https://continuo.example.com", True), ("https://continuo.example.com/", True), ("http://localhost:8090", True), ], ) def test_release_workflow_fails_closed_on_a_bad_endpoint(tmp_path, endpoint, ok): - """The first step rejects an unset or malformed RELEASE_ENDPOINT before any build.""" + """The first step rejects an unset or malformed RELEASE_ENDPOINT before any build. + + The whole value must match, so a multi-line value cannot smuggle extra + lines into $GITHUB_ENV, and the value is never echoed back. + """ _, job = _release_job() assert job["steps"][0]["name"] == "Check release endpoint" github_env = tmp_path / "github_env" @@ -125,7 +168,63 @@ def test_release_workflow_fails_closed_on_a_bad_endpoint(tmp_path, endpoint, ok) if ok: assert github_env.read_text() == f"CONTINUO_ORIGIN={endpoint.rstrip('/')}\n" else: + assert github_env.read_text() == "" assert "RELEASE_ENDPOINT" in result.stdout + assert "secret" not in result.stdout and "INJECTED" not in result.stdout + + +class _Continuo(BaseHTTPRequestHandler): + """A stub of the OIDC token endpoint and continuo's /api/v1/current-prod.""" + + current_prod_status = 200 + + def log_message(self, *args): + pass + + def do_GET(self): # noqa: N802 + if self.path.startswith("/oidc"): + status, body = 200, {"value": "stub-jwt"} + elif self.headers.get("Authorization") != "Bearer stub-jwt": + status, body = 401, {"error": "bad token", "code": "invalid_token"} + else: + status = type(self).current_prod_status + body = {"current_prod_release_id": "r1"} if status == 200 else {"error": "no", "code": "forbidden"} + raw = json.dumps(body).encode() + self.send_response(status) + self.send_header("content-length", str(len(raw))) + self.end_headers() + self.wfile.write(raw) + + +@pytest.mark.skipif(not (shutil.which("curl") and shutil.which("jq")), reason="needs curl and jq") +@pytest.mark.parametrize(("status", "ok"), [(200, True), (401, False), (403, False)]) +def test_release_workflow_preflight_names_the_binding_on_refusal(tmp_path, status, ok): + """The preflight calls current-prod with a bearer token and explains a refusal.""" + _, job = _release_job() + handler = type("Handler", (_Continuo,), {"current_prod_status": status}) + server = HTTPServer(("127.0.0.1", 0), handler) + threading.Thread(target=server.serve_forever, daemon=True).start() + origin = f"http://127.0.0.1:{server.server_port}" + try: + result = subprocess.run( + ["bash", "-eo", "pipefail", "-c", _step(job, "Preflight continuo API")["run"]], + env={ + "PATH": os.environ["PATH"], + "RUNNER_TEMP": str(tmp_path), + "CONTINUO_ORIGIN": origin, + "ACTIONS_ID_TOKEN_REQUEST_TOKEN": "request-token", + "ACTIONS_ID_TOKEN_REQUEST_URL": f"{origin}/oidc?api-version=2.0", + }, + capture_output=True, + text=True, + ) + finally: + server.shutdown() + assert (result.returncode == 0) is ok, result.stdout + result.stderr + assert "stub-jwt" not in result.stdout + result.stderr + if not ok: + assert "ciAuth.bindings" in result.stdout + assert "auth.publicUrl" in result.stdout def test_readme_and_template_name_images_the_publisher_emits():