Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 67 additions & 1 deletion .github/workflows/regen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ name: regen
#
# Loop guard: the job only commits/tags for patch or minor. `none` requires byte identity
# across IR, fixtures, and both emitted trees. Unsafe or unexplained changes are blocked.
#
# A blocked run needs a HUMAN, so it opens a GitHub issue assigned to the owner and keeps
# one issue per outage rather than one per run. The next healthy run closes it. Closing is
# the point: the upsert reuses an open issue by title, so a stale open issue would turn the
# next real failure into another silent comment on a thread nobody watches. Between
# 2026-09-01 and 2026-09-05 this job failed every night with no signal but GitHub's default
# Actions email, and the published SDKs sat five days stale.

on:
repository_dispatch:
Expand All @@ -24,11 +31,17 @@ on:
permissions:
contents: write
actions: write # to trigger release.yml via workflow_dispatch after tagging
issues: write # to open/close the blocked-regen incident

concurrency:
group: regen
cancel-in-progress: false

env:
# The upsert deduplicates by exact title, so the failure and recovery paths must name
# the same string. Changing it orphans any issue already open under the old title.
REGEN_ISSUE_TITLE: "regen is blocked: the SDKs are not being published"

jobs:
regen:
name: refresh catalog + classify + tag
Expand Down Expand Up @@ -65,7 +78,7 @@ jobs:
- name: classify generated diff -> release state + change summary
id: classify
run: |
bash scripts/release-notes.sh release-bump.txt release-notes.md
bash scripts/release-notes.sh release-bump.txt release-notes.md release-changes.json
echo "bump=$(cat release-bump.txt)" >> "$GITHUB_OUTPUT"

- name: prepare blocked-change evidence
Expand All @@ -77,6 +90,7 @@ jobs:
fi
cp generator/ir.json blocked-release/new-ir.json
cp release-notes.md blocked-release/release-notes.md
cp release-changes.json blocked-release/release-changes.json
git add -N -- generator/ir.json generator/fixtures.json \
packages/typescript/src/generated packages/python/src/getanyapi/platforms
git diff --binary --no-ext-diff HEAD -- \
Expand All @@ -85,6 +99,7 @@ jobs:
> blocked-release/generated.diff

- name: upload blocked-change evidence
id: evidence
if: ${{ always() && steps.classify.outputs.bump == 'blocked' }}
uses: actions/upload-artifact@v4
with:
Expand Down Expand Up @@ -160,3 +175,54 @@ jobs:
# `push` tag (for a human-pushed tag, e.g. the first manual v0.1.0). Both paths reach
# the same publish jobs.
run: gh workflow run release.yml --ref "v$NEXT" -f tag="v$NEXT"

# Every failing path lands here: a blocked classification (the stop step above exits
# 1) and any other failure, including one before the classifier ever ran.
- name: open or update the blocked-regen issue
if: failure()
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BUMP: ${{ steps.classify.outputs.bump }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
ARTIFACT_URL: ${{ steps.evidence.outputs.artifact-url }}
run: |
# The blocked and removed items ARE the block, and their slugs are what the
# reviewer decides from, so they are both the issue body and the fingerprint.
# A failure before or outside classification has no item list; it still files,
# under a single stable item, so the run link reaches a human.
if [ "$BUMP" = "blocked" ] && [ -f release-changes.json ]; then
jq -r '(.blocked + .removed)[] | "\(.slug): \(.detail)"' \
release-changes.json > regen-items.txt
else
printf 'regen failed with no classification (state: %s)\n' \
"${BUMP:-before the classifier ran}" > regen-items.txt
fi
{
printf '`regen` stopped and did not publish. The SDKs stay at the last released version until this is resolved.\n\n'
printf '## What is blocking\n\n'
sed 's/^/- /' regen-items.txt
printf '\n[Failing run](%s)\n' "$RUN_URL"
if [ -n "$ARTIFACT_URL" ]; then
printf '\n[Evidence artifact](%s) - old/new IR, release notes, and the generated diff.\n' "$ARTIFACT_URL"
fi
if [ -f release-notes.md ]; then
printf '\n## Full classifier output\n\n```\n'
cat release-notes.md
printf '```\n'
fi
printf '\nAccepting this batch is a human decision: review each blocked item against its upstream catalog change, then land a reviewed catalog-refresh PR with the version applied in lockstep. Do not relax the classifier to make it pass.\n'
} > regen-issue.md
node scripts/upsert-regen-issue.mjs \
--title "$REGEN_ISSUE_TITLE" \
--body-file regen-issue.md \
--items "$(cat regen-items.txt)"

- name: close the blocked-regen issue on recovery
if: success()
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
node scripts/upsert-regen-issue.mjs \
--title "$REGEN_ISSUE_TITLE" \
--resolve \
--body "regen recovered: [run]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID) classified \`${{ steps.classify.outputs.bump }}\`."
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,14 @@ Releases are automated from the live catalog. Two workflows drive it:
The workflow first proves that the committed generated trees match the old IR. Public
generated changes must then appear in both language trees, and the post-refresh drift check
proves that both match the new IR. A blocked run uploads the old/new IR, release notes, and
generated diff, then fails before versioning, committing, or tagging. On patch or minor it applies the version to BOTH
generated diff, then fails before versioning, committing, or tagging. Because a blocked run
needs a person, it also opens a GitHub issue assigned to the owner naming the blocked SKUs
and linking the run and that artifact (`scripts/upsert-regen-issue.mjs`); any other job
failure files the same issue. There is one issue per outage, not one per run: a repeat
rewrites the body, a changed block adds a delta comment, and the next healthy run closes
it. Closing matters - the upsert reuses an open issue by title, so leaving it open would
turn the next real failure into another silent comment.
On patch or minor it applies the version to BOTH
`packages/typescript/package.json` and `packages/python/pyproject.toml` in lockstep, commits
the regenerated tree, tags `v<X.Y.Z>`, pushes, and dispatches `release.yml`. The change
summary is the commit body and becomes the GitHub Release notes.
Expand Down
13 changes: 9 additions & 4 deletions generator/src/classify-cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
//
// Prints the state on stdout (none | patch | minor | blocked) so the workflow can read
// `$(tsx ...)`. With --summary-out it writes the human-readable change summary (commit body
// / release notes) to that path. Byte-change flags come from scripts/release-notes.sh.
// / release notes) to that path; with --json-out it writes the full classification, whose
// items carry the SKU slug the summary omits (the blocked-regen issue names them, so the
// issue alone is enough to decide). Byte-change flags come from scripts/release-notes.sh.
// Exit code is always 0 so a blocked result can be uploaded before the workflow fails.

import { readFileSync, writeFileSync } from "node:fs";
Expand All @@ -23,14 +25,16 @@ function main(): void {
console.error(
"usage: classify-cli <oldIrPath> <newIrPath> [--ir-changed] " +
"[--fixtures-changed] [--typescript-changed] [--python-changed] " +
"[--summary-out <path>] [--json]",
"[--summary-out <path>] [--json-out <path>] [--json]",
);
process.exitCode = 2;
return;
}

const summaryIdx = args.indexOf("--summary-out");
const summaryOut = summaryIdx >= 0 ? args[summaryIdx + 1] : null;
const jsonIdx = args.indexOf("--json-out");
const jsonOut = jsonIdx >= 0 ? args[jsonIdx + 1] : null;
const asJson = args.includes("--json");
const flags = new Set([
"--ir-changed",
Expand All @@ -41,10 +45,10 @@ function main(): void {
]);
for (let index = 2; index < args.length; index += 1) {
const arg = args[index] as string;
if (arg === "--summary-out") {
if (arg === "--summary-out" || arg === "--json-out") {
if (!args[index + 1]) {
// eslint-disable-next-line no-console
console.error("--summary-out requires a path");
console.error(`${arg} requires a path`);
process.exitCode = 2;
return;
}
Expand All @@ -65,6 +69,7 @@ function main(): void {
});

if (summaryOut) writeFileSync(summaryOut, result.summary);
if (jsonOut) writeFileSync(jsonOut, `${JSON.stringify(result, null, 2)}\n`);
if (asJson) {
// eslint-disable-next-line no-console
console.error(JSON.stringify(result, null, 2));
Expand Down
60 changes: 60 additions & 0 deletions generator/test/classify-cli.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import { execFileSync } from "node:child_process";
import { mkdtempSync, readFileSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import type { Classification } from "../src/classify.js";
import { base, ir } from "./classify-fixture.js";
import { sku } from "./factories.js";

const CLI = join(import.meta.dirname, "..", "src", "classify-cli.ts");

function runCli(args: string[]): { stdout: string; dir: string } {
const dir = mkdtempSync(join(tmpdir(), "classify-cli-"));
const oldPath = join(dir, "old.json");
const newPath = join(dir, "new.json");
writeFileSync(oldPath, JSON.stringify(ir([base])));
writeFileSync(
newPath,
JSON.stringify(ir([base, sku({ slug: "acme.extra", name: "Extra" })])),
);
const stdout = execFileSync(
process.execPath,
[
join(import.meta.dirname, "..", "node_modules", "tsx", "dist", "cli.mjs"),
CLI,
oldPath,
newPath,
...args.map((arg) => arg.replace("<dir>", dir)),
],
{ encoding: "utf8" },
);
return { stdout, dir };
}

describe("classify-cli", () => {
// The blocked-regen issue names the SKU each blocked item belongs to, and the rendered
// summary drops the slug. --json-out is the only surface that carries it.
it("--json-out writes the full classification, slugs included", () => {
const { stdout, dir } = runCli([
"--ir-changed",
"--typescript-changed",
"--python-changed",
"--fixtures-changed",
"--json-out",
join("<dir>", "changes.json"),
]);
expect(stdout.trim()).toBe("minor");
const parsed = JSON.parse(
readFileSync(join(dir, "changes.json"), "utf8"),
) as Classification;
expect(parsed.bump).toBe("minor");
expect(parsed.added).toContainEqual(
expect.objectContaining({ slug: "acme.extra" }),
);
});

it("rejects --json-out with no path instead of writing somewhere else", () => {
expect(() => runCli(["--json-out"])).toThrow();
});
});
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"generate": "pnpm --filter @anyapi/generator generate",
"generate:check": "pnpm --filter @anyapi/generator generate -- --check",
"dash-guard": "bash scripts/check-dashes.sh",
"release:test": "node --test scripts/check-registry-version.test.mjs",
"release:test": "node --test scripts/*.test.mjs",
"check": "pnpm run dash-guard && pnpm run release:test && pnpm run generate:check && pnpm -r --workspace-concurrency=1 run check"
},
"license": "MIT"
Expand Down
14 changes: 11 additions & 3 deletions scripts/release-notes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@
# release-bump.txt)
# * the human-readable change summary (commit body / release notes) to the file named
# by $2 (default: release-notes.md)
# * the full classification as JSON to the file named by $3 (default:
# release-changes.json). Its items carry the SKU slug the summary omits, which is
# what the blocked-regen issue names.
#
# Usage:
# scripts/release-notes.sh [bumpOutPath] [summaryOutPath]
# scripts/release-notes.sh [bumpOutPath] [summaryOutPath] [jsonOutPath]
#
# The classifier itself lives in generator/src/classify.ts (unit-tested). This wrapper
# extracts the old IR and proves which generator-owned files differ byte-for-byte from HEAD.
Expand All @@ -20,6 +23,7 @@ cd "$(dirname "$0")/.."

BUMP_OUT="${1:-release-bump.txt}"
SUMMARY_OUT="${2:-release-notes.md}"
JSON_OUT="${3:-release-changes.json}"

OLD_IR="$(mktemp)"
trap 'rm -f "$OLD_IR"' EXIT
Expand All @@ -35,6 +39,7 @@ fi
# to the intended location, not relative to the package dir.
case "$BUMP_OUT" in /*) ;; *) BUMP_OUT="$PWD/$BUMP_OUT" ;; esac
case "$SUMMARY_OUT" in /*) ;; *) SUMMARY_OUT="$PWD/$SUMMARY_OUT" ;; esac
case "$JSON_OUT" in /*) ;; *) JSON_OUT="$PWD/$JSON_OUT" ;; esac

# Byte equality covers every generator-consumed artifact. `git status` includes tracked,
# deleted, and untracked emitter output, unlike `git diff` alone.
Expand All @@ -56,14 +61,17 @@ fi
# 3.2 treats an empty array expansion as unset under `set -u`, so omit it explicitly.
if [ "${#CLASSIFY_ARGS[@]}" -eq 0 ]; then
BUMP="$(pnpm --silent --filter @anyapi/generator exec tsx src/classify-cli.ts \
"$OLD_IR" "$PWD/generator/ir.json" --summary-out "$SUMMARY_OUT")"
"$OLD_IR" "$PWD/generator/ir.json" --summary-out "$SUMMARY_OUT" \
--json-out "$JSON_OUT")"
else
BUMP="$(pnpm --silent --filter @anyapi/generator exec tsx src/classify-cli.ts \
"$OLD_IR" "$PWD/generator/ir.json" "${CLASSIFY_ARGS[@]}" --summary-out "$SUMMARY_OUT")"
"$OLD_IR" "$PWD/generator/ir.json" "${CLASSIFY_ARGS[@]}" --summary-out "$SUMMARY_OUT" \
--json-out "$JSON_OUT")"
fi

printf '%s\n' "$BUMP" > "$BUMP_OUT"

echo "Generated diff classified: state=$BUMP"
echo " bump -> $BUMP_OUT"
echo " summary -> $SUMMARY_OUT"
echo " json -> $JSON_OUT"
Loading
Loading