Skip to content

[AAASM-5688] 🔧 (ci): Run the modules ref drift gate on a schedule, not only on config changes - #143

Merged
Chisanan232 merged 3 commits into
mainfrom
v0.0.1/AAASM-5688/fix/modules_default_ref
Aug 13, 2026
Merged

[AAASM-5688] 🔧 (ci): Run the modules ref drift gate on a schedule, not only on config changes#143
Chisanan232 merged 3 commits into
mainfrom
v0.0.1/AAASM-5688/fix/modules_default_ref

Conversation

@Chisanan232

@Chisanan232 Chisanan232 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Description

Read this section first — otherwise the diff looks unrelated to the ticket.

AAASM-5688 reports that modules.json pins "ref": "master" for core, python-sdk, node-sdk and go-sdk while those repos' default branch is main. That flip already landed on main on 2026-07-23, in ecd3670, 9ce0e37 and ce64923, and a drift gate (.github/workflows/modules-ref-check.yml) landed on 2026-07-24 in d691254 — both roughly two weeks before the ticket was filed on 2026-08-07. All five modules on main today read "ref": "main". "ref": "master" survives only in a stale local checkout, which was 138 commits behind origin/main.

So this PR does not flip any ref. It closes the hole that the already-merged gate left open.

The ticket's central measurement was an artifact

The ticket states master and main are at identical SHAs in all five repos, and treats that as evidence a master branch still exists and is mirrored. It is not two branches — it is main read twice.

Probe (repo: agent-assembly) Result Verdict
git ls-remote --heads no refs/heads/master (25 heads, none named master) absent
gh api .../git/ref/heads/master HTTP 404 absent
gh api .../branches/master returns name: "main", sha ca2e5e4e9 rename redirect, not a branch

Two controls, because an absence proves nothing on its own:

  • Negative controlgh api .../branches/definitely-not-a-branch-xyz returns Branch not found, so the branches endpoint is not blanket-permissive. It specifically honours the mastermain rename and reports main's SHA under the old name.
  • Positive controlgit ls-remote refs/heads/main returns a SHA in all five repos, so the empty master result is a real negative rather than a broken or unauthenticated probe.

Same result in all five repos. There are no master mirrors. Consequence for risk: a rename fails loudly, not silently — aggregate.sh runs git clone then git checkout "$ref" under set -euo pipefail, so a vanished ref is a hard pathspec did not match. The gate's own header records this having broken aggregation three times in one week (AAASM-5047).

The actual residual defect: the gate is correct and never runs

modules-ref-check.yml triggers on push / pull_request filtered to paths: [modules.json, the workflow itself]. The event it exists to catch — a source repo changing its default branch — happens in another repo and touches neither file.

Measured 2026-08-13:

So the sharpest statement of the defect is not "it ran twice": the gate has never once evaluated a modules.json it did not ship with. Its only push-triggered run has headSha 4cb6729 — its own merge commit. Zero evaluations in the 20 days since.

A drift check that only runs when you edit the thing it checks cannot catch drift arriving from outside.

The two drift shapes are also not equally visible, which is why a scheduled run is required rather than merely tidy:

  • A renamed-away ref stops existing → git checkout fails loudly on the next aggregation cycle.
  • A ref naming a branch that still exists but is no longer the default (a stale release branch, a gh-pages pin) clones and builds green forever while publishing a frozen tree — silently stale, which is the failure this ticket is about. Only a time-based run catches that one.

Type of Change

  • 🔧 Configuration / CI change
  • 📝 New documentation page
  • 🐛 Bug fix (correcting inaccurate or missing content)
  • ♻️ Refactor / restructure (no content change)
  • ✨ New section or feature documentation

What changed

Three commits, 2 files, +22/−1.

0506b24 🔧 (ci): Run the modules.json ref-drift gate on a daily schedule
Adds schedule: cron "17 6 * * *" to modules-ref-check.yml. Daily rather than aggregate.yml's */30: this compares config against upstream default branches, which change on human timescales, and each run is five git ls-remote calls. Offset from the top of the hour because GitHub's scheduler is congested there and schedule is best-effort (AAASM-4508). The path filters are left in place — they still give fast feedback on config edits; the schedule covers what they structurally cannot.

eee583c 📝 (modules): Record the ref contract and its enforcing gate in the registry
The registry $comment advertised ref as "pins the checkout; use a tag/SHA to pin a release line". That affordance is no longer true — the gate admits no exceptions and would fail such a pin. Rather than paper over the contradiction, the comment now states the real contract: ref must name the repo's live default branch, the gate resolves that with git ls-remote --symref, and a deliberate departure needs both a recorded reason and an opt-out in the gate — not a weakened gate.

No opt-out mechanism is built, because no module pins a tag today. An unused mechanism is untested code that reads as coverage.

7ed65bd 📝 (modules): Name the modules whose ref does not control their build
The rewritten $comment still carried the clause "'ref' pins the checkout" inherited verbatim from the text it replaced — false for 2 of 5 modules. aggregate.sh:355 calls clone_module only when generator != mkdocs-material, and build_python hardcodes git clone --depth 1 -b gh-pages, so python-sdk and arena never check out at ref. Leaving that inside the very comment this PR rewrites to be a true record of the ref contract invites a concrete mistake: read the strengthened "MUST name the repo's live default branch", set arena.ref to a tag to freeze a release line, get a red gate over a field with no effect on that build, then weaken the gate. The clause now names the exception and points at AAASM-5728, which owns the inertness itself.

Every claim that clause makes was asserted against the code, not from memory: the mkdocs-material module set is exactly {python-sdk, arena}; the guard is at aggregate.sh:355; the gh-pages hardcode is at :159; and $ref appears only inside clone_module.

Why pin main at all, rather than stop naming a branch?

The tempting conclusion is that a hardcoded branch name caused this, so ref should be dropped entirely and aggregate.sh should just follow the remote HEAD. That inverts the actual lesson:

The defect was never the written-down name; it was that nothing checked it. Following HEAD implicitly doesn't remove the coupling, it removes the checkpoint.

An implicit-HEAD registry means an upstream repo could repoint its default branch and the hub would publish the new tree with no diff in this repo, no PR and no review. That trades a detectable inconsistency for an undetectable one — the exact failure class this ticket is about. ref also has to survive for release-line pinning, which the registry advertises.

The counter-case was checked rather than assumed: build_go already passes MASTER_REF="HEAD", and that is correct there, because it names no branch and no review value is lost by following HEAD.

So: an explicit pin, reviewed in a diff, plus a gate that resolves reality at check time. Both halves.

How to verify

The gate's logic was proven falsifiable before trusting it — it had never failed in CI, and a check that has never failed is measuring nothing.

The checker was extracted verbatim from the shipped workflow rather than retyped, so the thing under test is the artifact that decides, not a copy of it. Mutate → assert → restore ran in a single foreground command with restore guaranteed by a trap and verified by sha256.

Case Mutation Expected Got
A none (positive control) 0 0
B core.ref = master (absent upstream) 1 1
C core.ref = trunk (arbitrary name) 1 1
D arena.ref = gh-pages (exists upstream, but is not the default) 1 1
E core.ref = v0.0.1-rc.6 (a real release tag) 1 1

Case D is the proof that matters. B and C would also fail a check that merely verified "the branch exists", which would be indistinguishable from a correct check. Case D is the only one of those that separates "resolves the default branch" from "branch exists"gh-pages genuinely exists in arena, and must still be rejected because it is not the default.

Case E demonstrates the consequence the $comment now asserts rather than merely arguing it: the removed tag/SHA affordance really is removed. v0.0.1-rc.6 was confirmed to be a real upstream tag first — against a non-existent tag the case would collapse into C and prove nothing.

Case D failing output:

modules.json ref drift detected:
  - arena: modules.json pins ref 'gh-pages' but ai-agent-assembly/arena's live default branch is 'main'
---- exit code: 1
RESTORE OK   sha256=0814111913ed77ec3544ad59ac6b06a9567635a2ed058bfa2b21c329ed4da010

Restore hash matched the baseline in every case. Re-extracting the checker after the workflow edit confirmed it is byte-identical — only triggers changed.

Also verified:

  • The workflow parses. Asserted directly via actionlint (exit 0) in independent review, with two negative controls proving that exit code means something — a 4-field cron and 17 99 * * * each exit 1. This is asserted rather than inferred from absent red, because an unparseable workflow shows zero check runs rather than a failing one. Locally, python-yaml confirms all four triggers register (push, pull_request, workflow_dispatch, schedule).
  • modules.json parses under both python and jq (aggregate.sh reads it with jq).
  • generate_compatibility.py --check, generate_hub_components.py --check and check_repo_names.py all exit 0, captured as true exit codes rather than through a pipeline.
  • check_repo_names.py proven non-vacuous: injecting a stale repo name makes it exit 1.

Two things a future reader would otherwise assume

  • "On every modules.json change" is really "every PR targeting main, and every push to main." Both triggers carry branches: [main] in addition to the path filter, so a modules.json edit on a feature branch with no PR open is unchecked until the PR exists. Practically irrelevant here, but it is not the unconditional guarantee the phrasing suggests.
  • A failing scheduled run blocks nothing. There is no PR to go red and no required check to fail. Detection is the red Actions tab plus GitHub's failure email to whoever last edited the cron. That satisfies the AC and is a large improvement on never running, but nobody is required to look. Related: GitHub auto-disables schedule triggers after 60 days of repository inactivity. Low risk for docs, which is active, but it is worth naming as a silent-disable path for a gate whose entire purpose is not being silent.

Related Issues

Documentation Checklist

  • Commits follow GitEmoji convention and are small / atomic
  • Self-hosted instructions not included (SaaS-only scope)
  • cd docs && mdbook build — n/a, no book content changed (CI config + registry comment only)
  • All internal cross-links verified — n/a, no links changed
  • Page registered in docs/src/SUMMARY.md — n/a, no page added
  • "Last reviewed" footer updated — n/a, no page content changed

Out of scope

  • ref is inert for python-sdk and arena, and the branch that actually drives them is unguarded — now named in the $comment (commit 7ed65bd) but not fixed here. The gate validates a field with no effect on those builds while build_python's hardcoded git clone -b gh-pages goes unchecked; if a module's gh-pages stops being republished, the hub re-copies the old tree and deploys green. Being fair to the current state: python-sdk's gh-pages is 3 days behind its main, but that is expected if no docs changed and is not evidence of present staleness. The defect is that the coupling is neither enforced nor observable. Tracked in AAASM-5728.
  • Forbidden-framing text in docs/src/ is AAASM-5727. Those files are untouched here.
  • Two aggregate.yml failures on 2026-08-12 (20:33, 21:34) are the peaceiris/actions-hugo setup step, not a ref problem. No action.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XWLmA8FgULT9e6ntdCo1H2

The gate's path filters only fire when this repo edits modules.json, but the
event it exists to catch — a source repo changing its default branch — happens
elsewhere and touches nothing here. Measured 2026-08-13: 130 commits landed on
main after the gate was added, one touched modules.json, and the gate evaluated
0 times in 20 days.

This matters most for the drift shape that is invisible. A renamed-away ref
stops existing and breaks aggregate.sh's checkout loudly; a ref naming a branch
that still exists but is no longer the default builds green forever while
publishing a frozen tree. Only a time-based run catches the second.

Refs AAASM-5688
…gistry

The registry described `ref` as "pins the checkout; use a tag/SHA to pin a
release line" and its channel as "master/main HEAD" — neither states the rule
that actually governs the field, and the tag/SHA affordance is no longer true:
modules-ref-check.yml admits no exceptions and would fail such a pin.

State the real contract at the pin site: `ref` must name the repo's live
default branch, the gate resolves that with `git ls-remote --symref`, and a
deliberate departure needs both a recorded reason and an opt-out in the gate —
not a weakened gate. No module pins a tag today, so no opt-out is built.

Refs AAASM-5688
The corrected $comment still said "'ref' pins the checkout" unqualified, a
clause inherited verbatim from the text it replaced. It is false for 2 of 5
modules: aggregate.sh calls clone_module only when generator != mkdocs-material,
and build_python hardcodes `git clone --depth 1 -b gh-pages`, so python-sdk and
arena never check out at 'ref'.

Leaving that in the very comment this PR rewrites to be a true record of the ref
contract invites the concrete mistake: read "MUST name the repo's live default
branch", set arena.ref to a tag to freeze a release line, get a red gate over a
field with no effect on that build, then weaken the gate.

Names the exception and points at AAASM-5728, which owns the inertness itself.
Also tightens "on every modules.json change" to what the triggers actually do —
a PR to main or a push to main.

Refs AAASM-5688
@sonarqubecloud

Copy link
Copy Markdown

@Chisanan232
Chisanan232 merged commit 2664205 into main Aug 13, 2026
5 checks passed
@Chisanan232
Chisanan232 deleted the v0.0.1/AAASM-5688/fix/modules_default_ref branch August 13, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant