docs(research): CI change detection works; fourteen cache-write paths are disabled (#328) - #427
Conversation
…s fork is dead (#328) Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
serina-mcfall
left a comment
serina-mcfall
left a comment
There was a problem hiding this comment.
Blocker — "every cache-save on this fork is dead" is refuted by this note's own exemplar run
The note's headline, frontmatter description, PR title and escalation 1 all rest on:
This fork never writes a CI cache. Every pull request builds cold, and always will…
Six step-level conditions, every one of them a cache write
There are nine actions/cache/save steps, not six:
$ grep -n 'actions/cache/save' .github/workflows/ci.yml
229 276 386 459 575 845 879 895 1133
The three the note omits carry no push gate at all:
ci.yml:878 - name: Save Hermit package cache
if: always() && steps.hermit-cache.outputs.cache-hit != 'true'
ci.yml:894 - name: Save pub cache
if: always() && steps.pub-cache.outputs.cache-hit != 'true'
ci.yml:1132 - name: Save mesh llama build cache
if: steps.llama_cache.outputs.cache-hit != 'true'
And it is refuted by observation on run 32473521557 — the run the note itself uses as its exemplar:
$ gh run view 32473521557 --repo launchpad-26/buzz --json event,headBranch
pull_request on docs/adr-0018-relay-vps-specification
$ gh run view 32473521557 --json jobs --jq '...Save.*cache...'
{"conclusion":"success","job":"Desktop Build (macOS)",
"saves":[{"c":"success","n":"Save mesh llama build cache"}]}
A cache-save step ran to success, on a pull_request event, on this fork.
Your "Not checked" section honestly flags that cache restores were not observed to miss. The error is on the saves, which the note treats as established rather than as the six it actually inspected.
The narrower claim survives intact and is worth keeping: six pnpm/Playwright/relay-artifact saves are push-gated and therefore never fire on this fork, which is a real and useful finding. It is the universal quantifier that has to go — and it is currently the title.
Medium — the justfile filter entry can never match, and that is a live CI defect
This one is worth pulling out because it is not just a note defect. §2 quotes the paths-filter list accurately, including - 'justfile', then asserts that touching the justfile triggers the desktop E2E shards, the macOS build and the Windows Rust job.
The tracked filename is Justfile, capitalised. dorny/paths-filter matches case-sensitively against git diff --name-status output, so that entry never fires. The Cargo.lock and ci.yml halves of the claim hold.
I am filing this as an issue against ci.yml as well, since the fix belongs there rather than in the note.
Medium — ADR-0019 and ADR-0020 are cited as records; neither is merged
The note says "the '19 CI jobs' figure in #290 and ADR-0020 is 18" and "ADR-0019 records zero". Both ADRs are open pull requests (#281, #291), not records in launchpad/decisions/ — where the sequence currently runs 0018 → 0021 → 0022. The quotations are accurate; the status is not. Filed as a shared issue, since #424 and #437 do the same thing.
What is right — and this note is mechanically excellent almost everywhere
I re-checked the spine of it and it holds line for line:
ci.yml:2-5on: push: branches: [main, release]/pull_request:— exact.- All eight
save-if: ${{ github.event_name != 'pull_request' }}at 112, 134, 159, 355, 774, 966, 1010, 1084 — exact. token: ''at line 37;SCCACHE_GHA_RW_MODE … pull_request.number == 5224at line 331; therustfilter's 12-entry path list — all verbatim.- "18 jobs, not 19" is correct, and so is the diagnosis.
grep -n '^ [a-z0-9-]*:$'returns 19 lines, one of which ispush:at line 3 — exactly the naive-grep artefact you name. The "19 CI jobs" figure genuinely does appear in #290's body and in ADR-0020's draft. - Run
32473521557reproduces exactly: branch, event,10:38:03Z → 11:10:38Z, 23 job instances, 21 ran, 2 skipped.
Catching the 19→18 miscount in someone else's document, and diagnosing why the wrong number was produced, is the best thing in this note. It is the same class of error as the one above — which is worth sitting with.
Reviewed at head ee37ecbcb. The cache-save refutation was verified by me directly against run 32473521557, not taken on report.
🤖 Review drafted by Claude Code (claude-opus-5) for @serina-mcfall.
Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
tucktuck101
changed the title
serina-mcfall
left a comment
serina-mcfall
left a comment
There was a problem hiding this comment.
Re-review — the blocker is addressed. No blocker remains.
Agent-authored under @serina-mcfall's instruction. I do not approve; this records what I verified.
Blocker (the universal "this fork never writes a CI cache", refuted by the note's own exemplar run) — fixed, and fixed the honest way. The universal is gone from the frontmatter and the finding, the three ungated saves are no longer swept in, and the refuting observation is recorded rather than quietly dropped:
$ git show origin/research/328-ci-jobs-on-launchpad:launchpad/Research/328-ci-jobs-on-launchpad.md | sed -n '2p;79,80p'
description: ... change detection works, while six explicit saves and eight rust-cache writes are disabled on pull requests.
79:The mesh llama save completed successfully in pull-request run `32473521557`. The accurate result
80:is therefore narrower: the six saves above, all eight `rust-cache` writes, and sccache writes are
Naming the run that refuted your own claim, in the note, is the part that makes this reusable rather than merely corrected.
The two Mediums are tracked, not lost. #442 is open for the justfile/Justfile case-sensitivity defect in the paths filter — which is a live CI bug rather than a note defect, and the right place for it. The "ADR-0019/ADR-0020 cited as records while both are still open PRs" item I could not find a filed issue for; flagging that rather than assuming, since the review said it was filed as a shared issue across #424 and #437. Worth confirming before this closes.
Nothing new found. The spine of the note still reproduces line for line — the eight save-if conditions at 112/134/159/355/774/966/1010/1084, the token: '', the sccache PR-number gate, and the 19→18 job-count diagnosis that remains the best thing in it.
Reviewed at head 84836598c.
🤖 Review drafted by Claude Code (claude-opus-5) for @serina-mcfall.
2 participants
Summary
Answers #328 by reading
ci.yml's gates and comparing them against real runs. Change detection works — a docs-only pull request runs 2 jobs in under a minute — but nopushevent has ever run this workflow on the fork, which makes all six cache-save steps and all eightrust-cachesave-ifconditions dead code. The fork builds cold on every pull request and cannot stop.Related issue
Closes #328
Issue type
Task
Agent provenance
Objective
Add
launchpad/Research/328-ci-jobs-on-launchpad.mdrecording whichci.ymljobs execute on a pull request targetinglaunchpad, and what never executes at all.Impacted components
launchpad/Research/328-ci-jobs-on-launchpad.md
Approach and rejected alternatives
Read the gates out of
ci.ymlmechanically and then corroborated against real run data, rather than reasoning from the YAML alone. That corroboration is what surfaced the interesting part: a pull request whose GitHub "Files changed" shows one markdown file ran the full Rust and desktop pipeline, and only thechangesjob log explains why.Rejected sampling many runs for duration statistics — most recent runs are docs-only and finish in under a minute, so the sample of code-touching runs is inherently thin. Two are reported as two, not extrapolated.
Verification
Command run:
Raw output:
Not verified
Whether the cache restores actually miss. That the fork never writes a cache is established from the gate conditions; that the restores therefore miss is inference. I did not open a restore step's log to observe a miss, and that is the one measurement that would turn "the pipeline is cold" into an observation.
The stale-base mechanism rests on one run. The
changeslog for32473521557is unambiguous about what happened there, but the general rule — that a pull request's job set depends on how far the base branch moved since it was opened — is inferred from that pluspull_request.base.shasemantics, not reproduced a second time.Duration sampling is thin: two code-touching runs (32 min, 24 min), neither cache-warm, so there is no warm comparison point.
Only
ci.ymlwas audited. The other 22 workflow files were not examined.Whether any status check is currently required was not re-checked here; ADR-0019 records zero and #358 is looking at it separately.
Security implications
None. Adds one markdown research document. It quotes workflow line numbers and public run logs; no credentials, tokens or private hostnames. Note that the document records
SCCACHE_GHA_RW_MODEreferencing a hardcoded upstream pull request number, which is already public inci.yml.Escalations
pushtomain/release; this fork has neither branch. Fixing it means changingci.yml's triggers, which is an upstream file and therefore an ADR underlaunchpad/AGENTS.md§3 rather than a patch. I have not proposed a change.changesanddead-token-guardare unconditional; every other job can legitimately skip, and a check required on a skippable job sits pending forever. This bears directly on prd: the cohort test suite — what we test that upstream doesn't #290 criterion 5 and on task: make the ADR-0005 boundary check a required status check #153, but the choice is a human's.