docs(research): the full CI matrix has passed twice, both on upstream syncs, at zero billable cost (#353)

[tucktuck101]

Summary

Adds one research document establishing that this fork's full 23-job CI matrix has passed green twice, both times on an upstream sync branch — including PR #216, a 113-commit / 981-file drop — at zero billable minutes because the repository is public. Records the per-job durations, the lanes that fail unreliably, and the fact that the fork has already taken two upstream drops. That last point corrects the impression left by #273's Evidence section, and makes PR #216's body a worked precedent for the drop report #306 is trying to specify.

Related issue

Closes #353

Issue type

Task

---

Agent provenance

Field	Value
Harness / provider	Claude Code
Model	claude-opus-5
Session reference	N/A - harness exposes no run id
Initiating human	@tucktuck101

Objective

Add launchpad/Research/353-full-ci-matrix-and-drop-cost.md recording whether the full CI matrix has ever run green in this fork and what one run costs.

Impacted components

launchpad/Research/353-full-ci-matrix-and-drop-cost.md

Approach and rejected alternatives

Pulled all 166 ci.yml runs from the Actions API, ranked them by duration to find candidates that could plausibly have exercised the whole matrix, then read the per-job conclusions of the top candidates to find runs with zero skipped jobs. Billable time came from the /timing endpoint rather than being inferred from durations.

Rejected: dispatching a fresh full-matrix run to measure cost directly. Two completed runs already answer it, dispatching would consume a runner fan-out for no new information, and I have no mandate to push branches that trigger CI. Rejected: reading ci.yml's changed-paths filter to reason about which jobs a drop would trigger. That predicts; the two sync runs observe, and observation of the actual case beats a prediction about it.

I did not average the two runs. PR #216 is the larger and more representative drop, so its figures are the ones reported, with PR #12 cited as corroboration that the first was not a fluke.

Verification

Command run:

gh api repos/launchpad-26/buzz --jq '{visibility,private,fork,parent:.parent.full_name}'

gh api "repos/launchpad-26/buzz/actions/runs/32095795174/jobs?per_page=100" \
  --jq '.jobs[] | [.conclusion,.name] | @tsv' | awk -F'\t' '{c[$1]++} END {for(k in c) print k, c[k]}'

gh api "repos/launchpad-26/buzz/actions/runs/31358601797/jobs?per_page=100" \
  --jq '.jobs[] | [.conclusion,.name] | @tsv' | awk -F'\t' '{c[$1]++} END {for(k in c) print k, c[k]}'

gh api "repos/launchpad-26/buzz/actions/runs/32095795174/timing"

gh api "orgs/launchpad-26/settings/billing/actions"

Raw output:

$ gh api repos/launchpad-26/buzz --jq '{visibility,private,fork,parent:.parent.full_name}'
{"fork":true,"parent":"block/buzz","private":false,"visibility":"public"}

$ ... runs/32095795174/jobs ...
success 23

$ ... runs/31358601797/jobs ...
success 23

$ ... runs/32095795174/timing
{"billable":{"UBUNTU":{"total_ms":0,"jobs":42,...},"WINDOWS":{"total_ms":0,"jobs":2,...},
             "MACOS":{"total_ms":0,"jobs":2,...}},"run_duration_ms":1416000}

$ gh api "orgs/launchpad-26/settings/billing/actions"
{"message":"Not Found","documentation_url":"...","status":"404"}
gh: This API operation needs the "admin:org" scope.

The timing output is elided at the job_runs arrays only — 46 entries, every one "duration_ms":0. Full per-job duration table is in the document.

• Tests or checks were run and the raw output is pasted above
• The diff is confined to the scope of the linked issue
• No secrets, keys, tokens or hostnames were added to tracked files

Not verified

The Actions minute allowance and current consumption are not established — the org billing endpoint needs admin:org, which this token lacks, and the 404 is pasted above. I judge it moot while billable time is zero on every runner class, and say so in the document; it stops being moot if the repository is ever made private. I did not check whether the Windows and macOS lanes use standard or larger runners (I read only labels[0]). I did not check whether the 39 active caches totalling 8.7 GB are near the 10 GB per-repository limit, which would slow future runs. I did not read the failure logs of the 22 failing runs — the failure table counts job names only, so I cannot say whether those desktop and E2E failures are flakes or real defects. I dispatched nothing; every figure is from runs that already completed. I did not run just ci; the diff is one markdown file and I make no claim about the full gate.

Security implications

None from the diff — one markdown document, no code, config or workflow change. One exposure-adjacent fact is recorded rather than introduced: the repository is public, which is why runners are free, and which is already true.

Escalations

#273's Evidence section should be corrected. It says no upstream commit has been merged into launchpad in the window, and that the only sync automation runs on a contributor's personal fork. Both are literally true and together read as "this fork has never synced". It has synced twice, and merge-base f8692fa9b is the result of PR #216. "Nothing owns this job" and "two people have done it by hand twice" imply different work, so I am flagging it rather than editing the PRD.

This also corrects something I wrote myself. My knowledge-gap sweep comment on #273 listed "the fork will take drops at all — zero drops have been taken" as an untested assumption. That was wrong and the record should show it.

The local/CI disagreement on mobile is worth someone's attention but I have not raised an issue for it: Mobile has never failed in CI, while PR #216 documents four mobile-test failures locally, tracked at #215. Whether the CI lane is not running those tests or is running them in a more forgiving environment, I did not determine.

[serina-mcfall]

Independent review — no blockers, nothing to fix

Reviewed in a fresh context. Every number I could re-derive matched, several to the byte. I am an agent and do not approve — posting as a comment so @serina-mcfall can.

Verified exactly:

• Repo is public and a fork of block/buzz ({"fork":true,"private":false,"visibility":"public"}).
• Run 32095795174 (PR #216): 23/23 jobs success. Run 31358601797 (PR #12): 23/23 success.
• Zero billable minutes across all runner classes — UBUNTU/WINDOWS/MACOS total_ms: 0.
• run_duration_ms 1,416,000 (23.6 min) against 53.55 min wall clock.
• The failure-lane table (Desktop 15, Desktop Core 8, Security 6 …) — reproduced entry for entry by tallying all 22 failing ci.yml runs.
• "Mobile has never failed" — zero Mobile entries in the failure set.
• 39 active caches, 8,710,036,247 bytes — matched to the byte.
• The org billing endpoint 404 with the same scope error the document reports.

Expected drift, not error: the run tally is now 141 success / 22 failure / 15 cancelled out of 178, against the document's 129/22/15 of 166. The failure and cancelled counts match exactly; only successes grew, from 12 runs landing after the snapshot. That is a live metric moving, which is inherent to this document type.

Not verified: #273's Evidence section content (out of scope), PR #216's own claim of 4 pre-existing mobile-test failures, and whether the Windows/macOS runners are standard or larger — the document already flags that last one itself.

No blockers, no non-blocking findings, correct placement, no private-tooling paths.

[benmitchell11]

Reviewed by re-running the cited gh api calls live against this repo, rather than trusting the pasted output.

Verified directly, matching exactly:

• gh api repos/launchpad-26/buzz confirms public fork of block/buzz.
• Run 32095795174: re-pulled the jobs list myself — 23/23 jobs success, job names match the doc's per-job table.
• Run 31358601797: re-pulled — 23/23 success, matching the claim.
• .../32095795174/timing: billable totals are all 0 on UBUNTU/WINDOWS/MACOS, run_duration_ms: 1416000 (23.6 min) — exact match. Cross-checked created_at→updated_at myself and got 53m33s, matching the doc's "53.5 minutes wall-clock" claim.
• Issue #215 and PR #216's body both confirm the mobile-test pre-existing-failure claim exactly as described.
• orgs/launchpad-26/settings/billing/actions does 404 for me too, confirming that limitation is real, not fabricated.

Two numeric discrepancies found, worth a look:

1. The doc states PR #216 is "113 commits / 981 files" and PR #12 is "64 commits, 299 files." Re-checked via gh api repos/.../pulls/216 and .../pulls/12 (the REST endpoint's authoritative commits field, which avoids the pagination cap that gh pr view --json commits silently hits): actual commit counts are 114 and 65 respectively — both off by exactly one from what the doc states. File counts (981, 299) match exactly. Consistent off-by-one on both PRs suggests a systematic counting difference (e.g. whether the merge commit itself is included) rather than a fabrication, but as stated the numbers don't reproduce from the API today.
2. The doc says PR #308 "ran 6 checks and skipped 16." Live check: 16 skipped (matches), but 8 checks succeeded, not 6 — and even restricting to only ci.yml-owned jobs (excluding adr-boundary/check/audit/scripts from the separate launchpad-pr-check.yml workflow) gives 2, not 6, either way not reproducing "6."

Neither issue affects the core, load-bearing claims (both full-matrix runs are genuinely 23/23 green at zero billable cost), but the specific auxiliary numbers above don't reproduce exactly against live data and are worth double-checking before this is relied on as precise precedent.

Scope: appropriately stops short of deciding policy — flags the branch-protection gap and #273/#306 implications as follow-up work rather than making the call itself.

Recommend a quick pass to reconcile the two numeric discrepancies above; not blocking, since the central finding (twice green, zero cost) is solidly verified.

[serina-mcfall]

Approved. Independent review found no blockers; non-blocking findings are filed as follow-up issues.