docs(research): 19 of the 796 upstream files in the current drop can reach the deployed relay (#355) - #374
Conversation
serina-mcfall
left a comment
There was a problem hiding this comment.
Independent review — no blockers. Your numbers are right, and I want to be specific about that.
Reviewed in a fresh context. I am an agent and do not approve — posting as a comment so @serina-mcfall can.
An earlier reviewer reported the headline "796" as refuted. That was wrong, and I want it on the record here rather than left hanging. They re-ran your command a day later, got 912 files (and 635/134/83 for desktop/mobile/benchmarks), and concluded the figure did not reproduce.
Reconstructed at the 67-commit point you actually measured:
$ c67=$(git rev-list f8692fa9b..upstream/main | tail -67 | head -1)
$ git log -1 --format='%h %ci' $c67
9891e64f6 2026-08-21 07:07:36 -0400
$ git diff --name-only f8692fa9b $c67 | wc -l
796
$ git diff --name-only f8692fa9b $c67 | awk -F/ '{print $1}' | sort | uniq -c | sort -rn | head -4
575 desktop
110 mobile
52 benchmarks
35 crates
796, 575, 110, 52, 35 — every figure reproduces exactly. The 912 is what the same command returns 13 commits later; upstream moved, the document did not move with it, and that is not an error.
Also verified independently:
- The Dockerfile builds
buzz-relay,buzz-admin,buzz-pair-relay, copiesweb+admin-web, and pulls in no desktop or mobile — quoted output matches. - 30 workspace crates; merge-base
f8692fa9b52ddcfeb4b95fb4862109983509f131dated 2026-08-17. - All 19 named "deployed" files and all 24 named "build/gate" files are individually present in the diff.
- The per-crate touch table (buzz-acp 7, buzz-cli 6, …) reproduces exactly, same 11 crates, same counts, summing to 35.
The predicate for "can reach the deployed relay" is mechanical and falsifiable — the cargo tree closure of the three binaries the Dockerfile builds, plus web/, admin-web/, migrations/ and build inputs — and every one of the 19 checked out.
Non-blocker, filed as #384: the verification command pins the base SHA but leaves the tip as the floating upstream/main, and no tip SHA is recorded. That is precisely what let a competent reviewer mis-refute a correct document, so it has a demonstrated cost rather than a hypothetical one. One added line — TIP=$(git rev-parse upstream/main), printed alongside the base — makes the range fully pinned.
Not verified: I did not run cargo tree myself (no toolchain resolved here), so the 16-of-30 closure computation is unchecked; I verified its documented inputs and that its outputs appear in the diff.
Correct placement, no private-tooling paths. Ready for approval.
benmitchell11
left a comment
There was a problem hiding this comment.
Reviewed by re-checking the Dockerfile/ARCHITECTURE.md citations against the stated source commit, and by re-doing the arithmetic on the document's own published file lists.
Verified directly, exact matches:
Dockerfile:70-72,119,145-146,169(thecargo build,pnpm build, andCOPY --from=lines) all match verbatim against678008ea4.launchpad/ARCHITECTURE.md:99quotes the agent-execution-tree row verbatim.- The published 19-item "can reach the deployed relay" list and the 24-item "build/gate/ops" list both literally contain the number of items claimed (19 and 24 respectively, counted by hand).
Found a real internal inconsistency in the core classification table — this is the document's central deliverable, so it matters:
The four-tier table (Deployed 19 / Cohort tooling 17 / Build-and-gate 24 / Inert 736) sums to exactly 796, but the tiers are not actually disjoint. Cargo.lock, migrations/0032_channel_roster_snapshot_fence.sql, and web/package.json all appear verbatim in both the 19-item "Deployed" list and the 24-item "Build and gate" list (check the two lists in the PR body — all three filenames are in each). That's 3 files double-counted across two tiers, not partitioned as the "Files in the 796" column heading implies.
Working the numbers independently confirms this isn't just a labeling slip: the doc's own "796 classified" breakdown gives 14 non-deployed crate names, of which only 4 (buzz-acp/cli/agent/dev-mcp, totaling 17 touched files) are pulled into "Cohort tooling," leaving 2 leftover files (buzz-test-client, buzz-backend-kubernetes, per the doc's own per-crate touch-count table) unaccounted for elsewhere. The mathematically correct Inert total is therefore desktop(575) + mobile(110) + benchmarks(52) + leftover-crates(2) = 739, not 736 — and the gap between 739 and the stated 736 is exactly the 3 files double-counted into both Deployed and Build-and-gate. The Inert row's own descriptive text ("non-deployed crates 19") doesn't match its own stated total either (575+110+52+19 = 756 ≠ 736), which is the same underlying bug surfacing a second way.
Net effect: the table's "796" only balances because Inert was implicitly under-counted by 3 to absorb the double-count elsewhere, but the document presents it as a clean partition and a reader taking the table at face value would risk double-tallying Cargo.lock/the migration/web's package.json as both "deployed-relay-reaching" and merely "build tooling" risk, or miscount the true inert share.
Separately, a stale citation: the document cites "the cohort's own use of it as OPEN at #43" as evidence the cohort-tooling tier is "real, not hypothetical." Issue #43 was closed on 2026-08-21T02:39 UTC — roughly 18 hours before this PR's commit (2026-08-21T20:17 UTC / Aug 22 08:17 NZT). It was already closed when the document was written, not open. This doesn't invalidate the underlying point (upstream did touch those 4 crates 17 times), but the citation as written is factually wrong at time of authorship, not just stale since.
Could not verify: the underlying 796-file count and its classification against live data — upstream has moved (912 files now vs. 796 then) since this is a moving target, so I could not reproduce the base classification against current upstream/main; I instead checked the internal arithmetic of the numbers as published, which is independent of that drift. The "no VPS access, so this describes what's built, not what's deployed" caveat is honestly stated, as is the desktop-app inference being called "the weakest claim."
Requesting changes: the core classification table has a real double-counting bug (3 files counted in two tiers, with Inert silently adjusted to compensate) and cites issue #43 as open when it was already closed at time of writing. The underlying qualitative conclusion (a small, identifiable slice of the drop reaches the deployed relay, and one file is a live-risk migration) is probably still directionally right, but the specific tier numbers as published shouldn't be treated as a clean, non-overlapping partition without a fix.
serina-mcfall
left a comment
There was a problem hiding this comment.
Approved. Independent review found no blockers; non-blocking findings are filed as follow-up issues.
…reach the deployed relay (#355) Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
11a639b to
8aae852
Compare
Updated for the handbook source-material contractForce-pushed an amended commit bringing this document into line with two conventions introduced after it was written, since it is unmerged and retrofitting after merge is the expensive case: 1. Every reference pinned to a full 40-character SHA. Fork-side claims cite Two judgement calls made while doing it, flagged so a reviewer can overrule:
2. Recommendations separated from evidence and attributed. Each No finding, figure or caveat changed. The diff is pins, section labels, and one external link moved off AI agent (Claude Opus 5) on behalf of @tucktuck101, 2026-08-22. |
Requested change NOT yet done — worth a look soonHead moved to Three files counted in two tiers, under a column headed "Files in the 796" — so the tiers read as a partition when they aren't. His arithmetic on the leftover crate files stands too. To be explicit, because it was disputed: your headline numbers are correct, and I said so on this PR earlier. An earlier reviewer reported 796 as refuted after measuring at the live tip; reconstructing your 67-commit point gives 796 exactly, with 575/110/52/35 for every sub-total. That is not what needs fixing. What needs fixing is only the tier overlap — either mark the three files as counted once and note the overlap, or state plainly that the tiers are not a partition. The new commit strengthened the migration-risk framing, which is a genuinely good addition — The non-blocking tip-pinning item is filed as #384 and needs nothing here. Flagging for visibility rather than pressure — small edit, and it's the last thing between this and mergeable. |
) (#430) Reviewing a batch of PRs has two halves. One is judgement -- is this claim true, does the conclusion depend on this defect. The other is bookkeeping applied identically to every PR. This extracts the second half, which is the line ADR-0019 draws and the same extraction pr_body_check.py and adr_boundary_check.py already did for their own rules. Six classifiers, each one a rule applied by hand across three review batches on 2026-08-21/22, and each one applied WRONGLY at least once: - STALE/MISFILED reviews. Four PRs carried change-requests already satisfied. #262's blockers were fixed at 03:21 and the review restating them arrived at 03:57. #271's change-request was #275's review MISFILED -- textually identical including a "same as #271's" self-reference. No change to #271 could have addressed it. - CI triage. #268's red CI was setup-mold timing out on a one-markdown-file PR. #288's log printed four inherited warnings above the real blocker. - Independence. #265 carried a commit written in the reviewing session. - Leak scan. #281 quoted a private hook's header in a public file. - Placement, per AGENTS.md section 3. - Drift calibration. #374's "796 files" was reported REFUTED by a reviewer who measured at the live tip and got 912. The script emits the pinned SHA so a count that does not reproduce reads as drift, not error. It emits no severity. Five proposed blockers were demoted and one upheld across those batches, each turning on whether a conclusion depended on the defect; a script guessing that is the model-gating ADR-0019 forbids wearing automation's clothes. test_the_briefing_states_no_severity_anywhere asserts the absence. It also posts nothing, so it can run read-only. Three defects found by running it against live PRs rather than by reading it, each now a regression test: 1. FALSE MISFILED on #374. The only path token in a genuine review was `launchpad/ARCHITECTURE.md`, cited as corroborating evidence rather than as a defect site. Reviews cite files outside the diff constantly -- that is what checking a claim looks like. MISFILED now needs two or more cited paths, none in the diff, and no mention of any changed file. 2. Selecting log lines BY POSITION. The first draft took the last 80 lines; GitHub appends checkout teardown, so on #288 the size-guard line had scrolled past and a REAL failure classified as UNKNOWN. Selection is by content now. 3. The path regex required `:\d+` with no space, so it saw compiler output (`lib.rs:276:15`) but not the file-size guard (`lib.rs: 1000 -> 1001`). Verified against live PRs after the fixes: #288 classifies Desktop Core REAL on desktop/src-tauri/src/lib.rs and Desktop Smoke E2E PRE_EXISTING, matching the hand analysis; #374's calibration returns 796 files at the 67-commit point with tip 9891e64, matching the figure reconstructed by hand. 41 tests in test_pr_review_batch, 256 across launchpad/scripts. Registered in test_no_model.py's ALLOWLIST rather than NOT_OURS, deliberately: a script that prepares review material must be provably unable to call a model. DEVIATION from the issue's own done-when: it asked for registration in INTERFACE.md. Not done -- INTERFACE.md is #116's pre-flight record contract, not a script index, and an unrelated entry there would degrade a contract document. The script's module docstring is its interface. Refs #426 Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
…tus (#355) The four tiers double-counted Cargo.lock, the 0032 migration, and web/package.json in both Deployed and Build-and-gate, silently absorbing the overlap into Inert. Each file is now counted once: Deployed 19, Cohort tooling 17, Build-and-gate 21 (24 paths listed, three tallied under Deployed), Inert 739 (575+110+52+2), and the headline 737 — a third inconsistent figure — reads 739. #43 is cited as closed 2026-08-21, its state at this document's authorship, not OPEN. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
|
Re-review requested — pushed aa6c532. Disposition per finding (@benmitchell11's review, with @serina-mcfall's follow-up): Tier double-counting — fixed, as a true partition. Two independent verifiers re-enumerated both lists and confirmed your arithmetic exactly: the three files ( #43 cited as OPEN — fixed. Verified directly: #43 closed 2026-08-21T02:39Z, about 20 hours before this document's head commit. The citation now reads "recorded at #43 (closed 2026-08-21, before this document's commit)", keeping the verified 17-touches evidence (7+6+2+2) intact — the point survives, arguably stronger, since the adoption decision is settled rather than pending. Your framing was right on both: the headline numbers were never the problem (serina's reconstruction at the 67-commit point stands), and the qualitative conclusion was directionally sound — the defect was presenting overlapping tiers as a clean partition. Thanks for working the arithmetic all the way down to the two leftover crate files; that made the fix mechanical. |
|
Both blockers from @benmitchell11's review are still present at The double-count is confirmedI took the two published lists straight out of the document — the 19-item "can reach the deployed relay" list and the 24-item "build/gate/ops" list — and intersected them: Exactly the three files named in the review, and each list is the length it claims (19 and 24). So this is settled, and it does not need the The clean partition is:
Worth saying explicitly: the evidence block is already correct. Four call-sites, not twoThe stated total appears in three places and they don't agree with each other: line 18 — currently -**19 of the 796 files in the current drop can reach the deployed relay. 737 cannot reach anything this fork runs.**
+**19 of the 796 files in the current drop can reach the deployed relay. 739 cannot reach anything this fork runs.**line 26 — the -| **Build and gate** | `Justfile`, `lefthook.yml`, `bin/.lefthookrc`, `scripts/*`, `.github/workflows/*`, `schema/`, `renovate.json` | **24** |
+| **Build and gate** | `Justfile`, `lefthook.yml`, `bin/.lefthookrc`, `scripts/*`, `.github/workflows/*`, `schema/`, `renovate.json` — excluding `Cargo.lock`, `migrations/0032_…`, `web/package.json`, which are counted under Deployed | **21** |line 27 — the -| **Inert for this fork** | `desktop/` 575, `mobile/` 110, `benchmarks/` 52, non-deployed crates 19 | **736** |
+| **Inert for this fork** | `desktop/` 575, `mobile/` 110, `benchmarks/` 52, non-deployed crates 2 | **739** |The line 31 — the prose restating it: -The better number is that 736 of the 796 cannot affect anything the cohort operates *at all*, contested or not.
+The better number is that 739 of the 796 cannot affect anything the cohort operates *at all*, contested or not.The #43 citation (line 148)Issue #43 closed The substance survives: the successor issue is still open, so the cohort's use of the agent-execution tree genuinely is unresolved. Only the pointer is dead. -and the cohort's own use of it as `OPEN` at [#43](https://github.com/launchpad-26/buzz/issues/43). Upstream touched those four crates 17 times in this drop:
+and the cohort's own use of it as unresolved — #43 was closed `NOT_PLANNED` on 2026-08-21 and re-filed outside this repository, where the decision is now tracked. Upstream touched those four crates 17 times in this drop:I have deliberately not named the successor issue. It lives in a private repository and this one is public, so the number and link don't belong in a document here. Describing it as tracked elsewhere keeps the claim honest without exporting a private reference. What I did not checkI did not reproduce the underlying 796-file classification against live data — same limitation the review states, and upstream has moved since. I verified the internal arithmetic of the published numbers, the two lists' lengths and their intersection, the state and closure reason of #43, and the state of its successor. The qualitative conclusion — a narrow identifiable slice reaches the deployed relay, and one of those files is a migration applied on startup — is untouched by any of this. |
|
This comment crossed the fix in flight — it checks |
serina-mcfall
left a comment
There was a problem hiding this comment.
Re-review at aa6c53217 — both of @benmitchell11's findings are closed. His review is deliberately not being dismissed.
Agent-authored under @serina-mcfall's instruction. Per launchpad/AGENTS.md:165 this approves nothing.
@benmitchell11's change-request dates from 2026-08-21T21:09 and the head has moved to aa6c53217; the pre-pass classifies it STALE. This re-check exists so the finding is settled on the record — not to clear it. Ben is out sick, and @serina-mcfall's decision is to hold his review until he is back rather than dismiss it on his behalf. That is the right call: it was his finding, the fix is substantive rather than cosmetic, and he should get to see it.
Finding 1 — the four tiers were not disjoint; three files were counted twice and Inert was silently under-counted to compensate. Fixed, and fixed at the arithmetic rather than the label.
The table now reads:
| Tier | What it is | Files in the 796 (each file counted once) |
| **Build and gate** | …, less the three files already counted as Deployed | **21** |
| **Inert for this fork** | desktop/ 575, mobile/ 110, benchmarks/ 52, remaining non-deployed crate files 2 | **739** |
and states the overlap explicitly rather than leaving it to be inferred:
The build/gate/ops role is played by 24 paths in full (listed below); three of them —
Cargo.lock,migrations/0032_channel_roster_snapshot_fence.sql, andweb/package.json— are also deploy-reaching and are tallied under Deployed rather than counted twice: 19 + 17 + 21 + 739 = 796.
19 + 17 + 21 + 739 = 796. Checked.
Two things worth Ben seeing specifically. Inert is now 739, which is the number he derived independently — 575 + 110 + 52 + 2, including the two leftover crate files (buzz-test-client, buzz-backend-kubernetes) his own arithmetic surfaced. His "the mathematically correct Inert total is therefore 739, not 736" is what landed. And the full 24-path build/gate list is still printed, so nothing was hidden to make the sum work — the three files appear in both lists while being counted once in the tally, which is exactly the "mark them as counted once and note the overlap" option rather than the quieter one.
Finding 2 — #43 cited as evidence the cohort-tooling tier is real, when it was already closed at time of writing. Fixed, and more honestly than by deletion.
:153 and the cohort's own adoption of it recorded at [#43](…/issues/43)
:154 (closed 2026-08-21, before this document's commit). Upstream touched those four crates 17 times in this drop:
The closure and its timing relative to the commit are now both stated. Removing "OPEN" would have satisfied the finding; disclosing that the citation is to a closed issue, and when it closed, is the version a later reader can actually assess.
Not in question, and worth restating since it was disputed earlier in this PR's history: the headline 796 and the 575/110/52/35 sub-totals were confirmed correct by reconstructing the 67-commit point the document measured. An earlier reviewer reported 796 as refuted after measuring at the live tip; that was a calibration error, not a defect in the note.
No blocker. Nothing new found.
Re-reviewed at aa6c53217. The tier arithmetic and the #43 wording were checked by me at that head. I did not re-derive the 796 classification against upstream/main — upstream has moved, and that figure was already settled by pinning the document's own measurement point.
@benmitchell11 — nothing here needs action from you while you are unwell. Your change-request stays as it is until you are back to look at it.
🤖 Claude Code (claude-opus-5) for @serina-mcfall.
Summary
Adds one research document establishing this fork's operational surface from the relay image build, and classifying the 796 upstream files in the current backlog against it. Only 19 can reach the deployed relay; 575 are desktop, 110 mobile, 52 benchmarks. The finding cuts both ways for ADR-0022 — the affordability argument is far stronger than 8-of-796 suggests, but one of the 19 live files is a new database migration that the relay applies on startup, adopted unreviewed under the current scope ruling.
Related issue
Closes #355
Issue type
Task
Agent provenance
Objective
Add
launchpad/Research/355-what-the-fork-actually-operates.mdrecording which upstream paths can affect what the cohort operates, and how the current drop's 796 files divide against that boundary.Impacted components
Approach and rejected alternatives
Derived the boundary from the build rather than from documentation: read which packages the
Dockerfilecompiles and which bundles it copies, then took thecargo treeclosure of those package names and intersected it with the workspace member list. That gives a surface defined by what actually ships, not by what anyone believes ships.Rejected: classifying by directory intuition (
crates/= live,desktop/= not). It gets the headline right and the details wrong — 14 of the 30 workspace crates are not in the image, includingbuzz-cli, so "crates are live" would have overstated the deployed surface by nearly half.Rejected: a single live/inert split.
buzz-cliand thebuzz-acp→buzz-agent→buzz-dev-mcptree are not in the deployed image but are operational for this fork perlaunchpad/ARCHITECTURE.md:99and #43. Forcing them to one side would either inflate the deployed surface or hide tooling the cohort depends on, so the document reports three tiers and says which is which.Rejected: judging whether the new migration is dangerous. I recorded that a migration is present in the live 19 and that migrations apply on relay startup. Whether
0032_channel_roster_snapshot_fence.sqlis risky is a review of upstream's SQL, which is not this issue's question and would have been an opinion dressed as a finding.Verification
Command run:
Raw output:
Total workspace crates: 30. In closure: 16.
Not verified
I did not build the image — the closure is
cargo tree's answer, not an observed artifact, and I used--edges normal,build, which deliberately excludes dev-dependencies, so a crate reachable only through a test edge would not appear. I did not readmigrations/0032_channel_roster_snapshot_fence.sqlor assess it. I did not independently confirm againstbuzz-db's startup path that migrations apply automatically on startup; that claim comes from the rootAGENTS.md. I have no access to the deployed VPS, so I could not confirm what is running there, which image tag it carries, or whether it corresponds to this branch — the document describes what the repository builds, not what is deployed. I did not establish whether any contributor runs the desktop or mobile app;launchpad/ENVIRONMENTS.mdlists four environments and neither app is among them, and the relay serves a browserweb/client, so I inferred the desktop app is outside the cohort's operational path. That inference is the weakest claim in the document and is flagged as such in it. I did not runjust ci; the diff is one markdown file.Security implications
None from the diff. The document surfaces one security-relevant fact that already exists:
migrations/0032_channel_roster_snapshot_fence.sqlarrives clean in the current drop, is in no ledger, and would be applied to the live relay's database on the next startup after adoption without anyone having read it. That is a property of the current scope ruling, not something this PR changes, and it is recorded so it can be weighed rather than discovered.Escalations
The migration in the live 19 deserves a decision, not a mention. ADR-0022's stated counter-example is
bin/.lefthookrc, a developer-experience regression. A schema migration applied on startup against a live database is the same structural class with a much larger blast radius. I have not proposed changing the scope ruling — that record is accepted and it is not mine to reopen — but "clean-merging migrations are adopted unreviewed" should be an explicit position rather than an implication.The desktop-app question needs a person, not more archaeology. One sentence from a contributor — "nobody here runs the desktop app" or "we all do" — moves 575 files between tiers. I inferred the answer and flagged the inference rather than presenting it as established.