[Fix] Count withdrawn assets and drop prereleases from the downloads badge

[juanmaguitar]

Why

The README's downloads badge read shields' live GitHub total, and that number is wrong in both directions.

It undercounts, because a download counter belongs to the asset and dies with it. Four macOS .dmg files were deleted and re-uploaded when the signing key was rotated, so 89 downloads that really happened are gone from the API. They survive only in the weekly snapshots on metrics, which is the whole reason those snapshots exist.

It overcounts, because it includes release candidates and betas. A download of rc.1 two months after 1.0.0 shipped is not somebody adopting the app.

And badge.json was already being written every week with nothing reading it, despite the workflow's own comment claiming the README consumed data we control. It did not.

What changes

The badge is now computed from the full snapshot history instead of from a live query, by scripts/download-total.cjs, and the README points at badge.json through a shields endpoint badge.

Two decisions worth stating outright:

The snapshot records asset_id. The first version of this inferred "the file was replaced" from "the counter went down". That inference is one-directional and the snapshots are weekly: if a replacement passes the old count before the next Monday, the fall is never observed and the whole history is lost, which is exactly the case the script exists for. GitHub's asset id is unique per upload and was already in the JSON the workflow parses, so it is now a recorded fact rather than a guess. Rows taken before the column existed keep four fields and fall back to the asset name; the week an asset first gains an id its running count is handed over, or every asset alive that week would be counted twice, permanently.

A counter that falls now fails the run. GitHub cannot produce one, so it is bad data, and a badge that is quietly wrong is worse than a workflow that goes red. Same reasoning for an unreadable CSV row: refused, not skipped.

Stable tags are selected by the absence of a - in the tag rather than by GitHub's prerelease flag, which is set on v0.1.1 by mistake.

Deliberately not in this PR: the platform split (.dmg/.exe/everything else) is a catch-all that only feeds the job log, and isStableTag would silently exclude a tag that does not follow the v-semver convention. Both are noted as follow-ups below.

Against the snapshots as they stand the badge reads 220 rather than 149. Once this week's run adds today's counts it is 240 against the live total's 180 (macOS 106, Windows 90, Linux 44).

How to test this

Platforms: any. This is a build-time script and a workflow; nothing runs inside the app.

Starting state:

1. On this branch, with the repo's dependencies installed.
2. npm test and npm run lint both clean.

Then:

# The current four-column snapshot
git fetch origin metrics:metrics
git show metrics:downloads.csv > /tmp/downloads.csv
node scripts/download-total.cjs /tmp/downloads.csv    # 220, with the platform split on stderr

To watch the id handover, which is the part with no second chance: append today's counts with ids, as the workflow will, and confirm the total moves by the week's gains rather than doubling.

{ echo 'date,tag,asset,downloads,asset_id'; tail -n +2 /tmp/downloads.csv; } > /tmp/next.csv
gh api --paginate repos/WordPress/contributor-toolkit/releases \
  | jq -r '.[] | .tag_name as $t | .assets[] | [$t,.name,.download_count,.id] | @csv' \
  | sed "s/^/$(date -u +%F),/" >> /tmp/next.csv
node scripts/download-total.cjs /tmp/next.csv         # 240, not 460

Then run Download stats by hand from the Actions tab and check metrics: downloads.csv has a five-column header, today's rows carry an id, older rows still have four fields, and badge.json holds the adjusted total. The README badge follows a few minutes later, once shields' cache expires.

What must not have happened:

• The totals must not double. A five-column row and a four-column row for the same asset are the same counter; if the handover regressed, the badge roughly doubles in one week and nothing else looks wrong. Covered by downloadTotals hands a running count over when an asset first gains an id, which fails without it.
• The workflow must not commit a number on a bad read. readSnapshots throws on an unreadable row and downloadTotals throws on a falling counter, both under set -euo pipefail, so the job fails before git commit.
• Running the workflow twice on the same day must not double that day's rows. The existing grep -v "^$DATE," handles it; verified by re-running the append locally.

Risks and limitations

Review outcome: 5 [fix here] · 2 [follow-up], all 5 fixed, both follow-ups deferred with reasons below.

• The badge moves once a week, not live. That is the price of being able to adjust it at all. The number is stale by up to seven days by design.
• The 89 withdrawn downloads are frozen. Nobody can download those files any more, so macOS is slightly understated going forward while Windows and Linux keep accruing on old tags.
• Everything before 2026-07-31 is gone, and this change cannot recover it. GitHub never stored it.
• The workflow change is only provable by running it. The shell was read line by line against set -euo pipefail, including the orphan-branch first-run path, but CI does not exercise this job on a PR.

Related

Follow-up to the metrics branch and the weekly snapshot job it feeds. No issue.

---

Design decisions and alternatives considered

Keeping the live shields badge and accepting the wrong number. Rejected: the project has no other usage signal, so the one number it publishes should be defensible. It is also the number that goes into talks and P2 posts.

Inferring a re-upload from a falling counter, with no schema change. This is what the first version of this PR did, and the review killed it. Two failure modes, both silent and permanent: a replacement that overtakes the old count between two Mondays loses the whole history, and a single spurious low reading adds an asset's history a second time, forever, because the badge is recomputed from the full CSV every week. Worth recording that the heuristic had never once fired on the real data: it was dead code standing in for a case it did not actually handle.

Backfilling ids onto the existing rows. Not possible. Those snapshots were taken without the id, and the ids of the deleted assets no longer exist anywhere. The name fallback is as much as those rows can say.

Hardcoding the 89 as a constant. Simpler to write and wrong the next time an asset is withdrawn. The rule in the script covers every future case with no maintenance.

Review outcome (required — see AGENTS.md)

5 [fix here] · 2 [follow-up], all 5 fixed, 2 deferred.

Fixed:

1. Architecture 🟡: the falling-counter heuristic, replaced by asset_id (see above).
2. Tests 🟡: the assetKey test passed for every plausible implementation, bare concatenation included, so it proved nothing. Rewritten with pairs that actually separate them, plus the missing gap-then-return and handover cases.
3. Architecture 🟡: "98 downloads" was the figure across all tags; the withdrawn stable .dmg files are 89. Corrected in the script, STATS.md and the commit message.
4. Architecture 🔵: the stated rationale for excluding prereleases ("they grow at the rate the stable ones do, which looks like a crawler") is contradicted by the committed CSV: prereleases have been frozen at 18 since 2026-08-24. Claim withdrawn; the exclusion stands on its own terms.
5. Architecture 🟡: the reported 240 vs 180 does not reproduce against the snapshots, which give 220 vs 149. Both pairs are correct on their own date; the PR and commit now say both.

Deferred:

6. Architecture 🔵: platformOf returns Linux for anything that is not .dmg or .exe, so a .zip or a latest.yml would land there. It only feeds the stderr breakdown in the job log today, and the release matrix is stable; worth revisiting if a non-installer asset is ever published.
7. Architecture 🔵: isStableTag silently excludes a tag that does not follow v-semver (v1.0.0+build, toolkit-1.1.0). Anchoring on a semver regex and failing loudly on an unparseable tag is the right shape, but the repo has never tagged that way and the change is speculative until it does.

Deterministic layer: npm run lint clean, npm test 1261/1261.

• Review: completed, fresh agent context (judgement pass per .github/instructions/code-review.instructions.md, run before the PR existed); reviewed head 6b989cb / base dd8bc22; five dimensions, with the arithmetic checked against metrics@066ad92.
• CodeRabbit: not run, review limit reached (free OSS allowance exhausted; its check reports success anyway, which is why this is recorded rather than read as zero findings). Run IDs 64b65af7-568d-4b50-92d2-208988ec7a84 (head 2f2a7f5) and 5ff7af03-6e1d-421a-a28b-e604f08dde20 (head 539438f, after the rebase), both rate limited, no inline comments posted. The fresh-agent pass above covers the same revision against the same standard.
• Since review: 6b989cb → 2f2a7f5: every finding above addressed in that range. Then rebased onto trunk at ac31720 ([Fix] Stop a build from spawning processes without bound (#275) #395), giving head 539438f; that commit touches src/ only and this one touches scripts/, the workflow and the docs, so the diffs are disjoint and the reviewed change is unaltered. Re-verified after the rebase: lint clean, 1261 tests, and the script gives 220 on the committed snapshot and 240 with today's counts appended, stable across a further week with no new downloads.

Implementation notes

scripts/download-total.cjs accumulates per (date, asset) over the whole CSV: first sighting counts whole, a rise counts the gain, an absence keeps what the asset earned, a fall throws. The key is id:<asset id> when the row has one and JSON.stringify([tag, asset]) when it does not, which is also why the key is JSON rather than a joined string. No separator character is safe in a filename.

The workflow stages the script into $RUNNER_TEMP before git checkout metrics, because metrics is an orphan branch holding nothing but the two data files. If the script is ever moved, that step fails the job rather than skipping silently.

The CSV header is migrated in place on the first run after this merges: the header line is rewritten to five columns and the existing rows are left at four.

npm test count went 1258 → 1261 (the new file's 16 tests replace nothing; the delta is against this branch's own earlier revision).

---

🤖 Generated with Claude Code

https://claude.ai/code/session_0126Pv8GFR6dszndiaG58D4G

[coderabbitai]

Warning

Review limit reached

Next included review available in 10 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 5ff7af03-6e1d-421a-a28b-e604f08dde20

📥 Commits

Reviewing files that changed from the base of the PR and between ac31720 and 539438f.

📒 Files selected for processing (5)

• .github/workflows/download-stats.yml
• README.md
• STATS.md
• scripts/download-total.cjs
• tests/unit/download-total.test.cjs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[juanmaguitar]

Correcting the record on one claim in this PR.

The description and the merged code both say GitHub's prerelease flag is "set on v0.1.1 by mistake", and that v0.1.1 is a real release with real users. That is wrong. The tag is titled v0.1.1 draft, its body describes an experimental release, and it was never published as a release: this project has shipped four, v0.1.0, v0.1.2, v1.0.0 and v1.0.1.

The rule the sentence was defending is still right, for a better reason. All three assets under that tag are the v0.1.0 binaries (dist_WordPress.Contributor.Toolkit.Setup.0.1.0.exe, WordPress.Contributor.Toolkit-0.1.0.AppImage, and the withdrawn WordPress.Contributor.Toolkit-0.1.0-arm64.dmg). Their 42 downloads are downloads of the app as it then stood, so they belong in the total and they belong to v0.1.0. Filtering on GitHub's flag would silently drop them, which is exactly why the flag is not used.

No number moves: the badge still reads 240. The follow-up corrects the comment in scripts/download-total.cjs, the test name that asserted the same falsehood, and the bullet in STATS.md, and adds the per-release breakdown so that "four releases" beside a CSV listing five tags stops being confusing.