Automate the context-budget report on skill PRs, and gate on it

[MajorLift]

Size: M | Files: .github/workflows/, .github/scripts/, tools/skill-schema.mjs

---

Problem

A reviewer cannot read a skills PR. The nine currently open add 68 files, ~90,000 words, ~112k tokens — roughly six hours of sustained reading before any judgement. In practice that means skills merge on the strength of their PR description.

The part a reviewer most needs is also the part hardest to see by reading: what the skill costs. Measured by hand across the 26 new skills:

Tier	When paid	Median	Max
Frontmatter	every agent, every run, whether or not the skill is used	—	12,837 chars total (~3,209 tok)
Selected	agent opens the skill	~1,613 tok	9,059 tok
+ references & knowledge	agent follows the pointers	~3,511 tok	35,011 tok

Frontmatter is the tier that matters and the one nobody sees: it is paid by every agent on every run once installed, including runs that never touch the skill, and it is the only tier that cannot be opted out of after install.

The same measurement found five of 26 skills over the 1,024-character description ceiling that #47 enforces — each of which would have failed CI on merge, discovered by measurement rather than by review.

None of this is visible in a diff, and none of it is currently computed.

---

Solution

1. Post the report automatically. On any PR touching domains/**, have metamaskbot comment a context-budget table: per skill, its frontmatter chars, selected tokens, and ceiling with references followed — plus how those compare to the catalogue median. Idempotent upsert on re-push, so the comment tracks the head rather than accumulating.

Measurement has to run against an actual install (tools/install into a temp target), not source bytes. Source-byte sums are wrong in two ways found while doing this by hand: they miss the repo-overlay merge, and they double-count domain knowledge, which is copied per skill but read once per run.

Use main's installer rather than the PR branch's, or a branch behind main reports its knowledge as undelivered when on merge it ships.

2. Gate on the numbers. Fail CI where a threshold is objective, warn where it is a judgement call:

• Fail — any description over DESCRIPTION_MAX. Already enforced by ci: add lint-skill-entry structural validator #47 for changed skills; this extends the same number into the budget report so the two cannot disagree.
• Warn — a skill whose selected cost exceeds some multiple of the catalogue median. pr-validate was 5.6× before being restructured, which was worth knowing at review time rather than after merge.
• Warn — total always-on growth in a single PR above a threshold, since that is the tier every consumer pays forever.

Thresholds belong in tools/skill-schema.mjs beside DESCRIPTION_MAX, not in the workflow, so they are reviewable and testable like the rest of the schema.

---

Acceptance Criteria

• A PR touching domains/** receives a metamaskbot comment with per-skill frontmatter / selected / ceiling figures
• The comment upserts on re-push rather than posting a second one
• Measurement runs against a real install, using main's installer, not source bytes
• Knowledge copied per skill is counted once per run, not once per copy
• A description over DESCRIPTION_MAX fails CI, with the same number lint-skill-entry uses
• Cost thresholds live in tools/skill-schema.mjs and are covered by tests
• The check runs on changed skills only, so pre-existing catalogue weight never blocks an unrelated PR
• Verified to fire: a skill authored over each threshold fails or warns as specified

---

Notes

Deliberately cost, not value. A 35k-token skill that saves an hour is a bargain, and nothing here says which those are — it gives a reviewer one measured input, not a verdict. Judging whether the guidance is correct still needs a domain expert, or an A/B of agent output with and without the skill, which is a separate and much larger piece of work.

Companion to #47 (structural validation) and #87 (reference resolution): those check that a skill is well-formed and that what it points at exists. This checks what it costs.