claude-ops: audit-skill-visibility emits per-skill starvation verdicts when every score is zero

[kyle-sexton]

Summary

When no competing skill has any recorded usage (every fresh install, every cloud container), the product's stable sort keeps catalog order and the first-fit walk grants the first entries that happen to fit. audit-skill-visibility models this correctly (score_basis: "unscored", per-row confidence: "unscored") but still emits verdict: "likely-starved" and a numeric band on every competing row, and its Markdown says "roughly N of them are running name-only". Readers take that as a usage-derived ranking. It is not: in the reproducing run the "starved" list was simply the catalog from the letter C onward.

Evidence

• Binary (Claude Code 2.1.263): both truncators sort descending by usageCount * max(0.5^(days/7), 0.1) with a stable sort, then walk first-fit with no break. At all-zero scores the ordering is catalog position.
• reference/listing-scorer.md already states this ("When the band means nothing") and the classifier header says "A confidently wrong band is worse than no band", yet compute_listing (around lines 896-937) still writes the verdict and band, and the renderer (around lines 1316-1324) still prints the count-as-ranking sentence.
• SKILL.md's refusal doctrine: "routes any claim the span cannot support into a first-class withheld section with its reason. A declined verdict is reported, never omitted."
• The withheld list built in build_model (around line 1115) only carries observation claims today.

Expected behaviour

1. Keep the arithmetic: demand, budget, overflow, and the count of entries that cannot fit stay in both renders.
2. WHILE score_basis is unscored, no competing row carries verdict: likely-starved or a band; each such row's starvation object reads verdict: withheld with reason: unscored.
3. The per-skill "which ones" claim moves into the withheld section with reason "unscored: ordering is catalog-order tie, not usage", once, not once per skill.
4. The Markdown drops "roughly N of them are running name-only" in the unscored case and prints the cannot-fit count as a count.

Acceptance criteria

• [state-driven] WHILE every competing usage score is 0, the JSON contains zero rows with verdict: likely-starved and one withheld entry with reason unscored.
• [event-driven] WHEN at least one competing skill has non-zero usage, per-row verdicts and bands render exactly as today.
• [ubiquitous] starved_count and overflow_chars are unchanged in both cases.
• [ubiquitous] OverflowConsumptionTest and ListingScoreTest.test_all_zero_scores_report_an_unscored_basis are extended to assert the withheld shape.
• scripts/affected-tests.sh --run is green.

Out of scope

• The budget inputs themselves (settings fraction, window, bytes-per-token): claude-ops: audit-skill-visibility derives the listing budget from hardcoded defaults instead of the effective settings and model #4076.

Related

• Sibling: claude-ops: audit-skill-visibility derives the listing budget from hardcoded defaults instead of the effective settings and model #4076 (settings-derived budget inputs); the enabledPlugins reachability issue is linked in the first comment.
• Binary re-verification note: the tiebreak is catalog order via stable sort; listing-scorer.md says "alphabetical" in one place and "catalog order" in another, and should say the latter in both.

[kyle-sexton]

Sibling issues from the same audit run: #4076 (settings-derived budget inputs) and #4078 (enabledPlugins reachability).

---

Generated by Claude Code