claude-ops: audit-skill-visibility derives the listing budget from hardcoded defaults instead of the effective settings and model

[kyle-sexton]

Summary

claude-ops:audit-skill-visibility computes the listing budget as 200_000 * 4 * 0.01 = 8,000 characters regardless of the machine it runs on. It never reads skillListingBudgetFraction or skillListingMaxDescChars from settings, defaults the context window to 200k, and hardcodes 4 bytes per token. On a repository that sets the fraction to 0.05 the report claimed a 138,298-character overflow with 191 of 195 skills name-only; the effective budget on a 1M-window session is 200,000 characters and nothing was truncated.

Evidence

Reproduced in this repository (which sets skillListingBudgetFraction: 0.05 in .claude/settings.json) on Claude Code 2.1.263:

Run	Budget chars	Demand chars	Overflow	Starved
Bare invocation (hardcoded 0.01, 200k, 4 b/t)	8,000	146,298	138,298	191
Effective at 200k window	40,000	146,298	106,298	152
Effective at 1M window	200,000	146,298	0	0

Verified against official sources (fetched 2026-09-11) and the 2.1.263 bundle:

• Settings reference: skillListingBudgetFraction (default 0.01) and skillListingMaxDescChars (default 1536) are Any file scope keys. Settings precedence is managed > --settings > local > project > user; environment variables are not a level in that stack.
• Binary: budget = floor((window ?? 200000) * bytesPerToken * fraction), where the window is the active model's and bytesPerToken is 4 or 3 by model. SLASH_COMMAND_TOOL_CHAR_BUDGET returns before the fraction is consulted.
• model-config: Fable 5.1, Fable 5, Sonnet 5 and Opus 4.7+ run a 1M window by default on the Anthropic API; [1m] suffix, CLAUDE_CODE_DISABLE_1M_CONTEXT, and CLAUDE_CODE_MAX_CONTEXT_TOKENS change it.
• enabledPlugins schema text in the binary: "Settings precedence is user < project < local < flag < policy".

Where it goes wrong: ListingConfig defaults (scripts/audit_skill_visibility.py around line 741) and the main wiring that builds it (around line 1524); the argparse --context-window default of 200_000 (around line 1454).

Expected behaviour

1. Read the two settings keys from the merged settings scopes (user, project, local, managed) with per-key precedence user < project < local < policy, and render provenance in budget_basis naming the file that supplied each value. The --settings (flag) scope is unobservable from an out-of-process script; report it as "scope unread", never as absent.
2. Reuse the managed-policy enumeration by vendoring plugins/claude-config/lib/managed-scope.sh into plugins/claude-ops/lib/ through the same sync pattern claude-memory uses. Do not hand-roll managed paths.
3. Never resolve the model from disk. With no pin, report a two-axis band: window {200k, 1M} × bytes-per-token {4, 3}, each row labelled. Honor CLAUDE_CODE_DISABLE_1M_CONTEXT and CLAUDE_CODE_MAX_CONTEXT_TOKENS (the latter only when DISABLE_COMPACT is set) from the process environment with provenance, exactly as SLASH_COMMAND_TOOL_CHAR_BUDGET is honored today.
4. CLI pins collapse the band: add --budget-fraction, --max-desc-chars, and --bytes-per-token beside --context-window; drop the silent default=200_000.
5. Correct the ListingConfig docstring's four-part record (it calls 4 bytes per token "the product's own hardcoded estimate"; the recheck trigger has fired) and advance reference/listing-scorer.md's stamp to 2.1.263 with the two additions its binary re-verification found: two truncators with identical semantics, and per-model bytes-per-token.

Acceptance criteria

• [event-driven] WHEN the project sets skillListingBudgetFraction: 0.05 and the script runs bare, the report's budget row for the 1M/4 axis reads 200,000 characters and budget_basis names .claude/settings.json.
• [unwanted-behaviour] IF SLASH_COMMAND_TOOL_CHAR_BUDGET is set, THEN the fraction is ignored and budget_basis reads env-override, unchanged from today.
• [state-driven] WHILE no --context-window or --bytes-per-token pin is given, the Markdown and JSON carry all four band rows and name no single row as "this session".
• [ubiquitous] A managed-settings location is only ever enumerated through the vendored managed-scope.sh; the sync script reports the copy in sync.
• [ubiquitous] test_audit_skill_visibility.py (BudgetArithmeticTest) covers: settings precedence with a project value overriding a user value, local overriding project, the unread-flag-scope provenance line, the four-row band, and each env var's effect.
• scripts/affected-tests.sh --run is green for the change.

Out of scope

• Which skills lose descriptions when every score is zero (sibling issue: unscored ranking presentation).
• Reachability / enabledPlugins (sibling issue; it consumes this issue's settings reader).

Related

• Sibling issues: linked in the first comment once filed.
• plugins/skill-quality/scripts/check-listing-budget.sh deliberately does not read settings because it audits repository content; this skill audits the live machine, so reading settings is in scope here and not there.
• Official sources: code.claude.com/docs/en/settings (skillListingBudgetFraction, skillListingMaxDescChars, precedence), /docs/en/skills ("Skill descriptions are cut short"), /docs/en/env-vars (SLASH_COMMAND_TOOL_CHAR_BUDGET), /docs/en/model-config (1M default, [1m], env vars).

[kyle-sexton]

Sibling issues from the same audit run: #4077 (unscored ranking presentation) and #4078 (enabledPlugins reachability, which depends on this issue's settings reader). All three were scoped through an agent-validated interview backed by live official docs and the 2.1.263 binary; the research ledgers travel with the fix PR.

---

Generated by Claude Code