docs: add catalog schema reference

.github/PULL_REQUEST_TEMPLATE.md

@@ -6,6 +6,8 @@
 
 - [ ] The repo URL is public and reachable.
 - [ ] The entry has a specific category.
+- [ ] Required fields, category slug, artifact type, maturity, and labels match the [catalog schema reference](../docs/catalog-schema.md).
+- [ ] Source evidence is captured in the PR body using the worksheet in the [catalog schema reference](../docs/catalog-schema.md#evidence-capture-worksheet).
 - [ ] The `risk_notes` field explains what could go wrong.
 - [ ] The `operator_note` field explains why an infrastructure operator should care.
 - [ ] Labels match the observed behavior, not marketing claims.
@@ -21,11 +23,12 @@
 ## Validation
 
 ```bash
-python scripts/validate_repos_yaml.py
-python scripts/sync_readme_counts.py --check
-python -m pytest -q
-python scripts/run_mock_eval_scenarios.py
-python scripts/audit_github_repos.py --workers 12 --fail-on-unreachable
+python3 scripts/validate_repos_yaml.py
+python3 scripts/sync_readme_counts.py --check
+python3 scripts/sync_catalog_json.py --check
+python3 -m pytest -q
+python3 scripts/run_mock_eval_scenarios.py
+python3 scripts/audit_github_repos.py --workers 12 --fail-on-unreachable
 ```
 
 - [ ] Relevant commands above pass locally, or this PR explains why a command is not applicable.

CHANGELOG.md

@@ -9,6 +9,139 @@ documented here. The format is based on
 
 ### Added
 
+- **CI/CD and release automation guidance.** Added schema-reference guidance and
+  regression coverage for reviewing release agents against pipeline triggers,
+  artifact publication, protected environments, scoped CI tokens, log/artifact
+  exposure, approval records, rollback evidence, and public-safe fixture examples.
+- **PR validation checklist coverage.** Added regression tests for the pull
+  request template's catalog schema links and local validation command block, and
+  updated contributor guidance to require `python3` plus generated-sidecar checks.
+- **Policy-as-code and guardrail evaluation guidance.** Added schema-reference
+  guidance and regression coverage for reviewing policy agents against read-only
+  recommendations, exception approval, admission-controller changes, enforcement
+  paths, artifact hashes, audit evidence, remediation risks, and public-safe
+  synthetic policy examples.
+- **Cost, quota, and FinOps impact guidance.** Added schema-reference guidance
+  and regression coverage for reviewing cost and FinOps agents against billing
+  lookup, budget/quota/resource mutations, optimization tradeoffs, sensitive cost
+  exports, approval evidence, rollback, and public-safe synthetic billing
+  examples.
+- **Secrets and identity operations guidance.** Added schema-reference guidance
+  and regression coverage for reviewing secrets-manager, identity-platform, and
+  access-review agents against secret reveal/rotation/revocation, principal and
+  policy changes, redaction, audit evidence, expiration, rollback, and public-safe
+  synthetic identity examples.
+- **MLOps model operation and evaluation guidance.** Added schema-reference
+  guidance and regression coverage for reviewing MLOps agents against experiment
+  lookup, model training, registry promotion, endpoint deployment, feature-store
+  writes, prompt/eval data exposure, rollback, audit evidence, and public-safe
+  synthetic model-operation examples.
+- **Data platform operations and data movement guidance.** Added
+  schema-reference guidance and regression coverage for reviewing data-platform
+  agents against metadata lookup, production queries, writes/backfills/deletes,
+  dataset scope, masking, retention, audit evidence, rollback, and public-safe
+  synthetic warehouse examples.
+- **Compliance evidence and audit export guidance.** Added schema-reference
+  guidance and regression coverage for reviewing compliance/GRC agents against
+  control scope, evidence stores, read-only audit exports, remediation actions,
+  actor identity, retention boundaries, audit trails, and public-safe synthetic
+  compliance examples.
+- **Incident automation and escalation guidance.** Added schema-reference
+  guidance and regression coverage for scoring incident, on-call, and runbook
+  agents by paging, alert-silence, escalation, remediation, audit, rollback, and
+  public-safe incident-evidence boundaries.
+- **Approval evidence and change-control guidance.** Added schema-reference
+  guidance and regression coverage for checking durable approval artifacts,
+  actor identity, target resource context, change tickets, environment gates,
+  break-glass or auto-approval risks, and post-change audit/rollback evidence
+  before assigning approval or evidence labels to write-capable agents.
+- **Runtime isolation boundary guidance.** Added schema-reference guidance and
+  regression coverage for checking where agents and MCP servers execute,
+  including sandboxes, disposable containers, ephemeral CI runners, host
+  filesystem mounts, Docker sockets, kubeconfig contexts, browser profiles,
+  outbound egress, and public-safe fixture evidence.
+- **Dependency and supply-chain boundary guidance.** Added schema-reference
+  guidance and regression coverage for reviewing package, container, CI action,
+  plugin, generated-code, curl-to-shell, Docker socket, version-pinning, signing,
+  checksum, and rollback/uninstall risks before raising catalog maturity.
+- **Credential lifecycle and revocation guidance.** Added schema-reference
+  guidance and regression coverage for checking whether agent and MCP credentials
+  can be safely issued, rotated, revoked, audited, disconnected from webhooks or
+  integrations, and attributed to scoped bot/service identities instead of broad
+  human administrator access.
+- **Telemetry and retention boundary guidance.** Added schema-reference guidance
+  and regression coverage for checking prompt/tool-output telemetry, hosted logs,
+  retention/deletion/export controls, redaction, local logging, data residency,
+  and whether evidence signals are durable and safe to share.
+- **Tool permission and consent boundary guidance.** Added schema-reference
+  guidance and regression coverage for reviewing default tool permissions,
+  destructive-tool separation, per-tool allowlists, scoped runner identities,
+  revocable scopes, enforced approval gates, and most-privileged-tool scoring.
+- **Evaluation environment boundary guidance.** Added schema-reference guidance
+  and regression coverage for using sandbox projects, test tenants, fixture
+  repositories, read-only workspaces, narrow OAuth scopes, limited egress, and
+  redacted public-safe evidence before raising production-adjacent maturity.
+- **Agent instruction-boundary review guidance.** Added schema-reference guidance
+  and regression coverage for treating repository content, logs, tickets,
+  generated plans, and third-party MCP metadata as untrusted data rather than
+  executable instructions when scoring DevOps agents and MCP servers.
+- **Public-safe catalog metadata rules.** Added schema-reference guidance and
+  regression coverage that keeps tokens, customer data, tenant URLs, private
+  hostnames, production prompts, and private evidence out of catalog metadata,
+  README rows, generated reports, and PR notes.
+- **Catalog PR review checklist.** Added schema-reference guidance and regression
+  coverage for reviewing catalog pull requests against coherent scope,
+  reproducible source evidence, safety-score alignment, README/generated-sidecar
+  sync, no-secret hygiene, and recorded validation commands.
+- **Hosted MCP credential-boundary guidance.** Added schema-reference guidance and
+  regression coverage for checking hosted MCP authentication modes, OAuth or token
+  scopes, remote data-handling signals, read-only endpoints, and no-secret review
+  practices before cataloging vendor-hosted MCP endpoints.
+- **Catalog safety-score evidence rules.** Added schema-reference guidance and
+  regression coverage for assigning action level, approval, evidence tracing,
+  and labels from inspected tool-surface evidence instead of broad category
+  assumptions or marketing language.
+- **Catalog external-signal guidance.** Added schema-reference guidance and
+  regression coverage for using broad MCP indexes, registry mirrors, popularity
+  dashboards, and third-party evaluations as discovery prompts rather than
+  acceptance evidence or replacements for first-party source verification.
+- **Catalog change decision guide.** Added schema-reference guidance and
+  regression coverage for choosing whether a catalog PR should refresh, replace,
+  add, downgrade, or remove a row based on canonical-source and operator-safety
+  evidence.
+- **Catalog deprecation/removal guidance.** Added schema-reference guidance and
+  regression coverage for refreshing, downgrading, replacing, or removing
+  archived, deprecated, unreachable, unsafe, or superseded catalog entries
+  without preserving obsolete rows just to maintain counts.
+- **Catalog risk-note writing guide.** Added contributor guidance and regression
+  coverage for writing `risk_notes` as concrete operator warnings that name
+  credential boundaries, write/telemetry risks, dry-run-first controls, approval
+  expectations, and missing evidence.
+- **Catalog freshness audit guidance.** The schema reference now tells
+  contributors when and how to run the GitHub metadata audit, what report files
+  it produces, how to treat stale or archived warnings, and when manual
+  non-GitHub reachability checks are still required.
+- **Catalog schema reference.** Added a validator-backed reference for required
+  `data/repos.yaml` fields, allowed category slugs, artifact types, maturity
+  values, evaluation labels, score-to-label invariants, and pre-submit commands
+  so contributors can classify entries consistently before CI runs.
+- **Catalog entry template.** Added a minimal `data/repos.yaml` entry template
+  and review checklist to the schema reference so daily catalog additions start
+  with safe defaults, top-level list placement, credential-scoping notes, and
+  label-to-score consistency reminders.
+- **Catalog source verification checklist.** Added reproducible pre-submit checks
+  for reachability, freshness, tool surface, credential boundaries, and safety
+  signals so catalog reviewers can validate entries from public evidence before
+  trusting external indexes or marketing copy.
+- **Catalog evidence capture worksheet.** Added a PR-ready source-evidence note
+  template with harmless GitHub metadata checks and no-secret reminders so
+  reviewers can reproduce catalog claims without exposing credentials.
+- **Catalog identity rules.** Documented unique `name` and `url` expectations,
+  canonical source selection, single-row use-case handling, and the narrow case
+  where separate documentation and runnable artifact rows can coexist.
+- **README synchronization rules.** Documented the README surfaces that must stay
+  aligned with `data/repos.yaml`, including Recently added, catalog section
+  tables, quick picks, top-picks guidance, and the README count check.
 - **Operator safety checklist.** Added a practical preflight runbook for
   evaluating DevOps agents and MCP servers with read-only-first credentials,
   domain-specific credential boundaries, no-secret-in-context handling,
@@ -23,15 +156,24 @@ documented here. The format is based on
 
 ### Changed
 
+- **Guarded the operator safety checklist.** Added regression coverage so the
+  read-only-first, no-secret-in-context, dry-run/proposal, approval, blast-radius,
+  audit-evidence, and go/no-go guidance stays linked from the main entry points
+  and remains present during future documentation edits.
+- **Clarified catalog provenance classification.** The catalog schema reference now
+  defines when `official-*` categories are appropriate versus `community-*`
+  categories, and regression tests keep that contributor guidance present so
+  ecosystem-adjacent tools are not misclassified as official sources.
 - **Expanded the agent scorecard safety review.** The reusable scorecard now
   captures least-privilege credential scope, no-secret-in-context checks,
   redaction expectations, dry-run/preview commands, approval records, and audit
   artifacts, plus an explicit production-readiness decision, before recommending
   production-adjacent use.
 - **Expanded pull request safety checklist.** The PR template now asks
-  contributors to confirm no-secret-in-context handling, least-privilege
-  credential guidance, approval gates, dry-run/preview behavior, audit evidence,
-  rollback expectations, and telemetry/external API disclosure before review.
+  contributors to confirm schema-reference alignment, source-evidence capture,
+  no-secret-in-context handling, least-privilege credential guidance, approval
+  gates, dry-run/preview behavior, audit evidence, rollback expectations, and
+  telemetry/external API disclosure before review.
 - **Hardened catalog schema validation.** Required string fields now reject blank
   values, and `labels` / `use_cases` must contain at least one non-empty string
   item so incomplete catalog rows fail locally before reaching README generation

CONTRIBUTING.md

@@ -14,6 +14,7 @@ Thanks for helping make this a practical operator-grade index instead of a hype
 - Prefer entries that can be evaluated without real cloud credentials.
 - For write-capable, credentialed, or production-adjacent entries, use the [operator safety checklist](docs/operator-safety-checklist.md) to confirm domain-specific least-privilege credential boundaries, no-secret-in-context handling, dry-run/proposal behavior, approval gates, blast-radius limits, and audit evidence.
 - PRs should update [data/repos.yaml](data/repos.yaml) and [README.md](README.md) when the public index changes.
+- Use the [catalog schema reference](docs/catalog-schema.md) when choosing category slugs, artifact types, maturity values, and evaluation labels.
 
 ## Entry checklist
 
@@ -31,14 +32,15 @@ Thanks for helping make this a practical operator-grade index instead of a hype
 ## Local validation
 
 ```bash
-python -m venv .venv
+python3 -m venv .venv
 source .venv/bin/activate
 python -m pip install -e ".[dev]"
-python scripts/validate_repos_yaml.py
-python scripts/sync_readme_counts.py
-pytest -q
+python3 scripts/validate_repos_yaml.py
+python3 scripts/sync_readme_counts.py --check
+python3 scripts/sync_catalog_json.py --check
+python3 -m pytest -q
 ```
 
-`sync_readme_counts.py` refreshes the entry/category counts in the README intro from `data/repos.yaml`, so you never edit those numbers by hand.
+`sync_readme_counts.py` refreshes the entry/category counts in the README intro from `data/repos.yaml`, and `sync_catalog_json.py --check` verifies the skill-installer catalog sidecar stays aligned, so you never edit generated discovery metadata by hand.
 
-For a deeper freshness check before substantial catalog work, run `python scripts/audit_github_repos.py --stale-days 365`; it writes JSON and Markdown reports under `reports/` and warns on unreachable, archived, private, language-drifted, or stale GitHub repositories.
+For a deeper freshness check before substantial catalog work, run `python3 scripts/audit_github_repos.py --stale-days 365`; it writes JSON and Markdown reports under `reports/` and warns on unreachable, archived, private, language-drifted, or stale GitHub repositories.

README.md

@@ -19,6 +19,7 @@ Most agent lists stop at discovery. This one is built for operators:
 ## Contents
 
 - [Evaluation labels](#evaluation-labels)
+- [Catalog schema reference](#catalog-schema-reference)
 - [Compliance evidence checklist](#compliance-evidence-checklist)
 - [Operator safety checklist](#operator-safety-checklist)
 - [Top picks by use case](#top-picks-by-use-case)
@@ -41,6 +42,10 @@ Most agent lists stop at discovery. This one is built for operators:
 
 Labels are shorthand for structured fields recorded on every entry in [data/repos.yaml](data/repos.yaml) — [how entries are scored](docs/scoring.md) explains each field and how it is verified.
 
+## Catalog schema reference
+
+Catalog entries are validated against a curator-owned schema for required fields, allowed category slugs, artifact types, maturity values, evaluation labels, absolute `https://` URLs, duplicate names/URLs, and score-to-label consistency. See the [catalog schema reference](docs/catalog-schema.md) before adding or reclassifying entries.
+
 ## Compliance evidence checklist
 
 Production-adjacent agent runs need a reviewable evidence packet, not just a chat transcript. Use the [compliance evidence checklist](docs/compliance-evidence.md) to capture request context, identity and data boundaries, redacted tool calls, approval records, validation output, and follow-ups for MCP servers, skills, incident copilots, Terraform reviewers, and other DevOps agents.