Skip to content

ci: focus validation for content-only pull requests - #1999

Merged
mrbobbytables merged 3 commits into
cncf:mainfrom
mrbobbytables:docs/content-only-ci-adr
Oct 30, 2026
Merged

mrbobbytables merged 3 commits into
cncf:mainfrom
mrbobbytables:docs/content-only-ci-adr

Conversation

@mrbobbytables

Copy link
Copy Markdown
Member

Summary

  • Add ADR 0011 for focused validation of authored-document-only pull requests.
  • Classify changed paths and use focused document checks, contract tests, and a production build when eligible; preserve full validation for generated, mixed, code, and uncertain changes.
  • Update contributor guidance to explain the focused and full-validation paths.

Remit: Reduces CI friction for end users contributing authored documentation while preserving full checks for other changes.

Validation

  • Unit coverage gate and targeted workflow/classifier tests passed.
  • Format, spelling, Markdown, lint, and duplication checks passed.
  • Live GitHub Actions scheduling was not exercised locally.

mrbobbytables and others added 2 commits October 29, 2026 16:01
Document the proposed CI split for authored-document-only changes, retaining relevant content checks and the production build while reserving broad suites for code, generated data, mixed, and uncertain changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
Route authored-document-only changes through focused checks while preserving full validation for generated, mixed, and uncertain changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
@netlify

netlify Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for endusers ready!

Name Link
🔨 Latest commit 6e2b536
🔍 Latest deploy log https://app.netlify.com/projects/endusers/deploys/6ae3e47fc40ea542de385205
😎 Deploy Preview https://deploy-preview-1999--endusers.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@hivecommons-hive

Copy link
Copy Markdown
Contributor
7058eac

ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blocked

Hive flagged this PR (author @mrbobbytables, head 7058eac37854) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation — a maintainer should confirm the change is intended before it merges.

  • sensitive_path — changes 1 sensitive path(s) (Touches a sensitive path (OWNERS, workflows, policies, hive config, security docs, …))
    • .github/workflows/ci.yml

Hive did not add the sentinel-alert label because this author is trusted. The finding remains recorded for audit and the dashboard Security tab; set sentinel.trusted_authors_block: true to require the blocking label for trusted authors too.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One verified defect: blog-only PRs fail the focused lint this PR introduces, because --ignore-path '' disables .prettierignore.

  • scripts/lib/content-only-ci.mjs:55 classifies blog/ paths as authored documents, and ADR 0011 (line 58) explicitly includes blog posts in scope.
  • scripts/lib/content-only-ci.mjs:222 runs prettier --check --ignore-path '' <documents>, which disables .prettierignore.
  • .prettierignore deliberately excludes blog/: prettier's markdown parser mangles MDX truncation comments ({/*truncate*/} → {/_ _/}), breaking Docusaurus truncation and SSG (per the comment in that file).

Mechanism: a content-only PR touching only a blog post takes the focused path, prettier checks the blog file the full path deliberately skips, and the check fails. Verified at this head: npx prettier --check --ignore-path '' blog/2026-07-22-welcome-to-endusers-cncf-io.md blog/2026-08-08-month-in-metrics-august-2026.md → both [warn], exit 1 — i.e. every current blog post already fails. Consequence: the exact contributor class this PR optimizes for (blog authors) gets a guaranteed red check, and the natural remedy (prettier --write) corrupts the MDX comment. The unit tests mock spawnSync (tests/content-only-ci.test.mjs:417), so this isn't caught.

Fix is small: drop --ignore-path '' so prettier honors .prettierignore exactly as full validation does (the only other ignored pattern, docs/architectures/*/index.md, is already excluded by the classifier at content-only-ci.mjs:46-51), or filter documents against the ignore file before invoking prettier. Either way, a test with a real blog/ fixture would have failed without the fix.

What I verified as sound, reading the tree at this head:

  • The markdownlint arm is consistent: markdownlint-cli honors .markdownlintignore even for explicitly listed paths (verified: docs/community/technical-advisory-board/governance.md and generated architecture pages are skipped, exit 0), and cspell honors ignorePaths in .cspell.yml.
  • The classifier fails closed everywhere I probed: unknown git statuses → parseNameStatus returns null → full validation; T (typechange) and rename/copy statuses excluded; symlinks rejected via lstatSync (content-only-ci.mjs:30-32); ../absolute/backslash paths rejected (lines 36-42); base SHA must be 40-hex; classify job failure drives FULL_VALIDATION=true in ci.yml (needs.classify.result != 'success').
  • e2e gating is fail-safe: e2e runs whenever validate or lint did not succeed or the change is not document-only, so an upstream anomaly restores full coverage. Skipping e2e/e2e-coverage for doc-only PRs is a policy choice ADR 0011 documents, so I treat it as decided.
  • No permission or trust-surface regressions: checkout pinned by SHA, persist-credentials: false, no new permissions: blocks, document paths re-validated in decodeChangedDocuments before being passed as argv.

With the prettier ignore fix (plus a non-mocked blog fixture test), this looks sound to me.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Review finding on the focused lint path (scripts/lib/content-only-ci.mjs, runDocumentLint): a blog-only pull request will fail the required "Lint repository" check, and the failure message tells the contributor to make a change that breaks the MDX build.

runDocumentLint runs

npx prettier --check --ignore-path '' $CHANGED_DOCUMENTS

--ignore-path '' disables .prettierignore, which excludes blog/ precisely because prettier's markdown parser rewrites the MDX comment {/*truncate*/} into {/_truncate_/} (see the comment in .prettierignore and #681, #795). isAuthoredDocumentPath classifies blog/**/*.md as an authored document, so a blog post edit takes the focused path and gets prettier-checked against rules the full path deliberately skips.

Verified at b4ea678 (current main):

$ npx prettier --check --ignore-path '' blog/*.md
[warn] blog/2026-07-22-welcome-to-endusers-cncf-io.md
[warn] blog/2026-08-08-month-in-metrics-august-2026.md
[warn] Code style issues found in 2 files. Run Prettier with --write to fix.

Both existing posts already fail, so every blog-only PR starts red. Acting on the hint produces exactly the mangling .prettierignore exists to prevent:

-{/*truncate*/}
+{/_truncate_/}

docs/architectures/*/index.md is the other .prettierignore entry, and the classifier already excludes it, so blog/ is the only live collision.

Suggested fix — drop --ignore-path '' so prettier honours .prettierignore:

    ['npx', ['prettier', '--check', ...documents]],

Verified both directions at the same revision: with the repo default ignore file, npx prettier --check blog/2026-07-22-welcome-to-endusers-cncf-io.md exits 0 (and still exits 0 when a non-ignored file is passed alongside it), while a deliberately misformatted docs/*.md still exits 1 — so the check keeps its teeth on the documents it is meant to cover. A test asserting that a changed blog/ document passes runDocumentLint would pin this.

(Separately: the skipped End-to-end tests / End-to-end coverage jobs are safe for the required-status-check ruleset — GitHub reports a job skipped by a job-level if as "Success" and it does not block merging.)


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: unknown

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: mrbobbytables <mrbobbytables@users.noreply.github.com>
@mrbobbytables
mrbobbytables added this pull request to the merge queue Oct 30, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor
6e2b536

ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blocked

Hive flagged this PR (author @mrbobbytables, head 6e2b536becac) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation — a maintainer should confirm the change is intended before it merges.

  • sensitive_path — changes 1 sensitive path(s) (Touches a sensitive path (OWNERS, workflows, policies, hive config, security docs, …))
    • .github/workflows/ci.yml

Hive did not add the sentinel-alert label because this author is trusted. The finding remains recorded for audit and the dashboard Security tab; set sentinel.trusted_authors_block: true to require the blocking label for trusted authors too.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @castrojo should decide whether skipping the unit-coverage, e2e, and e2e-coverage gates on classifier-approved content-only PRs is an acceptable merge-policy change (skipped required checks count as passing under branch protection).

Conclusion: the implementation is careful and I found no defect, but this changes merge gating via .github/workflows/ci.yml, so the policy call belongs to a maintainer.

What I verified by reading the tree at 6e2b536:

  • Fail-closed classification. scripts/lib/content-only-ci.mjs returns full validation on any irregularity: non-PR/merge-group events and malformed base SHAs (classifyChanges, lines 87–94), statuses other than A/M/D (line 100), paths with \, absolute paths, or ../. segments (isAuthoredDocumentPath, lines 35–41), data-backed docs and generated architecture pages (lines 44–51), git-diff failure (classifyCurrentEvent catch, lines 155–160). workflow_dispatch and push also fall through to full validation.
  • Fail-open workflow wiring. validate/lint run if: !cancelled(), and FULL_VALIDATION is true whenever needs.classify.result != 'success' (ci.yml:46–57), so a broken classifier job yields the full suite, not a skip. e2e/e2e-coverage run whenever classify/validate/lint are not all green and document-only (ci.yml:163–169, 197–203).
  • Defense in depth. runDocumentLint re-validates every decoded path against the same allowlist and the filesystem before passing it to prettier/cspell/markdownlint (decodeChangedDocuments, lines 203–212), so the lint job does not trust the classify job's output blindly. Paths from the allowlist cannot begin with -, so no argument injection.
  • Retained gates. The production build still runs on the focused path (ci.yml, Build production site), content contracts (docs-contract, markdown-references, community-group-coverage) replace repo-wide validators, permissions: contents: read and persist-credentials: false are unchanged, and the Mutation testing check covers the new scripts/lib/ file (it ran green on this head).
  • Docs. ADR 0011, adr/README.md, and CONTRIBUTING.md all updated consistently with the workflow behaviour; tests/content-only-ci-workflow.test.mjs pins the workflow shape and tests/content-only-ci.test.mjs covers the classifier.

The one thing I cannot sign off: on a classifier-approved doc-only PR, the unit-coverage step and the End-to-end tests / End-to-end coverage jobs are skipped, and GitHub branch protection treats a skipped required check as satisfied. That is the ADR's stated intent, not a bug — but it is a deliberate reduction in what the required checks enforce, decided inside the same PR (ADR 0011 is self-accepted). Note this does not widen the existing GitHub trust boundary — any PR can already edit ci.yml on the pull_request event — but which checks gate a merge is a maintainer's call.

Looks correct to me as code; flagging for @castrojo to accept or decline the gating policy. I have not approved or changed anything.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant