✨ - Support repository owned verification setup hooks without editing generated workflows #647

Is there an existing issue or feature request for this?

• I have searched the existing issues and feature requests.

What area does this improvement affect?

New Feature

What actions does this improvement affect?

Issue, Pull Request, Commit

Description of the idea or improvement

Provide an officially supported extension point for each consumer repository to prepare its own verification toolchain and dependencies without editing generated Copilot event workflows.

Start with copilot_issue_comment.yml and inventory every other workflow/path that can run configured repository verification, including branch synchronization and automated fixes. The maintainer has selected a repository-owned native composite action at .github/actions/copilot-verification-setup/action.yml, with reviewed auxiliary assets. GitHub Actions executes its uses and run steps in the verifier job. Consumers should edit that auxiliary implementation while Copilot retains ownership of the generated workflows.

The preparation must be available in the same job/execution environment as verification. A successful application CI job, or an independent preparation job, does not make its Python environment, PATH, Node/pnpm tools or installed dependencies available to the Copilot job. Distinguish initial toolchain provisioning from refreshing dependencies for the actual candidate tree: synchronization can introduce a temporary merge, and an agent can change dependency manifests.

Expected consumer journey:

1. Install/update Copilot normally.
2. Review the consumer composite onto trusted main, update the generated callbacks, and enable VERIFICATION_SETUP=true (disabled by default).
3. Define only that project's runtimes and dependency installation, using its declared versions and lockfiles.
4. Copilot invokes preparation on authorized operations that need repository verification, then checks prerequisites and runs the existing verification commands.
5. A missing dependency, failed setup or failed verification prevents commit/push. Read-only/no-op/dry-run operations do not unnecessarily install project dependencies.

Current limitations or challenges

Real consumer evidence from vypdev/homeassistant-gateway:

• Issue: 🔧🧑‍💻 - Align Copilot automation with gateway verification and publication homeassistant-gateway#19
• Command: 🔧🧑‍💻 - Align Copilot automation with gateway verification and publication homeassistant-gateway#19 (comment)
• Failed run: https://github.com/vypdev/homeassistant-gateway/actions/runs/39570398824
• Job: https://github.com/vypdev/homeassistant-gateway/actions/runs/39570398824/job/119122814347
• Workflow: .github/workflows/copilot_issue_comment.yml
• The repository is authorized for runner group vypdev, with labels self-hosted, codex, Linux. Runner admission/authentication worked; this was a project verification environment failure.
• Configured verification begins with python3 -m pytest -m 'not live' --cov=homeassistant_gateway --cov-report=term-missing --cov-fail-under=85, followed by project-pinned pnpm frontend gates and git diff --check.

Sanitized observed result:

/usr/bin/python3 -m pytest ...
/usr/bin/python3: No module named pytest
/usr/bin/git merge --abort

The temporary synchronization was safely aborted without a push. This demonstrates the missing Python test dependency in that job; it does not establish that every runner is broken or that the group lacks repository access. Copilot itself does not require pytest universally: this consumer's verification policy does.

The consumer maintainer deliberately keeps generated Copilot workflows unchanged, except the documented release/hotfix extension points. Requiring direct edits to copilot_issue_comment.yml creates customization/update drift; globally changing shared runners is also a poor fit for repositories with different runtimes.

Related closed #620 delivered prerequisite declaration, version preflight and same-job setup guidance. That is useful and should be retained, but it does not provide the requested repository-owned extension point in the generated workflows. This request is the follow-up capability, not a request to suppress that diagnostic.

Expected impact

A consumer can prepare Python tests, Node/pnpm, JVM or other project tools through one reviewed auxiliary definition and use the standard sync/autofix flow without patching generated workflow files or changing a shared runner globally.

Acceptance:

• The helper contract is optional, bounded and documented; absence preserves compatible existing behavior and fail-closed verification.
• The setup CLI and generated templates wire the supported hook consistently for all operations that actually invoke repository verification. Updating Copilot preserves the consumer-owned helper and does not overwrite its contents.
• The helper's provenance, checkout revision and authorization are explicit. Fork/untrusted event handling cannot execute preparation with PAT, model credentials, persisted Git write credentials or access to agent auth stores.
• Prepared tools/dependencies work in the credential-free verification environment, including replaced HOME and the documented PATH behavior. Provision pnpm ahead of time rather than requiring an implicit Corepack download.
• Verification uses dependencies matching the tree being checked, including temporary synchronization merges or changed manifests; do not silently prepare an unrelated revision.
• Setup errors/timeouts have actionable sanitized diagnostics and withhold commit/push. Keep existing tests, coverage and authorization intact.
• Add focused coverage for the Python/pytest failure above, a correctly prepared Python consumer, a Node/pnpm consumer, absent/invalid helpers, setup failure/timeout, credential isolation, changed dependency manifests, relevant command routes and setup/update idempotency.
• Controlled acceptance demonstrates the original branch-sync verification with the same managed branch and no manual edits to the consumer's generated Copilot workflows. Do not launch a release as part of this work.

Alternatives considered

• Editing every generated event workflow: conflicts with clear generator ownership and repeats bootstrap logic across entry points.
• Globally installing every consumer's dependencies in the shared runner: introduces version conflicts and requires infrastructure administration for repository-specific checks.
• A separate preparation workflow/job without a shared execution environment: does not prepare the actual verifier.
• Weakening verification or removing pytest: hides the failure and reduces the existing quality bar.
• Selected: a native composite action in the same job, at the existing verification boundary after candidate changes. Rejected for this delivery: the unreleased Node ESM helper and reusable workflow candidate handoff.

Version of copilot

Consumer installed with CLI 3.9.3 and vypdev/copilot@v3; runtime failure observed on 2026-10-21. Existing prerequisite guidance and generic workflow contract inspected on current master. No claim of a new runtime retest after this proposal.

Additional context or comments

Requested by the consumer maintainer while completing vypdev/homeassistant-gateway#20. No generated Copilot event workflow was modified locally for this request.

Relevant contract: https://github.com/vypdev/copilot/blob/master/docs/security-operations/operations/verification.mdx#repository-verification-prerequisites

The maintainer explicitly authorized adapting PR #652 to the native composite design and merging to develop only after all checks and findings are resolved. No release is authorized or required.

Approved contract: an immutable trusted-main action-directory snapshot; fresh private HOME for every verification command; native five-minute setup steps; bounded same-job continuation of the admitted operation without agent replay; candidate/HEAD/index validation and validated PATH/JAVA_HOME before existing verification/publication gates. Native dependency-only profiles survive their post steps and are then removed. Read-only/dry-run/no-op/empty-command routes run no setup. The local CLI does not emulate Actions and fails closed on opted-in verification. Native logs use GitHub semantics rather than the retired Node output protocol.

Implementation and acceptance: PR #652 and its linked SDD, including real native Python/pytest + Node/pnpm acceptance on Linux, Windows and macOS, and controlled local same-managed-ref sync/abort coverage.