Skip to content

docs: make the exact-head live-gate status the live-evidence record #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mergify merged 2 commits into from
Oct 4, 2026
Merged

docs: make the exact-head live-gate status the live-evidence record #121

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c4b55bd
docs: make the exact-head live-gate status the live-evidence record
ericlitman Oct 4, 2026
71bc986
docs: align the A3 verifier row with the live-gate record
ericlitman Oct 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions .github/pull_request_template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,6 @@ Closes #
- [ ] Bun tests, strict typecheck, static invariants, and plugin validation pass.
- [ ] The exact candidate is installed in every affected harness.
- [ ] The changed behavior passes from each real user surface.
- [ ] The installed version, action, and observed result appear below.
- [ ] The `live-gate` status on the final head links the evidence comment (installed version, surface, action, observed result).

Live evidence:


A pull request without live evidence remains a draft. Do not merge, tag, release, or roll it out.
Mergify does not queue a pull request without `live-gate` on its exact head. Do not merge, tag, release, or roll out without it.
2 changes: 1 addition & 1 deletion AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@ Cursor's `cursor/plugins/pstack` tree is the content upstream. Keep one shared s

Before opening a pull request, run the Bun tests, strict typecheck, static invariants, and plugin validation.

Nothing merges, tags, releases, or rolls out until the exact candidate is installed and the changed behavior passes a live test from the real user surface in every affected harness. Unit tests, validators, source inspection, and self-reports do not satisfy this gate. Record the installed version, surface, action, and observed result in the pull request template. A pull request without that evidence remains a draft.
Nothing merges, tags, releases, or rolls out until the exact candidate is installed and the changed behavior passes a live test from the real user surface in every affected harness. Unit tests, validators, source inspection, and self-reports do not satisfy this gate. The record is the `live-gate` commit status on that exact head, which links an evidence comment naming the installed version, surface, action, and observed result. Mergify does not queue a pull request without it. A pull request may be ready for review before its live test, but every new head, including a queue-created head, needs its own.

Do not add an implicit runtime timeout or a weaker-model fallback.
12 changes: 8 additions & 4 deletions docs/plans/2026-10-01-triage-and-roadmap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Snapshot: `main` at `1c91e65` (Open Pstack 1.5.0, Cursor pstack 0.15.5). Cursor
|---|---|
| #100 / PR #101 Mergify queue | Add a required `live-gate` status first (section 6). The draft convention alone can't hold the queue, because Unfret reviews only ready PRs. Then land it, record the first queue-merged PR on #100, and close MASTRA-455, which duplicates it. |
| factory-run GitHub sources (MASTRA-737) | `begin` requires a Linear issue (`factory-ops.mjs` around line 1463), and `closeout` throws `Delivery item has no Linear issue source`. open-pstack cards have been GitHub-sourced since MASTRA-450. Add a GitHub issue source to `begin` (post prior findings as an issue comment) and to `closeout` (confirm the merged PR closed the issue, or close it with the evidence link). |
| Live-gate skill (#90, reopened) | #90 was closed as completed, but there's no project verification skill on `main` or in the main checkout. Factory sandboxes have no `bun`, `claude`, `codex`, or `grok` (see the factory triage comments), so the AGENTS.md installed-candidate gate has to run on the Mac. The skill should install a PR head into Claude Code and Codex, run the changed behavior, and write the PR template's evidence block. |
| Live-gate skill (#90, reopened) | #90 was closed as completed, but there's no project verification skill on `main` or in the main checkout. Factory sandboxes have no `bun`, `claude`, `codex`, or `grok` (see the factory triage comments), so the AGENTS.md installed-candidate gate has to run on the Mac. The skill should install a PR head into Claude Code and Codex, run the changed behavior, then post one evidence comment and the exact-SHA `live-gate` status, changing nothing else on the PR (see section 7). |
| Factory reachability | Resolved: `FACTORY_API_URL=https://studio2.tail062eee.ts.net:8444` reaches the API. The open-pstack project is `9af4ac1d-64a8-49d7-9d84-aa8d0e43c347`. |
| MASTRA-453 | #34's Build session is parked on an unresolved `request_access`. Cancel that run when #34 is planned again. |

Expand Down Expand Up @@ -133,9 +133,9 @@ Each issue goes through these steps:
2. Claude moves the card to Planning with `factory-run`.
3. The planner posts a plan.
4. `factory-adjudicate` judges the plan with a different model.
5. Build runs on the Factory branch. Before opening the PR, the builder runs the checks AGENTS.md requires: Bun tests, strict typecheck, static invariants, and plugin validation. Then it opens the PR as a draft. If the build host lacks Bun, as the triage sandbox did, fix the host before the first run. Don't skip the checks.
6. Claude runs the live gate on the Mac (A3) against the draft head, records the evidence in the PR, and sets the `live-gate` status on that exact head. Only then is the PR marked ready, as AGENTS.md requires.
7. Unfret reviews the ready PR. The builder fixes the findings. Each new head needs a fresh `live-gate` status, because the status is bound to the head SHA.
5. Build runs on the Factory branch. Before opening the PR, the builder runs the checks AGENTS.md requires: Bun tests, strict typecheck, static invariants, and plugin validation. Then it opens the PR, ready for review. If the build host lacks Bun, as the triage sandbox did, fix the host before the first run. Don't skip the checks.
6. Unfret reviews the PR. The builder fixes the findings until Unfret passes.
7. Claude runs the live gate (A3) on that final head. The verifier posts one evidence comment and the `live-gate` status on that exact head, and changes nothing else on the PR. Any later head, including a queue-created head, needs a fresh run.

@unfret-eal unfret-eal Bot Oct 4, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃數 Low 路 A3 skill spec left stale: still says write the PR template evidence block

Changed step 7 delegates the live gate to the A3 spec ("Claude runs the live gate (A3)"). It says the verifier posts one evidence comment plus the live-gate status and "changes nothing else on the PR", and section 7 (line 171) repeats this. The A3 row it points to (line 13, unchanged) still tells the #90 skill to "write the PR template's evidence block". This diff deletes that block from the template (-Live evidence:). The A3 row also never mentions posting the status or the comment. Scenario: the #90 builder implements from the A3 row. The skill then finds no block to write, or it edits the PR body, which section 7 says produced new review findings. It may also omit the live-gate status Mergify requires. The plan contradicts itself on the verifier contract.

Withdrawn in the latest review.

8. Mergify queues the PR once `verify`, `Unfret`, and `live-gate` pass on the same head.
9. The builder verifies the merged result and moves the card to Done.
10. `closeout` closes the GitHub issue (A2) and records findings.
Expand Down Expand Up @@ -165,3 +165,7 @@ Waves:
- `begin` and `closeout` accept only Linear sources (MASTRA-737). This blocks every open-pstack run.
- #101 gap: Unfret doesn't review drafts, so a PR has to be non-draft to get `Unfret`. Once it's non-draft, #101 queues it when `verify` and `Unfret` pass, which can happen before the live test. Fix requirement: the commit that merges must be the commit the installed-harness test ran on, and the queue must never wait on a check that nothing can post. The likely shape, to be confirmed in the #101 session against Mergify's docs: in-place queue checks (empty `merge_conditions`, `batch_size: 1`, so Mergify creates no draft batch PR), and `queue_conditions` that require `-draft`, `verify`, `Unfret`, `live-gate` (a commit status that the #90 skill posts on the exact head), and the branch being up to date with `main`. With the merge-commit method and an up-to-date branch, the merged tree is the tested tree. Any `main` change forces a rebase, which creates a new head that needs fresh `verify`, `Unfret`, and `live-gate`. Corrections are posted on the #101 Greptile thread.
- Order: MASTRA-737, then the live-gate check in #101 and its merge, then #90. After that, Wave 1. Until #90 exists, the operator posts `live-gate` by hand on each PR head, after running the installed-harness test, with a link to the evidence. #101 is merged that way as well.

## 7. Gate change (2026-10-03)

AGENTS.md now treats the exact-head `live-gate` status and its linked evidence comment as the live-evidence record, in place of the PR template and the draft rule. Mergify's required `live-gate` check enforces it. The change came out of the #111 step-back: every way the verifier changed draft, ready, or body state produced new review findings, and draft state was no longer a merge control. The verifier (#90) posts only the comment and the status. #106's runbook follows this order: review first, then the live gate on the final head.
Loading