Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
c635aa8
feat(pr-workflow): add pr-validate and falsifying-test skills
MajorLift Jul 30, 2026
26a7daf
Restore the MetaMask-planning link in the step-waiver item
MajorLift Jul 30, 2026
a66e72a
Move `falsifying-test` to the `testing` domain
MajorLift Jul 30, 2026
78a93b0
Add `D6` substitution A/B lane and sync `pr-validate` references
MajorLift Jul 30, 2026
36ea9c8
Move the AEP run procedure behind a reference
MajorLift Jul 30, 2026
2af1db1
Restore the full `pr-validate` description
MajorLift Jul 31, 2026
0b4bb26
Actually restore the `pr-validate` description
MajorLift Jul 31, 2026
704128a
Rename `pr-validate` to `evidence`
MajorLift Jul 31, 2026
e7e9248
Repair unresolvable references in `evidence` and `falsifying-test`
MajorLift Jul 31, 2026
1e927d5
Follow the engine renames in the catalog and `falsifying-test`
MajorLift Jul 31, 2026
14e670a
Add a lane index to the evidence catalog, and fix lane placement
MajorLift Jul 31, 2026
eceaf37
Add build-duration lanes `D7` and `G6`
MajorLift Jul 31, 2026
90d7082
Inline the publishing non-negotiables, which a real run ignored entirely
MajorLift Aug 1, 2026
d412caf
Add `falsify-probe.sh` — the runner that makes a lane reproducible
MajorLift Aug 1, 2026
559b4c0
Add `capture.sh` so the C9 and D3 analyses stop needing an operator t…
MajorLift Aug 1, 2026
85a575c
Add `selector-recompute.sh` — lane C4 gets a runner
MajorLift Aug 1, 2026
94a7640
Add `tsc-substitution.sh` — a runner for the tsc-blindspots lane
MajorLift Aug 1, 2026
6e38d6d
Diff the error sets rather than requiring a clean baseline
MajorLift Aug 1, 2026
3f861c7
Add `attest-gate.sh` — eight mechanical checks before anything is pub…
MajorLift Aug 1, 2026
2c0f286
Standardise the provenance marker across every runner
MajorLift Aug 1, 2026
51d6f94
Add `render-count.sh` — the component half of lane C4
MajorLift Aug 1, 2026
e5b6ce1
Guard the runners against the failures that masquerade as findings
MajorLift Aug 1, 2026
d84f304
Give the orchestrator a runner registry, with limits and synthesis rules
MajorLift Aug 1, 2026
0d8935f
State the bar: float concerns, do not close them
MajorLift Aug 1, 2026
6bdffd1
Report what each measurement runner did not cover
MajorLift Aug 1, 2026
30a03b6
Require a validation run to float something for review
MajorLift Aug 1, 2026
d5c6a61
Stop the runners from cutting the part a reader needed
MajorLift Aug 1, 2026
a3f1590
Reference artifacts by name in the publishable block
MajorLift Aug 1, 2026
d55b324
Stop the gate failing runs whose environment is not the repo's
MajorLift Aug 1, 2026
4ae5373
Let the caller say where its tool puts the finding
MajorLift Aug 1, 2026
d32bdf7
Write for the reviewer, not for a user of this skill
MajorLift Aug 1, 2026
7dee1f9
Move the output template into the skill, where corrections can land
MajorLift Aug 1, 2026
2c72486
Require a finding, not a printout
MajorLift Aug 1, 2026
4132951
Check where the disclaimer is, not just that it is there
MajorLift Aug 1, 2026
e58d1c0
Stop a script's summary of itself passing as a capture
MajorLift Aug 1, 2026
3c86466
Ask for a medium, not for better text
MajorLift Aug 1, 2026
dbc2754
Move the measurement to CI, where the run URL is the capture
MajorLift Aug 2, 2026
bdf5f39
Pin the runner source, and say so when it is missing
MajorLift Aug 2, 2026
5c41bfb
Require the target repo, make install opt-out, drop the consumer-inst…
MajorLift Aug 2, 2026
8d1ec29
Make every runner say whether a reader can verify it
MajorLift Aug 2, 2026
56578ce
Cite what exists; capture what you ran
MajorLift Aug 2, 2026
f6d76e8
Fold the runner-workflow fixes back from where they were found
MajorLift Aug 2, 2026
0b874cf
Name the commit in the command line, so an A/B pair is not one line t…
MajorLift Aug 2, 2026
a64ddb8
Stop a broken substitution reading as a divergence
MajorLift Aug 2, 2026
bfdf6b4
Require a run to measure the PR's range, and to say where its reach ends
MajorLift Aug 2, 2026
faff610
Require an instrument to publish the effect it had, not the instructi…
MajorLift Aug 3, 2026
46372af
Point `falsifying-test` at the harness, and separate it from its sibling
MajorLift Aug 3, 2026
1e7ef54
Rename `falsifying-test` to `red-on-base`
MajorLift Aug 3, 2026
50b52dc
Move `red-on-base` out; this PR is the instrument
MajorLift Aug 3, 2026
594a1a7
Check where the run is going, not only what it says
MajorLift Aug 3, 2026
0dee42c
Bring the runner fixes back from the branch CI was actually running
MajorLift Aug 3, 2026
cae0c08
Check that a figure in the prose traces to an exhibit
MajorLift Aug 4, 2026
a5bfb98
Stop check 12 passing when it could not run
MajorLift Aug 4, 2026
7da6ee1
Point references at the renamed `lavamoat-policy` skill
MajorLift Aug 4, 2026
38316ee
Give the diligence format a gate with `attest-gate.sh --diligence`
MajorLift Aug 4, 2026
76ecd0a
Remove private-repo and personal references from a public skill
MajorLift Aug 4, 2026
fbf8909
Take the artifact bucket and test fork out of the published text
MajorLift Aug 4, 2026
6634203
Restore what the privacy scrub broke: a working jq filter and bucket …
MajorLift Aug 4, 2026
d5fa3a5
Enforce the evidence rules where the model cannot route around them
MajorLift Aug 4, 2026
27be7ae
Add `attest` — the publish gate, ported from a personal command
MajorLift Aug 4, 2026
e069061
Add `gate-controls.sh`, and scope the gate to evidence artifacts
MajorLift Aug 4, 2026
2bb5a29
Check that the gate is wired, not only that it works
MajorLift Aug 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions domains/pr-workflow/skills/attest/references/dispatched-passes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Phase 1 — the three dispatched briefs

Send each to a **fresh instance** with the artifact and nothing else: not the transcript, not
your reasoning, not what you expect it to find. Context is what you are testing for. An instance
that knows what you meant will read what you meant.

Run them concurrently — they are independent, and sequencing lets the first one's findings frame
the others.

## outframe — contest the frame

> You are reading a finished set of findings you did not produce. Do not check whether the
> findings are correct. Ask what claim was chosen and what a different framing makes visible:
> what question would a reader with different priorities have asked of the same material, what
> does the chosen frame make it impossible to notice, and which of the findings only look
> significant because of how the problem was cut. Return findings the framing hid, not a
> critique of the writing.

## missing — contest the coverage

> You are auditing a completed run for what it did not do. Enumerate: a modality that was not
> run, a claim asserted but not verified, a source cited but not read, a case the method
> structurally cannot reach. For each, say what running it would cost and what it could change.
> Do not restate what the run found. Absence is the deliverable.

## press — read it as the stranger

> You are the reviewer this lands in front of, with no context and a decision to make. Read only
> the artifact. Say what you would have to take on trust, which number you could not check if you
> wanted to, what reads as a measurement but is a sentence, and anything that assumes you were
> present for work you were not. Flag register slips: hedging that reads as concealment,
> confidence that outruns the evidence, and any place the author's process shows through.

## Reading the returns

A finding from any pass that invalidates the claim is `BLOCKED`. A finding that qualifies it is
`ATTESTED WITH` — and the caveat goes **into the published artifact**, not just into the verdict,
or the reader never sees it.

Disagreement between passes is signal, not noise: `press` clearing something `outframe` flagged
usually means the artifact reads well and is framed wrong, which is the more dangerous state.
35 changes: 35 additions & 0 deletions domains/pr-workflow/skills/attest/references/phase-0-checks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Phase 0 — what each check catches

Generated from the checks in `mms-evidence/scripts/attest-gate.sh`; that script is the
authority. Each entry exists because a run shipped without it.

| # | check | run mode | diligence mode |
|---|---|---|---|
| 1 | marker pair | ✓ | ✓ |
| 2 | canonical header | ✓ | ✓ |
| 3 | verdict line | ✓ | ✓ |
| 4 | citations pinned | ✓ | ✓ |
| 5 | captured artifact | ✓ | ✓ |
| 6 | no prescriptions | ✓ | ✓ |
| 7 | no process narration | ✓ | ✓ |
| 8 | verdict is earned | ✓ | ✓ |
| 9 | verdict matches artifact | ✓ | ✓ |
| 10 | floats something for review | ✓ | ✓ |
| 11 | disclaimer present and early | ✓ | ✓ |
| 12 | destination is open | ✓ | ✓ |
| 13 | figures trace to an exhibit | ✓ | ✓ |

Checks 1–4 differ by mode: in `--diligence` they test that contract's own marker pair, its
header, and that citations are pinned to a tag or SHA rather than a branch head, and the
verdict-line check reports SKIP because a diligence artifact renders none. Checks 8 and 9 SKIP
for the same reason. Everything from 5 down is shared, because those defects are shared.

**Check 5 is the one that matters, and it asks for a medium.** Every earlier version tested a
property of the plaintext — does it carry a marker, does the command contain a placeholder — and
each caught one defect and missed the next, because every property of plaintext is forgeable by
whatever emits the plaintext. Four runs shipped that way. A `/blob/` permalink is a citation and
does not satisfy it: it witnesses a line in a file, never a run.

**Check 12 tests the destination**, which no property of the text reveals. Across one register of
published runs, 22 of 27 comments went to pull requests that had already merged — median 22 days
after the merge, gate-clean every time.
103 changes: 103 additions & 0 deletions domains/pr-workflow/skills/attest/skill.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
name: attest
description: The gate an evidence artifact passes before it is published to a pull request, issue or shared tracker. Two halves that do not substitute for each other — a mechanical pass that greps for the properties a reader needs (marker pair, pinned environment, a captured artifact rather than typed prose, a destination that is still open) and a dispatched pass sent to fresh instances that contest the framing, the coverage, and how it reads to a stranger. The author is the wrong checker: they remember running the check, and the memory supplies the provenance the text lacks. Verdicts are attested, attested with named caveats, blocked, or not a run — the last being common and legitimate, because a run that could not execute has produced nothing to publish. Triggers on mms-attest, or before posting any evidence, validation or diligence output to a public surface.
maturity: experimental
---

# /mms-attest

The gate an evidence artifact passes before it leaves your hands. Use before posting any
`/mms-evidence` or diligence output to a pull request, issue, or shared tracker.

## The author is the wrong reader, and the wrong checker

A validation run claims something was measured. Its characteristic failure is not a wrong number
— it is **prose that reads like a measurement**. An operator who ran the check cannot see this,
because they remember running it; the memory supplies the provenance the text lacks, before the
eye registers that it was missing.

This is not hypothetical. A run in this workflow shipped a results section whose commands, exit
codes and "reached 100%" were typed by hand, while the real logs sat unpublished on disk. The
author had the skill installed that forbids exactly that.

So the gate has two halves, and neither substitutes for the other.

**The mechanical half is not advisory.** Marker presence, a pinned environment, whether any
fenced block is a tool's output rather than the author's transcription, whether the destination
is still open — all greppable. Anything checkable is checked before a model is asked for
judgement, because a model asked "is this good evidence?" answers from inside the frame that
produced it.

**The dispatched half is positional.** Contesting the frame, the coverage, and the reading cannot
be self-run, for the same reason an author cannot proofread their own sentence for a word their
eye supplies.

## Phase 0 — mechanical, no model

```
scripts/attest-gate.sh <artifact.md> --target <owner/repo#N>
scripts/attest-gate.sh <artifact.md> --target <owner/repo#N> --diligence
```

Thirteen checks; every one a hard fail. `--diligence` swaps the four Validation-Run envelope
checks for a no-verdict contract's own and shares everything downstream. See
[references/phase-0-checks.md](references/phase-0-checks.md) for what each check exists to catch
and the run that caused it to be written.

**Run it as the same command that publishes, or it is a log line.** The gate and the write must
be one chain — `gate && publish`. Running both and reading the verdict afterwards is how a
blocked artifact reaches a public PR. The `hooks/pr-evidence-gate.py` PreToolUse hook enforces
this independently of your discipline, and fails closed; phase 0 is what you run to iterate
before it does.

## Phase 1 — dispatched, three lenses

| pass | reads for | returns |
|---|---|---|
| **outframe** | the frame — what claim was chosen, and what a different framing makes visible | findings the framing hid |
| **missing** | coverage — modality not run, claim unverified, source unread | the gap list |
| **press** | the text as it ships, as the stranger who has to act on it | leak and register findings |

Dispatch to fresh instances is the mechanism, not an optimisation: a self-run frame check is
composed inside the frame it is meant to test. Briefs in
[references/dispatched-passes.md](references/dispatched-passes.md).

Skipping a pass is allowed. Silently skipping it is not — name it as skipped in the verdict.

## Phase 2 — shape

Front-load the verdict, cut anything that does not change what the reader does, keep every
artifact and move only its placement. Shape only, after content is settled — a shape pass that
reaches content is how a capability table gets dissolved into paragraphs and the comment's
payload disappears.

## Verdict

```
ATTESTED phase 0 clean, no blocking finding from phase 1
ATTESTED WITH publishable, with named caveats carried INTO the artifact
BLOCKED phase 0 failure, or a phase 1 finding that invalidates the claim
NOT A RUN nothing was measured; there is no artifact to publish
```

`NOT A RUN` is legitimate and common. A run that could not execute its check produced no
evidence, and publishing the attempt with a disclaimer is worse than publishing nothing — the
disclaimer reads as hedging and the figure is kept anyway.

## Anti-patterns

| Bad | Good |
|---|---|
| Running phase 1 to decide phase 0 | Mechanical checks first; cheap and unarguable |
| Self-running the dispatched passes | Dispatch, or skip and say it was skipped |
| Attesting your own run | The gate is positional; an author attesting themselves attests nothing |
| Treating phase 0 items as advisory | Every one is a hard fail |
| `ATTESTED WITH` as a soft pass | The caveat goes *into the published artifact*, not just the verdict |
| Softening a check to fit the case in hand | If the new version could be satisfied by better prose alone, it is no longer the check |

## Related

- `mms-evidence` — produces the artifact this gates
- `mms-instrument-check` — prove the instrument fires before its output counts
- `mms-unmeasured-join` — audit the inference between the facts
- `mms-scope-of-search` — what a negative result is a fact about
Loading
Loading