Skip to content

Commit f3a220d

Browse files
ejntaylorclaudemariojgt
authored
Add MAINTAINING.md — a guide for editing the install prompt and setup docs (#54)
Consolidates the load-bearing invariants (previously split across CLAUDE.md and field-test/README.md) into one map a maintainer opens first: a traffic-light table of the five artifacts and their edit rules, the three-places-must-match rule for the prompt, the field-test gate to run before shipping, and the refusal-trigger don'ts. Adds discoverability pointers from CLAUDE.md (for the LLM) and a source-only HTML comment in GETTING-STARTED.md step 1 (for a colleague editing the onboarding prose) so both land on the guide before touching the prompt. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Mario Tarosso <mariojgt2@gmail.com>
1 parent 06a02d8 commit f3a220d

3 files changed

Lines changed: 48 additions & 0 deletions

File tree

‎CLAUDE.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
npm package that connects a JS/Node project to Patchstack vulnerability monitoring: scans the lockfile, POSTs package names + versions, provisions a site UUID, prints a claim URL. CLI entry: `src/cli.ts`; the state-aware setup checklist lives in `src/guide.ts`.
44

5+
Editing any onboarding, prompt, or setup-guide content? Read `MAINTAINING.md` first — it maps which files are load-bearing and how to ship a change safely.
6+
57
## Commands
68

79
- `npm test` — vitest suite

‎GETTING-STARTED.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Getting started — connect a project to Patchstack
22

3+
<!-- Editing this file? The prompt block in step 1 is a load-bearing, adversarially-tuned artifact that must stay byte-identical with README.md and field-test/prompt.txt. Read MAINTAINING.md before changing it or any setup-guide content. -->
4+
35
The fastest path from "I have a JS/Node project" to "Patchstack is monitoring its dependencies". Share this doc with anyone setting up a project; the whole flow is 2–5 minutes.
46

57
## 1. Paste this prompt into your AI coding tool

‎MAINTAINING.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Maintaining the docs — a guide for the guide
2+
3+
Read this before editing any onboarding, prompt, or setup-guide content. The install flow is an adversarial-UX artifact: AI coding agents actively try to refuse it, and several files are load-bearing in ways that aren't obvious from reading them. This doc is the map of what's safe to change and what isn't.
4+
5+
The deep "why" — the AI-agent refusal modes each clause guards against — lives in [`field-test/README.md`](field-test/README.md). This doc is the shorter "what do I touch, and how do I ship it safely."
6+
7+
## The five artifacts and their edit rules
8+
9+
| Artifact | Rule |
10+
|---|---|
11+
| **The install prompt** (1 sentence) — in `README.md`, `GETTING-STARTED.md` (step 1), and `field-test/prompt.txt` | 🔴 **Don't casually edit.** Load-bearing and adversarially tuned; every clause exists because an agent refused a shorter version. Must be **byte-identical** in all three places, and any change must pass the hostile field-test gate. |
12+
| **`src/guide.ts`** — the `guide` checklist output | 🟠 **Edit with the gate.** Agents read this live and act on it; wrong commands or claims cause refusals. |
13+
| **`AGENT-INSTALL.md`** — ships inside the npm tarball | 🟠 **Edit with the gate.** Must disclose **every** capability in `dist/` (e.g. the `protect` command); an undisclosed capability or overbroad privacy claim is read as misrepresentation and refused. |
14+
| **`GETTING-STARTED.md`** — teammate-facing onboarding prose (steps 2–4) | 🟢 **Safe to improve** — *except* the quoted prompt block in step 1, which is the 🔴 artifact above. |
15+
| **`field-test/README.md`** — dev-process rationale | 🟢 **Safe.** Dev-only; never ships in the package. |
16+
17+
**The trap:** "update the onboarding steps" sounds like a 🟢 job, but step 1 of `GETTING-STARTED.md` *contains* the 🔴 prompt. Improving the surrounding prose is fine; changing the quoted prompt is not, unless you change it in all three places and re-gate.
18+
19+
## The prompt lives in three places — keep them identical
20+
21+
`README.md` (Install prompt section), `GETTING-STARTED.md` (step 1), and `field-test/prompt.txt`. `prompt.txt` is the artifact the harness actually tests. If you change one, change all three — a drift means the docs advertise one prompt while the tested one is another.
22+
23+
## Before shipping a 🔴 or 🟠 change
24+
25+
Run the gate:
26+
27+
```bash
28+
node field-test/run.mjs --persona hostile --rounds 3
29+
```
30+
31+
It runs a real AI agent through the full install in a throwaway fixture against a mocked API and scores the outcome. Read each round's `report.md` (the HESITATIONS / DECISION ANALYSIS sections) — anything an agent pauses on is a future refusal. See [`field-test/README.md`](field-test/README.md) for the improve-and-retest loop, the safety model, and what to do when the gate is red for environmental reasons (stale published docs, release freshness).
32+
33+
Caveat: the fixture installs the **published** package, so an unpublished `guide`/CLI change can't be exercised end-to-end — publish first, or accept that the run validates only the prompt shape.
34+
35+
## Don'ts (these are refusal triggers, not style nits)
36+
37+
- **Don't add reassurance language** ("it's safe", "nothing is executed remotely"). Agents fact-check it against the code and flag the gap as a manipulation signal.
38+
- **Don't narrate how the prompt survives refusals** in any shipped doc (`README.md`, `AGENT-INSTALL.md`, `GETTING-STARTED.md`). Agents `npm pack` the tarball and read everything; being told the message was tuned to get past them is itself read as the signal to refuse. That rationale belongs in `field-test/README.md`, which doesn't ship.
39+
- **Don't ask the agent to "follow the guide/instructions it prints"** unbounded — name the concrete steps (build hooks + widget snippet) instead. Unbounded delegation reads as a blank check.
40+
- **Don't let the shipped docs claim more privacy than the code delivers**, or omit a capability `dist/` ships. Every contradiction between docs and code is treated as misrepresentation.
41+
42+
## When a new refusal shows up in the wild
43+
44+
A real-world refusal report with a *new* reason becomes a new persona in [`field-test/personas/`](field-test/personas/) so the regression stays covered. Encode the pressure the agent applied, don't just patch the prompt.

0 commit comments

Comments
 (0)