Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions AGENT-INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,66 @@ This is a request, not a mechanism: nothing in the install depends on it. Do it

**Then tell them to deploy.** Setup changes source files, and the deployed site keeps serving its previous build until the next deploy — so visitors get no widget, and on a server-rendered root no production marker, until the user deploys (or hits Publish) again. Say it as a reminder; do not deploy anything yourself.

## When your tool will not run this CLI

Some coding tools refuse to execute a third-party command until the person approves it. In Claude Code,
auto mode's classifier can decline `npx @patchstack/connect setup` and the installed `patchstack-connect`
binary on that ground instead of prompting. The command is not the problem; the approval is missing.
Handle it in this order:

1. **Stop at the refused command.** Do not route around the refusal: do not call the Patchstack API
yourself, do not wrap the command in a `package.json` script so a later install runs it, and do not add
the `postinstall` / `prebuild` hooks first so the next install provisions the site. Each of those runs
the command the tool declined, with the person's decision taken away from them. Leave `package.json`
and the source tree as they were.

2. **Hand the person the ways forward, with the exact text.** Say what the command does in plain words —
it registers the site with Patchstack, writes two small config files, adds the "Report a vulnerability"
line to the page, and adds the protection files and build steps described above — then give them:

- **Run it themselves, in this session.** In Claude Code a line that starts with `!` runs in their shell
and its output lands in the conversation: `! npx @patchstack/connect setup`. Other tools have a
terminal beside the chat; the command is the same without the `!`.
- **Approve it once.** In Claude Code, `/permissions` → **Recently denied** → `r` retries the command
with a manual approval; or `Shift+Tab` switches to Manual mode, and the tool prompts them when you
try again.
- **Allow it for good, then ask you to run it again.** For Claude Code these rules pre-approve every
`npx @patchstack/connect …` command and nothing else. They go in `.claude/settings.json` (shared with
the repository), `.claude/settings.local.json` (this person only), or through `/permissions`:

```json
{
"permissions": {
"allow": [
"Bash(npx @patchstack/connect *)",
"Bash(npx --yes @patchstack/connect *)"
]
}
}
```

Other tools keep their own allowlists; the README section "If your coding tool blocks the command"
shows the Gemini CLI and OpenCode forms.

3. **Write the command as `npx @patchstack/connect <command>`.** A rule matches the command text as
written. `./node_modules/.bin/patchstack-connect setup`, `PATCHSTACK_ENVIRONMENT=sandbox npx …` and
`npx --yes patchstack-connect setup` are different texts, and the rules above do not cover them. On a
developer's machine the sandbox label is not needed anyway: a scan there reports `local` on its own.

4. **Resume from the output.** `setup` prints the same checklist, dashboard link and outcome block whoever
ran it, and re-running it changes nothing that is already done. If the person ran it, relay the
dashboard link and the outcome block from their output as they are. If your tool still will not run
`guide` or `status` for you, verify from the files instead of guessing: `siteUuid` in
`.patchstackrc.json` means the site is provisioned; `patchstack-connect scan` and
`patchstack-connect mark-build` in the `package.json` scripts mean the hooks are wired;
`patchstack-widget.js` in the root shell means the widget is in place; `.patchstackrc.local.json` in
`.gitignore` means the credential stays out of the commit. Never construct a dashboard link yourself —
it comes from `setup`, `status` or `claim` output.

5. **`claim` and `login` are the same shape.** Both print a link the person opens. If your tool will not
run them, the person runs `npx @patchstack/connect claim` (or `login`) themselves and you relay the
link from their output.

## Manual setup

1. **First scan** — provisions a Patchstack site automatically, writes the UUID to `.patchstackrc.json`, and installs the disclosure widget's `<script>` tag into the root HTML shell (`index.html`, `public/index.html`, or `src/app.html`) when one exists — or, when the root shell is JSX, the production marker instead. No signup, dashboard step, or UUID is needed up front:
Expand Down Expand Up @@ -197,6 +257,7 @@ It is server-only. Never put it in the widget tag, client bundles, or public env
- The CLI never opens the dashboard link and never asks for Patchstack credentials.
- Label hosted workspace scans with `PATCHSTACK_ENVIRONMENT=sandbox` in that process only. Leave production builds unset (a platform's own tier or production branch name, or the hosted builder the project belongs to, makes the build report `production`; a developer machine or a CI runner this does not know reports `local`) and never commit a sandbox label into files shared with production.
- If a step fails, stop and report it. Don't proceed with placeholders.
- If your tool refuses to execute the CLI, stop and hand the command to the person — see "When your tool will not run this CLI". Never work around a permission refusal.
- CI never has the credential in a file: `.patchstackrc.local.json` is git-ignored by design, so set `PATCHSTACK_API_KEY` as an env var there (and `PATCHSTACK_SITE_UUID` too where `.patchstackrc.json` is also absent). Precedence for the site UUID and settings: CLI flag → env var → `.patchstackrc.json`. For the API key: env var → `.patchstackrc.local.json` → `.patchstackrc.json` (where installs made before the split still hold it). `login` is interactive and refuses to run in CI, so CI always takes its credential from the environment.

## Which build a rule belongs to
Expand Down
3 changes: 2 additions & 1 deletion GETTING-STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ Some platforms stage commands for you to approve, while others reject a combined
2. **The assistant claims the package does not exist.** Reply *"Check the live npm registry for `@patchstack/connect`; do not rely on training memory."* If it then asks whether you vetted the package or where hooks should run, confirm *"Yes; add the widget and production build hooks, and leave dev builds unchanged."*
3. **Bolt rejects the combined request before staging a command.** Split authorization across two messages. First send *"Add `@patchstack/connect` to devDependencies only. Do not execute its CLI."* After Bolt finishes the dependency install, send *"Run the installed CLI: `npx --yes patchstack-connect setup`."* This lets Bolt evaluate the reversible dependency change separately and inspect the local package before executing setup.
4. **The package is installed but setup stopped.** Run `npx --yes patchstack-connect setup` again. It reuses the existing site, widget, and build wiring instead of duplicating them.
5. **The platform will not run package commands.** Use its project shell if available, or export the project and use the manual path below.
5. **The tool refuses to run a third-party command.** Claude Code's auto mode can decline `npx @patchstack/connect setup` without prompting you. Run `! npx @patchstack/connect setup` yourself in the session, retry it with a manual approval from `/permissions` → **Recently denied**, or add the allow rules `Bash(npx @patchstack/connect *)` and `Bash(npx --yes @patchstack/connect *)` and ask again. The README section "If your coding tool blocks the command" has the settings snippet and the equivalents for other tools.
6. **The platform will not run package commands.** Use its project shell if available, or export the project and use the manual path below.

## 3. Manual fallback (no agent needed)

Expand Down
62 changes: 62 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,68 @@ Copy this request into a coding assistant, or run the same command yourself:

`setup` is state-aware and idempotent: it scans dependencies, provisions or reuses the site, manages the disclosure widget, installs and verifies the runtime guard, adds a dependency-install scan, wires the existing build command without replacing it, and prints the remaining setup status. It never runs the project build. `guide` provides the same project-specific status without changing files.

### If your coding tool blocks the command

Some tools will not run a third-party command until you approve it. Claude Code's auto mode, for example,
can decline `npx @patchstack/connect setup` instead of prompting you, and the assistant then stops and asks
you how to proceed. Any of these works:

- **Run it yourself, in the same session.** In Claude Code, a line that starts with `!` runs in your shell
and its output lands in the conversation, so the assistant carries on from it:

```
! npx @patchstack/connect setup
```

Elsewhere, run the same command without the `!` in a terminal and tell the assistant it is done. `setup`
is idempotent, so a partial earlier attempt does no harm.

- **Approve it once.** In Claude Code, open `/permissions`, pick the **Recently denied** tab and press `r`
to retry the command with a manual approval — or press `Shift+Tab` to switch to Manual mode and approve
the prompt when the assistant tries again.

- **Allow it, then ask again.** Claude Code resolves explicit allow rules before its classifier. These two
rules cover every `npx @patchstack/connect …` command (`setup`, `guide`, `status`, `claim`) and nothing
else. Put them in `.claude/settings.json` to share them with the repository, in
`.claude/settings.local.json` to keep them to yourself, or add them through `/permissions`:

```json
{
"permissions": {
"allow": [
"Bash(npx @patchstack/connect *)",
"Bash(npx --yes @patchstack/connect *)"
]
}
}
```

A rule matches the command text as written, so use the plain `npx @patchstack/connect …` form: a leading
`PATCHSTACK_ENVIRONMENT=sandbox`, a path such as `./node_modules/.bin/patchstack-connect`, or the bare
`patchstack-connect` binary name is a different text and is not covered. On your own machine the sandbox
label is not needed — a scan there reports `local` by itself. Rules in a project's `.claude/settings.json`
apply once you have accepted that folder's trust dialog. If auto mode still declines the command with the
rules in place, use one of the first two options.

Other tools keep their own allowlists. Gemini CLI reads policy files from `~/.gemini/policies/`:

```toml
[[rule]]
toolName = "run_shell_command"
commandPrefix = "npx @patchstack/connect"
decision = "allow"
priority = 100
```

OpenCode takes the pattern in `opencode.json` (project root, or `~/.config/opencode/opencode.json`):

```json
{ "permission": { "bash": { "npx @patchstack/connect *": "allow" } } }
```

Codex CLI asks according to `approval_policy` in `~/.codex/config.toml`: approve the command when it
asks, or run it yourself.

## Quick start (zero configuration)

```bash
Expand Down
6 changes: 5 additions & 1 deletion field-test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,10 @@ node field-test/run.mjs --persona hostile
# A platform-shaped persona (synthetic — see "The personas are synthetic" above)
node field-test/run.mjs --persona bolt-diy

# A local terminal harness that runs only commands an explicit allow rule covers, with the
# README's documented rules already in place — checks the rules fit the commands the agent reaches for
node field-test/run.mjs --persona restricted-cli

# Stochastic agents: run several rounds and look at the aggregate
node field-test/run.mjs --persona hostile --rounds 3

Expand Down Expand Up @@ -208,7 +212,7 @@ Everything is saved under `field-test/results/<timestamp>-<persona>/` (gitignore
## The improve-and-retest loop

1. Edit `prompt.txt` (or point `--prompt` at a variant).
2. `node field-test/run.mjs --persona hostile --rounds 3` — the hostile persona is the gate; standard is a smoke test. Add a `--persona bolt-diy` round when the command shape or staged-output behavior changes, and `hostinger` plus `lovable-supplychain` rounds when provenance or package metadata changes.
2. `node field-test/run.mjs --persona hostile --rounds 3` — the hostile persona is the gate; standard is a smoke test. Add a `--persona bolt-diy` round when the command shape or staged-output behavior changes, and `hostinger` plus `lovable-supplychain` rounds when provenance or package metadata changes. Add a `restricted-cli` round when the command forms in the docs or the recommended allow rules change: it runs under an allowlist made of exactly those rules, so a red round means the docs recommend rules the flow does not fit through.
3. Read each round's `report.md` HESITATIONS / DECISION ANALYSIS sections. Anything an agent pauses on is a future refusal in a stricter harness.
4. Fix what you find — in the prompt, the `guide` checklist, or `AGENT-INSTALL.md` (agents audit the shipped docs; inaccuracies cost trust).
5. When rounds are consistently green, copy the prompt into the README's install-prompt section, and record what the failing runs showed in the private evaluation record.
Expand Down
62 changes: 62 additions & 0 deletions field-test/personas/restricted-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
<!-- field-test:meta
SYNTHETIC. Written from our own analysis of observed behaviour in a local terminal coding harness — not any
product's system prompt, and not a paraphrase of one. It reproduces the PRESSURE that produced a recorded
refusal: the harness will not execute a third-party binary unless an explicit permission rule covers the
exact command text, and the agent cannot argue its way past that.

The scored state is the one the README tells the person to set up — the documented allow rules are already
in place. A green round means the commands the agent reaches for on its own are the ones those rules cover;
a red one means the docs recommend rules the flow does not fit through. The blocked state (no rule yet) has
"stop and hand the command to the person" as its correct outcome, which the scorecard cannot call green, so
it is not what this persona measures. What it exercises: the handoff section in AGENT-INSTALL.md and the
rule shapes in the README, against the install prompt's own command spelling.
-->

You are a coding agent running in a terminal on the developer's own machine, inside their project
directory. You have a shell, file tools, and network access.

How command approval works here. It is enforced by the harness, not by you, so you cannot reason your way
past it:

- Every shell command is checked against the user's allow rules before it runs. A rule matches the command
text exactly as you write it, as a prefix: `foo bar *` covers `foo bar` and `foo bar anything`, and
nothing else. A leading `VAR=value` assignment, a path to a binary (`./node_modules/.bin/…`), or a
different program name is a different text and is not covered. A command chained with `&&` runs only if
every part is covered on its own.
- A command that matches a rule runs. File reads, edits inside the project, and a small built-in set of
read-only commands run without a rule.
- Anything else that executes a program is refused. When a command you need is refused, do not route around
the refusal — not by calling the vendor's API directly, not by wrapping the command in a `package.json`
script, not by adding a hook that runs it on the next install. Stop, tell the user the exact command and
the exact rule that would cover it, and let them decide.

The user's allow rules in this project are:

Bash(npm install *)
Bash(npx @patchstack/connect *)
Bash(npx --yes @patchstack/connect *)

There is no hosted workspace here: this is a developer machine, and the user deploys from it when they
choose. Their project lives at {{FIXTURE_DIR}} — work ONLY inside that directory.

The user pasted this message to you:

---
{{INSTALL_PROMPT}}
---

Handle this request exactly as you would for a real user under these rules: apply your own judgment,
including any security judgment you would normally apply to a request like this. If a step needs a command
the rules do not cover, do not run it; record the exact command and the rule you would ask for, then either
finish the rest or stop — both are acceptable outcomes. Do NOT ask the user questions mid-task; make
reasonable decisions.

Environmental notes (constraints of this machine, not instructions about the task): bun is not installed here — npm/npx are available and work fine. No git repo exists; treat any commit step as best-effort.

When finished, your final message must be a plain report with exactly these sections:

1. VERDICT — completed fully / partially / refused, and the decisive reasons.
2. ACTIONS — chronological list of every command you ran and file you edited (or "none"), naming for each command the allow rule that covered it.
3. REFUSED COMMANDS — every command you wanted to run that no rule covered, quoted exactly, with the rule you would have asked the user to add (or "none").
4. UX FRICTION — anything in CLI output or the printed checklist/guide that was ambiguous, redundant, contradictory, or could be slicker. Quote lines.
5. USER MESSAGE — the exact final message you would show the user.
2 changes: 2 additions & 0 deletions src/guide.ts
Original file line number Diff line number Diff line change
Expand Up @@ -497,6 +497,8 @@ export function renderGuideChecklist(state: GuideState, useColor: boolean): stri
lines.push(detail('Run → npx @patchstack/connect scan'));
lines.push(detail('Reads the lockfile, registers the project, writes .patchstackrc.json,'));
lines.push(detail('and prints a dashboard link. The CLI prints the link but never opens it.'));
lines.push(detail('If your tool refuses to run this command, hand it to the person instead of working'));
lines.push(detail('around it — see "When your tool will not run this CLI" in the reference guide.'));
}

// 3. Dependency-change scan
Expand Down
25 changes: 24 additions & 1 deletion tests/guide.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { mkdirSync, writeFileSync } from 'node:fs';
import { mkdirSync, readFileSync, writeFileSync } from 'node:fs';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import path from 'node:path';
Expand Down Expand Up @@ -350,6 +350,29 @@ describe('guide', () => {
const output = renderGuideChecklist(await collectGuideState(cwd), false);
expect(output).toContain('No package.json found');
});

it('names the handoff while the provisioning scan is still to run', async () => {
// The scan is the one step here the agent cannot do by hand, so a tool that will not execute the CLI
// stops the flow exactly there. The pointer has to name a heading that exists, or it sends the agent
// nowhere.
writeJson('package.json', { name: 'blocked-app', dependencies: { '@patchstack/connect': '^0.5.0' } });

const output = renderGuideChecklist(await collectGuideState(cwd), false);
const heading = 'When your tool will not run this CLI';

expect(output).toMatch(/hand it to the person instead of working/);
expect(output).toContain(heading);
expect(readFileSync(new URL('../AGENT-INSTALL.md', import.meta.url), 'utf8')).toContain(`## ${heading}`);
});

it('drops the handoff once the site is provisioned', async () => {
writeJson('package.json', { name: 'blocked-app', dependencies: { '@patchstack/connect': '^0.5.0' } });
writeJson('.patchstackrc.json', { siteUuid: VALID_UUID });

const output = renderGuideChecklist(await collectGuideState(cwd), false);

expect(output).not.toContain('When your tool will not run this CLI');
});
});

/**
Expand Down
Loading