diff --git a/.agents/skills/verify-open-pstack b/.agents/skills/verify-open-pstack new file mode 120000 index 00000000..1f803db0 --- /dev/null +++ b/.agents/skills/verify-open-pstack @@ -0,0 +1 @@ +../../.claude/skills/verify-open-pstack \ No newline at end of file diff --git a/.claude/skills/verify-open-pstack/SKILL.md b/.claude/skills/verify-open-pstack/SKILL.md new file mode 100644 index 00000000..771085bc --- /dev/null +++ b/.claude/skills/verify-open-pstack/SKILL.md @@ -0,0 +1,94 @@ +--- +name: verify-open-pstack +description: Verify an Open Pstack PR's exact candidate head in native Claude Code and Codex sessions, retain operator-reviewed real-surface evidence, and publish the exact-SHA live gate. +--- + +# Verify Open Pstack + +## Launch + +This repository-local, non-shipped skill is shared with Codex through `.agents/skills/verify-open-pstack`. Verifier-changing PRs run from a checkout of their reviewed candidate head; all other PRs run from trusted `main`. The trusted parent owns GitHub reads and publication; candidate processes never receive publisher credentials. Its only GitHub writes are one structured evidence comment and `live-gate` status on the exact candidate SHA; it never queues or merges. + +Install development dependencies separately when testing this skill; `verify.sh` does not install at runtime: + +```sh +(cd .claude/skills/verify-open-pstack && bun install --frozen-lockfile --ignore-scripts) +``` + +Then run from the trusted publisher checkout: + +```sh +PR=111 # supplied delivery PR number +SESSION="$(mktemp -d "${TMPDIR:-/tmp}/open-pstack-evidence.XXXXXX")" +EVIDENCE="$SESSION/live" # must not exist yet +.claude/skills/verify-open-pstack/scripts/verify.sh doctor --output "$SESSION/probe" +.claude/skills/verify-open-pstack/scripts/verify.sh run --pr "$PR" --self-test --output "$EVIDENCE" +``` + +Reuse the Claude Code and Codex logins you already have; no extra login or credential source flags. Omit `--self-test` for ordinary plugin verification; it is mandatory for this skill's delivery. + +The publisher revision is recorded separately from the candidate SHA and must equal it when classification selects `project-skill`. Run those PRs from their reviewed candidate head; run all other PRs from trusted `main`. Use a fresh private output directory for every run. The verifier requires an open same-repository PR, pins exact head and base SHAs, classifies immutable Git objects, and rechecks them at phase and publication boundaries. Unknown paths abort; evidence never transfers to another head. + +## Doctor + +The parent doctor checks Bun, git, both native harness CLIs, and required capabilities. There are no credential-source flags or source-directory authentication checks. Candidate-mode doctor checks only candidate-safe capabilities, not publisher authentication. Capability checks do not prove installation or live behavior; missing requirements fail closed. + +Candidates retain real `HOME`, `USER`, and `LOGNAME`. Claude uses normal operator config/login. Codex uses a run-owned `CODEX_HOME` with `auth.json` symlinked to the operator's `${CODEX_HOME:-$HOME/.codex}/auth.json`, resolving the daily home before overriding the environment. If that source file is absent, stop with a plain message that this tool needs file-based Codex auth; do not start login. Refresh writes through the symlink to the single existing login file. + +Temporary, GitHub, and fixture state remain run-owned; Git configuration is disabled or run-owned. This is operational state separation, not an OS security boundary. Do not read, parse, or copy credential contents, register tokens, or check public comments against credentials. No extra login directories are needed. + +## Drive + +Read `features/README.md` and every selected feature document. Runtime instructions, consumed references, shipped tools, and installed assets require coverage on their actual consumers. Shared paths select consumers conservatively. `no runtime change` launches no harness unless `--self-test` is requested. + +Create detached exact-head candidate checkouts and run-owned temporary/fixture state. Launch Claude with normal operator config/login and session-only `claude --plugin-dir /plugins/pstack --settings '{"enabledPlugins":{"pstack@open-pstack":false}}'`. Codex installs the exact-head marketplace/plugin in its run-owned home with `codex plugin marketplace add … --ref ` and `codex plugin add`. Verify candidate plugin and project-skill sources against immutable Git provenance before and after exercise; version strings alone are insufficient. + +Candidate commands and provider descendants use the recorded native-login arrangement, never GitHub publisher credentials. No extra login, implicit timeout, or weaker-model fallback is permitted. If Claude's plugin flags fail to exclude installed pstack or Codex replaces the auth symlink instead of writing through it, stop and reopen the design. + +For every selected feature, exercise changed sub-features on each actual consuming native surface and retain real native tool calls and concrete fixture effects. Asset consumers come from the pinned plugin manifests; the current `plugins/pstack/assets/logo.png` is consumed by the Codex manifest only and therefore requires the Codex installed surface, not an invented Claude asset exercise. Save raw transcripts and artifacts in the private run root. Require operator-reviewed surface, action, expected/observed result, and retained artifact paths. Model self-reports and direct CLI tests are not installed-harness evidence. + +For setup only, snapshot real `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/pstack-models.md` and `CLAUDE.md`, including original absence, before exercise. Restore exact bytes or absence afterwards on success or failure and verify restoration before any publication. Restoration failure stops the run and publishes nothing. + +For `project-skill`, invoke `/verify-open-pstack` in Claude and `$verify-open-pstack` in Codex. Ask the discovered pinned skill to run `verify.sh doctor --candidate --output `. Preserve the invocation and canonical-path doctor output; forbid recursive verification or publication. This does not replace required plugin exercises. + +## Evidence + +Create a fresh private mode-0700 run root. Keep raw receipts, transcripts, artifacts, immutable source/installation provenance, and setup restoration results private through merge. Raw evidence is not a public artifact; after archival, only the operator deletes the named run root. + +Before publication, revalidate retained transcript and artifact hashes. The trusted parent publishes one bounded, structured PR comment for the exact candidate SHA and sets `live-gate` on that same SHA with the comment URL as its target. It never edits the PR body, marks ready or draft, reverses a transition, or performs PR-state compensation. No other PR write is permitted. Head movement requires a fresh run and status on the new exact SHA. + +Re-prompt operator `surface`, `action`, and `observed` values until each is printable ASCII without backtick or angle brackets and at most 160 characters. Publish accepted values verbatim using the shared predicate. Other unsafe fields remain omitted whole. Failure details remain private; never publish raw evidence or claim it is secret-free. Failed setup restoration forbids all publication. + +## Cleanup + +Quit native sessions normally. Verify any setup restoration before publication. Removing the Codex run home removes only its auth symlink, never the operator's target file. Preserve private raw evidence through merge, archive it afterwards, and let the operator delete only the named run root. No credential-copy cleanup or separate `~/.pstack-verify` login directories are needed. Do not kill unrelated processes. + +## Helpers + +- `scripts/verify.sh doctor --output `: parent capability report, no installation/publication. +- `scripts/verify.sh doctor --candidate --output `: candidate-safe self-test probe; no publisher authentication probing. +- `scripts/verify.sh run --pr [--self-test] --output `: supervised exact-head verification; mapped exercises require an interactive terminal. +- `bun run test` and `bun run typecheck`: development checks after a separate frozen dependency install, not live proof. +- `features/registry.json`: maintained path ownership; unknown runtime paths block. + +Read receipts as data, never shell input. Do not put credentials in prompts or public artifacts. + +## Scripts +- scripts/cli.test.ts +- scripts/cli.ts +- scripts/core.test.ts +- scripts/core.ts +- scripts/doctor.ts +- scripts/github.ts +- scripts/harness.test.ts +- scripts/harness.ts +- scripts/io.test.ts +- scripts/io.ts +- scripts/isolation.test.ts +- scripts/isolation.ts +- scripts/provenance.test.ts +- scripts/provenance.ts +- scripts/types.ts +- scripts/verify.sh +- scripts/verify.test.ts +- scripts/verify.ts diff --git a/.claude/skills/verify-open-pstack/bun.lock b/.claude/skills/verify-open-pstack/bun.lock new file mode 100644 index 00000000..d1145521 --- /dev/null +++ b/.claude/skills/verify-open-pstack/bun.lock @@ -0,0 +1,22 @@ +{ + "lockfileVersion": 2, + "configVersion": 1, + "workspaces": { + "": { + "name": "@open-pstack/project-verification", + "devDependencies": { + "bun-types": "1.4.0", + "typescript": "5.9.3", + }, + }, + }, + "packages": { + "@types/node": ["@types/node@26.6.4", "", { "dependencies": { "undici-types": "~8.9.0" } }, "sha512-ldVPDCzj7fsaGZrLB0NuHuTvJcsNasysBAqMolr/cgxrLd1xbqxIr3XJiPnHHJUCxj5sNF1vnRj9aWnrVh5Jcg=="], + + "bun-types": ["bun-types@1.4.0", "", { "dependencies": { "@types/node": "*" } }, "sha512-iIKw23BspnQQYd3prITOBxeUsxBHnwzX6YJfGMuNOZzeNcMmVqzIIVGRm1l69ogaPQmb4wB6BN8mA5bE9YuC5Q=="], + + "typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="], + + "undici-types": ["undici-types@8.9.0", "", {}, "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg=="], + } +} diff --git a/.claude/skills/verify-open-pstack/features/README.md b/.claude/skills/verify-open-pstack/features/README.md new file mode 100644 index 00000000..d3d00939 --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/README.md @@ -0,0 +1,33 @@ +# Maintained feature map + +`registry.json` is the machine-readable ownership map. `core.ts` supports exact paths and terminal `/**` only. Unknown paths fail closed; rename classification includes both paths. Runtime markdown, references, playbooks, hooks, manifests, and scripts are runtime even when their extension looks documentary. + +| Ownership | Required real surfaces | +| --- | --- | +| `skills` | One `skill-invocation:` exercise per selected skill, in both harnesses | +| `shared` | All listed skills, setup, runner, and shipped tools in both harnesses | +| `assets` | Installed UI/resource surface for each actual consumer found in the pinned plugin manifests | +| `setup` | Installed setup/model configuration with byte-exact daily Claude file restoration before publication | +| `runner` | Installed parent invoking real configured provider lanes | +| `tools` | Installed parent invoking the changed shipped CLI | +| `project` | Native discovery/invocation of this verifier in both harnesses | +| `nonRuntime` | Explicit ordinary repository documentation, policy, non-shipped assets, and unit-test paths; no plugin harness launch | + +Asset ownership is consumer-driven, not blanket shared ownership. Inspect the exact candidate's plugin manifests. At present, `plugins/pstack/assets/logo.png` is referenced only by `plugins/pstack/.codex-plugin/plugin.json`, so a logo-only change selects the installed Codex asset surface and does not invent a Claude exercise. New shipped assets must be added to `assets` and must have a real manifest consumer; missing, ambiguous, or unknown consumption fails closed. + +`bootstrap.ts` belongs to `tools`; exercise its actual consumers, `orch` and `watch-pr`, including dependency bootstrapping rather than the unrelated external runner. + +Skill-owned `SKILL.md`, `references/**`, and `playbooks/**` select their skill. Additional executable paths must be registered explicitly. Shared consumed mappings select every dependent skill conservatively. The coverage test inventories tracked plugin files; new unmapped files block CI and runs. Deleted skill surfaces cannot silently disappear from evidence: choose a surviving documented entry point demonstrating the intended removal, or report blocked. + +The feature documents below define sub-features, user entry points, driving instructions, and gotchas. Review each changed sub-feature in the diff; one aggregate observation cannot replace distinct selected skills. Record all changed sub-feature actions in the feature's transcript and artifact. Do not run destructive application work in the maintainer checkout. + +- `skill-invocation.md`: installed native skills, principle leaves, and project-skill self-test. +- `setup.md`: setup/routing/model configuration and byte-exact restoration. +- `isolation.md`: existing native logins, Codex auth symlink, and private run state. +- `runner.md`: parent-to-child provider dispatch and receipts. +- `shipped-tools.md`: orchestration, watch, plan checks, audit, and evidence logging. +- `assets.md`: manifest-declared installed assets on actual consuming harnesses. + +Consumed `AGENTS.md`/`CLAUDE.md` instructions and `tests/skill-collision-repro.sh` select `project-skill`. `.mergify.yml` is explicitly `nonRuntime`. There are no blanket no-runtime exemptions for `.github/**`, `tests/**`, or `scripts/**`: unregistered instruction/enforcement paths fail closed until their ownership is reviewed and mapped. + +Explicit non-runtime repository changes are `no runtime change`. Changes to this verifier always select `project-skill`; they cannot bypass native proof. `--self-test` also selects that exercise, deduplicated in each harness, and is mandatory for this skill's delivery PR. Plugin-runtime proof and local CLI proof remain separate. diff --git a/.claude/skills/verify-open-pstack/features/assets.md b/.claude/skills/verify-open-pstack/features/assets.md new file mode 100644 index 00000000..d4da52ae --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/assets.md @@ -0,0 +1,19 @@ +# Installed assets + +## Sub-features + +Manifest declaration, packaged presence, installed resolution, and the user-visible resource on every actual consuming harness. Determine consumers from the immutable candidate plugin manifests; do not assume all harnesses consume every asset. + +## How to get to it (user POV) + +Install the candidate plugin through each manifest that references the changed asset, then open or inspect the native installed surface that renders or exposes that resource. The current `plugins/pstack/assets/logo.png` is referenced by `plugins/pstack/.codex-plugin/plugin.json` at `interface.logo`; no Claude manifest references it. + +## Driving it with verify-open-pstack + +Inspect the pinned manifests and retain the exact asset-reference evidence. Confirm the referenced path resolves inside the pinned installed plugin, the installed bytes match candidate Git provenance, and the consuming native surface loads the resource. Retain a transcript plus a concrete UI capture or native inspection artifact. Record each actual consumer and why non-consumers were not selected. + +A logo-only change currently requires Codex installed-surface evidence only. If a manifest adds another consumer, exercise that consumer too. If a changed asset has no manifest reference, escapes the plugin, has ambiguous ownership, or cannot be observed on its declared native surface, fail closed rather than substituting unrelated skill, setup, runner, or tool exercises. + +## Gotchas + +Repository file existence alone is not installed behavior. Do not select every skill merely because an asset lives under `plugins/pstack`. Do not claim Claude coverage for the current logo when its manifest does not consume it. Direct image inspection may supplement, but cannot replace, the actual installed consumer observation. diff --git a/.claude/skills/verify-open-pstack/features/isolation.md b/.claude/skills/verify-open-pstack/features/isolation.md new file mode 100644 index 00000000..257ce32c --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/isolation.md @@ -0,0 +1,31 @@ +# Native login and run-state contract + +Reuse the operator's existing Claude Code and Codex logins. No extra login, credential source flags, credential reading/copying/parsing, token registration, or exact-secret public-comment check. This is operational state separation, not an OS security boundary. + +## Trusted parent + +Verifier-changing PRs run from their reviewed candidate head, with publisher revision equal to candidate SHA; all other PRs run from trusted `main`. Record publisher revision separately. GitHub authentication and publication remain exclusively in the parent; candidate processes never receive publisher credentials. + +## Candidate process state + +- Preserve real `HOME`, `USER`, and `LOGNAME`. +- Claude uses normal operator login/config with `claude --plugin-dir /plugins/pstack --settings '{"enabledPlugins":{"pstack@open-pstack":false}}'`. These flags load only the candidate for that session, without persistent installation or settings changes. +- Codex installs the exact-head marketplace with `codex plugin marketplace add … --ref ` and `codex plugin add` in a run-owned `CODEX_HOME`. Symlink its `auth.json` to the operator's `${CODEX_HOME:-$HOME/.codex}/auth.json`, resolving the daily home before overriding the environment. +- If the daily home has no `auth.json`, fail closed: this tool needs file-based Codex auth. Do not read credentials, copy them, or start login. Native refresh writes through the symlink to the single existing login file. +- Keep temporary, GitHub, and fixture state run-owned; disable global/system Git configuration and credential helpers for candidate commands. Provider descendants use the same native-login arrangement, never publisher credentials. + +Do not claim Seatbelt enforcement, source-write protection, Keychain denial, daily-home denial, or OS-backed secrecy. Provenance rechecks detect changed candidate/installed files but are not prevention. + +## Setup boundary + +For setup only, snapshot `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/pstack-models.md` and `CLAUDE.md` before exercise, including original absence. Restore exact bytes or absence afterwards, on success or failure, and verify restoration before any publication. Failed restoration stops the run and publishes nothing. Do not substitute a mock or generated configuration alone for installed-surface evidence. + +## Private evidence and cleanup + +Use a fresh external mode-0700 run root. Retain raw transcripts, artifacts, receipts, and restoration results privately through merge. Quit native sessions normally. Removing the Codex run home removes its auth symlink, never the operator's target file. No copied-credential cleanup or separate `~/.pstack-verify` login directories are needed. + +After merge, archive retained evidence and let the operator delete only the named run root. Never recursively delete an operator directory or kill unrelated processes. + +## Native verification + +Confirm a real Mac session loads only the candidate in Claude, Codex authenticates through the symlink, daily Codex still works after any refresh, and setup restoration is byte-exact. If Codex replaces the symlink or installed pstack leaks into Claude's candidate session, stop and reopen the design rather than improvise. diff --git a/.claude/skills/verify-open-pstack/features/registry.json b/.claude/skills/verify-open-pstack/features/registry.json new file mode 100644 index 00000000..ac211599 --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/registry.json @@ -0,0 +1,53 @@ +{ + "skills": [ + "architect", "arena", "automate-me", "babysit", "blast-radius", "bro", + "create-verification-skill", "deslop", "figure-it-out", "fix-ci", "fix-merge-conflicts", + "get-pr-comments", "how", "interrogate", "maintain-verification-skill", + "make-pr-easy-to-review", "no-comments", "poteto-mode", + "principle-attack-the-premise", "principle-boundary-discipline", "principle-build-the-lever", + "principle-encode-lessons-in-structure", "principle-exhaust-the-design-space", + "principle-experience-first", "principle-fix-root-causes", "principle-foundational-thinking", + "principle-guard-the-context-window", "principle-laziness-protocol", + "principle-make-operations-idempotent", "principle-migrate-callers-then-delete-legacy-apis", + "principle-minimize-reader-load", "principle-model-the-domain", "principle-never-block-on-the-human", + "principle-outcome-oriented-execution", "principle-prove-it-works", + "principle-redesign-from-first-principles", "principle-separate-before-serializing-shared-state", + "principle-sequence-verifiable-units", "principle-subtract-before-you-add", + "principle-test-behavior-not-implementation", "principle-type-system-discipline", + "recall", "reflect", "setup-pstack", "show-me-your-work", "swarm", "tdd", "teach", + "technical-writing", "thermo-nuclear-code-quality-review", "typescript-best-practices", + "unslop", "what-did-i-get-done", "why" + ], + "shared": [ + "plugins/pstack/.claude-plugin/plugin.json", "plugins/pstack/.codex-plugin/plugin.json", + ".claude-plugin/marketplace.json", ".agents/plugins/marketplace.json", + "plugins/pstack/agents/**", "plugins/pstack/hooks/**", + "plugins/pstack/skills/poteto-mode/references/**" + ], + "assets": ["plugins/pstack/assets/**"], + "setup": ["plugins/pstack/skills/setup-pstack/SKILL.md"], + "runner": [ + "plugins/pstack/skills/poteto-mode/scripts/runner/**" + ], + "tools": [ + "plugins/pstack/skills/poteto-mode/scripts/bootstrap.ts", + "plugins/pstack/skills/poteto-mode/scripts/orch/**", + "plugins/pstack/skills/poteto-mode/scripts/watch-pr/**", + "plugins/pstack/skills/poteto-mode/scripts/check-plan.mjs", + "plugins/pstack/skills/poteto-mode/scripts/worktree-audit.sh", + "plugins/pstack/skills/poteto-mode/scripts/package.json", + "plugins/pstack/skills/poteto-mode/scripts/bun.lock", + "plugins/pstack/skills/poteto-mode/scripts/check-plan.tsconfig.json", + "plugins/pstack/skills/show-me-your-work/scripts/log.sh" + ], + "project": [ + ".claude/skills/verify-open-pstack/**", ".agents/skills/verify-open-pstack", + "AGENTS.md", "CLAUDE.md", "tests/skill-collision-repro.sh" + ], + "nonRuntime": [ + "README.md", "CHANGES.md", "LICENSE", "LICENSE.md", "UPSTREAM.md", + ".gitignore", ".mergify.yml", "docs/**", "assets/**", + "plugins/pstack/skills/poteto-mode/scripts/bootstrap.test.ts", + "plugins/pstack/skills/poteto-mode/scripts/check-plan.test.ts" + ] +} diff --git a/.claude/skills/verify-open-pstack/features/runner.md b/.claude/skills/verify-open-pstack/features/runner.md new file mode 100644 index 00000000..c92d34a2 --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/runner.md @@ -0,0 +1,19 @@ +# Runner and provider dispatch + +## Sub-features + +Strict argument validation, parent/provider routing, external child launch, prompt delivery, output and receipt persistence, and failure reporting; inspect the diff for which branches changed. + +## How to get to it (user POV) + +Invoke the installed parent workflow (`poteto-mode`, `arena`, or `swarm` as documented by the candidate) in the fresh native harness. Have that parent dispatch its documented external-provider lane. + +## Driving it with verify-open-pstack + +Read the candidate's `pstack-runner --help` and installed mapping through the native skill. Use a run-owned fixture and fresh output/receipt paths. Exercise all changed provider lanes without changing parent routing. Retain the native parent/child transcript, provider receipt, output, and concrete fixture effect. Confirm provider/model/effort/access mode match the requested route and output belongs to this invocation. Exercise changed invalid-input/failure branches as well. Keep direct runner tests in a separately labeled local-CLI artifact, never as the installed-harness result. + +## Gotchas + +Provider children retain real `HOME`, `USER`, and `LOGNAME` and the native-login arrangement in `isolation.md`: normal Claude config/login with session-only candidate plugin flags, and run-owned Codex config with auth symlinked to the daily login. Temporary and GitHub state remain run-owned; never pass publisher credentials. No credential reading/copying, source flags, or extra login is required. Missing file-based Codex auth fails closed. State separation is not an OS access-denial guarantee. + +Do not add implicit timeouts or weaker-model fallbacks. A model assertion that a child ran is insufficient; inspect the actual receipt and output. Never reuse output from another SHA or run. diff --git a/.claude/skills/verify-open-pstack/features/setup.md b/.claude/skills/verify-open-pstack/features/setup.md new file mode 100644 index 00000000..b209df9d --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/setup.md @@ -0,0 +1,19 @@ +# Setup and model configuration + +## Sub-features + +Installed setup invocation, harness-to-provider mapping, generated routing/model preferences, unavailable-provider handling, and changed configuration references. + +## How to get to it (user POV) + +Invoke `/pstack:setup-pstack` in Claude or `$setup-pstack` in Codex. The installed instructions are the authority for setup inputs. + +## Driving it with verify-open-pstack + +For setup only, snapshot the operator's real `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/pstack-models.md` and `CLAUDE.md` before invocation, preserving exact bytes and whether each file existed. Exercise every changed setup branch with explicit model/provider inputs. Retain the native transcript and generated configuration privately, then invoke a consuming installed skill to observe the configured route. + +Restore both files byte for byte afterwards, including restoring original absence, on success or failure. Verify restoration before any evidence comment or status publication. Restoration failure stops the run and publishes nothing. Retain snapshot and restoration results in the private mode-0700 evidence root through merge. + +## Gotchas + +Claude uses the operator's normal config/login; Codex uses a run-owned home with auth symlinked to the daily login. Do not redirect `HOME`, copy credentials, start login, or use a mock as setup proof. Generated configuration alone does not prove the consuming surface. Setup restoration is the only permitted temporary change to these daily Claude files. diff --git a/.claude/skills/verify-open-pstack/features/shipped-tools.md b/.claude/skills/verify-open-pstack/features/shipped-tools.md new file mode 100644 index 00000000..b3cf91fd --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/shipped-tools.md @@ -0,0 +1,19 @@ +# Shipped CLI tools from installed workflows + +## Sub-features + +`orch` state transitions, `watch-pr` observations, `check-plan` verdict boundaries, read-only worktree audit, and `show-me-your-work` logging. Exercise every tool and branch touched by selected paths; shared package changes select all tools. + +## How to get to it (user POV) + +Invoke the installed `poteto-mode` parent for orchestration/watch/plan/audit, or installed `show-me-your-work` for evidence logging, through each harness's native skill surface. + +## Driving it with verify-open-pstack + +Use the candidate's installed help and documented arguments. Create fixture state/output under the private run root; use an explicitly named safe test PR where remote reads are required. Have the native installed workflow drive the changed tool, then inspect its real structured output and persisted state. For the audit assert no mutation; for logging check the written TSV and escaping; for orchestration check the stored transition; for watch/plan preserve the observed result and verdict. Retain native transcripts and these state artifacts. Label any direct helper invocation `local CLI` separately. Document every touched tool and changed branch in the operator-reviewed observation. + +`bootstrap.ts` changes require its real shipped-tool consumers, `orch` and `watch-pr`, including their clean dependency bootstrap path. They do not select the unrelated external runner. + +## Gotchas + +A mock test is not live proof. Never perform PR writes from a candidate process or edit daily workflow state as a fixture. Do not infer a successful remote read from an empty response. Candidates retain real identity and reuse normal Claude login/config and Codex's symlinked daily auth as described in `isolation.md`; GitHub and temporary state remain run-owned. No credential contents are read or copied and no extra login is started. Candidates never receive publisher credentials. This does not claim OS denial of daily-home or Keychain access. Missing access fails the feature; preserve its private reason and publish only the bounded exact-SHA result, never after failed setup restoration. diff --git a/.claude/skills/verify-open-pstack/features/skill-invocation.md b/.claude/skills/verify-open-pstack/features/skill-invocation.md new file mode 100644 index 00000000..4291a0f3 --- /dev/null +++ b/.claude/skills/verify-open-pstack/features/skill-invocation.md @@ -0,0 +1,21 @@ +# Installed skill invocation + +## Sub-features + +Every registry skill is its own required feature. Include changed arguments, routing, consumed references, helper actions, and removal behavior. Principle leaves are model-invocable, not slash-menu entries. `project-skill` separately verifies this repository-local skill's discovery. + +## How to get to it (user POV) + +In a fresh Claude session invoke `/pstack:`. In Codex use the installed skill selector or `$`. Invoke principle leaves through the model's native skill tool. For self-test invoke `/verify-open-pstack` in Claude and `$verify-open-pstack` in Codex. + +## Driving it with verify-open-pstack + +Launch with real `HOME`, `USER`, and `LOGNAME`, run-owned temporary/GitHub state, and no publisher credentials. Claude uses normal operator login/config with `claude --plugin-dir /plugins/pstack --settings '{"enabledPlugins":{"pstack@open-pstack":false}}'`. Codex installs the exact-head candidate in a run-owned `CODEX_HOME` with `auth.json` symlinked to the daily `${CODEX_HOME:-$HOME/.codex}/auth.json`. Missing file-based auth fails closed. No source flags, credential reading/copying, or extra login; see `isolation.md`. + +For each selected name, use a disposable fixture appropriate to its documented input, invoke the installed surface, and exercise every changed sub-feature. Observe native tool/skill loading and a concrete fixture effect, not the assistant saying it passed. Save the raw transcript and an artifact showing the actual result (diff, generated file, tool receipt, or UI capture) under the private mode-0700 evidence root. Include input, expected result, observed result, and each changed sub-feature in the operator review. + +For `project-skill`, invoke the discovered skill and ask it to run `verify.sh doctor --candidate --output `; candidate mode must not inspect source credentials or probe publisher authentication. Forbid recursive `run` and publication. Preserve the native invocation transcript plus `doctor.json`, checking the canonical executable path and immutable source provenance belong to the pinned project skill. The Codex symlink and Claude canonical directory must both be discovered in fresh sessions. If candidate mode is unavailable, stop; do not substitute trusted-parent doctor. + +## Gotchas + +A generic prompt about a skill is not native invocation evidence. A list of discovered skills is not behavioral proof. Environment/config separation does not establish Seatbelt, daily-home, Keychain, or source-write denial, and no such claim belongs in evidence. If an invocation would merge, queue, release, mutate a PR, or write daily state, use a safe disposable fixture or record failure; do not weaken the gate. Never use the child verifier to publish a comment or status. diff --git a/.claude/skills/verify-open-pstack/package.json b/.claude/skills/verify-open-pstack/package.json new file mode 100644 index 00000000..5d275464 --- /dev/null +++ b/.claude/skills/verify-open-pstack/package.json @@ -0,0 +1,7 @@ +{ + "name": "@open-pstack/project-verification", + "private": true, + "type": "module", + "scripts": { "test": "bun test scripts", "typecheck": "tsc --noEmit --strict" }, + "devDependencies": { "bun-types": "1.4.0", "typescript": "5.9.3" } +} diff --git a/.claude/skills/verify-open-pstack/scripts/cli.test.ts b/.claude/skills/verify-open-pstack/scripts/cli.test.ts new file mode 100644 index 00000000..ea32f423 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/cli.test.ts @@ -0,0 +1,72 @@ +import { expect, test } from 'bun:test'; +import { parse } from './cli.ts'; +import { mkdtemp, readFile, realpath, rm, stat } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { doctor } from './doctor.ts'; +import type { Command } from './io.ts'; +import type { GitHub } from './types.ts'; +import { REPO } from './types.ts'; +import { verify } from './verify.ts'; +import { validateRegistry } from './core.ts'; +import registry from '../features/registry.json'; + +test('publisher revision is recorded independently of candidate proof', async () => { + const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-publisher-revision-'))); + const candidate = 'a'.repeat(40), base = 'b'.repeat(40), publisherRevision = 'c'.repeat(40); + const github: GitHub = { + async pull() { return { number: 111, head: { sha: candidate }, base: { sha: base }, state: 'open', headRepo: REPO }; }, + async files() { return [{ filename: 'README.md' }]; }, + async comment() { return `https://github.com/${REPO}/pull/111#issuecomment-1`; }, + async status() {}, + }; + try { + const receipt = await verify({ pr: 111, selfTest: false, root, publisherRevision, registry: validateRegistry(registry), github, + driver: { async prepare() { throw new Error('Docs must not prepare harnesses'); }, async exercise() { throw new Error('Docs must not exercise harnesses'); } }, persist: async () => {} }); + expect(receipt.sha).toBe(candidate); + expect(receipt.publisherRevision).toBe(publisherRevision); + expect(receipt.publisherRevision).not.toBe(receipt.sha); + } finally { await rm(root, { recursive: true, force: true }); } +}); +test('candidate mode is an explicit doctor-only flag', () => { + expect(parse(['doctor', '--candidate', '--output', '/fresh/probe'])).toEqual({ mode: 'doctor', output: '/fresh/probe', pr: 0, selfTest: false, candidate: true }); + expect(() => parse(['doctor', '--candidate', '--candidate', '--output', '/fresh/probe'])).toThrow('Duplicate option'); + expect(() => parse(['run', '--candidate', '--pr', '111', '--output', '/fresh/run'])).toThrow('Unknown option'); +}); +test('candidate doctor probes only help and versions with private roots already present', async () => { + const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-doctor-'))), calls: string[][] = []; + const run: Command = async (args, options = {}) => { + calls.push(args); + const env = options.env!; + expect(env.HOME === process.env.HOME).toBe(true); + expect(env.USER === process.env.USER).toBe(true); + expect(env.LOGNAME === process.env.LOGNAME).toBe(true); + expect(env.CLAUDE_CONFIG_DIR === process.env.CLAUDE_CONFIG_DIR).toBe(true); + for (const key of ['TMPDIR', 'CODEX_HOME', 'XDG_CONFIG_HOME', 'XDG_CACHE_HOME', 'GH_CONFIG_DIR']) { + expect(env[key]!.startsWith(root + '/probe-home')).toBe(true); + const info = await stat(env[key]!); + expect(info.isDirectory()).toBe(true); + expect(info.mode & 0o777).toBe(0o700); + } + return args.includes('--version') ? 'version' : '--plugin-dir --settings --setting-sources --json local path'; + }; + try { + await doctor(root, run, 'darwin', true); + const report = JSON.parse(await readFile(join(root, 'doctor.json'), 'utf8')); + expect(report.candidate).toBe(true); expect(report.result).toBe('pass'); + expect(calls.every(args => args.includes('--version') || args.includes('--help'))).toBe(true); + expect(calls.some(args => args[0] === 'gh' || args.includes('auth') || args.includes('login'))).toBe(false); + } finally { await rm(root, { recursive: true, force: true }); } +}); +test('run and parent doctor need no credential flags', () => { + expect(parse(['doctor', '--output', '/fresh/probe'])).toEqual({ mode: 'doctor', output: '/fresh/probe', pr: 0, selfTest: false, candidate: false }); + expect(parse(['run', '--pr', '123', '--self-test', '--output', '/fresh/run'])).toEqual({ mode: 'run', output: '/fresh/run', pr: 123, selfTest: true }); + for (const args of [[], ['wat'], ['run', '--pr', '0', '--output', '/tmp/a'], ['run', '--pr', '1.5', '--output', '/tmp/a'], ['run', '--pr', '9007199254740992', '--output', '/tmp/a'], ['doctor', '--pr', '1', '--output', '/tmp/a'], ['doctor', '--output', '/tmp/a', '--output', '/tmp/b'], ['run', '--pr', '1'], ['doctor', '--output', '--self-test'], ['doctor', '--output', '/tmp/a', '--publish']]) expect(() => parse(args)).toThrow(); +}); +test('credential-directory and account options are removed', () => { + for (const option of ['--claude-config', '--codex-home', '--claude-account', '--codex-account']) { + for (const args of [['run', '--pr', '111'], ['doctor'], ['doctor', '--candidate']]) { + expect(() => parse([...args, '--output', '/fresh/run', option, '/operator/config'])).toThrow(`Unknown option: ${option}`); + } + } +}); diff --git a/.claude/skills/verify-open-pstack/scripts/cli.ts b/.claude/skills/verify-open-pstack/scripts/cli.ts new file mode 100644 index 00000000..145ddc7c --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/cli.ts @@ -0,0 +1,63 @@ +import { readFile } from 'node:fs/promises'; +import { join, resolve } from 'node:path'; +import { validateRegistry } from './core.ts'; +import { doctor } from './doctor.ts'; +import { Publisher } from './github.ts'; +import { MacDriver } from './harness.ts'; +import { command, freshRoot, interruption, interruptCommands, save } from './io.ts'; +import { verify } from './verify.ts'; +type Options = { mode: 'doctor'; output: string; pr: number; selfTest: boolean; candidate: boolean } + | { mode: 'run'; output: string; pr: number; selfTest: boolean }; +export function parse(args: string[]): Options { + const mode = args[0]; + if (mode !== 'doctor' && mode !== 'run') throw new Error('Usage: verify.sh doctor [--candidate] --output /fresh/path | run --pr NUMBER [--self-test] --output /fresh/path'); + let output = '', pr = 0, selfTest = false, candidate = false; + const seen = new Set(); + for (let i = 1; i < args.length; i++) { + const arg = args[i]!; + if (seen.has(arg)) throw new Error(`Duplicate option: ${arg}`); seen.add(arg); + if (arg === '--self-test' && mode === 'run') selfTest = true; + else if (arg === '--candidate' && mode === 'doctor') candidate = true; + else if (arg === '--output' || mode === 'run' && arg === '--pr') { + const value = args[++i]; if (!value || value.startsWith('--')) throw new Error(`Missing value: ${arg}`); + if (arg === '--output') output = value; + else { if (!/^[1-9]\d*$/.test(value) || !Number.isSafeInteger(Number(value))) throw new Error('PR must be a positive integer'); pr = Number(value); } + } else throw new Error(`Unknown option: ${arg}`); + } + if (!output || mode === 'run' && !pr) throw new Error('Output and (for run) PR are required'); + if (mode === 'run') return { mode, output, pr, selfTest }; + return { mode, output, pr, selfTest, candidate }; +} +async function main(): Promise { + let root: string | undefined, interrupted: 'SIGINT' | 'SIGTERM' | undefined; + const onInterrupt = (signal: 'SIGINT' | 'SIGTERM'): void => { + interrupted ??= signal; + process.exitCode = interrupted === 'SIGINT' ? 130 : 143; + void interruptCommands(signal).catch(() => { + console.error('Failed to stop active verifier commands; inspect retained evidence.'); process.exitCode = 1; + }); + }; + const onInt = (): void => onInterrupt('SIGINT'), onTerm = (): void => onInterrupt('SIGTERM'); + process.on('SIGINT', onInt); process.on('SIGTERM', onTerm); + try { + const options = parse(process.argv.slice(2)), repository = resolve(import.meta.dir, '../../../..'); + root = await freshRoot(options.output, repository); + if (options.mode === 'doctor') await doctor(root, undefined, undefined, options.candidate); + else { + const publisherRevision = (await command(['git', 'rev-parse', 'HEAD'], { cwd: repository })).trim(); + const registry = validateRegistry(JSON.parse(await readFile(join(import.meta.dir, '../features/registry.json'), 'utf8'))); + const receipt = await verify({ pr: options.pr, selfTest: options.selfTest, root, registry, github: new Publisher(), driver: new MacDriver(), persist: r => save(join(root!, 'receipt.json'), r), publisherRevision }); + console.log(`Pinned ${receipt.sha}: live-gate=${receipt.status}; evidence ${receipt.commentUrl}`); + } + interruption.signal.throwIfAborted(); + console.log(`Retained evidence: ${root}`); + } catch (error) { + const failure = error instanceof Error ? error.message : String(error); + if (root) await save(join(root, 'failure.json'), { result: 'failed', reason: failure }); + console.error(root ? 'Verification failed; inspect retained failure.json.' : 'Verification failed before evidence could be retained.'); + process.exitCode = interrupted === 'SIGINT' ? 130 : interrupted === 'SIGTERM' ? 143 : 1; + } finally { + process.off('SIGINT', onInt); process.off('SIGTERM', onTerm); + } +} +if (import.meta.main) await main(); diff --git a/.claude/skills/verify-open-pstack/scripts/core.test.ts b/.claude/skills/verify-open-pstack/scripts/core.test.ts new file mode 100644 index 00000000..a3e2223e --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/core.test.ts @@ -0,0 +1,193 @@ +import { describe, expect, test } from 'bun:test'; +import { execFileSync } from 'node:child_process'; +import { resolve } from 'node:path'; +import registryData from '../features/registry.json'; +import { classify, completeEvidence, matches, newReceipt, requiredFeatures, requiredHarnesses, validateRegistry } from './core.ts'; +import { type Receipt } from './types.ts'; +import codexManifest from '../../../../plugins/pstack/.codex-plugin/plugin.json'; +import claudeManifest from '../../../../plugins/pstack/.claude-plugin/plugin.json'; + +export const SHA = 'a'.repeat(40), BASE = 'b'.repeat(40); +export const registry = validateRegistry(registryData); +export function receipt(selfTest = false): Receipt { + const r = newReceipt(90, SHA, BASE, selfTest, '/tmp/evidence'); + r.selection = classify([{ filename: 'README.md' }], registry); + return r; +} +export function evidence(r: Receipt): void { + for (const harness of requiredHarnesses(r)) { + r.installations.push({ harness, sha: r.sha, cliVersion: 'test', pluginVersion: '1.5.0', + location: '/tmp/candidate/plugin', home: `/tmp/${harness}`, treeHash: 'c'.repeat(64) }); + for (const feature of requiredFeatures(r, harness)) { + r.observations.push({ harness, feature, surface: 'native skill', action: 'invoke', observed: 'fixture changed', + transcript: '/tmp/log', transcriptHash: 'c'.repeat(64), reviewer: 'operator', + artifacts: [{ path: '/tmp/artifact', sha256: 'd'.repeat(64) }] }); + } + } +} + +describe('registry and ownership', () => { + test('all tracked plugin files are covered, including consumed markdown', () => { + const root = resolve(import.meta.dir, '../../../..'); + const paths = execFileSync('git', ['ls-files', '-z', 'plugins/pstack', '.claude-plugin', '.agents/plugins'], { cwd: root, encoding: 'utf8' }).split('\0').filter(Boolean); + expect(paths.length).toBeGreaterThan(100); + for (const filename of paths) expect(() => classify([{ filename }], registry)).not.toThrow(); + }); + test('validates external registry shape', () => { + for (const value of [null, {}, { ...registry, skills: ['../bad'] }, { ...registry, tools: ['x', 'x'] }, { ...registry, shared: [3] }, { ...registry, assets: undefined }, { ...registry, assets: ['../asset'] }, { ...registry, assets: ['x', 'x'] }]) { + expect(() => validateRegistry(value)).toThrow(); + } + }); + test('terminal prefix cannot match sibling directory', () => { + expect(matches('skills/x/file', 'skills/x/**')).toBe(true); + expect(matches('skills/xyz/file', 'skills/x/**')).toBe(false); + }); + test('markdown instructions are runtime', () => { + for (const filename of ['plugins/pstack/skills/architect/SKILL.md', 'plugins/pstack/skills/architect/references/guide.md']) { + const s = classify([{ filename }], registry); + expect(s.features).toContain('skill-invocation:architect'); + expect(s.noRuntime).toBe(false); + } + }); + test('shared references select every dependent skill and executable surface', () => { + const s = classify([{ filename: 'plugins/pstack/skills/poteto-mode/references/codex-tools.md' }], registry); + expect(s.skills).toEqual([...registry.skills].sort()); + for (const feature of ['setup', 'runner', 'shipped-tools']) expect(s.features).toContain(feature); + }); + test('installed logo changes require only their actual Codex manifest consumer', () => { + expect(codexManifest.interface.logo).toBe('./assets/logo.png'); + expect(JSON.stringify(claudeManifest)).not.toContain('assets/logo.png'); + const r = receipt(); + r.selection = classify([{ filename: 'plugins/pstack/assets/logo.png' }], registry); + expect(r.selection.noRuntime).toBe(false); + expect(r.selection.skills).toEqual([]); + expect(r.selection.features).toEqual(['assets:codex']); + expect(requiredHarnesses(r)).toEqual(['codex']); + expect(requiredFeatures(r, 'claude')).toEqual([]); + expect(requiredFeatures(r, 'codex')).toEqual(['assets:codex']); + expect(() => completeEvidence(r)).toThrow('Missing installation: codex'); + evidence(r); + expect(r.installations.map(i => i.harness)).toEqual(['codex']); + expect(() => completeEvidence(r)).not.toThrow(); + r.observations.push({ ...r.observations[0]!, harness: 'claude' }); + expect(() => completeEvidence(r)).toThrow('Extra installation or evidence'); + }); + test('unknown assets fail closed rather than inheriting logo consumers', () => { + for (const filename of ['plugins/pstack/assets/new.png', 'plugins/pstack/assets/nested/logo.png']) { + expect(() => classify([{ filename }], registry)).toThrow('Unmapped asset consumer'); + expect(() => classify([{ filename: 'README.md', previous_filename: filename }], registry)).toThrow('Unmapped asset consumer'); + } + }); + test('manifest changes retain shared coverage in both harnesses', () => { + for (const filename of ['plugins/pstack/.claude-plugin/plugin.json', 'plugins/pstack/.codex-plugin/plugin.json']) { + const r = receipt(); + r.selection = classify([{ filename }], registry); + expect(requiredHarnesses(r)).toEqual(['claude', 'codex']); + expect(requiredFeatures(r, 'claude')).toContain('setup'); + expect(requiredFeatures(r, 'codex')).toContain('setup'); + } + }); + test('asset self-test requires both harnesses without fictitious Claude asset proof', () => { + const r = receipt(true); + r.selection = classify([{ filename: 'plugins/pstack/assets/logo.png' }], registry); + expect(requiredHarnesses(r)).toEqual(['claude', 'codex']); + expect(requiredFeatures(r, 'claude')).toEqual(['project-skill']); + expect(requiredFeatures(r, 'codex')).toEqual(['assets:codex', 'project-skill']); + evidence(r); + expect(r.observations).toHaveLength(3); + expect(() => completeEvidence(r)).not.toThrow(); + r.observations = r.observations.filter(o => o.harness !== 'claude'); + expect(() => completeEvidence(r)).toThrow('Missing/duplicate evidence: claude/project-skill'); + }); + test('bootstrap changes require its shipped tool consumers, not the unrelated runner', () => { + const s = classify([{ filename: 'plugins/pstack/skills/poteto-mode/scripts/bootstrap.ts' }], registry); + expect(s.noRuntime).toBe(false); + expect(s.features).toEqual(['shipped-tools']); + expect(s.features).not.toContain('runner'); + }); + test('rename and deletion preserve old runtime ownership', () => { + const s = classify([{ filename: 'docs/old.md', previous_filename: 'plugins/pstack/skills/architect/SKILL.md' }], registry); + expect(s.features).toContain('skill-invocation:architect'); + expect(classify([{ filename: 'plugins/pstack/skills/architect/SKILL.md' }], registry).noRuntime).toBe(false); + }); + test('unmapped paths and new executable leaves fail closed', () => { + for (const filename of ['plugins/pstack/new.ts', 'plugins/pstack/skills/architect/scripts/new.ts', '/absolute', '../escape', 'plugins/pstack/skills/new/SKILL.md']) { + expect(() => classify([{ filename }], registry)).toThrow(); + } + }); + test('project-local verification changes cannot bypass their native self-test', () => { + const r = receipt(true); + r.selection = classify([{ filename: '.claude/skills/verify-open-pstack/scripts/verify.ts' }], registry); + expect(r.selection.noRuntime).toBe(false); + expect(r.selection.features).toEqual(['project-skill']); + evidence(r); + expect(() => completeEvidence(r)).not.toThrow(); + }); + test('ordinary repository documentation selects no harnesses', () => { + const s = classify([{ filename: 'README.md' }, { filename: 'docs/guide.md' }], registry); + expect(s.noRuntime).toBe(true); + expect(s.features).toEqual([]); + }); + test('consumed instructions and known verifier enforcement require native project proof', () => { + for (const filename of ['AGENTS.md', 'CLAUDE.md', 'tests/skill-collision-repro.sh']) { + const s = classify([{ filename }], registry); + expect(s.noRuntime).toBe(false); + expect(s.features).toEqual(['project-skill']); + expect(classify([{ filename: 'docs/moved.md', previous_filename: filename }], registry).noRuntime).toBe(false); + } + }); + test('Mergify enforcement is not plugin runtime, including renamed paths', () => { + for (const file of [{ filename: '.mergify.yml' }, { filename: 'docs/moved.md', previous_filename: '.mergify.yml' }]) { + const r = receipt(); + r.selection = classify([file], registry); + expect(r.selection.noRuntime).toBe(true); + expect(requiredHarnesses(r)).toEqual([]); + expect(() => completeEvidence(r)).not.toThrow(); + } + }); + test('unregistered CI, tests, and scripts fail closed instead of bypassing runtime proof', () => { + for (const filename of ['.github/workflows/ci.yml', '.github/pull_request_template.md', 'tests/new-gate.sh', 'scripts/disable-proof.ts']) { + expect(() => classify([{ filename }], registry)).toThrow(); + expect(() => classify([{ filename: 'README.md', previous_filename: filename }], registry)).toThrow(); + } + }); +}); + +describe('receipt and evidence boundaries', () => { + test('validates exact immutable SHAs, PR, and optional publisher revision', () => { + for (const pr of [0, -1, 1.1, NaN]) expect(() => newReceipt(pr, SHA, BASE, false, '/tmp')).toThrow(); + for (const sha of ['main', 'a'.repeat(39), 'A'.repeat(40)]) expect(() => newReceipt(90, sha, BASE, false, '/tmp')).toThrow(); + expect(newReceipt(90, SHA, BASE, false, '/tmp', 'c'.repeat(40)).publisherRevision).toBe('c'.repeat(40)); + for (const revision of ['', 'main', 'c'.repeat(39), 'c'.repeat(64), 'C'.repeat(40)]) expect(() => newReceipt(90, SHA, BASE, false, '/tmp', revision)).toThrow('publisher revision'); + }); + test('ordinary no-runtime has no install requirement', () => expect(() => completeEvidence(receipt())).not.toThrow()); + test('self-test is separate and required in both harnesses', () => { + const r = receipt(true); + expect(() => completeEvidence(r)).toThrow('Missing installation'); + evidence(r); + expect(() => completeEvidence(r)).not.toThrow(); + r.observations.pop(); + expect(() => completeEvidence(r)).toThrow('Missing/duplicate'); + }); + test('each selected skill must have its own observation', () => { + const r = receipt(); + r.selection = classify(registry.skills.slice(0, 2).map(s => ({ filename: `plugins/pstack/skills/${s}/SKILL.md` })), registry); + evidence(r); + expect(() => completeEvidence(r)).not.toThrow(); + r.observations.pop(); + expect(() => completeEvidence(r)).toThrow(); + }); + test('wrong SHA, duplicate and incomplete evidence cannot pass', () => { + for (const damage of [ + (r: Receipt) => { r.installations[0]!.sha = BASE; }, + (r: Receipt) => { r.observations.push(r.observations[0]!); }, + (r: Receipt) => { r.observations[0]!.transcriptHash = ''; }, + (r: Receipt) => { r.observations[0]!.observed = ''; }, + (r: Receipt) => { r.observations[0]!.artifacts = []; }, + (r: Receipt) => { r.failure = 'provider unavailable'; }, + ]) { + const r = receipt(true); evidence(r); damage(r); + expect(() => completeEvidence(r)).toThrow(); + } + }); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/core.ts b/.claude/skills/verify-open-pstack/scripts/core.ts new file mode 100644 index 00000000..8bf4edac --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/core.ts @@ -0,0 +1,89 @@ +import { HARNESSES, REPO, type ChangedFile, type Harness, type Receipt, type Registry, type Selection } from './types.ts'; + +export function validateRegistry(value: unknown): Registry { + if (!value || typeof value !== 'object') throw new Error('Invalid feature registry'); + const r = value as Registry; + for (const key of ['skills', 'shared', 'assets', 'setup', 'runner', 'tools', 'project', 'nonRuntime'] as const) { + if (!Array.isArray(r[key]) || r[key].some(v => typeof v !== 'string' || !v || v.includes('..'))) { + throw new Error(`Invalid registry.${key}`); + } + if (new Set(r[key]).size !== r[key].length) throw new Error(`Duplicate registry.${key}`); + } + if (!r.skills.length || r.skills.some(v => !/^[a-z][a-z0-9-]+$/.test(v))) throw new Error('Invalid skills'); + return r; +} + +// Only a terminal /** is supported: unknown paths cannot match an accidental broad glob. +export function matches(path: string, pattern: string): boolean { + return pattern.endsWith('/**') ? path.startsWith(pattern.slice(0, -2)) : path === pattern; +} + +export function classify(files: ChangedFile[], registry: Registry): Selection { + const paths = [...new Set(files.flatMap(f => [f.filename, ...(f.previous_filename ? [f.previous_filename] : [])]))].sort(); + const skills = new Set(), features = new Set(); + for (const path of paths) { + if (!path || path.startsWith('/') || path.split('/').includes('..')) throw new Error(`Unsafe path: ${path}`); + let covered = false; + if (registry.assets.some(p => matches(path, p))) { + if (path !== 'plugins/pstack/assets/logo.png') throw new Error(`Unmapped asset consumer: ${path}`); + features.add('assets:codex'); covered = true; + } + if (registry.shared.some(p => matches(path, p))) { + registry.skills.forEach(s => skills.add(s)); + ['setup', 'runner', 'shipped-tools'].forEach(f => features.add(f)); + covered = true; + } + for (const [key, feature] of [['setup', 'setup'], ['runner', 'runner'], ['tools', 'shipped-tools'], ['project', 'project-skill']] as const) { + if (registry[key].some(p => matches(path, p))) { features.add(feature); covered = true; } + } + const skill = /^plugins\/pstack\/skills\/([^/]+)\/(.+)$/.exec(path); + if (skill && registry.skills.includes(skill[1]!)) { + const leaf = skill[2]!; + if (leaf === 'SKILL.md' || leaf.startsWith('references/') || leaf.startsWith('playbooks/')) { + skills.add(skill[1]!); covered = true; + } + } + // Runtime ownership always takes precedence over documentation exemptions. + if (!covered && !registry.nonRuntime.some(p => matches(path, p))) throw new Error(`Unmapped path: ${path}`); + } + skills.forEach(s => features.add(`skill-invocation:${s}`)); + return { paths, skills: [...skills].sort(), features: [...features].sort(), noRuntime: !features.size }; +} + +export function newReceipt(pr: number, sha: string, base: string, selfTest: boolean, artifactRoot: string, publisherRevision?: string): Receipt { + if (!Number.isSafeInteger(pr) || pr < 1 || !/^[a-f0-9]{40}$/.test(sha) || !/^[a-f0-9]{40}$/.test(base)) throw new Error('Invalid PR/SHA'); + if (publisherRevision !== undefined && !/^[a-f0-9]{40}$/.test(publisherRevision)) throw new Error('Invalid publisher revision'); + return { schema: 1, repo: REPO, pr, sha, base, selfTest, artifactRoot, ...(publisherRevision ? { publisherRevision } : {}), phase: 'resolve', + selection: { paths: [], skills: [], features: [], noRuntime: false }, installations: [], observations: [], + started: new Date().toISOString(), cleanup: 'Run state retained; operator owns cleanup.' }; +} + +export function requiredFeatures(receipt: Receipt, harness?: Harness): string[] { + const features = [...new Set([...receipt.selection.features, ...(receipt.selfTest ? ['project-skill'] : [])])]; + return harness ? features.filter(feature => feature !== 'assets:codex' || harness === 'codex') : features; +} + +export function requiredHarnesses(receipt: Receipt): Harness[] { + return HARNESSES.filter(harness => requiredFeatures(receipt, harness).length > 0); +} + +export function completeEvidence(receipt: Receipt): void { + const features = requiredFeatures(receipt); + if (!features.length && !receipt.selection.noRuntime) throw new Error('No classification'); + const harnesses = requiredHarnesses(receipt); + for (const harness of harnesses) { + const installs = receipt.installations.filter(i => i.harness === harness && i.sha === receipt.sha); + if (installs.length !== 1 || !installs[0]!.location || !/^[a-f0-9]{64}$/.test(installs[0]!.treeHash) || !installs[0]!.cliVersion || !installs[0]!.pluginVersion || !installs[0]!.home) throw new Error(`Missing installation: ${harness}`); + for (const feature of requiredFeatures(receipt, harness)) { + const records = receipt.observations.filter(o => o.harness === harness && o.feature === feature); + if (records.length !== 1) throw new Error(`Missing/duplicate evidence: ${harness}/${feature}`); + const o = records[0]!; + if (o.reviewer !== 'operator' || !o.surface.trim() || !o.action.trim() || !o.observed.trim() || + !o.transcript || !/^[a-f0-9]{64}$/.test(o.transcriptHash) || !o.artifacts.length || + o.artifacts.some(a => !a.path || !/^[a-f0-9]{64}$/.test(a.sha256))) throw new Error(`Incomplete evidence: ${harness}/${feature}`); + } + } + const observationCount = harnesses.reduce((count, harness) => count + requiredFeatures(receipt, harness).length, 0); + if (receipt.installations.length !== harnesses.length || receipt.observations.length !== observationCount) throw new Error('Extra installation or evidence'); + if (receipt.failure) throw new Error(receipt.failure); +} diff --git a/.claude/skills/verify-open-pstack/scripts/doctor.test.ts b/.claude/skills/verify-open-pstack/scripts/doctor.test.ts new file mode 100644 index 00000000..f04411ca --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/doctor.test.ts @@ -0,0 +1,39 @@ +import { afterEach, expect, test } from 'bun:test'; +import { mkdtemp, readFile, realpath, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { doctor } from './doctor.ts'; +import type { Command } from './io.ts'; + +const roots: string[] = []; +afterEach(async () => { for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); }); +async function fixture(): Promise { + const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-doctor-'))); roots.push(root); + return root; +} +for (const candidate of [false, true]) { + test(`doctor is capabilities-only (candidate=${candidate})`, async () => { + const root = await fixture(), calls: string[][] = []; + const run: Command = async args => { + calls.push(args); + if (args.includes('--version')) return 'version'; + if (args.includes('--help')) return '--plugin-dir --settings --setting-sources --json local path'; + throw new Error(`Unexpected command: ${args.join(' ')}`); + }; + await doctor(root, run, 'darwin', candidate); + expect(calls.every(args => args.includes('--version') || args.includes('--help'))).toBe(true); + expect(calls.some(args => args.includes('auth') || args.includes('login') || args.includes('status'))).toBe(false); + expect(calls.some(args => args[0] === 'gh')).toBe(!candidate); + expect(JSON.parse(await readFile(join(root, 'doctor.json'), 'utf8')).result).toBe('pass'); + }); +} +test('doctor records blocked capability checks', async () => { + const root = await fixture(); + await expect(doctor(root, async () => { throw new Error('Missing binary'); }, 'darwin')).rejects.toThrow('Missing binary'); + expect(JSON.parse(await readFile(join(root, 'doctor.json'), 'utf8')).result).toBe('blocked'); +}); +test('doctor refuses non-Mac live verification before commands', async () => { + const root = await fixture(), calls: string[][] = []; + await expect(doctor(root, async args => { calls.push(args); return ''; }, 'linux')).rejects.toThrow('operator Mac'); + expect(calls).toEqual([]); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/doctor.ts b/.claude/skills/verify-open-pstack/scripts/doctor.ts new file mode 100644 index 00000000..4881304b --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/doctor.ts @@ -0,0 +1,28 @@ +import { mkdir, realpath } from 'node:fs/promises'; +import { join } from 'node:path'; +import { command, isolatedEnv, save, type Command } from './io.ts'; + +export async function doctor(root: string, run: Command = command, platform = process.platform, candidate = false): Promise { + const report: Record = { platform, candidate, date: new Date().toISOString(), skill: await realpath(join(import.meta.dir, '..')), checks: {} }; + const checks = report.checks as Record; + try { + if (platform !== 'darwin') throw new Error('Live gate requires an operator Mac (Darwin)'); + const home = join(root, 'probe-home'); + for (const dir of ['tmp', '.codex', '.config/gh', '.cache']) await mkdir(join(home, dir), { recursive: true, mode: 0o700 }); + const env = isolatedEnv(home); + for (const binary of candidate ? ['bun', 'git', 'claude', 'codex'] : ['bun', 'git', 'claude', 'codex', 'gh']) checks[binary] = (await run([binary, '--version'], { env })).trim(); + const claude = await run(['claude', '--help'], { env }); + for (const flag of ['--plugin-dir', '--settings']) if (!claude.includes(flag)) throw new Error(`Claude isolation missing ${flag}`); + checks.claudeHelp = claude; + for (const args of [['codex', 'plugin', 'marketplace', 'add', '--help'], ['codex', 'plugin', 'add', '--help']]) { + const help = await run(args, { env }); + if (!help.includes('--json')) throw new Error('Codex local plugin installation interface unavailable'); + if (args.includes('marketplace') && !/local path|local or Git|local marketplace/i.test(help)) throw new Error('Codex local marketplace interface unavailable'); + checks[args.join(' ')] = help; + } + report.result = 'pass'; + } catch (error) { + report.result = 'blocked'; report.reason = String(error); + throw error; + } finally { await save(join(root, 'doctor.json'), report); } +} diff --git a/.claude/skills/verify-open-pstack/scripts/github.ts b/.claude/skills/verify-open-pstack/scripts/github.ts new file mode 100644 index 00000000..a6d452c6 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/github.ts @@ -0,0 +1,55 @@ +import { mkdtemp, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { command, type Command } from './io.ts'; +import { REPO, type ChangedFile, type GitHub, type Pull } from './types.ts'; + +export function parseChangedFiles(output: string): ChangedFile[] { + if (output === '') return []; + if (!output.endsWith('\0')) throw new Error('Truncated changed-file diff'); + const fields = output.slice(0, -1).split('\0'), files: ChangedFile[] = []; + for (let i = 0; i < fields.length;) { + const status = fields[i++]; + if (!/^(?:[AMDT]|[RC]\d{1,3})$/.test(status)) throw new Error('Invalid changed-file status'); + const previous = /^[RC]/.test(status) ? fields[i++] : undefined, filename = fields[i++]; + if (!filename || previous === '') throw new Error('Incomplete changed-file diff'); + files.push({ filename, ...(previous !== undefined ? { previous_filename: previous } : {}) }); + } + if (new Set(files.map(f => f.filename)).size !== files.length) throw new Error('Duplicate changed-file diff'); + return files; +} +export class Publisher implements GitHub { + constructor(private run: Command = command) {} + private async api(path: string, args: string[] = [], allowInterrupted = false): Promise { + return JSON.parse(await this.run(['gh', 'api', `repos/${REPO}/${path}`, ...args], { allowInterrupted })); + } + async pull(pr: number): Promise { + const data = await this.api(`pulls/${pr}`) as { number: number; head: { sha: string; repo?: { full_name: string } }; base: { sha: string }; state: string }; + if (data.number !== pr || typeof data.state !== 'string') throw new Error('Invalid PR response'); + return { number: pr, head: { sha: data.head.sha }, base: { sha: data.base.sha }, state: data.state, headRepo: data.head.repo?.full_name ?? '' }; + } + async files(base: string, head: string): Promise { + if (![base, head].every(sha => /^[a-f0-9]{40}$/.test(sha))) throw new Error('Invalid diff SHA'); + const repository = await mkdtemp(join(tmpdir(), 'open-pstack-diff-')); + const env = { PATH: process.env.PATH ?? '', GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null', GIT_CONFIG_NOSYSTEM: '1', GIT_CONFIG_COUNT: '0', GIT_TERMINAL_PROMPT: '0' }; + try { + await this.run(['git', 'init', '--bare', '--template=', repository], { env }); + await this.run(['git', 'fetch', '--no-tags', '--', `https://github.com/${REPO}.git`, base, head], { cwd: repository, env }); + for (const sha of [base, head]) { + if ((await this.run(['git', 'rev-parse', '--verify', `${sha}^{commit}`], { cwd: repository, env })).trim() !== sha) throw new Error('Pinned diff commit unavailable'); + } + const output = await this.run(['git', 'diff', '--no-ext-diff', '--no-textconv', '--name-status', '-z', '--find-renames', `${base}...${head}`, '--'], { cwd: repository, env }); + return parseChangedFiles(output); + } finally { await rm(repository, { recursive: true, force: true }); } + } + async comment(pr: number, body: string): Promise { + if (body.length > 60000) throw new Error('Evidence exceeds comment budget'); + const result = await this.api(`issues/${pr}/comments`, ['--method', 'POST', '-f', `body=${body}`]) as { html_url: string }; + if (typeof result.html_url !== 'string' || !result.html_url.startsWith(`https://github.com/${REPO}/issues/`) && !result.html_url.startsWith(`https://github.com/${REPO}/pull/`)) throw new Error('Missing same-repository evidence comment URL'); + return result.html_url; + } + async status(sha: string, state: 'success' | 'failure', target: string, description: string): Promise { + if (!/^[a-f0-9]{40}$/.test(sha)) throw new Error('Invalid status SHA'); + await this.api(`statuses/${sha}`, ['--method', 'POST', '-f', 'context=live-gate', '-f', `state=${state}`, '-f', `target_url=${target}`, '-f', `description=${description.slice(0, 140)}`], state === 'failure'); + } +} diff --git a/.claude/skills/verify-open-pstack/scripts/harness.test.ts b/.claude/skills/verify-open-pstack/scripts/harness.test.ts new file mode 100644 index 00000000..4242f5b8 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/harness.test.ts @@ -0,0 +1,309 @@ +import { afterEach, describe, expect, test } from 'bun:test'; +import { cp, lstat, mkdtemp, mkdir, readFile, readlink, realpath, rm, stat, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { doctor } from './doctor.ts'; +import { codexInstallation, launch, MacDriver, verifyCodexEnabled, verifyProjectDoctor } from './harness.ts'; +import { newReceipt } from './core.ts'; +import { sourceDigest } from './provenance.ts'; +import { evidence } from './verify.ts'; +import { command, freshRoot, isolatedEnv, retainedFile, treeHash, type Command } from './io.ts'; +const roots: string[] = []; +async function fixture(): Promise { const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-test-'))); roots.push(root); return root; } +const originalEnv = { HOME: process.env.HOME, USER: process.env.USER, LOGNAME: process.env.LOGNAME, + CLAUDE_CONFIG_DIR: process.env.CLAUDE_CONFIG_DIR, CODEX_HOME: process.env.CODEX_HOME }; +afterEach(async () => { + for (const [name, value] of Object.entries(originalEnv)) { + if (value === undefined) delete process.env[name]; else process.env[name] = value; + } + for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); +}); +async function loginFixture(root: string): Promise { + const home = join(root, 'daily-codex'); + await mkdir(home, { mode: 0o700 }); + await writeFile(join(home, 'auth.json'), 'opaque existing login'); + await writeFile(join(home, 'daily-state'), 'do not touch'); + process.env.CODEX_HOME = home; + return join(home, 'auth.json'); +} + +describe('isolated harness boundaries', () => { + test('candidate environment keeps operator identity while isolating provider configuration', () => { + const previous = { HOME: process.env.HOME, USER: process.env.USER, LOGNAME: process.env.LOGNAME, + GH_TOKEN: process.env.GH_TOKEN, CODEX_HOME: process.env.CODEX_HOME, CLAUDE_CONFIG_DIR: process.env.CLAUDE_CONFIG_DIR, + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY, OPENAI_API_KEY: process.env.OPENAI_API_KEY }; + process.env.HOME = '/operator/home'; process.env.USER = 'operator-user'; process.env.LOGNAME = 'operator-login'; + process.env.GH_TOKEN = 'test-publisher-token'; process.env.CODEX_HOME = '/daily/codex'; + process.env.CLAUDE_CONFIG_DIR = '/daily/claude'; process.env.ANTHROPIC_API_KEY = 'daily-anthropic'; process.env.OPENAI_API_KEY = 'daily-openai'; + try { + for (const harness of ['claude', 'codex'] as const) { + const env = isolatedEnv('/run/state', harness); + expect(env.GH_TOKEN).toBeUndefined(); expect(env.GITHUB_TOKEN).toBeUndefined(); + expect(env.ANTHROPIC_API_KEY).toBeUndefined(); expect(env.OPENAI_API_KEY).toBeUndefined(); + expect(env.HOME).toBe('/operator/home'); expect(env.USER).toBe('operator-user'); expect(env.LOGNAME).toBe('operator-login'); + expect(env.GIT_CONFIG_GLOBAL).toBe('/dev/null'); expect(env.TMPDIR).toBe('/run/state/tmp'); + expect(env.CLAUDE_CONFIG_DIR).toBe('/daily/claude'); expect(env.CODEX_HOME).toBe('/run/state/.codex'); + expect(env.GH_CONFIG_DIR).toBe('/run/state/.config/gh'); + expect(JSON.stringify(env)).not.toContain('/daily/codex'); + } + expect(launch('claude', '/run/state', '/candidate')).toEqual(['claude', '--plugin-dir', '/candidate/plugins/pstack', '--settings', '{"enabledPlugins":{"pstack@open-pstack":false}}']); + expect(launch('claude', '/run/state', '/candidate').join(' ')).not.toContain('/run/state'); + delete process.env.CLAUDE_CONFIG_DIR; + expect(isolatedEnv('/run/state', 'claude').CLAUDE_CONFIG_DIR).toBeUndefined(); + expect(launch('codex', '/run/state', '/candidate')).toEqual(['codex']); + } finally { + for (const [name, value] of Object.entries(previous)) { + if (value === undefined) delete process.env[name]; else process.env[name] = value; + } + } + }); + test('candidate environment requires the real operator identity', () => { + const previous = { HOME: process.env.HOME, USER: process.env.USER, LOGNAME: process.env.LOGNAME }; + try { + delete process.env.HOME; + expect(() => isolatedEnv('/run/state', 'claude')).toThrow('Real HOME'); + process.env.HOME = '/operator/home'; delete process.env.USER; + expect(() => isolatedEnv('/run/state', 'claude')).toThrow('USER and LOGNAME'); + process.env.USER = 'operator-user'; delete process.env.LOGNAME; + expect(() => isolatedEnv('/run/state', 'claude')).toThrow('USER and LOGNAME'); + } finally { + for (const [name, value] of Object.entries(previous)) { + if (value === undefined) delete process.env[name]; else process.env[name] = value; + } + } + }); + test('doctor blocks non-Mac and missing isolation interfaces, retaining reasons', async () => { + const root = await fixture(); + const run: Command = async args => args.includes('--version') ? 'version' : ''; + await expect(doctor(root, run, 'linux')).rejects.toThrow('operator Mac'); + expect(JSON.parse(await readFile(join(root, 'doctor.json'), 'utf8')).result).toBe('blocked'); + await expect(doctor(root, run, 'darwin', true)).rejects.toThrow('isolation missing'); + }); + test('doctor probes without installing or reading daily authentication', async () => { + const calls: string[][] = [], root = await fixture(); + const run: Command = async args => { calls.push(args); return args.includes('--version') ? 'version' : '--plugin-dir --settings --setting-sources --json local path'; }; + await doctor(root, run, 'darwin', true); + expect(JSON.parse(await readFile(join(root, 'doctor.json'), 'utf8')).result).toBe('pass'); + expect(calls.every(c => c.includes('--help') || c.includes('--version'))).toBe(true); + }); + test('project self-test accepts real candidate doctor output through canonical workspace paths', async () => { + const root = await fixture(), workspace = join(root, 'workspace'); + const skill = await realpath(join(import.meta.dir, '..')); + await mkdir(join(workspace, '.claude/skills'), { recursive: true }); + await symlink(skill, join(workspace, '.claude/skills/verify-open-pstack')); + const alias = join(root, 'workspace-alias'); await symlink(workspace, alias); + const run: Command = async args => args.includes('--version') ? 'version' : '--plugin-dir --settings --setting-sources --json local path'; + await doctor(root, run, 'darwin', false); + const parentText = await readFile(join(root, 'doctor.json'), 'utf8'); + await expect(verifyProjectDoctor([parentText], workspace)).rejects.toThrow('passing child doctor'); + await doctor(root, run, 'darwin', true); + const text = await readFile(join(root, 'doctor.json'), 'utf8'); + expect(JSON.parse(text).candidate).toBe(true); expect(JSON.parse(text).skill).toBe(skill); + await verifyProjectDoctor(['not JSON', text], workspace); + await verifyProjectDoctor([text], alias); + const other = join(root, 'other-workspace'); + await mkdir(join(other, '.claude/skills/verify-open-pstack'), { recursive: true }); + await expect(verifyProjectDoctor([text], other)).rejects.toThrow('passing child doctor'); + await expect(verifyProjectDoctor(['{}'], workspace)).rejects.toThrow('passing child doctor'); + await expect(verifyProjectDoctor([JSON.stringify({ ...JSON.parse(text), result: 'blocked' })], workspace)).rejects.toThrow('passing child doctor'); + }); + + test.each([false, true])('prepare pins local Git candidate and links existing Codex login (install failure=%s)', async failInstall => { + const root = await fixture(), repository = join(root, 'repository'), operatorHome = join(root, 'operator-home'), calls: string[][] = []; + const auth = await loginFixture(root); + for (const manifest of ['.claude-plugin', '.codex-plugin']) { + const dir = join(repository, 'plugins/pstack', manifest); await mkdir(dir, { recursive: true }); + await writeFile(join(dir, 'plugin.json'), JSON.stringify({ version: 'test' })); + } + await mkdir(join(repository, '.claude/skills/verify-open-pstack'), { recursive: true }); + await writeFile(join(repository, '.claude/skills/verify-open-pstack/SKILL.md'), 'pinned project skill'); + await mkdir(join(repository, '.agents/skills'), { recursive: true }); + await symlink('../../.claude/skills/verify-open-pstack', join(repository, '.agents/skills/verify-open-pstack')); + await command(['git', 'init', repository]); + await command(['git', 'add', '.'], { cwd: repository }); + await command(['git', '-c', 'user.name=Fixture', '-c', 'user.email=fixture@example.com', '-c', 'commit.gpgsign=false', 'commit', '-m', 'fixture'], { cwd: repository }); + const sha = (await command(['git', 'rev-parse', 'HEAD'], { cwd: repository })).trim(), actualGit: Command = command; + await mkdir(operatorHome, { mode: 0o700 }); + const platform = Object.getOwnPropertyDescriptor(process, 'platform')!; + const previous = { HOME: process.env.HOME, USER: process.env.USER, LOGNAME: process.env.LOGNAME }; + const run: Command = async (args, options = {}) => { + calls.push(args); + const env = options.env, candidate = Boolean(env?.CODEX_HOME?.startsWith(join(root, 'state') + '/')); + if (candidate) { + const stateHome = join(env!.CODEX_HOME!, '..'); + expect(env!.HOME).toBe(operatorHome); expect(env!.USER).toBe('operator-user'); expect(env!.LOGNAME).toBe('operator-login'); + expect(env!.CLAUDE_CONFIG_DIR).toBe(join(operatorHome, 'claude-config')); expect(env!.CODEX_HOME).toBe(join(stateHome, '.codex')); + expect(env!.TMPDIR).toBe(join(stateHome, 'tmp')); expect(env!.GH_CONFIG_DIR).toBe(join(stateHome, '.config/gh')); + expect(env!.GH_TOKEN).toBeUndefined(); expect(env!.GITHUB_TOKEN).toBeUndefined(); + for (const dir of [stateHome, env!.TMPDIR!, env!.CODEX_HOME!, env!.GH_CONFIG_DIR!]) { + const info = await stat(dir); expect(info.isDirectory()).toBe(true); expect(info.mode & 0o777).toBe(0o700); + } + await expect(stat(join(stateHome, '.claude'))).rejects.toThrow('ENOENT'); + await expect(stat(join(stateHome, 'settings.json'))).rejects.toThrow('ENOENT'); + } + if (args[0] === '/usr/bin/script') { + expect(candidate).toBe(true); expect(options.interactive).toBe(true); + const linked = join(env!.CODEX_HOME!, 'auth.json'); + expect((await lstat(linked)).isSymbolicLink()).toBe(true); + expect(await readlink(linked)).toBe(auth); + await writeFile(args[2]!, 'retained raw native evidence'); + throw new Error('fixture native surface reached'); + } + if (args[0] === 'git') { + const local = [...args]; + if (args[1] === 'clone') local[local.length - 2] = repository; + return actualGit(local, options); + } + if (args.includes('--version')) return 'version'; + if (args.includes('--help')) return '--plugin-dir --settings --setting-sources --json local path'; + if (args[0] === 'codex' && args[1] === 'plugin') { + const linked = join(env!.CODEX_HOME!, 'auth.json'); + expect((await lstat(linked)).isSymbolicLink()).toBe(true); + expect(await readlink(linked)).toBe(auth); + if (failInstall) throw new Error('fixture installation failed'); + if (args[2] === 'marketplace') return '{}'; + if (args[2] === 'add') { + const installedPath = join(env!.CODEX_HOME!, 'plugins/pstack'); + await cp(join(env!.CODEX_HOME!, '../workspace/plugins/pstack'), installedPath, { recursive: true }); + return JSON.stringify({ name: 'pstack', marketplaceName: 'open-pstack', installedPath }); + } + if (args[2] === 'list') return JSON.stringify({ installed: [{ name: 'pstack', marketplaceName: 'open-pstack', installed: true, enabled: true }] }); + } + throw new Error(`Unexpected fixture command: ${args.join(' ')}`); + }; + const driver = new MacDriver(run, async () => ''); + try { + Object.defineProperty(process, 'platform', { ...platform, value: 'darwin' }); + process.env.HOME = operatorHome; process.env.USER = 'operator-user'; process.env.LOGNAME = 'operator-login'; + process.env.CLAUDE_CONFIG_DIR = join(operatorHome, 'claude-config'); + const receipt = newReceipt(111, sha, sha, true, root), preparation = driver.prepare(receipt); + if (failInstall) await expect(preparation).rejects.toThrow('fixture installation failed'); + else { + const installs = await preparation; receipt.installations = installs; + expect(installs.map(i => i.harness)).toEqual(['claude', 'codex']); + expect(installs.every(i => i.sha === sha)).toBe(true); + expect(installs.every(i => /^[a-f0-9]{64}$/.test(i.sourceHash!))).toBe(true); + expect(installs[0]!.sourceHash).toBe(installs[1]!.sourceHash); + expect(calls.some(c => c.join(' ') === 'codex plugin add pstack@open-pstack --json')).toBe(true); + for (const installation of installs) { + const one = newReceipt(111, sha, sha, true, root); one.installations = [installation]; + await expect(driver.exercise(one)).rejects.toThrow('fixture native surface reached'); + } + expect(calls.filter(c => c[0] === '/usr/bin/script')).toHaveLength(2); + } + } finally { + await mkdir(join(root, 'artifacts'), { mode: 0o700 }); + await writeFile(join(root, 'artifacts/private.txt'), 'private raw evidence', { mode: 0o600 }); + for (const harness of ['claude', 'codex']) await writeFile(join(root, 'state', harness, 'native-state.db'), 'retained candidate state'); + await driver.cleanup(); await driver.cleanup(); + Object.defineProperty(process, 'platform', platform); + for (const [name, value] of Object.entries(previous)) { + if (value === undefined) delete process.env[name]; else process.env[name] = value; + } + } + expect(calls.some(c => c.includes('auth') || c.includes('login'))).toBe(false); + expect(calls.some(c => c[0] === '/usr/bin/sandbox-exec')).toBe(false); + expect(await readFile(join(root, 'artifacts/private.txt'), 'utf8')).toBe('private raw evidence'); + expect(await readFile(auth, 'utf8')).toBe('opaque existing login'); + expect(await readFile(join(auth, '../daily-state'), 'utf8')).toBe('do not touch'); + expect((await stat(join(root, 'artifacts'))).mode & 0o777).toBe(0o700); + expect((await stat(join(root, 'artifacts/private.txt'))).mode & 0o777).toBe(0o600); + for (const harness of ['claude', 'codex']) { + const home = join(root, 'state', harness); expect((await stat(home)).isDirectory()).toBe(true); + expect(await readFile(join(home, 'native-state.db'), 'utf8')).toBe('retained candidate state'); + await expect(stat(join(home, '.codex'))).rejects.toThrow('ENOENT'); + await expect(stat(join(home, '.claude'))).rejects.toThrow('ENOENT'); + if (!failInstall) expect(await readFile(join(home, 'surface.raw'), 'utf8')).toBe('retained raw native evidence'); + } + expect(JSON.parse(await readFile(join(root, 'doctor.json'), 'utf8')).result).toBe('pass'); + }); + test.each([['claude', false], ['claude', true], ['codex', false], ['codex', true]] as const)('%s setup interruption restores exact bytes or forbids publication (restore failure=%s)', async (harness, failRestore) => { + const root = await fixture(), home = join(root, 'state', harness), workspace = join(home, 'workspace'); + process.env.HOME = root; + const config = join(root, harness === 'claude' ? 'daily-claude' : '.codex'); process.env.CLAUDE_CONFIG_DIR = join(root, 'daily-claude'); + const instructions = harness === 'claude' ? 'CLAUDE.md' : 'AGENTS.md'; + for (const path of ['plugins/pstack', '.claude/skills/verify-open-pstack', '.agents/skills']) await mkdir(join(workspace, path), { recursive: true }); + await symlink('../../.claude/skills/verify-open-pstack', join(workspace, '.agents/skills/verify-open-pstack')); + await writeFile(join(workspace, 'plugins/pstack/SKILL.md'), 'candidate setup'); + await mkdir(config); + const original = Buffer.from([0, 13, 10, 255, 97]); + await writeFile(join(config, instructions), original); + if (harness === 'codex') await writeFile(join(config, 'pstack-models.md'), original); + const receipt = newReceipt(111, 'a'.repeat(40), 'b'.repeat(40), false, root); + receipt.selection = { paths: [], skills: [], features: ['setup'], noRuntime: false }; + receipt.installations = [{ harness, home, location: join(workspace, 'plugins/pstack'), sha: receipt.sha, cliVersion: 'test', pluginVersion: 'test', treeHash: await treeHash(join(workspace, 'plugins/pstack')), sourceHash: await sourceDigest(workspace) }]; + const driver = new MacDriver(async args => { + expect(args[0]).toBe('/usr/bin/script'); + await writeFile(join(config, instructions), 'changed by setup'); + await writeFile(join(config, 'pstack-models.md'), 'created by setup'); + if (harness === 'codex') await writeFile(join(config, 'CLAUDE.md'), 'created by setup'); + if (failRestore) { await rm(join(config, instructions)); await mkdir(join(config, instructions)); } + throw new Error('interrupted setup'); + }, async () => ''); + await expect(driver.exercise(receipt)).rejects.toThrow(failRestore ? 'Setup restoration requires a regular file' : 'interrupted setup'); + if (failRestore) { + await expect(driver.cleanup()).rejects.toThrow('Setup restoration requires a regular file'); + await rm(join(config, instructions), { recursive: true }); + await expect(driver.cleanup()).rejects.toThrow('Setup restoration failed; publication forbidden'); + return; + } + await driver.cleanup(); + expect(await readFile(join(config, instructions))).toEqual(original); + if (harness === 'codex') { + expect(await readFile(join(config, 'pstack-models.md'))).toEqual(original); + await expect(stat(join(config, 'CLAUDE.md'))).rejects.toThrow('ENOENT'); + } else await expect(stat(join(config, 'pstack-models.md'))).rejects.toThrow('ENOENT'); + }); + test('exercise re-prompts unsafe evidence and publishes accepted answers verbatim', async () => { + const root = await fixture(), home = join(root, 'state/claude'), workspace = join(home, 'workspace'); + for (const path of ['plugins/pstack', '.claude/skills/verify-open-pstack', '.agents/skills']) await mkdir(join(workspace, path), { recursive: true }); + await symlink('../../.claude/skills/verify-open-pstack', join(workspace, '.agents/skills/verify-open-pstack')); + await writeFile(join(workspace, 'plugins/pstack/SKILL.md'), 'candidate plugin'); + await writeFile(join(root, 'transcript.txt'), 'reviewed native transcript'); + await writeFile(join(root, 'artifact.txt'), 'reviewed effect'); + const receipt = newReceipt(111, 'a'.repeat(40), 'b'.repeat(40), false, root); + receipt.selection = { paths: [], skills: ['architect'], features: ['skill-invocation:architect'], noRuntime: false }; + receipt.installations = [{ harness: 'claude', home, location: join(workspace, 'plugins/pstack'), sha: receipt.sha, cliVersion: 'test', pluginVersion: 'test', treeHash: await treeHash(join(workspace, 'plugins/pstack')), sourceHash: await sourceDigest(workspace) }]; + const long = 's'.repeat(54), answers = ['transcript.txt', long, '`unsafe`', long, '✓', long, 'artifact.txt', 'PASS skill-invocation:architect']; + const questions: string[] = []; + const driver = new MacDriver(async () => '', async question => { questions.push(question); return answers.shift()!; }); + receipt.observations = await driver.exercise(receipt); + expect(receipt.observations[0]!.surface).toBe(long); + expect(questions.filter(question => question.startsWith('Concrete action'))).toHaveLength(2); + expect(questions.filter(question => question.startsWith('Observed assertion'))).toHaveLength(2); + for (const question of questions.slice(1, 6)) expect(question).toContain('printable ASCII, no backtick, < or >, and at most 160 characters'); + const comment = evidence(receipt); + expect(comment).toContain(`surface ${long}; action ${long}; result ${long};`); + expect(comment).not.toContain('[value omitted]'); + expect(answers).toHaveLength(0); + }); + + test('Codex exact installed tree and enabled listing are required', async () => { + const root = await fixture(), plugin = join(root, 'config/plugins/pstack'); + await mkdir(plugin, { recursive: true }); await writeFile(join(plugin, 'SKILL.md'), 'candidate'); + const hash = await treeHash(plugin), receipt = JSON.stringify({ name: 'pstack', marketplaceName: 'open-pstack', installedPath: plugin }); + expect(await codexInstallation(receipt, root, hash)).toBe(plugin); + await expect(codexInstallation(receipt, root, 'bad')).rejects.toThrow('differs'); + await expect(codexInstallation('{}', root, hash)).rejects.toThrow('Unrecognized'); + expect(() => verifyCodexEnabled(JSON.stringify({ installed: [{ name: 'pstack', marketplaceName: 'open-pstack', installed: true, enabled: true }] }))).not.toThrow(); + expect(() => verifyCodexEnabled(JSON.stringify({ installed: [{ name: 'pstack', marketplaceName: 'open-pstack', installed: true, enabled: false }] }))).toThrow('enabled'); + const outside = await fixture(); await writeFile(join(outside, 'SKILL.md'), 'candidate'); + await expect(codexInstallation(JSON.stringify({ name: 'pstack', marketplaceName: 'open-pstack', installedPath: outside }), root, hash)).rejects.toThrow('escaped'); + }); + test('plugin symlinks and evidence outside retained root are rejected', async () => { + const root = await fixture(), outside = await fixture(); + await writeFile(join(outside, 'secret'), 'outside'); await symlink(join(outside, 'secret'), join(root, 'escape')); + await expect(treeHash(root)).rejects.toThrow('symlink'); + await expect(retainedFile(root, 'escape')).rejects.toThrow('within output'); + await writeFile(join(root, 'reviewed.txt'), 'native surface'); + expect((await retainedFile(root, 'reviewed.txt')).sha256).toMatch(/^[a-f0-9]{64}$/); + await mkdir(join(root, 'state')); await writeFile(join(root, 'state/raw'), 'raw'); + await expect(retainedFile(root, 'state/raw')).rejects.toThrow('outside isolated state'); + }); + test('output must be fresh and outside the repository', async () => { + const root = await fixture(), repo = join(root, 'repo'); await mkdir(repo); + await expect(freshRoot(join(repo, 'output'), repo)).rejects.toThrow('outside'); + await expect(freshRoot(root, repo)).rejects.toThrow(); + expect(await freshRoot(join(root, 'output'), repo)).toBe(join(root, 'output')); + }); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/harness.ts b/.claude/skills/verify-open-pstack/scripts/harness.ts new file mode 100644 index 00000000..2a4e3634 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/harness.ts @@ -0,0 +1,158 @@ +import { mkdir, readFile, realpath } from 'node:fs/promises'; +import { join } from 'node:path'; +import { createInterface } from 'node:readline/promises'; +import { requiredFeatures, requiredHarnesses } from './core.ts'; +import { command, interruption, isolatedEnv, retainedFile, save, treeHash, type Command } from './io.ts'; +import { doctor } from './doctor.ts'; +import { assertCodexAuth, linkCodexAuth, removeCodexHome, restoreSetup, snapshotSetup, type SetupSnapshot } from './isolation.ts'; +import { sourceDigest, sourceHash } from './provenance.ts'; +import { EVIDENCE_LIMIT, publishable } from './verify.ts'; +import { HARNESSES, REPO, type Driver, type Harness, type Installation, type Observation, type Receipt } from './types.ts'; + +export type Ask = (question: string) => Promise; +export const ask: Ask = async question => { + if (!process.stdin.isTTY) throw new Error('Operator review requires an interactive terminal'); + const reader = createInterface({ input: process.stdin, output: process.stdout }); + try { return (await reader.question(question + '\n> ', { signal: interruption.signal })).trim(); } finally { reader.close(); } +}; +export function launch(harness: Harness, home: string, workspace: string): string[] { + return harness === 'claude' + ? ['claude', '--plugin-dir', join(workspace, 'plugins/pstack'), '--settings', '{"enabledPlugins":{"pstack@open-pstack":false}}'] + : ['codex']; +} +export async function codexInstallation(output: string, home: string, expected: string): Promise { + const result = JSON.parse(output); + if (result.name !== 'pstack' || result.marketplaceName !== 'open-pstack' || typeof result.installedPath !== 'string') { + throw new Error('Unrecognized Codex installation receipt; isolation cannot be established'); + } + const path = await realpath(result.installedPath); + if (!path.startsWith(await realpath(home) + '/')) throw new Error('Codex installation escaped isolated home'); + if (await treeHash(path, ['skills/poteto-mode/scripts/node_modules']) !== expected) throw new Error('Codex installed tree differs from pinned candidate'); + return path; +} +export function verifyCodexEnabled(output: string): void { + const result = JSON.parse(output); + const plugins = Array.isArray(result.installed) ? result.installed.filter((p: Record) => p.name === 'pstack' && p.marketplaceName === 'open-pstack') : []; + if (plugins.length !== 1 || plugins[0].installed !== true || plugins[0].enabled !== true) throw new Error('Isolated Codex plugin is not installed and enabled'); +} +export async function verifyProjectDoctor(texts: string[], workspace: string): Promise { + const expected = await realpath(join(workspace, '.claude/skills/verify-open-pstack')); + for (const text of texts) { + try { + const d = JSON.parse(text); + if (d.result === 'pass' && d.candidate === true && typeof d.skill === 'string' && await realpath(d.skill) === expected) return; + } catch { /* Other reviewed artifacts need not be doctor reports. */ } + } + throw new Error('Self-test requires the pinned project skill\'s passing child doctor.json'); +} +export class MacDriver implements Driver { + private codexHomes: string[] = []; + private setup?: SetupSnapshot; + private restorationFailed = false; + constructor(private run: Command = command, private review: Ask = ask) {} + async cleanup(): Promise { + if (this.setup) await restoreSetup(this.setup); + if (this.restorationFailed) throw new Error('Setup restoration failed; publication forbidden'); + for (const home of this.codexHomes) await removeCodexHome(home); + this.codexHomes = []; + } + private candidate(home: string): Command { + return (args, options = {}) => this.run(args, { ...options, env: isolatedEnv(home) }); + } + async prepare(receipt: Receipt): Promise { + const root = receipt.artifactRoot; + await doctor(root, this.run); + // Fetch a separate immutable reference for pinned-source comparisons. + const referenceHome = join(root, 'source-home'), reference = join(referenceHome, 'workspace'); + await mkdir(join(referenceHome, 'tmp'), { recursive: true, mode: 0o700 }); + const trustedRun: Command = (args, options = {}) => this.run(args, { ...options, env: isolatedEnv(referenceHome) }); + await trustedRun(['git', 'clone', '--no-checkout', '--', `https://github.com/${REPO}.git`, reference]); + await trustedRun(['git', 'fetch', 'origin', receipt.sha], { cwd: reference }); + await trustedRun(['git', '-c', 'core.hooksPath=/dev/null', 'checkout', '--detach', receipt.sha], { cwd: reference }); + const pinnedSource = await sourceHash(reference, receipt.sha, trustedRun); + if (process.platform !== 'darwin') throw new Error('Live proof requires the operator Mac'); + const installs: Installation[] = []; + for (const harness of requiredHarnesses(receipt)) { + const home = join(root, 'state', harness), workspace = join(home, 'workspace'); + await mkdir(join(home, 'tmp'), { recursive: true, mode: 0o700 }); + for (const dir of ['.codex', '.config/gh', '.cache']) await mkdir(join(home, dir), { recursive: true, mode: 0o700 }); + const env = isolatedEnv(home, harness); + const candidateRun = this.candidate(home); + await candidateRun(['git', 'clone', '--no-checkout', '--', `https://github.com/${REPO}.git`, workspace], { env }); + await candidateRun(['git', 'fetch', 'origin', receipt.sha], { cwd: workspace, env }); + await candidateRun(['git', 'checkout', '--detach', receipt.sha], { cwd: workspace, env }); + if ((await candidateRun(['git', 'rev-parse', 'HEAD'], { cwd: workspace, env })).trim() !== receipt.sha) throw new Error('Candidate checkout SHA mismatch'); + const candidate = join(workspace, 'plugins/pstack'), expected = await treeHash(candidate, ['skills/poteto-mode/scripts/node_modules']); + if (await sourceDigest(workspace) !== pinnedSource) throw new Error('Candidate files differ from trusted pinned source'); + let location = candidate; + await linkCodexAuth(home); + this.codexHomes.push(home); + if (harness === 'codex') { + const added = await candidateRun(['codex', 'plugin', 'marketplace', 'add', workspace, '--json'], { env, cwd: home }); + await save(join(root, 'codex-marketplace.json'), JSON.parse(added)); + const installed = await candidateRun(['codex', 'plugin', 'add', 'pstack@open-pstack', '--json'], { env, cwd: home }); + await save(join(root, 'codex-install.json'), JSON.parse(installed)); + location = await codexInstallation(installed, home, expected); + const listed = await candidateRun(['codex', 'plugin', 'list', '--marketplace', 'open-pstack', '--json'], { env, cwd: home }); + verifyCodexEnabled(listed); + await save(join(root, 'codex-plugins.json'), JSON.parse(listed)); + } + const manifest = JSON.parse(await readFile(join(location, harness === 'claude' ? '.claude-plugin/plugin.json' : '.codex-plugin/plugin.json'), 'utf8')); + if (await sourceDigest(workspace) !== pinnedSource) throw new Error('Pinned source changed during installation'); + installs.push({ harness, sha: receipt.sha, home, location, treeHash: expected, sourceHash: pinnedSource, pluginVersion: manifest.version, + cliVersion: (await candidateRun([harness, '--version'], { env })).trim() }); + } + return installs; + } + async exercise(receipt: Receipt): Promise { + const observations: Observation[] = []; + for (const installation of receipt.installations) { + const { harness, home } = installation, workspace = join(home, 'workspace'); + if (await sourceDigest(workspace) !== installation.sourceHash) throw new Error('Pinned source changed before exercise'); + const env = isolatedEnv(home, harness), candidateRun = this.candidate(home); + const request = { sha: receipt.sha, harness, workspace, features: requiredFeatures(receipt, harness), + featureMap: join(workspace, '.claude/skills/verify-open-pstack/features'), + selfTest: requiredFeatures(receipt, harness).includes('project-skill') ? `Invoke the candidate project skill natively; run doctor --candidate --output "${join(home, 'self-test')}". Do not invoke run, login, or publication recursively.` : false, + isolation: 'Existing operator login; Claude uses session-only candidate plugin settings. Codex uses run-owned configuration with a symlink to existing auth.json. Do not log in. Exercise real requests; quota/auth failures fail closed.' }; + await save(join(receipt.artifactRoot, `${harness}-request.json`), request); + console.log(JSON.stringify(request, null, 2)); + const raw = join(home, 'surface.raw'); + console.log('Exercise every requested feature from the native surface, using the maintained feature map. Exit when done.'); + const setup = requiredFeatures(receipt, harness).some(feature => feature === 'setup' || feature === 'skill-invocation:setup-pstack'); + if (setup) this.setup = await snapshotSetup(); + try { + await candidateRun(['/usr/bin/script', '-q', raw, ...launch(harness, home, workspace)], { cwd: workspace, env, interactive: true }); + } finally { + if (this.setup) { + try { await restoreSetup(this.setup); this.setup = undefined; } + catch (error) { this.restorationFailed = true; throw error; } + } + } + if (harness === 'codex') await assertCodexAuth(home); + if (await treeHash(installation.location, ['skills/poteto-mode/scripts/node_modules']) !== installation.treeHash) throw new Error('Installed tree changed during exercise'); + if (await sourceDigest(workspace) !== installation.sourceHash) throw new Error('Pinned project/plugin source changed during exercise'); + const transcript = await retainedFile(receipt.artifactRoot, await this.review(`Retain a private copy of ${raw} inside output (outside state), review it, and enter its path. Do not publish raw content:`)); + for (const feature of requiredFeatures(receipt, harness)) { + const answers = { surface: '', action: '', observed: '' }; + for (const [field, question] of [['surface', `${harness}/${feature}: native surface/discovery entry point?`], ['action', 'Concrete action exercised?'], ['observed', 'Observed assertion/result (not the model\'s success claim)?']] as const) { + do { answers[field] = await this.review(`${question} Use printable ASCII, no backtick, < or >, and at most ${EVIDENCE_LIMIT} characters.`); } while (!publishable(answers[field])); + } + const { surface, action, observed } = answers; + const artifacts = []; + for (const path of (await this.review('Reviewed artifact paths inside output, one or more separated by commas?')).split(',')) { + artifacts.push(await retainedFile(receipt.artifactRoot, path.trim())); + } + if (feature === 'project-skill') { + const doctors = await Promise.all(artifacts.map(a => readFile(join(receipt.artifactRoot, a.path), 'utf8'))); + await verifyProjectDoctor(doctors, workspace); + } + if (await this.review(`Operator: type PASS ${feature} only after reviewing the native transcript and artifacts; anything else fails.`) !== `PASS ${feature}`) { + throw new Error(`Operator rejected ${harness}/${feature}`); + } + observations.push({ harness, feature, surface, action, observed, + reviewer: 'operator', transcript: transcript.path, transcriptHash: transcript.sha256, artifacts }); + } + } + return observations; + } +} diff --git a/.claude/skills/verify-open-pstack/scripts/io.test.ts b/.claude/skills/verify-open-pstack/scripts/io.test.ts new file mode 100644 index 00000000..7a04301c --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/io.test.ts @@ -0,0 +1,175 @@ +import { afterEach, describe, expect, test } from 'bun:test'; +import { chmod, mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { command, isolatedEnv, retainedFile, save, sha256, treeHash } from './io.ts'; + +const roots: string[] = []; +async function fixture(): Promise { + const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-io-'))); + roots.push(root); + return root; +} +afterEach(async () => { for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); }); + +describe('private values and retained file boundaries', () => { + test('saves canonical private JSON unchanged with private permissions', async () => { + const root = await fixture(), path = join(root, 'receipt.json'); + const evidence = { workspace: '/tmp/sk-review/live', nested: ['Bearer private",}', 'github_pat_sensitive'], + checks: { explanation: 'sk-private' }, count: 2 }; + await save(path, evidence); + expect(await readFile(path, 'utf8')).toBe(JSON.stringify(evidence, null, 2) + '\n'); + expect((await stat(path)).mode & 0o777).toBe(0o600); + }); + + test('keeps private command errors unchanged', async () => { + let failure: unknown; + try { await command(['/bin/sh', '-c', "printf '%s' 'private sk-review diagnostic' >&2; exit 7"]); } + catch (error) { failure = error; } + expect(String(failure)).toBe('Error: /bin/sh failed: private sk-review diagnostic'); + }); + + test('uses real account metadata, run-owned config paths, and normal PATH', () => { + const previous = { HOME: process.env.HOME, USER: process.env.USER, LOGNAME: process.env.LOGNAME, PATH: process.env.PATH, CLAUDE_CONFIG_DIR: process.env.CLAUDE_CONFIG_DIR }; + process.env.HOME = '/Users/operator'; process.env.USER = 'operator-user'; process.env.LOGNAME = 'operator-login'; + process.env.PATH = '/usr/local/bin:/usr/bin:/bin'; + try { + expect(() => isolatedEnv('relative/home')).toThrow('absolute run-owned'); + const env = isolatedEnv('/run/home', 'claude'); + expect(env).toMatchObject({ HOME: '/Users/operator', USER: 'operator-user', LOGNAME: 'operator-login', + PATH: '/usr/local/bin:/usr/bin:/bin', TMPDIR: '/run/home/tmp', XDG_CONFIG_HOME: '/run/home/.config', + XDG_CACHE_HOME: '/run/home/.cache', GH_CONFIG_DIR: '/run/home/.config/gh', + CODEX_HOME: '/run/home/.codex' }); + delete process.env.CLAUDE_CONFIG_DIR; + expect(isolatedEnv('/run/home').CLAUDE_CONFIG_DIR).toBeUndefined(); + process.env.CLAUDE_CONFIG_DIR = '/Users/operator/custom-claude'; + for (const harness of ['claude', 'codex'] as const) { + const inherited = isolatedEnv('/run/home', harness); + expect(inherited.CLAUDE_CONFIG_DIR).toBe('/Users/operator/custom-claude'); + expect(inherited.CODEX_HOME).toBe('/run/home/.codex'); + for (const key of ['GH_TOKEN', 'GITHUB_TOKEN', 'OPENAI_API_KEY', 'ANTHROPIC_API_KEY', 'CLAUDE_CODE_OAUTH_TOKEN']) expect(inherited[key]).toBeUndefined(); + } + expect(env.PATH).not.toContain('/run/home/bin'); + process.env.HOME = 'relative/home'; + expect(() => isolatedEnv('/run/home')).toThrow('Real HOME must be absolute'); + process.env.HOME = '/Users/operator'; delete process.env.USER; + expect(() => isolatedEnv('/run/home', 'codex')).toThrow('Real USER and LOGNAME are required'); + } finally { + for (const name of ['HOME', 'USER', 'LOGNAME', 'PATH', 'CLAUDE_CONFIG_DIR'] as const) { + if (previous[name] === undefined) delete process.env[name]; else process.env[name] = previous[name]; + } + } + }); + + test('retains binary artifacts unchanged without inspecting copied secrets', async () => { + const root = await fixture(), path = join(root, 'artifact.bin'), token = 'known-copied-binary-opaque-credential'; + const bytes = Buffer.concat([Buffer.from([0, 255, 128]), Buffer.from(token), Buffer.from([0, 254])]); + await writeFile(path, bytes); + expect(await retainedFile(root, path)).toEqual({ path: 'artifact.bin', sha256: sha256(bytes) }); + expect(Buffer.from(await readFile(path))).toEqual(bytes); + }); + + test('permits evidence below an ancestor named state and canonicalizes output aliases', async () => { + const root = await fixture(), output = join(root, 'state', 'run'), alias = join(root, 'alias'); + await mkdir(output, { recursive: true }); + await writeFile(join(output, 'transcript.txt'), 'native surface evidence'); + await symlink(output, alias); + expect(await retainedFile(output, 'transcript.txt')).toEqual(await retainedFile(alias, 'transcript.txt')); + expect((await retainedFile(alias, 'transcript.txt')).path).toBe('transcript.txt'); + }); + + test('rejects this run state and aliases into it, but not another directory named state', async () => { + const root = await fixture(); + await mkdir(join(root, 'state')); + await mkdir(join(root, 'artifacts', 'state'), { recursive: true }); + await writeFile(join(root, 'state', 'secret.txt'), 'isolated authentication'); + await writeFile(join(root, 'artifacts', 'state', 'result.txt'), 'retained result'); + await symlink(join(root, 'state', 'secret.txt'), join(root, 'transcript.txt')); + await expect(retainedFile(root, 'state')).rejects.toThrow('outside isolated state'); + await expect(retainedFile(root, 'state/secret.txt')).rejects.toThrow('outside isolated state'); + await expect(retainedFile(root, 'transcript.txt')).rejects.toThrow('outside isolated state'); + expect((await retainedFile(root, 'artifacts/state/result.txt')).path).toBe('artifacts/state/result.txt'); + }); +}); + +describe('candidate tree digests', () => { + test('dependency bootstrap is excluded only at explicitly supplied generated directories', async () => { + const root = await fixture(), scripts = join(root, 'skills', 'poteto-mode', 'scripts'); + await mkdir(scripts, { recursive: true }); + await writeFile(join(scripts, 'bootstrap.ts'), 'export const candidate = 1;'); + const exclusions = ['skills/poteto-mode/scripts/node_modules']; + const pinned = await treeHash(root, exclusions); + const dependencies = join(scripts, 'node_modules'); + await mkdir(dependencies); + await writeFile(join(dependencies, 'generated.js'), 'generated dependency'); + await symlink(join(dependencies, 'generated.js'), join(dependencies, 'package')); + expect(await treeHash(root, exclusions)).toBe(pinned); + await expect(treeHash(root)).rejects.toThrow('symlink refused'); + await writeFile(join(scripts, 'bootstrap.ts'), 'export const candidate = 2;'); + expect(await treeHash(root, exclusions)).not.toBe(pinned); + await expect(treeHash(root, ['skills/poteto-mode/scripts'])).rejects.toThrow('Only explicit generated'); + await expect(treeHash(root, ['**/node_modules'])).rejects.toThrow('Only explicit generated'); + }); + + test('source executable permissions affect the hash and non-excluded symlinks are rejected', async () => { + const root = await fixture(), source = join(root, 'verify.sh'); + await writeFile(source, '#!/bin/sh\nexit 0\n', { mode: 0o644 }); + const pinned = await treeHash(root); + await chmod(source, 0o755); + expect(await treeHash(root)).not.toBe(pinned); + await symlink(source, join(root, 'alias.sh')); + await expect(treeHash(root)).rejects.toThrow('symlink refused'); + const alias = join(await fixture(), 'tree'); + await symlink(root, alias); + await expect(treeHash(alias)).rejects.toThrow('real directory'); + }); +}); + +describe('coordinated process interruption', () => { + test.each(['SIGINT', 'SIGTERM'] as const)('%s kills real process groups and permits only explicit cleanup commands', async signal => { + const root = await fixture(), home = join(root, 'candidate'), credential = join(home, '.claude/.credentials.json'); + const vault = join(root, 'vault.json'), report = join(root, 'report.json'), heartbeat = join(root, 'heartbeat'), ready = join(root, 'ready.json'); + await mkdir(join(home, '.claude'), { recursive: true, mode: 0o700 }); + await writeFile(credential, 'disposable token', { mode: 0o600 }); + await writeFile(vault, 'original vault sentinel', { mode: 0o600 }); + const grandchild = join(root, 'grandchild.ts'), dummy = join(root, 'dummy.ts'), worker = join(root, 'worker.ts'); + await writeFile(grandchild, `let counter = 0; await Bun.write(${JSON.stringify(heartbeat)}, String(counter));\nsetInterval(() => Bun.write(${JSON.stringify(heartbeat)}, String(++counter)), 10);\n`); + await writeFile(dummy, `const child = Bun.spawn([process.execPath, ${JSON.stringify(grandchild)}], { stdout: 'ignore', stderr: 'ignore' });\nwhile (!(await Bun.file(${JSON.stringify(heartbeat)}).exists())) await Bun.sleep(5);\nawait Bun.write(${JSON.stringify(ready)}, JSON.stringify({ pid: process.pid, grandchild: child.pid }));\nsetInterval(() => {}, 1000);\n`); + await writeFile(worker, `import { command, interruption, interruptCommands, save } from ${JSON.stringify(join(import.meta.dir, 'io.ts'))}; +process.on('SIGINT', () => { void interruptCommands('SIGINT'); }); +process.on('SIGTERM', () => { void interruptCommands('SIGTERM'); }); +let failure = '', rejected = false; +try { await command([process.execPath, ${JSON.stringify(dummy)}]); } +catch (error) { + failure = String(error); + try { await command(['/bin/true']); } catch { rejected = true; } +} finally { + await command(['/bin/rm', '-f', '--', ${JSON.stringify(credential)}], { allowInterrupted: true }); + await save(${JSON.stringify(report)}, { failure, rejected, aborted: interruption.signal.aborted }); + process.exitCode = 1; +} +`); + const child = Bun.spawn([process.execPath, worker], { stdout: 'pipe', stderr: 'pipe' }); + const stderr = new Response(child.stderr).text(), stdout = new Response(child.stdout).text(); + try { + for (let attempt = 0; attempt < 500 && !(await Bun.file(ready).exists()); attempt++) await Bun.sleep(5); + expect(await Bun.file(ready).exists()).toBe(true); + child.kill(signal); + expect(await child.exited).toBe(1); + expect(await stderr).toBe(''); expect(await stdout).toBe(''); + expect(JSON.parse(await readFile(report, 'utf8'))).toEqual({ failure: `Error: Interrupted by ${signal}`, rejected: true, aborted: true }); + await expect(stat(credential)).rejects.toThrow('ENOENT'); + expect(await readFile(vault, 'utf8')).toBe('original vault sentinel'); + const stopped = await readFile(heartbeat, 'utf8'); + await Bun.sleep(100); + expect(await readFile(heartbeat, 'utf8')).toBe(stopped); + } finally { + child.kill('SIGKILL'); + if (await Bun.file(ready).exists()) { + const { pid } = JSON.parse(await readFile(ready, 'utf8')); + try { process.kill(-pid, 'SIGKILL'); } catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ESRCH') throw error; } + } + await child.exited; + } + }, 10000); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/io.ts b/.claude/skills/verify-open-pstack/scripts/io.ts new file mode 100644 index 00000000..2434dc77 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/io.ts @@ -0,0 +1,107 @@ +import { createHash } from 'node:crypto'; +import { lstat, mkdir, readFile, readdir, realpath, rename, writeFile } from 'node:fs/promises'; +import { isAbsolute, join, relative, resolve } from 'node:path'; + +export type Command = (args: string[], options?: { cwd?: string; env?: Record; interactive?: boolean; allowInterrupted?: boolean }) => Promise; +export const interruption = new AbortController(); +const activeCommands = new Map>(); +let interrupting: Promise | undefined; +export function interruptCommands(signal: 'SIGINT' | 'SIGTERM'): Promise { + if (interrupting) return interrupting; + interruption.abort(new Error(`Interrupted by ${signal}`)); + interrupting = (async () => { + const children = [...activeCommands]; + for (const [pid] of children) { + try { process.kill(-pid, 'SIGKILL'); } + catch (error) { if ((error as NodeJS.ErrnoException).code !== 'ESRCH') throw error; } + } + await Promise.all(children.map(([, exited]) => exited)); + })(); + return interrupting; +} +export const command: Command = async (args, options = {}) => { + if (!options.allowInterrupted) interruption.signal.throwIfAborted(); + const child = Bun.spawn(args, { cwd: options.cwd, env: options.env, detached: true, + stdin: options.interactive ? 'inherit' : 'ignore', stdout: options.interactive ? 'inherit' : 'pipe', + stderr: options.interactive ? 'inherit' : 'pipe' }); + if (!options.allowInterrupted) activeCommands.set(child.pid, child.exited); + try { + const [output, error] = options.interactive ? ['', ''] : await Promise.all([ + new Response(child.stdout).text(), new Response(child.stderr).text(), + ]); + const code = await child.exited; + if (!options.allowInterrupted) interruption.signal.throwIfAborted(); + if (code !== 0) throw new Error(`${args[0]} failed: ${error}`); + return output; + } finally { + activeCommands.delete(child.pid); + } +}; +export function isolatedEnv(state: string, _harness?: 'claude' | 'codex'): Record { + if (!isAbsolute(state)) throw new Error('Candidate state must be an absolute run-owned directory'); + const { HOME, USER, LOGNAME } = process.env; + if (!HOME || !isAbsolute(HOME)) throw new Error('Real HOME must be absolute'); + if (!USER || !LOGNAME) throw new Error('Real USER and LOGNAME are required'); + return { PATH: process.env.PATH ?? '', HOME, USER, LOGNAME, + TMPDIR: join(state, 'tmp'), TERM: process.env.TERM ?? 'xterm-256color', + XDG_CONFIG_HOME: join(state, '.config'), XDG_CACHE_HOME: join(state, '.cache'), + GH_CONFIG_DIR: join(state, '.config/gh'), + GIT_CONFIG_GLOBAL: '/dev/null', GIT_CONFIG_SYSTEM: '/dev/null', GIT_CONFIG_NOSYSTEM: '1', + GIT_CONFIG_COUNT: '1', GIT_CONFIG_KEY_0: 'credential.helper', GIT_CONFIG_VALUE_0: '', + ...(process.env.CLAUDE_CONFIG_DIR !== undefined ? { CLAUDE_CONFIG_DIR: process.env.CLAUDE_CONFIG_DIR } : {}), + CODEX_HOME: join(state, '.codex') }; +} +export async function save(path: string, value: unknown): Promise { + const json = JSON.stringify(value, null, 2); + await writeFile(path + '.tmp', json + '\n', { mode: 0o600 }); + await rename(path + '.tmp', path); +} +export async function freshRoot(path: string, repository: string): Promise { + if (!isAbsolute(path)) throw new Error('Output must be an absolute fresh directory outside the repository'); + const parent = await realpath(resolve(path, '..')); + const root = join(parent, path.split('/').at(-1)!); + const repo = await realpath(repository); + if (root === repo || root.startsWith(repo + '/')) throw new Error('Output must be outside the repository'); + await mkdir(root, { mode: 0o700 }); // EEXIST deliberately rejects reuse and symlinks. + return root; +} +export function sha256(data: string | Uint8Array): string { return createHash('sha256').update(data).digest('hex'); } +export async function treeHash(root: string, excludePaths: string[] = []): Promise { + const generated = new Set([ + 'skills/poteto-mode/scripts/node_modules', + 'plugins/pstack/skills/poteto-mode/scripts/node_modules', + '.claude/skills/verify-open-pstack/node_modules', + ]); + if (excludePaths.some(path => !generated.has(path))) throw new Error('Only explicit generated dependency directories may be excluded'); + const excluded = new Set(excludePaths); + const rootStat = await lstat(root); + if (rootStat.isSymbolicLink() || !rootStat.isDirectory()) throw new Error('Plugin tree root must be a real directory'); + const entries: string[] = []; + async function walk(dir: string): Promise { + for (const name of (await readdir(dir)).sort()) { + const path = join(dir, name), entry = relative(root, path); + if (excluded.has(entry)) continue; + const stat = await lstat(path); + if (stat.isSymbolicLink()) throw new Error(`Plugin tree symlink refused: ${path}`); + if (stat.isDirectory()) await walk(path); + else if (stat.isFile()) entries.push(`${entry}\0${stat.mode & 0o111 ? '100755' : '100644'}\0${sha256(await readFile(path))}`); + else throw new Error(`Non-file in plugin tree: ${path}`); + } + } + await walk(root); + if (!entries.length) throw new Error('Empty plugin tree'); + return sha256(entries.join('\n')); +} +export async function retainedFile(root: string, path: string): Promise<{ path: string; sha256: string }> { + const output = await realpath(root); + const actual = await realpath(isAbsolute(path) ? path : join(output, path)); + const state = await realpath(join(output, 'state')).catch((error: NodeJS.ErrnoException) => { + if (error.code !== 'ENOENT') throw error; + return join(output, 'state'); + }); + if (!actual.startsWith(output + '/') || actual === state || actual.startsWith(state + '/')) throw new Error('Evidence must be retained outside isolated state, within output'); + const stat = await lstat(actual); + if (!stat.isFile() || !stat.size) throw new Error('Evidence file must be a nonempty regular file'); + const bytes = await readFile(actual); + return { path: relative(output, actual), sha256: sha256(bytes) }; +} diff --git a/.claude/skills/verify-open-pstack/scripts/isolation.test.ts b/.claude/skills/verify-open-pstack/scripts/isolation.test.ts new file mode 100644 index 00000000..1260e270 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/isolation.test.ts @@ -0,0 +1,143 @@ +import { afterEach, describe, expect, test } from 'bun:test'; +import { lstat, mkdir, mkdtemp, readFile, readlink, realpath, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { assertCodexAuth, linkCodexAuth, removeCodexHome, restoreSetup, snapshotSetup } from './isolation.ts'; + +const roots: string[] = []; +const environment = { HOME: process.env.HOME, CODEX_HOME: process.env.CODEX_HOME, CLAUDE_CONFIG_DIR: process.env.CLAUDE_CONFIG_DIR }; +afterEach(async () => { + for (const [name, value] of Object.entries(environment)) { + if (value === undefined) delete process.env[name]; else process.env[name] = value; + } + for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); +}); +async function fixture() { + const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-isolation-'))); + roots.push(root); + process.env.HOME = root; + process.env.CODEX_HOME = join(root, 'daily-codex'); + process.env.CLAUDE_CONFIG_DIR = join(root, 'daily-claude'); + const home = join(root, 'run'); + for (const directory of [home, process.env.CODEX_HOME, process.env.CLAUDE_CONFIG_DIR]) await mkdir(directory, { mode: 0o700 }); + return { root, home, auth: join(process.env.CODEX_HOME, 'auth.json'), claude: process.env.CLAUDE_CONFIG_DIR }; +} + +describe('existing native login', () => { + test('links opaque auth without parsing or copying it and removes only run-owned Codex state', async () => { + const { home, auth } = await fixture(); + await writeFile(auth, 'opaque non-JSON authentication'); + await linkCodexAuth(home); + const link = join(home, '.codex/auth.json'); + expect((await lstat(link)).isSymbolicLink()).toBe(true); + expect(await readlink(link)).toBe(auth); + await assertCodexAuth(home); + await writeFile(join(home, '.codex/state.db'), 'candidate state'); + await writeFile(join(home, 'surface.raw'), 'private evidence'); + await removeCodexHome(home); + expect(await Bun.file(link).exists()).toBe(false); + expect(await readFile(auth, 'utf8')).toBe('opaque non-JSON authentication'); + expect(await readFile(join(home, 'surface.raw'), 'utf8')).toBe('private evidence'); + }); + + test('defaults to the real HOME Codex login', async () => { + const { root, home } = await fixture(); + delete process.env.CODEX_HOME; + await mkdir(join(root, '.codex')); + await writeFile(join(root, '.codex/auth.json'), 'opaque'); + await linkCodexAuth(home); + expect(await readlink(join(home, '.codex/auth.json'))).toBe(join(root, '.codex/auth.json')); + }); + + test('missing or non-file source auth refuses keyring-only authentication', async () => { + const { home, auth } = await fixture(); + await expect(linkCodexAuth(home)).rejects.toThrow('existing file-based authentication'); + await mkdir(auth); + await expect(linkCodexAuth(home)).rejects.toThrow('keyring-only authentication'); + }); + + test('replaced auth blocks cleanup/publication and preserves the operator source', async () => { + const { home, auth } = await fixture(); + await writeFile(auth, 'operator login'); + await linkCodexAuth(home); + const link = join(home, '.codex/auth.json'); + await rm(link); await writeFile(link, 'replacement'); + await expect(assertCodexAuth(home)).rejects.toThrow('stop and report before publication'); + await expect(removeCodexHome(home)).rejects.toThrow('Codex replaced or redirected auth.json'); + expect(await readFile(auth, 'utf8')).toBe('operator login'); + expect(await readFile(link, 'utf8')).toBe('replacement'); + }); + + test('redirected run-owned Codex directory is refused without touching the source', async () => { + const { home, auth } = await fixture(); + await writeFile(auth, 'operator login'); + await symlink(process.env.CODEX_HOME!, join(home, '.codex')); + await expect(linkCodexAuth(home)).rejects.toThrow('Codex state directory redirected'); + expect(await readFile(auth, 'utf8')).toBe('operator login'); + }); +}); + +describe('setup-only snapshot and restoration', () => { + test('restores both files byte-exact including binary bytes and restores absence', async () => { + const { claude } = await fixture(); + const models = join(claude, 'pstack-models.md'), instructions = join(claude, 'CLAUDE.md'); + const original = Buffer.from([0, 255, 13, 10, 65]); + await writeFile(models, original); + const snapshot = await snapshotSetup(); + await writeFile(models, 'setup mutation'); await writeFile(instructions, 'new setup instructions'); + await restoreSetup(snapshot); + expect(await readFile(models)).toEqual(original); + expect(await Bun.file(instructions).exists()).toBe(false); + }); + + test('restores all default and custom setup targets, including absence', async () => { + const { root, claude } = await fixture(); + const directories = [join(root, '.claude'), join(root, '.codex'), claude, process.env.CODEX_HOME!]; + const original = Buffer.from([0, 255, 13, 10, 65]); + for (const directory of directories) { + await mkdir(directory, { recursive: true }); + for (const name of ['pstack-models.md', 'CLAUDE.md']) await writeFile(join(directory, name), original); + } + const snapshot = await snapshotSetup(); + expect(snapshot).toHaveLength(12); + for (const { path } of snapshot) await writeFile(path, 'setup mutation'); + await restoreSetup(snapshot); + for (const directory of directories) { + for (const name of ['pstack-models.md', 'CLAUDE.md']) expect(await readFile(join(directory, name))).toEqual(original); + expect(await Bun.file(join(directory, 'AGENTS.md')).exists()).toBe(false); + } + process.env.CLAUDE_CONFIG_DIR = directories[0]; process.env.CODEX_HOME = directories[1]; + expect(await snapshotSetup()).toHaveLength(6); + }); + + test('defaults to HOME/.claude and restores deleted existing files', async () => { + const { root } = await fixture(); + delete process.env.CLAUDE_CONFIG_DIR; + const directory = join(root, '.claude'); await mkdir(directory); + const path = join(directory, 'CLAUDE.md'); await writeFile(path, 'original\r\n'); + const snapshot = await snapshotSetup(); + await rm(path); + await restoreSetup(snapshot); + expect(await readFile(path, 'utf8')).toBe('original\r\n'); + }); + + test('rejects symlink snapshots and failed restoration without following redirected files', async () => { + const { root, claude } = await fixture(); + const path = join(claude, 'pstack-models.md'), external = join(root, 'external'); + await writeFile(external, 'untouched'); + await symlink(external, path); + await expect(snapshotSetup()).rejects.toThrow('regular file'); + await rm(path); await writeFile(path, 'original'); + const snapshot = await snapshotSetup(); + await rm(path); await symlink(external, path); + await expect(restoreSetup(snapshot)).rejects.toThrow('regular file'); + expect(await readFile(external, 'utf8')).toBe('untouched'); + }); + + test('non-file setup mutations fail restoration instead of publishing success', async () => { + const { claude } = await fixture(); + const snapshot = await snapshotSetup(); + await mkdir(join(claude, 'CLAUDE.md')); + await expect(restoreSetup(snapshot)).rejects.toThrow('regular file'); + }); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/isolation.ts b/.claude/skills/verify-open-pstack/scripts/isolation.ts new file mode 100644 index 00000000..94af84a0 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/isolation.ts @@ -0,0 +1,76 @@ +import { lstat, mkdir, readFile, readlink, realpath, rm, stat, symlink, writeFile } from 'node:fs/promises'; +import { isAbsolute, join, resolve } from 'node:path'; + +function codexAuth(): string { + return resolve(process.env.CODEX_HOME ?? join(process.env.HOME!, '.codex'), 'auth.json'); +} + +async function codexDirectory(home: string): Promise { + if (!isAbsolute(home) || await realpath(home) !== home) throw new Error('Codex state must be a canonical run-owned directory'); + const directory = join(home, '.codex'); + const info = await lstat(directory); + if (!info.isDirectory() || info.isSymbolicLink() || await realpath(directory) !== directory) throw new Error('Codex state directory redirected'); + return directory; +} + +export async function linkCodexAuth(home: string): Promise { + const source = codexAuth(); + const info = await stat(source).catch((error: NodeJS.ErrnoException) => { + if (error.code !== 'ENOENT') throw error; + return undefined; + }); + if (!info?.isFile()) throw new Error('Codex verification requires existing file-based authentication (auth.json); keyring-only authentication is not supported'); + await mkdir(join(home, '.codex'), { recursive: true, mode: 0o700 }); + await symlink(source, join(await codexDirectory(home), 'auth.json')); +} + +export async function assertCodexAuth(home: string): Promise { + const path = join(await codexDirectory(home), 'auth.json'); + const info = await lstat(path); + if (!info.isSymbolicLink() || await readlink(path) !== codexAuth()) { + throw new Error('Codex replaced or redirected auth.json instead of writing through its symlink; stop and report before publication'); + } +} + +export async function removeCodexHome(home: string): Promise { + await assertCodexAuth(home); + await rm(await codexDirectory(home), { recursive: true }); +} + +export type SetupSnapshot = { path: string; bytes?: Buffer; mode?: number }[]; + +async function setupFile(path: string) { + const info = await lstat(path).catch((error: NodeJS.ErrnoException) => { + if (error.code !== 'ENOENT') throw error; + return undefined; + }); + if (info && (!info.isFile() || info.isSymbolicLink())) throw new Error(`Setup restoration requires a regular file: ${path}`); + return info; +} + +export async function snapshotSetup(): Promise { + const directories = new Set([join(process.env.HOME!, '.claude'), join(process.env.HOME!, '.codex'), + process.env.CLAUDE_CONFIG_DIR, process.env.CODEX_HOME].filter((directory): directory is string => !!directory).map(directory => resolve(directory))); + const snapshot: SetupSnapshot = []; + for (const directory of directories) { + for (const name of ['pstack-models.md', 'CLAUDE.md', 'AGENTS.md']) { + const path = join(directory, name), info = await setupFile(path); + snapshot.push(info ? { path, bytes: await readFile(path), mode: info.mode & 0o777 } : { path }); + } + } + return snapshot; +} + +export async function restoreSetup(snapshot: SetupSnapshot): Promise { + for (const { path, bytes, mode } of snapshot) { + await setupFile(path); + if (bytes === undefined) await rm(path, { force: true }); + else await writeFile(path, bytes, { mode }); + } + for (const { path, bytes } of snapshot) { + const info = await setupFile(path); + if (bytes === undefined ? info !== undefined : !info || !(await readFile(path)).equals(bytes)) { + throw new Error(`Setup restoration verification failed: ${path}`); + } + } +} diff --git a/.claude/skills/verify-open-pstack/scripts/provenance.test.ts b/.claude/skills/verify-open-pstack/scripts/provenance.test.ts new file mode 100644 index 00000000..d8aae7c0 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/provenance.test.ts @@ -0,0 +1,177 @@ +import { afterEach, expect, test } from 'bun:test'; +import { chmod, mkdtemp, mkdir, realpath, rename, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { command } from './io.ts'; +import { sourceDigest, sourceHash } from './provenance.ts'; + +const roots: string[] = []; +const project = '.claude/skills/verify-open-pstack'; +const plugin = 'plugins/pstack'; +const alias = '.agents/skills/verify-open-pstack'; +afterEach(async () => { for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }); }); +async function fixture(): Promise<{ root: string; sha: string }> { + const root = await realpath(await mkdtemp(join(tmpdir(), 'pstack-provenance-'))); roots.push(root); + await mkdir(join(root, project, 'scripts'), { recursive: true }); + await mkdir(join(root, plugin, 'skills/poteto-mode/scripts'), { recursive: true }); + await mkdir(join(root, '.agents/skills'), { recursive: true }); + await writeFile(join(root, project, 'SKILL.md'), 'native project skill\n'); + await writeFile(join(root, project, 'scripts/verify.sh'), '#!/bin/sh\ntrue\n', { mode: 0o755 }); + await writeFile(join(root, plugin, 'plugin.json'), '{"version":"1"}\n'); + await writeFile(join(root, plugin, 'skills/poteto-mode/scripts/bootstrap.ts'), 'bootstrap\n'); + await symlink('../../.claude/skills/verify-open-pstack', join(root, alias)); + for (const args of [ + ['git', 'init', '--quiet'], ['git', 'config', 'user.name', 'Provenance Test'], + ['git', 'config', 'user.email', 'test@example.invalid'], ['git', 'add', '.'], + ['git', '-c', 'commit.gpgsign=false', 'commit', '--quiet', '-m', 'fixture'], + ]) await command(args, { cwd: root }); + return { root, sha: (await command(['git', 'rev-parse', 'HEAD'], { cwd: root })).trim() }; +} + +test('real pinned source remains stable after both permitted dependency bootstraps, including symlinks', async () => { + const { root, sha } = await fixture(), before = await sourceHash(root, sha); + const dependencyTarget = join(root, 'generated-dependency'); + await mkdir(dependencyTarget); await writeFile(join(dependencyTarget, 'runtime.js'), 'generated\n'); + for (const path of [`${project}/node_modules`, `${plugin}/skills/poteto-mode/scripts/node_modules`]) await symlink(dependencyTarget, join(root, path)); + expect(await sourceHash(root, sha)).toBe(before); + await writeFile(join(dependencyTarget, 'runtime.js'), 'generated update\n'); + expect(await sourceHash(root, sha)).toBe(before); + expect(before).toMatch(/^[a-f0-9]{64}$/); + expect(await sourceDigest(root)).toBe(before); +}); + +test('canonical project code mutation and added source fail against pinned Git blobs', async () => { + const { root, sha } = await fixture(); + await writeFile(join(root, project, 'SKILL.md'), 'changed instructions\n'); + await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); + await writeFile(join(root, project, 'SKILL.md'), 'native project skill\n'); + await writeFile(join(root, project, 'scripts/new.ts'), 'untracked candidate code\n'); + await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); +}); + +test('plugin code mutation, executable-bit changes and missing source fail against the pinned tree', async () => { + const { root, sha } = await fixture(); + await writeFile(join(root, plugin, 'plugin.json'), '{}\n'); + await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); + await writeFile(join(root, plugin, 'plugin.json'), '{"version":"1"}\n'); + await command(['git', 'config', 'core.filemode', 'false'], { cwd: root }); + await chmod(join(root, project, 'scripts/verify.sh'), 0o644); + await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); + await chmod(join(root, project, 'scripts/verify.sh'), 0o755); + await rm(join(root, project, 'SKILL.md')); + await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); +}); + +test('Codex alias target must be exact and canonical even when alternate spelling resolves identically', async () => { + const { root, sha } = await fixture(); + await rm(join(root, alias)); + await symlink('../../.claude/skills/../skills/verify-open-pstack', join(root, alias)); + await expect(sourceHash(root, sha)).rejects.toThrow('Codex alias'); + await rm(join(root, alias)); + await mkdir(join(root, 'other-skill')); + await symlink('../../other-skill', join(root, alias)); + await expect(sourceHash(root, sha)).rejects.toThrow('Codex alias'); +}); + +test('generated exclusions are narrow and source symlinks are not accepted', async () => { + const { root, sha } = await fixture(); + await mkdir(join(root, project, 'scripts/node_modules')); + await writeFile(join(root, project, 'scripts/node_modules/hidden.ts'), 'unexpected\n'); + await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); + await rm(join(root, project, 'scripts/node_modules'), { recursive: true }); + await rm(join(root, project, 'SKILL.md')); + await symlink(join(root, plugin, 'plugin.json'), join(root, project, 'SKILL.md')); + await expect(sourceHash(root, sha)).rejects.toThrow('Source differs from pinned Git tree'); +}); + +test('filesystem-only rechecks cannot be spoofed by modified Git metadata or a forged Git executable', async () => { + const { root, sha } = await fixture(), pinned = await sourceHash(root, sha); + expect(await sourceDigest(root)).toBe(pinned); + await rm(join(root, '.git'), { recursive: true }); + await mkdir(join(root, '.git')); + await writeFile(join(root, '.git/HEAD'), 'forged pinned metadata\n'); + const bin = join(root, 'bin'), invoked = join(root, 'git-invoked'); + await mkdir(bin); + await writeFile(join(bin, 'git'), `#!/bin/sh\nprintf invoked > '${invoked}'\nprintf '${sha}\\n'\n`, { mode: 0o755 }); + const path = process.env.PATH; + try { + process.env.PATH = bin; + expect(await sourceDigest(root)).toBe(pinned); + await writeFile(join(root, project, 'SKILL.md'), 'forged project source\n'); + expect(await sourceDigest(root)).not.toBe(pinned); + expect(await Bun.file(invoked).exists()).toBe(false); + } finally { + if (path === undefined) delete process.env.PATH; + else process.env.PATH = path; + } +}); + +test('filesystem digest detects plugin/source additions, deletions, and executable mode changes', async () => { + const { root, sha } = await fixture(), pinned = await sourceHash(root, sha); + const manifest = join(root, plugin, 'plugin.json'); + await writeFile(manifest, 'changed plugin\n'); + expect(await sourceDigest(root)).not.toBe(pinned); + await writeFile(manifest, '{"version":"1"}\n'); + expect(await sourceDigest(root)).toBe(pinned); + const added = join(root, project, 'scripts/extra.ts'); + await writeFile(added, 'extra code\n'); + expect(await sourceDigest(root)).not.toBe(pinned); + await rm(added); + await chmod(join(root, project, 'scripts/verify.sh'), 0o644); + expect(await sourceDigest(root)).not.toBe(pinned); + await chmod(join(root, project, 'scripts/verify.sh'), 0o755); + expect(await sourceDigest(root)).toBe(pinned); + await rm(manifest); + expect(await sourceDigest(root)).not.toBe(pinned); +}); + +test('filesystem rechecks reject a candidate workspace redirected to an untouched pinned copy', async () => { + const { root, sha } = await fixture(), pinned = await sourceHash(root, sha); + const untouched = `${root}-untouched`; + await rename(root, untouched); roots.push(untouched); + await symlink(untouched, root); + expect(await sourceDigest(untouched)).toBe(pinned); + await expect(sourceDigest(root)).rejects.toThrow('workspace root must not be redirected'); +}); + +test('filesystem rechecks reject noncanonical aliases and source symlinks including ancestor directories', async () => { + const { root } = await fixture(); + await rm(join(root, alias)); + await symlink('../../.claude/skills/../skills/verify-open-pstack', join(root, alias)); + await expect(sourceDigest(root)).rejects.toThrow('Codex alias'); + await rm(join(root, alias)); + await symlink('../../.claude/skills/verify-open-pstack', join(root, alias)); + await rm(join(root, project, 'SKILL.md')); + await symlink(join(root, plugin, 'plugin.json'), join(root, project, 'SKILL.md')); + await expect(sourceDigest(root)).rejects.toThrow('source symlink'); + await rm(join(root, project, 'SKILL.md')); + await writeFile(join(root, project, 'SKILL.md'), 'native project skill\n'); + await rm(join(root, '.agents/skills'), { recursive: true }); + await mkdir(join(root, 'redirected-skills')); + await symlink('../../.claude/skills/verify-open-pstack', join(root, 'redirected-skills/verify-open-pstack')); + await symlink(join(root, 'redirected-skills'), join(root, '.agents/skills')); + await expect(sourceDigest(root)).rejects.toThrow('source directory'); +}); + +test('filesystem digest does not exclude dependency-like directories outside the two permitted paths', async () => { + const { root, sha } = await fixture(), pinned = await sourceHash(root, sha); + await mkdir(join(root, project, 'scripts/node_modules')); + await writeFile(join(root, project, 'scripts/node_modules/source.ts'), 'unexpected code\n'); + expect(await sourceDigest(root)).not.toBe(pinned); + await rm(join(root, project, 'scripts/node_modules/source.ts')); + await symlink(join(root, plugin, 'plugin.json'), join(root, project, 'scripts/node_modules/source.ts')); + await expect(sourceDigest(root)).rejects.toThrow('source symlink'); +}); + +test('HEAD must match exact pinned source and committed dependency directories cannot be excluded', async () => { + const { root, sha } = await fixture(); + await expect(sourceHash(root, 'a'.repeat(40))).rejects.toThrow('HEAD differs'); + await expect(sourceHash(root, 'HEAD')).rejects.toThrow('exact pinned SHA'); + await mkdir(join(root, project, 'node_modules')); + await writeFile(join(root, project, 'node_modules/committed.ts'), 'tracked runtime\n'); + await command(['git', 'add', '.'], { cwd: root }); + await command(['git', '-c', 'commit.gpgsign=false', 'commit', '--quiet', '-m', 'tracked dependencies'], { cwd: root }); + const changed = (await command(['git', 'rev-parse', 'HEAD'], { cwd: root })).trim(); + await expect(sourceHash(root, sha)).rejects.toThrow('HEAD differs'); + await expect(sourceHash(root, changed)).rejects.toThrow('Pinned source files missing'); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/provenance.ts b/.claude/skills/verify-open-pstack/scripts/provenance.ts new file mode 100644 index 00000000..13933e8e --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/provenance.ts @@ -0,0 +1,98 @@ +import { createHash } from 'node:crypto'; +import { lstat, readFile, readlink, readdir, realpath } from 'node:fs/promises'; +import { join, resolve } from 'node:path'; +import { command, sha256, treeHash, type Command } from './io.ts'; + +const plugin = 'plugins/pstack'; +const project = '.claude/skills/verify-open-pstack'; +const alias = '.agents/skills/verify-open-pstack'; +const aliasTarget = '../../.claude/skills/verify-open-pstack'; +const generated = new Set([ + 'plugins/pstack/skills/poteto-mode/scripts/node_modules', + '.claude/skills/verify-open-pstack/node_modules', +]); + +export async function sourceDigest(workspace: string): Promise { + const root = await realpath(workspace); + if (root !== resolve(workspace) || (await lstat(workspace)).isSymbolicLink()) throw new Error('Pinned workspace root must not be redirected'); + for (const path of ['plugins', plugin, '.claude', '.claude/skills', project, '.agents', '.agents/skills']) { + const stat = await lstat(join(root, path)); + if (stat.isSymbolicLink() || !stat.isDirectory()) throw new Error(`Pinned source directory refused: ${path}`); + } + const aliasPath = join(root, alias); + if (!(await lstat(aliasPath)).isSymbolicLink() || await readlink(aliasPath) !== aliasTarget || await realpath(aliasPath) !== join(root, project)) { + throw new Error('Codex alias does not resolve to the canonical pinned project skill'); + } + const entries: string[] = []; + async function walk(path: string): Promise { + if (generated.has(path)) return; + const actual = join(root, path), stat = await lstat(actual); + if (stat.isSymbolicLink()) throw new Error(`Pinned source symlink refused: ${path}`); + if (stat.isDirectory()) { + for (const name of (await readdir(actual)).sort()) await walk(`${path}/${name}`); + } else if (stat.isFile()) { + entries.push(`${path}\0${stat.mode & 0o111 ? '100755' : '100644'}\0${sha256(await readFile(actual))}`); + } else throw new Error(`Non-file in pinned source tree: ${path}`); + } + await walk(plugin); + await walk(project); + entries.push([alias, '120000', sha256(Buffer.from(await readlink(aliasPath)))].join('\0')); + return sha256(entries.sort().join('\n')); +} + +export async function sourceHash(workspace: string, sha: string, run: Command = command): Promise { + if (!/^[a-f0-9]{40}$/.test(sha)) throw new Error('Source provenance requires an exact pinned SHA'); + const git = ['git', '--no-replace-objects', '-c', 'core.fsmonitor=false', '-c', 'core.hooksPath=/dev/null']; + if ((await run([...git, 'rev-parse', 'HEAD'], { cwd: workspace })).trim() !== sha) { + throw new Error('Project source HEAD differs from pinned SHA'); + } + const aliasPath = join(workspace, alias); + if (!(await lstat(aliasPath)).isSymbolicLink() || await readlink(aliasPath) !== aliasTarget || await realpath(aliasPath) !== await realpath(join(workspace, project))) { + throw new Error('Codex alias does not resolve to the canonical pinned project skill'); + } + try { + await run([...git, 'diff', '--quiet', '--no-ext-diff', '--no-textconv', sha, '--', plugin, project, alias], { cwd: workspace }); + } catch { + throw new Error('Source differs from pinned Git tree'); + } + const tree = await run([...git, 'ls-tree', '-rz', '--full-tree', sha, '--', plugin, project, alias], { cwd: workspace }); + const expected = new Map(); + for (const entry of tree.split('\0').filter(Boolean)) { + const tab = entry.indexOf('\t'), fields = entry.slice(0, tab).split(' '), path = entry.slice(tab + 1); + if (tab < 0 || fields.length !== 3 || fields[1] !== 'blob' || !/^(100644|100755|120000)$/.test(fields[0]!) || !/^[a-f0-9]{40}$/.test(fields[2]!)) { + throw new Error('Unrecognized pinned source tree'); + } + if (path !== alias && !path.startsWith(plugin + '/') && !path.startsWith(project + '/')) throw new Error('Pinned source path escaped coverage'); + if (expected.has(path)) throw new Error('Duplicate pinned source path'); + expected.set(path, { mode: fields[0]!, blob: fields[2]! }); + } + if (!expected.has(alias) || ![...expected.keys()].some(path => path.startsWith(project + '/')) || ![...expected.keys()].some(path => path.startsWith(plugin + '/'))) { + throw new Error('Pinned source tree lacks plugin, canonical project skill, or Codex alias'); + } + const entries: string[] = []; + function accept(path: string, mode: string, bytes: Uint8Array): void { + const pinned = expected.get(path); + const blob = createHash('sha1').update(`blob ${bytes.length}\0`).update(bytes).digest('hex'); + if (!pinned || pinned.mode !== mode || pinned.blob !== blob) throw new Error(`Source differs from pinned Git tree: ${path}`); + expected.delete(path); + entries.push(`${path}\0${mode}\0${sha256(bytes)}`); + } + async function walk(path: string): Promise { + // Only these two untracked dependency directories may vary after bootstrap. + if (generated.has(path)) return; + const actual = join(workspace, path), stat = await lstat(actual); + if (stat.isSymbolicLink()) throw new Error(`Pinned source symlink refused: ${path}`); + if (stat.isDirectory()) { + for (const name of (await readdir(actual)).sort()) await walk(`${path}/${name}`); + } else if (stat.isFile()) { + accept(path, stat.mode & 0o111 ? '100755' : '100644', await readFile(actual)); + } else throw new Error(`Non-file in pinned source tree: ${path}`); + } + const pluginHash = await treeHash(join(workspace, plugin), ['skills/poteto-mode/scripts/node_modules']); + await walk(plugin); + if (sha256(entries.map(entry => entry.slice(plugin.length + 1)).join('\n')) !== pluginHash) throw new Error('Plugin source changed while checking provenance'); + await walk(project); + accept(alias, '120000', Buffer.from(await readlink(aliasPath))); + if (expected.size) throw new Error(`Pinned source files missing: ${[...expected.keys()].join(', ')}`); + return sha256(entries.sort().join('\n')); +} diff --git a/.claude/skills/verify-open-pstack/scripts/types.ts b/.claude/skills/verify-open-pstack/scripts/types.ts new file mode 100644 index 00000000..d360948f --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/types.ts @@ -0,0 +1,46 @@ +export const REPO = 'ericlitman/open-pstack'; +export const HARNESSES = ['claude', 'codex'] as const; +export type Harness = typeof HARNESSES[number]; +export type Phase = 'resolve' | 'classify' | 'prepare' | 'exercise' | 'publish' | 'failed' | 'head-moved'; +export interface Pull { + number: number; head: { sha: string }; base: { sha: string }; + state: string; headRepo: string; +} +export interface ChangedFile { filename: string; previous_filename?: string } +export interface Registry { + skills: string[]; + shared: string[]; + assets: string[]; + setup: string[]; + runner: string[]; + tools: string[]; + project: string[]; + nonRuntime: string[]; +} +export interface Selection { paths: string[]; skills: string[]; features: string[]; noRuntime: boolean } +export interface Observation { + harness: Harness; feature: string; surface: string; action: string; observed: string; + transcript: string; transcriptHash: string; reviewer: 'operator'; + artifacts: { path: string; sha256: string }[]; +} +export interface Installation { + harness: Harness; cliVersion: string; pluginVersion: string; sha: string; + location: string; treeHash: string; home: string; sourceHash?: string; +} +export interface Receipt { + schema: 1; repo: string; pr: number; sha: string; base: string; phase: Phase; + selection: Selection; installations: Installation[]; observations: Observation[]; + selfTest: boolean; artifactRoot: string; publisherRevision?: string; failure?: string; + commentUrl?: string; status?: 'success' | 'failure'; cleanup: string; started: string; +} +export interface GitHub { + pull(pr: number): Promise; + files(base: string, head: string): Promise; + comment(pr: number, body: string): Promise; + status(sha: string, state: 'success' | 'failure', target: string, description: string): Promise; +} +export interface Driver { + prepare(receipt: Receipt): Promise; + exercise(receipt: Receipt): Promise; + cleanup?(receipt: Receipt): Promise; +} diff --git a/.claude/skills/verify-open-pstack/scripts/verify.sh b/.claude/skills/verify-open-pstack/scripts/verify.sh new file mode 100755 index 00000000..4cfacbf8 --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/verify.sh @@ -0,0 +1,5 @@ +#!/usr/bin/env bash +set -euo pipefail +HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)" +command -v bun >/dev/null || { printf '%s\n' 'Bun is required; no fallback runtime.' >&2; exit 1; } +exec bun "$HERE/scripts/cli.ts" "$@" diff --git a/.claude/skills/verify-open-pstack/scripts/verify.test.ts b/.claude/skills/verify-open-pstack/scripts/verify.test.ts new file mode 100644 index 00000000..c730f70f --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/verify.test.ts @@ -0,0 +1,219 @@ +import { afterEach, describe, expect, test } from 'bun:test'; +import { execFileSync } from 'node:child_process'; +import { mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import data from '../features/registry.json'; +import { requiredFeatures, validateRegistry } from './core.ts'; +import { parseChangedFiles, Publisher } from './github.ts'; +import { command, sha256 } from './io.ts'; +import { evidence, verify } from './verify.ts'; +import { HARNESSES, REPO, type Driver, type GitHub, type Pull, type Receipt } from './types.ts'; + +const SHA = 'a'.repeat(40), BASE = 'b'.repeat(40), NEXT = 'c'.repeat(40), HASH = 'd'.repeat(64), REVISION = 'e'.repeat(40); +const URL = `https://github.com/${REPO}/pull/123#issuecomment-1`; +const roots: string[] = []; +function temporary(): string { const root = realpathSync(mkdtempSync(join(tmpdir(), 'pstack-publish-test-'))); roots.push(root); return root; } +afterEach(() => { for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true }); }); + +function fixture(runtime = false, selfTest = false) { + const root = temporary(); + writeFileSync(join(root, 'retained.log'), 'reviewed transcript'); + writeFileSync(join(root, 'fixture.json'), 'reviewed artifact'); + const calls: string[] = [], saved: Receipt[] = [], comments: string[] = []; + const pull: Pull = { number: 123, head: { sha: SHA }, base: { sha: BASE }, state: 'open', headRepo: REPO }; + let hook: (call: string) => void = () => {}; + const mark = (call: string) => { calls.push(call); hook(call); }; + const github: GitHub = { + async pull() { mark('pull'); return structuredClone(pull); }, + async files(base, head) { expect(base).toBe(BASE); expect(head).toBe(SHA); mark('files'); return [{ filename: runtime ? 'plugins/pstack/skills/architect/SKILL.md' : 'README.md' }]; }, + async comment(_pr, body) { mark('comment'); comments.push(body); expect(body).toContain(SHA); return URL; }, + async status(sha, state, target) { mark(`status:${state}:${sha}`); expect(target).toBe(state === 'success' || comments.length ? URL : `https://github.com/${REPO}/pull/123`); }, + }; + const driver: Driver = { + async prepare(r) { mark('prepare'); return HARNESSES.map(harness => ({ harness, sha: r.sha, cliVersion: 'test', pluginVersion: '1.5.0', treeHash: HASH, location: `/isolated/${harness}/plugin`, home: `/isolated/${harness}` })); }, + async exercise(r) { mark('exercise'); return HARNESSES.flatMap(harness => requiredFeatures(r).map(feature => ({ harness, feature, surface: 'native surface', action: 'invoke', observed: 'fixture changed', reviewer: 'operator' as const, transcript: 'retained.log', transcriptHash: sha256('reviewed transcript'), artifacts: [{ path: 'fixture.json', sha256: sha256('reviewed artifact') }] }))); }, + }; + const options = { pr: 123, root, selfTest, publisherRevision: REVISION, registry: validateRegistry(data), github, driver, persist: async (r: Receipt) => { saved.push(structuredClone(r)); } }; + return { options, calls, saved, comments, pull, hook: (fn: typeof hook) => { hook = fn; } }; +} + +describe('exact-head evidence publication', () => { + test('docs-only publishes one no-runtime comment and exact-SHA live-gate without harnesses or PR mutation', async () => { + const f = fixture(), receipt = await verify(f.options); + expect(receipt.status).toBe('success'); expect(receipt.sha).toBe(SHA); expect(receipt.publisherRevision).toBe(REVISION); + expect(f.calls).not.toContain('prepare'); expect(f.calls).not.toContain('exercise'); + expect(f.calls.filter(call => call === 'comment')).toHaveLength(1); + expect(f.calls).toContain(`status:success:${SHA}`); + expect(f.comments[0]).toContain('PASSED — no runtime change'); expect(f.comments[0]).toContain(REVISION); + expect(Object.keys(receipt)).not.toContain('proposedTemplate'); expect(Object.keys(receipt)).not.toContain('madeReady'); expect(Object.keys(receipt)).not.toContain('compensation'); + }); + + test('runtime selection and explicit self-test exercise every feature in both harnesses', async () => { + const f = fixture(true, true), receipt = await verify(f.options); + expect(requiredFeatures(receipt).sort()).toEqual(['project-skill', 'skill-invocation:architect']); + expect(receipt.installations.map(record => record.harness).sort()).toEqual([...HARNESSES]); + expect(receipt.observations).toHaveLength(4); + for (const feature of requiredFeatures(receipt)) for (const harness of HARNESSES) { + expect(receipt.observations.filter(record => record.feature === feature && record.harness === harness)).toHaveLength(1); + } + expect(f.comments).toHaveLength(1); expect(f.comments[0]).toContain('operator reviewed'); + }); + + test('self-test alone runs project proof in both harnesses', async () => { + const receipt = await verify(fixture(false, true).options); + expect(receipt.selection.noRuntime).toBe(true); + expect(receipt.observations.map(record => `${record.harness}/${record.feature}`).sort()).toEqual(['claude/project-skill', 'codex/project-skill']); + }); + + test('verifier changes require the candidate publisher before any driver call or GitHub write', async () => { + for (const revision of [REVISION, SHA]) { + const f = fixture(); f.options.publisherRevision = revision; + f.options.github.files = async () => [{ filename: '.claude/skills/verify-open-pstack/scripts/verify.ts' }]; + f.options.driver.cleanup = async () => { f.calls.push('cleanup'); }; + if (revision !== SHA) { + await expect(verify(f.options)).rejects.toThrow('rerun from that checkout'); + expect(f.calls.some(call => ['prepare', 'exercise', 'cleanup', 'comment'].includes(call) || call.startsWith('status:'))).toBe(false); + } else { + expect((await verify(f.options)).status).toBe('success'); + expect(f.calls).toContain('prepare'); expect(f.calls).toContain('exercise'); + } + } + }); + + test('non-verifier changes do not require a candidate publisher revision', async () => { + const f = fixture(true); + expect((await verify(f.options)).status).toBe('success'); + expect(f.calls).toContain('prepare'); + }); + + test('successful verification restores setup and removes Codex run state before any publication', async () => { + const f = fixture(true); f.options.driver.cleanup = async () => { f.calls.push('cleanup'); }; + const receipt = await verify(f.options); + expect(f.calls.filter(call => call === 'cleanup')).toHaveLength(1); + expect(f.calls.indexOf('exercise')).toBeLessThan(f.calls.indexOf('cleanup')); + expect(f.calls.indexOf('cleanup')).toBeLessThan(f.calls.indexOf('comment')); + expect(receipt.cleanup).toContain('removed'); + }); + + test('persistent cleanup failure prevents every public write and remains private', async () => { + const f = fixture(true); f.options.driver.cleanup = async () => { f.calls.push('cleanup'); throw new Error('RAW_CREDENTIAL_CLEANUP_EXCEPTION'); }; + await expect(verify(f.options)).rejects.toThrow('RAW_CREDENTIAL_CLEANUP_EXCEPTION'); + expect(f.calls.filter(call => call === 'cleanup')).toHaveLength(2); + expect(f.calls).not.toContain('comment'); expect(f.calls.some(call => call.startsWith('status:'))).toBe(false); + expect(f.saved.at(-1)?.failure).toContain('RAW_CREDENTIAL_CLEANUP_EXCEPTION'); + }); + + test('failed setup restoration forbids publication even if cleanup retry succeeds', async () => { + const f = fixture(true); let attempts = 0; + f.options.driver.cleanup = async () => { f.calls.push('cleanup'); if (++attempts === 1) throw new Error('Setup restoration failed'); }; + await expect(verify(f.options)).rejects.toThrow('Setup restoration failed'); + expect(f.calls.filter(call => call === 'cleanup')).toHaveLength(2); + expect(f.comments).toHaveLength(0); + expect(f.calls.some(call => call.startsWith('status:'))).toBe(false); + expect(f.saved.at(-1)?.failure).toContain('Setup restoration failed'); + }); + + test('failure publication is structured and excludes raw exceptions, transcripts, and artifact paths', async () => { + const f = fixture(true); + f.options.driver.exercise = async () => { f.calls.push('exercise'); throw new Error('RAW_PROVIDER_EXCEPTION secret detail'); }; + f.options.driver.cleanup = async () => { f.calls.push('cleanup'); }; + await expect(verify(f.options)).rejects.toThrow('RAW_PROVIDER_EXCEPTION'); + expect(f.comments).toHaveLength(1); + const body = f.comments[0]!; + expect(body).toContain('## Live evidence: Open Pstack'); expect(body).toContain(`Candidate SHA: \`${SHA}\``); + for (const forbidden of ['RAW_PROVIDER_EXCEPTION', 'secret detail', 'reviewed transcript', 'retained.log', 'fixture.json']) expect(body).not.toContain(forbidden); + expect(f.saved.at(-1)?.failure).toContain('RAW_PROVIDER_EXCEPTION secret detail'); + }); + + test('an ambiguous comment response is never retried', async () => { + const f = fixture(true); f.options.driver.cleanup = async () => { f.calls.push('cleanup'); }; + f.hook(call => { if (call === 'comment') throw new Error('response lost'); }); + await expect(verify(f.options)).rejects.toThrow('response lost'); + expect(f.calls.filter(call => call === 'comment')).toHaveLength(1); + expect(f.calls).toContain(`status:failure:${SHA}`); expect(f.calls).not.toContain(`status:success:${SHA}`); + }); + + test('transcript or artifact overwrite at publication boundaries revokes or blocks exact-SHA success', async () => { + for (const file of ['retained.log', 'fixture.json']) for (const boundary of ['comment', `status:success:${SHA}`]) { + const f = fixture(true); f.options.driver.cleanup = async () => { f.calls.push('cleanup'); }; + f.hook(call => { if (call === boundary) writeFileSync(join(f.options.root, file), 'overwritten'); }); + await expect(verify(f.options)).rejects.toThrow('changed after acceptance'); + expect(f.comments).toHaveLength(1); expect(f.calls).toContain(`status:failure:${SHA}`); + if (boundary === 'comment') expect(f.calls).not.toContain(`status:success:${SHA}`); + } + }); + + test('head/base races never publish success and failure gate stays on the pinned candidate', async () => { + for (const boundary of ['files', 'prepare', 'exercise', 'comment', `status:success:${SHA}`]) { + const f = fixture(true); f.options.driver.cleanup = async () => { f.calls.push('cleanup'); }; + f.hook(call => { if (call === boundary) f.pull.head.sha = NEXT; }); + await expect(verify(f.options)).rejects.toThrow('changed'); + if (boundary !== `status:success:${SHA}`) expect(f.calls).not.toContain(`status:success:${SHA}`); + expect(f.calls).toContain(`status:failure:${SHA}`); expect(f.calls.join()).not.toContain(`status:failure:${NEXT}`); + expect(f.comments.length).toBeLessThanOrEqual(1); + } + }); + + test('closed or foreign PR never enters publication', async () => { + for (const field of ['state', 'headRepo'] as const) { + const f = fixture(); f.pull[field] = 'other'; + await expect(verify(f.options)).rejects.toThrow('open same-repository'); expect(f.calls).toEqual(['pull']); + } + }); + + test('large evidence remains bounded and exposes no raw transcript content', async () => { + const receipt = await verify(fixture(true).options), observation = receipt.observations[0]!; + receipt.observations = Array.from({ length: 400 }, (_, index) => ({ ...observation, feature: `skill:${index}`, action: 'a'.repeat(3000), observed: 'r'.repeat(3000), artifacts: Array.from({ length: 20 }, (_, n) => ({ path: `artifact-${n}`, sha256: HASH })) })); + const comment = evidence(receipt); + expect(comment.length).toBeLessThan(60000); expect(comment).toContain('Additional observations'); expect(comment).toContain('Evidence-set SHA-256'); + expect(comment).not.toContain('reviewed transcript'); expect(comment).not.toContain('retained.log'); + receipt.failure = 'RAW '.repeat(10000); expect(evidence(receipt).length).toBeLessThan(60000); expect(evidence(receipt)).not.toContain('RAW'); + }); +}); + +describe('publisher request boundaries', () => { + test('NUL diff parser preserves rename provenance and rejects malformed results', () => { + expect(parseChangedFiles('R100\0plugins/pstack/skills/architect/SKILL.md\0docs/new.md\0M\0README.md\0')).toEqual([{ filename: 'docs/new.md', previous_filename: 'plugins/pstack/skills/architect/SKILL.md' }, { filename: 'README.md' }]); + expect(parseChangedFiles('')).toEqual([]); + for (const output of ['M\0README.md', 'R100\0old\0', 'M\0same\0M\0same\0', 'X\0file\0']) expect(() => parseChangedFiles(output)).toThrow(); + }); + + test('fresh local git diff uses immutable base/head even when a source branch moves', async () => { + const repository = temporary(), env = { ...process.env, GIT_AUTHOR_NAME: 'test', GIT_AUTHOR_EMAIL: 'test@example.com', GIT_COMMITTER_NAME: 'test', GIT_COMMITTER_EMAIL: 'test@example.com' }; + const git = (...args: string[]) => execFileSync('git', args, { cwd: repository, env, encoding: 'utf8' }).trim(); + git('init', '-q'); writeFileSync(join(repository, 'README.md'), 'base'); git('add', '.'); git('commit', '-qm', 'base'); const base = git('rev-parse', 'HEAD'); + mkdirSync(join(repository, 'runtime')); writeFileSync(join(repository, 'runtime', 'skill.md'), 'immutable runtime'); git('add', '.'); git('commit', '-qm', 'runtime'); const head = git('rev-parse', 'HEAD'); + git('rm', '-q', 'runtime/skill.md'); git('commit', '-qm', 'later docs-only branch'); + const calls: string[][] = []; + const publisher = new Publisher(async (args, options) => { calls.push(args); if (args[0] === 'git' && args[1] === 'fetch') args = args.map(arg => arg === `https://github.com/${REPO}.git` ? repository : arg); return command(args, options); }); + expect(await publisher.files(base, head)).toEqual([{ filename: 'runtime/skill.md' }]); + expect(calls.some(args => args[0] === 'gh')).toBe(false); expect(calls.at(-1)).toContain(`${base}...${head}`); + await expect(publisher.files('main', head)).rejects.toThrow('Invalid diff SHA'); + }); + + test('interruption blocks ordinary status writes but permits a pinned failure gate', () => { + for (const signal of ['SIGINT', 'SIGTERM']) { + const script = ` + import { command, interruptCommands } from ${JSON.stringify(new globalThis.URL('./io.ts', import.meta.url).pathname)}; + import { Publisher } from ${JSON.stringify(new globalThis.URL('./github.ts', import.meta.url).pathname)}; + await interruptCommands(${JSON.stringify(signal)}); + const executed = []; + const publisher = new Publisher(async (args, options) => { const output = await command([process.execPath, '-e', 'console.log("{}")'], options); executed.push(args); return output; }); + try { await publisher.status(${JSON.stringify(SHA)}, 'success', ${JSON.stringify(URL)}, 'passed'); throw new Error('ordinary write escaped interruption'); } + catch (error) { if (!String(error).includes('Interrupted by')) throw error; } + await publisher.status(${JSON.stringify(SHA)}, 'failure', ${JSON.stringify(URL)}, 'failed'); + console.log(JSON.stringify(executed)); + `; + const calls = JSON.parse(execFileSync(process.execPath, ['-e', script], { encoding: 'utf8' }).trim()) as string[][]; + expect(calls).toHaveLength(1); expect(calls[0]).toContain('state=failure'); + } + }); + + test('status targets the supplied exact SHA and evidence URL, never a branch', async () => { + const calls: string[][] = [], publisher = new Publisher(async args => { calls.push(args); return '{}'; }); + await publisher.status(SHA, 'success', URL, 'passed'); + expect(calls[0]).toContain(`repos/${REPO}/statuses/${SHA}`); expect(calls[0]).toContain('context=live-gate'); expect(calls[0]).toContain(`target_url=${URL}`); + await expect(publisher.status('main', 'success', URL, 'passed')).rejects.toThrow('Invalid status SHA'); + }); +}); diff --git a/.claude/skills/verify-open-pstack/scripts/verify.ts b/.claude/skills/verify-open-pstack/scripts/verify.ts new file mode 100644 index 00000000..d250d18b --- /dev/null +++ b/.claude/skills/verify-open-pstack/scripts/verify.ts @@ -0,0 +1,139 @@ +import { classify, completeEvidence, newReceipt } from './core.ts'; +import { retainedFile, sha256 } from './io.ts'; +import { REPO, type Driver, type GitHub, type Pull, type Receipt, type Registry } from './types.ts'; + +class HeadMoved extends Error {} +// Omit unsafe or oversized values whole rather than rewriting operator evidence. +export const EVIDENCE_LIMIT = 160; +export const publishable = (text: string, limit = EVIDENCE_LIMIT): boolean => text.length <= limit && /^[\x20-\x7e]*$/.test(text) && !/[`<>]/.test(text); +const safe = (text: string, limit = EVIDENCE_LIMIT): string => publishable(text, limit) ? text : '[value omitted]'; + +/** Build the single structured public record for this run. Failure details remain private in receipt.json. */ +export function evidence(receipt: Receipt): string { + const failed = Boolean(receipt.failure); + const lines = [ + '## Live evidence: Open Pstack', + `- Candidate SHA: \`${receipt.sha}\``, + `- Base SHA: \`${receipt.base}\``, + `- Result: ${failed ? 'FAILED — verification did not complete; rerun required' : receipt.selection.noRuntime && !receipt.selfTest ? 'PASSED — no runtime change' : 'PASSED — all mapped features verified'}`, + `- Self-test: ${receipt.selfTest ? failed ? 'required in both harnesses' : 'completed in both harnesses' : 'not requested'}`, + `- Publisher revision: ${receipt.publisherRevision ? `\`${receipt.publisherRevision}\`` : 'not supplied'}`, + `- Evidence-set SHA-256: \`${sha256(JSON.stringify({ installations: receipt.installations, observations: receipt.observations }))}\``, + ]; + if (!failed) { + for (const install of receipt.installations) { + lines.push(`- Installation (${install.harness}): CLI ${safe(install.cliVersion)}, plugin ${safe(install.pluginVersion)}, candidate \`${install.sha}\`, tree \`${install.treeHash}\`.`); + } + let shown = 0; + for (const record of receipt.observations) { + const line = `- Observation (${record.harness} / ${safe(record.feature, 72)}): surface ${safe(record.surface)}; action ${safe(record.action)}; result ${safe(record.observed)}; operator reviewed; transcript SHA-256 \`${record.transcriptHash}\`; artifact-set SHA-256 \`${sha256(JSON.stringify(record.artifacts))}\`.`; + if (lines.join('\n').length + line.length > 48000) break; + lines.push(line); shown++; + } + if (shown < receipt.observations.length) lines.push(`- Additional observations: ${receipt.observations.length - shown} retained privately; bound by the evidence-set digest.`); + } + lines.push('- Policy: every new head requires a fresh run; this verifier does not change PR state, queue, or merge.'); + return lines.join('\n'); +} + +export async function revalidateEvidence(receipt: Receipt): Promise { + for (const record of receipt.observations) { + const transcript = await retainedFile(receipt.artifactRoot, record.transcript); + if (transcript.path !== record.transcript || transcript.sha256 !== record.transcriptHash) throw new Error('Reviewed transcript changed after acceptance'); + for (const artifact of record.artifacts) { + const actual = await retainedFile(receipt.artifactRoot, artifact.path); + if (actual.path !== artifact.path || actual.sha256 !== artifact.sha256) throw new Error('Reviewed artifact changed after acceptance'); + } + } +} + +type VerifyOptions = { + pr: number; + selfTest: boolean; + root: string; + registry: Registry; + github: GitHub; + driver: Driver; + persist: (receipt: Receipt) => Promise; + publisherRevision?: string; +}; + +export async function verify(options: VerifyOptions): Promise { + const { github, driver, persist } = options; + const initial = await github.pull(options.pr); + if (initial.state !== 'open' || initial.headRepo !== REPO) throw new Error('Require an open same-repository PR'); + const receipt = newReceipt(options.pr, initial.head.sha, initial.base.sha, options.selfTest, options.root, options.publisherRevision); + let successAttempted = false, commentAttempted = false, cleanupComplete = false, cleanupFailed = false; + const cleanup = async (): Promise => { + if (cleanupComplete) return; + try { await driver.cleanup?.(receipt); } + catch (error) { cleanupFailed = true; throw error; } + cleanupComplete = true; + receipt.cleanup = 'Setup files restored and verified; run-owned Codex home removed; private raw evidence retained'; + }; + const current = async (): Promise => { + const pull = await github.pull(receipt.pr); + if (pull.head.sha !== receipt.sha || pull.base.sha !== receipt.base || pull.state !== 'open' || pull.headRepo !== REPO) throw new HeadMoved('PR head/base or open repository identity changed; run again'); + return pull; + }; + const phase = async (next: Receipt['phase']): Promise => { + receipt.phase = next; + await persist(receipt); + await current(); + }; + const publishComment = async (): Promise => { + const body = evidence(receipt); + commentAttempted = true; + receipt.commentUrl = await github.comment(receipt.pr, body); + await persist(receipt); + }; + const appendFailure = (label: string, error: unknown): void => { + receipt.failure = `${receipt.failure}; ${label}: ${error instanceof Error ? error.message : String(error)}`; + }; + await phase('classify'); + receipt.selection = classify(await github.files(receipt.base, receipt.sha), options.registry); + if (receipt.selection.features.includes('project-skill') && receipt.publisherRevision !== receipt.sha) { + throw new Error('Verifier-changing PRs must run from a checkout of the candidate head; rerun from that checkout'); + } + try { + await current(); await persist(receipt); + if (!receipt.selection.noRuntime || receipt.selfTest) { + await phase('prepare'); receipt.installations = await driver.prepare(receipt); await current(); await persist(receipt); + await phase('exercise'); receipt.observations = await driver.exercise(receipt); await current(); await persist(receipt); + } + await cleanup(); + completeEvidence(receipt); + await phase('publish'); await revalidateEvidence(receipt); await current(); + await publishComment(); + await current(); await revalidateEvidence(receipt); + successAttempted = true; + await github.status(receipt.sha, 'success', receipt.commentUrl!, receipt.selection.noRuntime && !receipt.selfTest ? 'No runtime change' : 'All mapped features passed in both harnesses'); + receipt.status = 'success'; await persist(receipt); await current(); await revalidateEvidence(receipt); + return receipt; + } catch (error) { + receipt.failure = error instanceof Error ? error.message : String(error); + receipt.phase = error instanceof HeadMoved ? 'head-moved' : 'failed'; + try { await cleanup(); } + catch (cleanupError) { + receipt.cleanup = `FAILED: ${cleanupError instanceof Error ? cleanupError.message : String(cleanupError)}`; + appendFailure('setup restoration or Codex cleanup', cleanupError); + } + try { await persist(receipt); } + catch (persistError) { appendFailure('persist failure receipt', persistError); } + + // Failed setup restoration or Codex auth-link cleanup forbids all publication. + if (cleanupComplete && !cleanupFailed) { + if (!commentAttempted) { + try { await current(); await publishComment(); } + catch (publicationError) { appendFailure('publish failure evidence', publicationError); } + } + try { + await github.status(receipt.sha, 'failure', receipt.commentUrl ?? `https://github.com/${REPO}/pull/${receipt.pr}`, successAttempted ? 'Verification aborted; rerun required' : 'Verification failed; rerun required'); + receipt.status = 'failure'; + } catch (statusError) { appendFailure('publish failure live-gate', statusError); } + } + try { await persist(receipt); } + catch (persistError) { appendFailure('persist final failure receipt', persistError); } + throw new Error(receipt.failure); + } +} diff --git a/.claude/skills/verify-open-pstack/tsconfig.json b/.claude/skills/verify-open-pstack/tsconfig.json new file mode 100644 index 00000000..4aba2486 --- /dev/null +++ b/.claude/skills/verify-open-pstack/tsconfig.json @@ -0,0 +1,8 @@ +{ + "compilerOptions": { + "target": "ESNext", "module": "ESNext", "moduleResolution": "bundler", + "allowImportingTsExtensions": true, "noEmit": true, "strict": true, + "types": ["bun-types"], "skipLibCheck": true + }, + "include": ["scripts/**/*.ts"] +} diff --git a/.gitignore b/.gitignore index 008e57b5..10fb2a92 100644 --- a/.gitignore +++ b/.gitignore @@ -11,3 +11,6 @@ # bun deps for the vendored poteto-mode scripts (bootstrap.ts installs them on first run) plugins/pstack/skills/poteto-mode/scripts/node_modules/ + +# Non-shipped repository verification helper dependencies +.claude/skills/verify-open-pstack/node_modules/ diff --git a/AGENTS.md b/AGENTS.md index 8a1917b2..6a284340 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,4 +8,6 @@ Before opening a pull request, run the Bun tests, strict typecheck, static invar Nothing merges, tags, releases, or rolls out until the exact candidate is installed and the changed behavior passes a live test from the real user surface in every affected harness. Unit tests, validators, source inspection, and self-reports do not satisfy this gate. The record is the `live-gate` commit status on that exact head, which links an evidence comment naming the installed version, surface, action, and observed result. Mergify does not queue a pull request without it. A pull request may be ready for review before its live test, but every new head, including a queue-created head, needs its own. +For Open Pstack PRs, use the shared repository-local `.claude/skills/verify-open-pstack/` skill (Codex: `.agents/skills/verify-open-pstack`). Verifier-changing PRs run from a checkout of their reviewed candidate head, with the publisher revision equal to the candidate SHA; all other PRs run from trusted `main`. Record the publisher revision separately from the candidate SHA. Reuse the operator's existing logins without credential source flags, extra logins, or credential reading/copying. Claude uses normal operator config with session-only `--plugin-dir /plugins/pstack --settings '{"enabledPlugins":{"pstack@open-pstack":false}}'`. Codex installs the exact-head candidate in a run-owned `CODEX_HOME`, with `auth.json` symlinked to the operator's `${CODEX_HOME:-$HOME/.codex}/auth.json`; absent file-based auth fails closed. Removing the run home removes only the symlink, never its target. Setup alone snapshots real `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/pstack-models.md` and `CLAUDE.md`, restores their exact bytes or original absence afterwards, and verifies restoration before any publication; restoration failure publishes nothing. Preserve private mode-0700 raw evidence through merge. Publication is one structured evidence comment plus exact-SHA `live-gate`; no credential parsing, token registration, or exact-secret comment checks. No PR-body, ready/draft, reversal, or compensation writes, publisher-code attestation, or OS filesystem-denial claims are permitted. Builders do not publish the gate, queue, or merge. + Do not add an implicit runtime timeout or a weaker-model fallback. diff --git a/tests/skill-collision-repro.sh b/tests/skill-collision-repro.sh index 9ca752b2..40f54d2e 100755 --- a/tests/skill-collision-repro.sh +++ b/tests/skill-collision-repro.sh @@ -434,6 +434,37 @@ else note "ok: codex logo path resolves" fi +verification="$repo/.claude/skills/verify-open-pstack" +verification_bad="" +if [ ! -L "$repo/.agents/skills/verify-open-pstack" ] || + [ "$(readlink "$repo/.agents/skills/verify-open-pstack" 2>/dev/null || true)" != "../../.claude/skills/verify-open-pstack" ]; then + verification_bad="Codex must link to the canonical Claude project skill" +fi +for section in Launch Doctor Drive Evidence Cleanup Helpers; do + grep -q "^## $section$" "$verification/SKILL.md" || verification_bad="$verification_bad missing $section;" +done +for file in features/registry.json features/README.md package.json bun.lock tsconfig.json; do + [ -f "$verification/$file" ] || verification_bad="$verification_bad missing $file;" +done +[ -x "$verification/scripts/verify.sh" ] || verification_bad="$verification_bad helper is not executable;" +if [ -e "$plugin/skills/verify-open-pstack" ] || [ -e "$plugin/commands/verify-open-pstack.md" ]; then + verification_bad="$verification_bad project verifier must not ship in the plugin;" +fi +if [ -n "$verification_bad" ]; then + note "FAIL: repository-local verification skill: $verification_bad" + fail=1 +else + note "ok: shared project verification skill has executable helper and maintained feature map" +fi + +note "Checking non-shipped verification helper" +( + cd "$verification" + bun install --frozen-lockfile + bun run test + bun run typecheck +) + if [ "${PSTACK_STATIC_ONLY:-0}" = "1" ]; then exit "$fail" fi