diff --git a/CHANGES.md b/CHANGES.md index 92906368..1b90d84c 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -2,6 +2,12 @@ This port applies the Cursor → Claude Code substitutions in skill bodies. Earlier drafts left them flagged; this revision resolves them. A later pass added a Codex build that shares the same skills; see [Codex port](#codex-port) below. +## PR 59 setup changes awaiting installed verification + +Setup accepts optional Astra and explicitly loaded provider extensions while retaining the default four-family panel. Native Codex dispatch preserves the selected model and effort with a compatible history setting. Claude output parsing supports terminal results inside event arrays. Grok preflight recognizes API-key authentication, verifies the exact model token, and rejects explicit unavailable-model responses before execution. + +This branch includes the 1.4.1 upstream sync below. Its changed setup behavior still requires installed live verification in both harnesses before readiness. + ## 1.4.1 syncs to Cursor pstack 0.15.1 Open Pstack 1.4.1 tracks Cursor pstack 0.15.1 at `f8abeddd1862dc73704e3d719dd73df0d51b8c71`. Poteto-mode now requires each claim to include its evidence or a measured, inferred, or guess label in the same sentence. Agents also run any check they can run themselves instead of handing that check to the user. No playbook, model, runtime, or dependency changed. @@ -18,6 +24,7 @@ Opening a PR adopts the "briefing, not the lab notebook" guidance and links deta Existing exclusions remain: `make-bot-ui`, Benny automations, Cursor-only guide and sticky-mode content, Cursor-only solo-model defaults, invocation-blocking flags on How, Why, Unslop, and TypeScript best practices, and the unsupported Claude manifest logo field. Those four skills stay user-invocable and model-invocable. The principle leaves keep `user-invocable: false`, which hides them from the slash menu but leaves model invocation open. The Cursor manifest is not imported. Watcher and orchestrator directories, package metadata, and lockfile have no upstream changes in this range. The unrelated Grok Voice plugin remains outside this sync. + ## 1.3.0 syncs to Cursor pstack 0.14.7 Open Pstack now tracks Cursor pstack 0.14.7 at `efa2a531985e0a8084d36ff3cf87233be8a9f34b`. diff --git a/plugins/pstack/skills/poteto-mode/references/codex-tools.md b/plugins/pstack/skills/poteto-mode/references/codex-tools.md index b967458d..f924206a 100644 --- a/plugins/pstack/skills/poteto-mode/references/codex-tools.md +++ b/plugins/pstack/skills/poteto-mode/references/codex-tools.md @@ -1,6 +1,6 @@ # Codex tool mapping for pstack -pstack skills retain Claude Code tool language (`Skill`, `Agent`, `AskUserQuestion`) in shared prose. On Codex the files are the same; only those tool names resolve differently. Model execution is not translated here. Read [`provider-dispatch.md`](provider-dispatch.md) for the parent-owned Claude/Codex/Grok route table and provider-qualified descriptors. +pstack skills retain Claude Code tool language (`Skill`, `Agent`, `AskUserQuestion`) in shared prose. On Codex the files are the same; only those tool names resolve differently. Model execution is not translated here. Read [`provider-dispatch.md`](provider-dispatch.md) for the parent-owned built-in and optional extension routes and provider-qualified descriptors. ## Tool actions @@ -35,6 +35,7 @@ Without it, the native Codex lane is a named dropout. Independent external lanes poteto-mode's Subagents section sets Claude-specific defaults (`subagent_type: "poteto-agent"`, `run_in_background: true`). On Codex: - There is no `poteto-agent` subagent type. Route an ad-hoc subagent through poteto-mode's style by dispatching a `spawn_agent` whose instructions tell it to read the `poteto-mode` skill in full first. +- Model or effort overrides use `fork_turns: "none"` and a complete task with grounding paths; full-history forks inherit the parent and cannot carry these overrides. - `spawn_agent` calls already run concurrently with your turn, so `run_in_background: true` has no separate flag. Issue the dispatch and continue. - There is no `comment-sicko` subagent type either. The **no-comments** skill spawns it on Claude Code; on Codex dispatch a `spawn_agent` whose instructions tell it to read `agents/comment-sicko.md` in full first. - Claude Code runs every subagent on this machine, so the **swarm** skill's workers and the fan-out playbooks (`orchestrate`, `autopilot-full`, `autopilot-stack`) isolate writers with worktrees. The same holds on Codex. @@ -42,7 +43,7 @@ poteto-mode's Subagents section sets Claude-specific defaults (`subagent_type: " ## Models and providers -Do not replace every configured entry with a Codex model. `/setup-pstack` writes portable descriptors such as `claude:fable@max`, `codex:gpt-5.6-sol@max`, and `grok:grok-4.6@xhigh`. In a Codex parent, only `codex:*` is native. Route Claude and Grok descriptors through the external launcher exactly as `provider-dispatch.md` specifies. The current default panel intentionally keeps four-provider frontier diversity and contains no older GPT or Claude substitute. +Do not replace every configured entry with a Codex model. `/setup-pstack` writes portable descriptors such as `claude:fable@max`, `codex:gpt-5.6-sol@max`, and `grok:grok-4.6@xhigh`. In a Codex parent, only `codex:*` is native. Route Claude and Grok descriptors through the external launcher exactly as `provider-dispatch.md` specifies. The default panel has four model families across three providers. Optional Astra is also native. If the sheet contains `Dispatch extension: `, read it before validating or dispatching extension descriptors and invoke its declared launcher directly. The sheet remains the only role configuration; the extension supplies routes and evidence. User choices and effort caps override examples. ## Claude built-in skills pstack references diff --git a/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md b/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md index 74ab90da..f94d7d53 100644 --- a/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md +++ b/plugins/pstack/skills/poteto-mode/references/provider-dispatch.md @@ -8,17 +8,50 @@ pstack model choices are provider-qualified descriptors: ## Model matrix -| Family | Upstream pstack choice | Provider | Model | Default effort | Selectable efforts | Claude-native agent stem | -|---|---|---|---|---|---|---| -| fable | fable | claude | fable | max | low medium high xhigh max | fable | -| sol | gpt-5.6-sol-max | codex | gpt-5.6-sol | max | low medium high xhigh max | - | -| grok | grok-4.6-fast-xhigh | grok | grok-4.6 | xhigh | low medium high xhigh max | - | -| opus | opus | claude | opus | xhigh | low medium high xhigh max | opus | +| Family | Upstream pstack choice | Provider | Model | Default effort | Selectable efforts | Claude-native agent stem | First-run active | +|---|---|---|---|---|---|---|---| +| fable | fable | claude | fable | max | low medium high xhigh max | fable | yes | +| sol | gpt-5.6-sol-max | codex | gpt-5.6-sol | max | low medium high xhigh max | - | yes | +| grok | grok-4.6-fast-xhigh | grok | grok-4.6 | xhigh | low medium high xhigh max | - | yes | +| opus | opus | claude | opus | xhigh | low medium high xhigh max | opus | yes | +| astra | - | codex | gpt-6-astra | medium | low medium high xhigh max | - | no | -The allowed effort universe is exactly `low`, `medium`, `high`, `xhigh`, `max`. First-run requested efforts are the Default effort cell of each row. A Claude-native agent stem of `-` means the family has no Claude-native agent. Otherwise the shipped agent name is `pstack--`. +The allowed effort universe is exactly `low`, `medium`, `high`, `xhigh`, `max`. The default panel selects Fable, Sol, Grok, and Opus in that order. Rows with First-run active `yes` seed a missing sheet; otherwise derive selected families from the loaded role map and explicit user changes. Optional families are selected only by a loaded role descriptor or an explicit user choice. Defaults propose effort only for selected families without a supplied or current value. Explicit user model/effort choices override examples; user effort caps constrain every selection and probe, including loaded values. Never probe above a cap. Reuse choices already supplied in the conversation. A Claude-native agent stem of `-` means the family has no Claude-native agent. Otherwise the shipped agent name is `pstack--`. `fable` and `opus` are Claude Code's rolling aliases. Claude resolves each alias to the latest available family revision. A runner receipt keeps the requested alias in `model` and the concrete provider-reported revision in `reportedModel`; verification accepts only a numeric `claude-fable-*` or `claude-opus-*` revision from the matching family. +## Optional dispatch extensions + +When the current model sheet contains a line `Dispatch extension: `, read that reference before validating or dispatching any role. Setup may also load a path explicitly supplied by the user and persist this pointer in the sheet. Expand `~` for the current user; resolve relative paths against the model sheet directory. A missing reference is inconsistent state. + +An extension declares additional provider/model families, selectable efforts, a proposed effort, parent-specific launcher argv, and authentication/model/completion evidence. It adds routing capabilities, never role assignments or a second mutable model configuration. Require unique families and provider/model pairs; extensions cannot override built-in families, routes, aliases, or the user's effort caps. Only families selected by a role descriptor or the user participate in setup. Persist all selected families in the role map. + +The reference is UTF-8 JSON with `schemaVersion: 1` and a `families` array. This example describes the format; `example` and the executable path are illustrative placeholders, not an installed provider: + +```json +{ + "schemaVersion": 1, + "families": [{ + "family": "example", + "provider": "example", + "model": "example-model", + "selectableEfforts": ["medium", "high"], + "defaultEffort": "medium", + "launcherArgv": { + "claude": ["/absolute/path/to/extension-runner", "--parent", "{parent}", "--provider", "{provider}", "--model", "{model}", "--effort", "{effort}", "--mode", "{mode}", "--prompt", "{promptPath}", "--cwd", "{cwd}", "--output", "{outputPath}", "--receipt", "{receiptPath}"], + "codex": ["/absolute/path/to/extension-runner", "--parent", "{parent}", "--provider", "{provider}", "--model", "{model}", "--effort", "{effort}", "--mode", "{mode}", "--prompt", "{promptPath}", "--cwd", "{cwd}", "--output", "{outputPath}", "--receipt", "{receiptPath}"] + }, + "receiptSchema": "pstack-runner-v1" + }] +} +``` + +The parent validates the reference before invoking a launcher. Reject unknown fields, duplicate JSON keys, wrong types, empty families, built-in collisions, and duplicate families or provider/model pairs. Family, provider, and model names match `[a-z0-9][a-z0-9.-]*`. Efforts are unique members of the matrix's effort universe and include `defaultEffort`. `launcherArgv` has exactly `claude` and `codex` arrays of nonempty strings, beginning with an absolute executable path. Require each placeholder shown above exactly once per argv. Placeholders occupy a whole argument; embedded or unknown placeholders are invalid. Substitute each parent-owned value as one argv element, without shell evaluation or environment expansion. Only the extension reference path supports the tilde/relative-path rule above. + +`receiptSchema` is exactly `pstack-runner-v1`: the launcher writes the existing `RunnerReceipt` fields in `scripts/runner/types.ts`, with `provider` set to its declared extension provider. It performs authentication/model preflight and records sanitized evidence, requested identity and effort, actual argv, exit code, model report, usage, and terminal status. Failed preflight must not execute the model. Success requires process exit 0, receipt `status: complete`, `preflight.status: passed`, `modelVerified: true`, matching parent/provider/model/effort/access/path fields, and nonempty output. Provider reports establish identity; `pinned-argv` is allowed only when the CLI does not report it and actual argv pins the exact model. Login alone or a child self-report is insufficient. The parent verifies the receipt against the assigned lane. Invalid or missing evidence fails setup without writing the sheet. + +The parent routes an extension descriptor directly to its declared launcher, with the same unique paths, retained background handle, access boundary, and receipt checks as built-in external lanes. Read the launcher's help before first use. Never send an extension provider to the built-in runner or reinterpret it as a native model. Shared skills consume this contract through the model sheet; no runtime resolver or machine-specific dependency is required. + ## Read-time normalization Normalize configured descriptors before matching them to the matrix or choosing a route. If a provider-qualified Claude model starts with `claude-fable-` or `claude-opus-` and its remaining revision contains only digits and hyphens, replace that model component in memory with `fable` or `opus`. Preserve provider, effort, role, and lane order. Use only the normalized descriptor for native dispatch or runner argv. Never pass the versioned predecessor to Claude. @@ -43,7 +76,7 @@ The top-level harness resolves the route once. A child receives an assigned prov Native dispatch avoids a second CLI startup and its base context. - Claude Code: match the descriptor's `(provider, model)` to one model-matrix row, then dispatch it through `pstack--` using that row's Claude-native agent stem and the descriptor's effort. Those definitions select the rolling model alias, requested effort, and `background: true`. `pstack-fable-max` and `pstack-opus-xhigh` remain in that set. Pass the complete task, grounding paths, access mode, and unique output location in the `Agent` prompt. Retain the task handle and drain it only after fan-out. -- Codex: call `spawn_agent` with the descriptor's model and `reasoning_effort`, the complete task, grounding paths, access mode, and unique output location. Use an isolated worktree for a writer. Codex subagents already run concurrently. +- Codex: call `spawn_agent` with the descriptor's model, `reasoning_effort`, and `fork_turns: "none"` (full-history forks cannot override model or effort), the complete task, grounding paths, access mode, and unique output location. Use an isolated worktree for a writer. Codex subagents already run concurrently. Do not send a same-provider descriptor to the external runner. It rejects that call because the native route is cheaper and already available. diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/extension-contract.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/extension-contract.test.ts new file mode 100644 index 00000000..a29eeb97 --- /dev/null +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/extension-contract.test.ts @@ -0,0 +1,46 @@ +import { expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { EFFORTS, PARENTS, PROVIDERS } from "./types.ts"; + +const reference = readFileSync( + new URL("../../references/provider-dispatch.md", import.meta.url), + "utf8" +); + +test("the documented extension example supplies each lane's complete argv and receipt contract", () => { + const section = reference.split("## Optional dispatch extensions")[1].split("## Read-time normalization")[0]; + const example = JSON.parse(section.match(/```json\n([\s\S]*?)\n```/)![1]); + expect(example.schemaVersion).toBe(1); + expect(example.families).toHaveLength(1); + const family = example.families[0]; + expect(PROVIDERS).not.toContain(family.provider); + expect(family.receiptSchema).toBe("pstack-runner-v1"); + expect(new Set(family.selectableEfforts).size).toBe(family.selectableEfforts.length); + expect(family.selectableEfforts).toContain(family.defaultEffort); + for (const effort of family.selectableEfforts) expect(EFFORTS).toContain(effort); + expect(Object.keys(family.launcherArgv).sort()).toEqual([...PARENTS].sort()); + const values: Record = { + parent: "codex", provider: family.provider, model: family.model, effort: "medium", + mode: "read-only", promptPath: "/tmp/prompt with spaces.md", cwd: "/tmp/work with spaces", + outputPath: "/tmp/output.txt", receiptPath: "/tmp/receipt.json", + }; + for (const parent of PARENTS) { + const argv: string[] = family.launcherArgv[parent]; + expect(argv[0].startsWith("/")).toBe(true); + const placeholders = argv.filter((arg) => arg.startsWith("{")); + expect(placeholders.sort()).toEqual(Object.keys(values).map((key) => `{${key}}`).sort()); + const substituted = argv.map((arg) => arg.startsWith("{") ? values[arg.slice(1, -1)] : arg); + expect(substituted[argv.indexOf("{promptPath}")]).toBe(values.promptPath); + expect(substituted[argv.indexOf("{cwd}")]).toBe(values.cwd); + expect(substituted).toHaveLength(argv.length); + } +}); + +test("setup requires an independent provider before any probe or write", () => { + const setup = readFileSync(new URL("../../../setup-pstack/SKILL.md", import.meta.url), "utf8"); + const step2 = setup.split("### 2. Load current state")[1].split("### 3.")[0]; + expect(step2).toContain("at least two distinct providers"); + expect(step2).toContain("before probing or writing"); + expect(setup).not.toContain("Require at least one selected family"); + expect(setup.split("### 9. Behavioral smoke")[1]).toContain("independent cross-judge"); +}); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/model-matrix.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/model-matrix.test.ts index e0d6df1e..a2e6a914 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/model-matrix.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/model-matrix.test.ts @@ -19,9 +19,10 @@ const MATRIX_HEADER = [ "Default effort", "Selectable efforts", "Claude-native agent stem", + "First-run active", ] as const; -const FAMILY_ORDER = ["fable", "sol", "grok", "opus"] as const; +const DEFAULT_FAMILY_ORDER = ["fable", "sol", "grok", "opus"] as const; const PROVIDERS = ["claude", "codex", "grok"] as const; const DESCRIPTOR_RE = /(claude|codex|grok):[a-z0-9.-]+@(low|medium|high|xhigh|max)/g; @@ -52,7 +53,7 @@ const SETUP_SECTION_ORDER = [ "### 2. Load current state", "### 3. Parse per-family efforts", "### 4. Collect one requested effort per family", - "### 5. Probe the four requested pairs", + "### 5. Probe the selected pairs", "### 6. Render, preserving role families", "### 7. Confirm and commit", ] as const; @@ -65,6 +66,7 @@ interface MatrixRow { defaultEffort: Effort; selectableEfforts: Effort[]; claudeNativeAgentStem: string | null; + firstRunActive: boolean; } function splitRow(line: string): string[] { @@ -89,6 +91,12 @@ function asEffort(value: string): Effort { throw new Error(`not an effort: ${value}`); } +function asFirstRunActive(value: string): boolean { + if (value === "yes") return true; + if (value === "no") return false; + throw new Error(`invalid First-run active value: ${value}`); +} + function parseModelMatrix(markdown: string): MatrixRow[] { const lines = markdown.split(/\r?\n/); const start = lines.findIndex((line) => line.trim() === "## Model matrix"); @@ -106,9 +114,10 @@ function parseModelMatrix(markdown: string): MatrixRow[] { .slice(start + 1, end) .map((line) => line.trim()) .filter((line) => line.startsWith("|")); - if (table.length !== 6) { + const minimumRows = DEFAULT_FAMILY_ORDER.length + 2; + if (table.length < minimumRows) { throw new Error( - `model matrix must be header, separator, and 4 data rows, got ${table.length}` + `model matrix must contain the default families, got ${table.length - 2} data rows` ); } const header = splitRow(table[0]); @@ -118,7 +127,7 @@ function parseModelMatrix(markdown: string): MatrixRow[] { if (!isSeparator(splitRow(table[1]))) { throw new Error("matrix header separator missing"); } - return table.slice(2).map((line) => { + const rows = table.slice(2).map((line) => { const cells = splitRow(line); if (cells.length !== MATRIX_HEADER.length) { throw new Error(`matrix row has ${cells.length} cells: ${line}`); @@ -131,10 +140,14 @@ function parseModelMatrix(markdown: string): MatrixRow[] { defaultEffortRaw, selectableRaw, stemRaw, + firstRunActiveRaw, ] = cells; if (!(PROVIDERS as readonly string[]).includes(provider)) { throw new Error(`invalid provider: ${provider}`); } + if (!/^[a-z0-9.-]+$/.test(model)) { + throw new Error(`invalid model for ${family}: ${model}`); + } const selectableEfforts = selectableRaw.split(/\s+/).map(asEffort); const claudeNativeAgentStem = stemRaw === "-" ? null : stemRaw; if (claudeNativeAgentStem !== null && !/^[a-z0-9-]+$/.test(claudeNativeAgentStem)) { @@ -144,6 +157,7 @@ function parseModelMatrix(markdown: string): MatrixRow[] { throw new Error(`${family} stem must be present iff provider is claude`); } const defaultEffort = asEffort(defaultEffortRaw); + const firstRunActive = asFirstRunActive(firstRunActiveRaw); if (!selectableEfforts.includes(defaultEffort)) { throw new Error(`${family} default effort is not selectable`); } @@ -155,8 +169,35 @@ function parseModelMatrix(markdown: string): MatrixRow[] { defaultEffort, selectableEfforts, claudeNativeAgentStem, + firstRunActive, }; }); + + const families = new Set(); + const providerModels = new Set(); + for (const row of rows) { + if (families.has(row.family)) { + throw new Error(`duplicate family: ${row.family}`); + } + families.add(row.family); + + const providerModel = `${row.provider}:${row.model}`; + if (providerModels.has(providerModel)) { + throw new Error(`duplicate provider/model: ${providerModel}`); + } + providerModels.add(providerModel); + } + + const firstRunFamilies = rows + .filter((row) => row.firstRunActive) + .map((row) => row.family); + if (firstRunFamilies.join("|") !== DEFAULT_FAMILY_ORDER.join("|")) { + throw new Error( + `unexpected first-run families: ${firstRunFamilies.join(", ")}` + ); + } + + return rows; } function defaultDescriptors(rows: MatrixRow[]): string[] { @@ -198,13 +239,17 @@ function firstRunSheet(setup: string): string { } describe("model matrix", () => { - const rows = parseModelMatrix(readFileSync(DISPATCH_PATH, "utf8")); + const dispatch = readFileSync(DISPATCH_PATH, "utf8"); + const rows = parseModelMatrix(dispatch); + const defaultRows = rows.filter((row) => row.firstRunActive); const setup = readFileSync(SETUP_PATH, "utf8"); - const quad = defaultDescriptors(rows); + const quad = defaultDescriptors(defaultRows); it("owns the effort universe and first-run defaults", () => { expect([...EFFORTS]).toEqual(["low", "medium", "high", "xhigh", "max"]); - expect(rows.map((row) => row.family)).toEqual([...FAMILY_ORDER]); + expect(defaultRows.map((row) => row.family)).toEqual([ + ...DEFAULT_FAMILY_ORDER, + ]); for (const row of rows) { expect(row.upstreamChoice.length).toBeGreaterThan(0); expect(row.model.length).toBeGreaterThan(0); @@ -214,7 +259,7 @@ describe("model matrix", () => { ); } expect( - rows.map((row) => [row.family, row.defaultEffort]) + defaultRows.map((row) => [row.family, row.defaultEffort]) ).toEqual([ ["fable", "max"], ["sol", "max"], @@ -229,6 +274,31 @@ describe("model matrix", () => { ["fable", "fable"], ["opus", "opus"], ]); + expect( + rows + .filter((row) => !row.firstRunActive) + .map((row) => [row.family, row.provider, row.model, row.defaultEffort]) + ).toContainEqual(["astra", "codex", "gpt-6-astra", "medium"]); + }); + + it("rejects duplicate families and provider/model pairs", () => { + expect(() => + parseModelMatrix( + dispatch.replace( + "| astra | - | codex | gpt-6-astra |", + "| sol | - | codex | gpt-6-astra |" + ) + ) + ).toThrow("duplicate family: sol"); + + expect(() => + parseModelMatrix( + dispatch.replace( + "| astra | - | codex | gpt-6-astra |", + "| astra | - | codex | gpt-5.6-sol |" + ) + ) + ).toThrow("duplicate provider/model: codex:gpt-5.6-sol"); }); it("ships exactly the declared Claude-native frontier agents", () => { @@ -317,8 +387,8 @@ describe("model matrix", () => { expect(setup).toContain("Do not invent a precedence rule."); expect(setup).toContain("Do not probe or write while any inconsistency is unresolved."); expect(setup).toContain("A failed probe writes nothing:"); - expect(setup).toContain("Run one probe per family"); - expect(setup).toContain("normalized complete role map from step 2"); + expect(setup).toContain("Run one probe per selected family"); + expect(setup).toContain("final in-memory assignments from step 2"); expect(setup).toContain("starts with `claude-fable-` or `claude-opus-`"); expect(setup).toContain("preserving the provider, effort, role, and lane order"); expect(setup).toContain("Show any rolling-alias migrations"); diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.test.ts index b4ebc041..191a96ef 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.test.ts @@ -24,6 +24,69 @@ describe("parseProviderOutput", () => { }); }); + it("extracts Claude terminal evidence from an event array", () => { + const parsed = parseProviderOutput( + "claude", + JSON.stringify([ + { type: "system", subtype: "init", session_id: "init-session" }, + { + type: "assistant", + message: { content: [{ type: "text", text: "progress" }] }, + }, + { + type: "result", + subtype: "success", + is_error: false, + result: "CLAUDE_ARRAY_OK", + session_id: "claude-array-session", + usage: { input_tokens: 14, output_tokens: 4 }, + total_cost_usd: 0.07, + modelUsage: { + "claude-haiku-4-5-20251001": {}, + "claude-fable-9-9": {}, + }, + }, + ]), + "", + "fable" + ); + + expect(parsed).toEqual({ + text: "CLAUDE_ARRAY_OK", + reportedModel: "claude-fable-9-9", + sessionId: "claude-array-session", + usage: { inputTokens: 14, outputTokens: 4 }, + costUsd: 0.07, + }); + }); + + it("rejects Claude event arrays with an error or no final text", () => { + expect(() => + parseProviderOutput( + "claude", + JSON.stringify([ + { + type: "result", + subtype: "error_during_execution", + is_error: true, + result: "partial output", + }, + ]), + "", + "fable" + ) + ).toThrow("reported an error result"); + + expect(() => + parseProviderOutput( + "claude", + JSON.stringify([{ type: "result", subtype: "success", is_error: false }]), + "", + "fable" + ) + ).toThrow("did not contain final text"); + }); + it("extracts Codex JSONL without inventing a provider-reported model", () => { const parsed = parseProviderOutput( "codex", diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.ts index 81ed53d4..78984a7c 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.ts @@ -68,12 +68,24 @@ function parseClaude(stdout: string, requestedModel: string): ParsedOutput { } catch { throw new Error("claude did not emit valid JSON"); } - const value = object(raw); - if (value === null) throw new Error("claude emitted a non-object result"); + let value: JsonObject | null; + if (Array.isArray(raw)) { + value = null; + for (const candidate of raw) { + const event = object(candidate); + if (event?.type === "result") value = event; + } + if (value === null) { + throw new Error("claude result did not contain a terminal event"); + } + } else { + value = object(raw); + if (value === null) throw new Error("claude emitted a non-object result"); + } + if (value.is_error === true) throw new Error("claude reported an error result"); const text = nullableString(value.result); if (text === null) throw new Error("claude result did not contain final text"); - if (value.is_error === true) throw new Error("claude reported an error result"); return { text, diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts index 20743b52..d73b4d6b 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts @@ -76,14 +76,22 @@ if (name === "grok" && args[0] === "models") { console.error("You are not authenticated."); process.exit(0); } + if (process.env.FAKE_GROK_UNSUPPORTED_MODEL === "1") { + console.log("You are using XAI_API_KEY.\\nmodel grok-4.6 is not supported"); + process.exit(0); + } if (process.env.FAKE_GROK_MISSING_MODEL === "1") { - console.log("You are logged in with grok.com.\\nAvailable models:\\n * grok-4.5 (default)"); + console.log("You are logged in with grok.com.\\nAvailable models:\\n * grok-4.60 (default)"); process.exit(0); } if (process.env.FAKE_GROK_UNAUTH === "1") { console.error("Not logged in. Run grok auth login."); process.exit(1); } + if (process.env.FAKE_GROK_API_KEY_AUTH === "1") { + console.log("You are using XAI_API_KEY.\\nAvailable models:\\n * grok-4.6 (default)"); + process.exit(0); + } console.log("You are logged in with grok.com.\\nAvailable models:\\n * grok-4.6 (default)"); process.exit(0); } @@ -238,9 +246,11 @@ beforeEach(() => { delete process.env.FAKE_MODEL_EXITING_PATH; delete process.env.FAKE_REMOVE_EXECUTABLE_AFTER_PREFLIGHT; delete process.env.FAKE_GROK_UNAUTH; + delete process.env.FAKE_GROK_API_KEY_AUTH; delete process.env.FAKE_GROK_TRANSIENT_UNAUTH_PATH; delete process.env.FAKE_GROK_PREFLIGHT_LOG_PATH; delete process.env.FAKE_GROK_MISSING_MODEL; + delete process.env.FAKE_GROK_UNSUPPORTED_MODEL; delete process.env.FAKE_DESCENDANT_HOLDS_PIPES_MS; delete process.env.FAKE_DESCENDANT_PID_PATH; delete process.env.FAKE_SELF_SIGNAL; @@ -262,9 +272,11 @@ afterEach(() => { delete process.env.FAKE_MODEL_EXITING_PATH; delete process.env.FAKE_REMOVE_EXECUTABLE_AFTER_PREFLIGHT; delete process.env.FAKE_GROK_UNAUTH; + delete process.env.FAKE_GROK_API_KEY_AUTH; delete process.env.FAKE_GROK_TRANSIENT_UNAUTH_PATH; delete process.env.FAKE_GROK_PREFLIGHT_LOG_PATH; delete process.env.FAKE_GROK_MISSING_MODEL; + delete process.env.FAKE_GROK_UNSUPPORTED_MODEL; delete process.env.FAKE_DESCENDANT_HOLDS_PIPES_MS; delete process.env.FAKE_DESCENDANT_PID_PATH; delete process.env.FAKE_SELF_SIGNAL; @@ -322,6 +334,27 @@ describe("runLane", () => { }); }); + it("accepts Grok API-key authentication when the requested model is available", async () => { + process.env.FAKE_GROK_API_KEY_AUTH = "1"; + const preflightLog = join(scratch, "grok-api-key.log"); + process.env.FAKE_GROK_PREFLIGHT_LOG_PATH = preflightLog; + const modelStarted = join(scratch, "grok-api-key-model.started"); + process.env.FAKE_MODEL_STARTED_PATH = modelStarted; + const input = options("grok", "grok-api-key"); + const result = await runLane(input); + + expect(result.exitCode).toBe(0); + expect(readFileSync(preflightLog, "utf8")).toBe("attempt\n"); + expect(existsSync(modelStarted)).toBe(true); + expect(receipt(input.receiptPath)).toMatchObject({ + status: "complete", + preflight: { + status: "passed", + evidence: "authenticated; model grok-4.6 available", + }, + }); + }, 10_000); + it("retries a contradictory Grok authentication preflight before running the model", async () => { const transientMarker = join(scratch, "grok-transient-unauth.seen"); const preflightLog = join(scratch, "grok-transient-unauth.log"); @@ -444,6 +477,24 @@ describe("runLane", () => { }); }); + it("rejects an authenticated Grok preflight that explicitly rejects the requested model", async () => { + process.env.FAKE_GROK_UNSUPPORTED_MODEL = "1"; + const preflightLog = join(scratch, "grok-unsupported-model.log"); + process.env.FAKE_GROK_PREFLIGHT_LOG_PATH = preflightLog; + const modelStarted = join(scratch, "grok-unsupported-model.started"); + process.env.FAKE_MODEL_STARTED_PATH = modelStarted; + const input = options("grok", "grok-unsupported-model"); + const result = await runLane(input); + + expect(result.exitCode).toBe(69); + expect(readFileSync(preflightLog, "utf8")).toBe("attempt\n"); + expect(existsSync(modelStarted)).toBe(false); + expect(receipt(input.receiptPath)).toMatchObject({ + status: "unavailable-model", + preflight: { status: "failed" }, + }); + }); + it("kills a timed-out child and preserves a failure receipt", async () => { process.env.FAKE_TIMEOUT = "1"; const input = { ...options("claude"), timeoutMs: 30 }; diff --git a/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts b/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts index 054564a4..1cd2dded 100644 --- a/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts +++ b/plugins/pstack/skills/poteto-mode/scripts/runner/run.ts @@ -350,6 +350,13 @@ async function waitForGrokPreflightRetry( } } +function grokModelAvailable(value: string, model: string): boolean { + const escapedModel = model.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + return new RegExp( + `(^|[^A-Za-z0-9._-])${escapedModel}(?=$|[^A-Za-z0-9._-])` + ).test(value); +} + function preflightPassed(provider: Provider, model: string, result: ProcessResult): boolean { if (result.exitCode !== 0 || result.timedOut) return false; const combined = `${result.stdout}\n${result.stderr}`; @@ -368,8 +375,16 @@ function preflightPassed(provider: Provider, model: string, result: ProcessResul } case "codex": return /logged in/i.test(combined); - case "grok": - return /logged in/i.test(combined) && combined.includes(model); + case "grok": { + const authenticated = + /logged in/i.test(combined) || + /You are using XAI_API_KEY\./.test(combined); + return ( + unavailableStatus(combined) === "child-failed" && + authenticated && + grokModelAvailable(combined, model) + ); + } } } @@ -396,7 +411,7 @@ function preflightFailureStatus( ): ReceiptStatus { const status = unavailableStatus(value); if (status !== "child-failed") return status; - return provider === "grok" && !value.includes(model) + return provider === "grok" && !grokModelAvailable(value, model) ? "unavailable-model" : "unauthenticated"; } diff --git a/plugins/pstack/skills/setup-pstack/SKILL.md b/plugins/pstack/skills/setup-pstack/SKILL.md index 9a0e7441..a35f17a2 100644 --- a/plugins/pstack/skills/setup-pstack/SKILL.md +++ b/plugins/pstack/skills/setup-pstack/SKILL.md @@ -1,11 +1,11 @@ --- name: setup-pstack -description: Configure pstack's provider-qualified models, per-family requested effort, and parent-owned routes per role. Verifies native and external Claude, Codex, and Grok lanes before writing the override sheet. Use for /setup-pstack, "configure pstack models", or changing pstack's model choices. +description: Configure pstack's provider-qualified models, per-family requested effort, and parent-owned routes per role. Verifies selected native and external lanes before writing the override sheet. Use for /setup-pstack, "configure pstack models", or changing pstack's model choices. --- # Setup pstack -Configure one portable model sheet for the current parent harness. Read [`provider-dispatch.md`](../poteto-mode/references/provider-dispatch.md) before probing or writing anything. Its model matrix, descriptor grammar, and route table are the contract. Choose one requested effort per matrix family. Do not add a second configuration file, a runtime resolver, or a weaker-model fallback. +Configure one portable model sheet for the current parent harness. Read [`provider-dispatch.md`](../poteto-mode/references/provider-dispatch.md) before probing or writing anything. Its model matrix, descriptor grammar, and route table are the contract. Choose one requested effort per selected family. Do not add a second configuration file, a runtime resolver, or a weaker-model fallback. Claude Code writes `~/.claude/pstack-models.md` and loads it from `~/.claude/CLAUDE.md` with: @@ -29,25 +29,27 @@ Use the harness and tool surface running this skill: Claude Code or Codex. Envir ### 2. Load current state -Read the current parent-specific sheet when it exists. Before matrix validation, normalize only the rolling-alias predecessors that earlier pstack releases generated. A provider-qualified Claude model is migratable when its model component starts with `claude-fable-` or `claude-opus-` and the remaining revision contains only digits and hyphens. Replace that component in memory with `fable` or `opus`, preserving the provider, effort, role, and lane order. Record each original and normalized descriptor for the confirmation in step 7. This migration is valid loaded state and does not require a separate operator choice. +Read the current parent-specific sheet when it exists. Load its dispatch extension pointer, or the path explicitly supplied by the user, under the provider-dispatch extension contract. Built-in and loaded extension rows form the available family set. A missing sheet starts with the First-run active families. Otherwise derive selected families from loaded role descriptors, then apply explicit user additions or removals. Unselected optional families require no question or probe. Before matrix validation, normalize only the rolling-alias predecessors that earlier pstack releases generated. A provider-qualified Claude model is migratable when its model component starts with `claude-fable-` or `claude-opus-` and the remaining revision contains only digits and hyphens. Replace that component in memory with `fable` or `opus`, preserving the provider, effort, role, and lane order. Record each original and normalized descriptor for the confirmation in step 7. This migration is valid loaded state and does not require a separate operator choice. -Treat the normalized values as current role-to-family assignments. Overlay those rows on the complete first-run role map in step 7. Materialize any missing documented role row from that map on the next successful write. A duplicate or unknown role row is inconsistent state; report it and resolve it before probing. A bare host-native slug from an older sheet is also invalid because it does not say which provider owns it. A versioned Claude model outside the two migration families remains inconsistent state. If the sheet is missing, use the complete first-run role map and the model matrix's Default effort cells. +Treat the normalized values as current role-to-family assignments. Overlay those rows on the complete first-run role map in step 7. Materialize any missing documented role row from that map on the next successful write; if its seed names an unselected family, resolve that lane to a selected family or alias before probing. A duplicate or unknown role row is inconsistent state; report it and resolve it before probing. A bare host-native slug from an older sheet is also invalid because it does not say which provider owns it. A versioned Claude model outside the two migration families remains inconsistent state. If the sheet is missing, use the complete first-run role map and the default families' Default effort cells, subject to the user's supplied choices and caps. + +Before effort collection or probing, apply the user's role changes in memory. Every added family must occupy a role; replace every removed family's occurrence with a selected family or alias. Require selected families from at least two distinct providers so the mandatory independent cross-judge in step 9 can run. Reject a reduced selection without that diversity before probing or writing, and report the missing independent provider. Require the resulting role map's family set to equal the selected set. Keep the loaded assignments by default; resolve missing assignments before probing. Membership persists only through role descriptors. ### 3. Parse per-family efforts -Read the model matrix. Every non-alias value must match `:@`. Map it to exactly one matrix family by `(provider, model)`, require its effort to appear in that row's Selectable efforts cell, and collect the effort. `inherit-parent` and `auto` rows carry no family effort. +Read the available family rows. Treat `Dispatch extension:` as metadata, not a role. Every non-alias value must match `:@`. Map it to exactly one available family by `(provider, model)`, require its effort to appear in that row's Selectable efforts cell, and collect the effort. `inherit-parent` and `auto` rows carry no family effort. -An unmatched provider/model, out-of-domain effort, duplicate role, or unknown role is inconsistent state. Stop, show the conflicting rows verbatim, and ask for an explicit matrix family or alias replacement. If one or more families have mixed efforts, show every conflicting family and role row, then ask for one normalized effort per family from its Selectable efforts cell. Do not invent a precedence rule. Do not probe or write while any inconsistency is unresolved. +Apply explicit user choices and caps before validation. An explicit replacement resolves an older loaded effort; otherwise a value above a cap requires a permitted selection before probing. An unmatched provider/model, out-of-domain effort, duplicate role, or unknown role is inconsistent state. Stop, show the conflicting rows verbatim, and ask for an explicit available family or alias replacement. If one or more selected families still have mixed efforts after explicit user choices, show every conflicting family and role row, then ask for one normalized effort per family from its Selectable efforts cell. Do not invent a precedence rule. Do not probe or write while any inconsistency is unresolved. -One distinct effort per family is the current value. A family with no non-alias occurrence is unassigned; use its matrix Default effort as the proposed value and label it unassigned rather than calling it current. +One distinct effort per family is the current value. For a newly selected family, use its declared default effort constrained by the user's cap as the proposed value; label it proposed rather than current. ### 4. Collect one requested effort per family -Ask exactly four effort questions, one each for Fable, Sol, Grok, and Opus. Name each model, its current or proposed value, and the Selectable efforts from its matrix row. Empty input keeps a current value or accepts the matrix proposal for an unassigned family. On a first run, state the four matrix defaults before asking. On a rerun, state the four parsed values without offering to reset customized role lanes. +Reuse explicit family and effort choices already supplied by the user. For each selected family still missing a permitted choice, ask once, naming its current or proposed value and selectable efforts within the user's cap. Empty input keeps a permitted current value or accepts the permitted proposal. Examples and defaults never override explicit choices or caps; if a default exceeds a cap, propose the highest selectable effort within that cap. Preserve customized role lanes. Continue only when every selected family has one permitted requested effort. -### 5. Probe the four requested pairs +### 5. Probe the selected pairs -Probe only the four selected `provider:model@effort` pairs. Run one probe per family, even when two families share a provider. Do not enumerate or offer older models as substitutes. A failed probe writes nothing: report the failing pair and provider, stop, and keep the active sheet plus parent integration bytes unchanged. A failed first run creates neither artifact. +Probe only the selected `provider:model@effort` pairs. Run one probe per selected family, even when two families share a provider. Do not enumerate or offer older models as substitutes. A failed probe writes nothing: report the failing pair and provider, stop, and keep the active sheet plus parent integration bytes unchanged. A failed first run creates neither artifact. | Family | Pair source | Claude parent route | Codex parent route | Availability proof | |---|---|---|---|---| @@ -56,7 +58,9 @@ Probe only the four selected `provider:model@effort` pairs. Run one probe per fa | Grok | Grok matrix row + selected effort | Grok CLI | Grok CLI | `grok models` must list the requested model; one-turn probe | | Opus | Opus matrix row + selected effort | native Agent `pstack-opus-` | Claude CLI | native one-turn probe or `claude auth status --json` plus one-turn probe | -Use a tiny read-only probe that returns a unique marker. A login-status command alone proves credentials, not that the requested model and effort flags run. Record native and external results separately. Never call the external launcher for the parent's own provider. On a Claude parent, the Fable and Opus probes are one-turn runs of the mapped `pstack--` agent. On a Codex parent, the Sol probe is native `spawn_agent` with the selected `reasoning_effort`. Every other pair uses the external runner with the selected effort flag. +The table describes the default four families. Selected Astra uses `codex exec` from Claude and native `spawn_agent` from Codex, with the same availability proof as Sol. Selected extension families use their loaded contract's route and evidence. + +Use a tiny read-only probe that returns a unique marker. A login-status command alone proves credentials, not that the requested model and effort flags run. Record native and external results separately. Never call the external launcher for the parent's own provider. On a Claude parent, the Fable and Opus probes are one-turn runs of the mapped `pstack--` agent. On a Codex parent, Sol and selected Astra probes use native `spawn_agent` with the selected model, `reasoning_effort`, and `fork_turns: "none"`. Every other pair uses its declared external launcher with the selected effort flag. Receipts and native transcripts prove the requested effort and the route. They do not prove a provider's hidden applied reasoning depth. There is no implicit timeout, weaker-model fallback, same-provider external fallback, or second mutable configuration source. @@ -64,14 +68,13 @@ Receipts and native transcripts prove the requested effort and the route. They d Build the new sheet in memory. Do not write it yet. -- First run: start from the complete role assignments in step 7. -- Rerun: start from the normalized complete role map from step 2, preserving each loaded row's lane order and family (or alias) per lane. +Use the final in-memory assignments from step 2, preserving each lane's family, alias, and order. -After effort selection, ask whether to keep those role-to-family assignments or change named roles. Keeping them is the default. Apply only role changes the operator names; never offer a reset of a customized sheet to the first-run assignments. A changed role may use one of the four probed matrix families, `inherit-parent`, or `auto`. +Role assignments were settled before probing. Any later family change returns to step 2 and requires a probe of the new selected pair before rendering. -Require the final role map to contain at least one descriptor from each matrix family. The sheet stores effort only in role descriptors, so an unassigned family's selection cannot persist without adding a second source of truth. +Require the final role map's family set to equal the selected and successfully probed family sets. The sheet stores effort only in role descriptors, so an unassigned family's selection cannot persist without adding a second source of truth. -Rewrite every matrix-family descriptor to `provider:model@`. Leave `inherit-parent` and `auto` unchanged. An effort-only rerun cannot change a role's family. Changing Grok's effort updates every Grok occurrence and does not move a Sol role onto Grok. Refuse an unqualified slug, an unavailable route, a model other than the four matrix families, or a provider/model mismatch. +Rewrite every selected-family descriptor to `provider:model@`. Leave `inherit-parent` and `auto` unchanged. An effort-only rerun cannot change a role's family. Changing Grok's effort updates every Grok occurrence and does not move a Sol role onto Grok. Refuse an unqualified slug, an unavailable route, a model outside the selected families, or a provider/model mismatch. ### 7. Confirm and commit @@ -79,7 +82,7 @@ Show any rolling-alias migrations as original and normalized descriptors. Then s Why and Reflect require the parent's live MCP surface. Keep their investigator, reviewer, and synthesizer roles on `inherit-parent` or `auto`; the bounded external runner deliberately omits ambient MCPs. `inherit-parent` and `auto` always validate, but say when they reduce a panel's provider diversity. For panel roles, one lane runs per entry. The list length is the fan-out count. `arena cross-judge pool` is a list from which Arena chooses a provider different from the parent and base candidate when possible. `swarm workers` is the default for every worker unless a race explicitly assigns another descriptor. -Every non-alias value must match `:@` and must have passed step 5. +Preserve the loaded extension pointer in the rendered sheet so later dispatch can read its contract. Every non-alias value must match `:@` and must have passed step 5. After the operator confirms, write the in-memory render from step 6. Never paste the example below as the result. It is only the complete first-run role map used to seed step 2; selected efforts and explicit role changes always replace its example values before writing. @@ -109,12 +112,12 @@ interrogate reviewers: claude:fable@max, codex:gpt-5.6-sol@max, grok:grok-4.6@xh Render the parent integration in memory before either write. On Claude, the integration is the single `@~/.claude/pstack-models.md` include in `~/.claude/CLAUDE.md`. On Codex, it is the exact sheet bytes between one `` and `` pair in `~/.codex/AGENTS.md`. Replace that whole bounded block on a rerun. Insert one block at the end on first run. If either marker is missing, duplicated, or reversed, stop and report inconsistent state instead of guessing a boundary. -Snapshot every target's current bytes. Write the sheet and parent integration only after all four probes pass and the operator confirms. Read both targets back and compare them with the in-memory render. If either write or readback fails, restore every snapshot and report the failure. An unchanged rerun must produce byte-identical sheet and integration content after normalization. +Snapshot every target's current bytes. Write the sheet and parent integration only after all selected-family probes pass and the operator confirms. Read both targets back and compare them with the in-memory render. If either write or readback fails, restore every snapshot and report the failure. An unchanged rerun must produce byte-identical sheet and integration content after normalization. Do not copy the model sheet between harnesses without rerunning the parent-specific probes; route availability can differ even on the same host. ### 9. Behavioral smoke -Before declaring setup complete, run one small read-only mixed panel from this parent: all four chosen descriptors, distinct output/receipt paths, and an independent cross-judge. Launch Claude-native agents and every external process in the background with retained handles, then drain them. Verify the native transcript entries and every external receipt. A structural config check or unit test is not a substitute. +Before declaring setup complete, run one small read-only mixed panel from this parent: all selected descriptors, distinct output/receipt paths, and an independent cross-judge chosen from the selected set. Launch Claude-native agents and every external process in the background with retained handles, then drain them. Verify the native transcript entries and every external receipt. A structural config check or unit test is not a substitute. Report the sheet path, parent route table, requested-effort probe results, smoke results, and external elapsed/token/cost receipts. Re-running this skill re-probes and updates the same sheet. Do not claim the provider exposed hidden applied-effort observability. diff --git a/tests/skill-collision-repro.sh b/tests/skill-collision-repro.sh index 22db0d8e..da870dce 100755 --- a/tests/skill-collision-repro.sh +++ b/tests/skill-collision-repro.sh @@ -91,6 +91,7 @@ canon_quad="$(awk ' } family = cells[1] if (family == "Family" || family ~ /^:?-+:?$/) next + if (cells[8] != "yes") next provider = cells[3] model = cells[4] effort = cells[5]