Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ Start the background process, continue launching the other lanes, then drain the

The runner and its preflight have no implicit timeout. Do not invent a duration from role, mode, or a convenient round number; real implementation lanes can run for 90 minutes or much longer. Pass `--timeout` only when the user, an external service deadline, or a measured task contract supplies a real bound. That value starts at wrapper entry, before module loading and argument parsing, and remains one absolute deadline across setup, preflight, model execution, and output capture. It is never a fresh allowance per child, and long waits are armed in runtime-safe chunks without shortening the supplied deadline. Otherwise supervise liveness through the retained background task/session handle and cancel manually only on evidence that the run is dead. Cancel through that retained handle so the runner receives SIGINT or SIGTERM, sends it to an active child when one remains, stops waiting on inherited output pipes, removes the empty output reservation, and writes a `cancelled` receipt. Preserve that receipt; a retry is a new attempt with new unique output and receipt paths. Unchanged running state is not a dropout, and Claude's ten-minute foreground ceiling is never a reason to terminate a healthy lane.

Read-only mode maps to Claude plan mode with project-only settings and an explicit tool list, Codex's read-only sandbox, and Grok plan mode plus its `read-only` sandbox and read-oriented tool list. Grok's built-in read-only profile deliberately keeps its own state and system temporary directories writable, so point a read-only Grok lane at the actual checkout rather than a worktree under `/tmp`, `/var/tmp`, or the host's temporary directory. `isolated-write` maps to Claude `acceptEdits` with project-only settings, Codex `workspace-write`, and Grok `acceptEdits` plus its `workspace` sandbox and write-capable tool list. Give every writer only a dedicated worktree or output directory. Never route a writer into the primary checkout.
Read-only mode maps to Claude plan mode with project-only settings and an explicit tool list, Codex's read-only sandbox, and Grok auto mode plus its `read-only` sandbox and read-oriented tool list. Headless Grok uses `auto` in both modes to report blocked tool calls to the model without cancelling the turn on a permission prompt. Grok's built-in read-only profile deliberately keeps its own state and system temporary directories writable, so point a read-only Grok lane at the actual checkout rather than a worktree under `/tmp`, `/var/tmp`, or the host's temporary directory. `isolated-write` maps to Claude `acceptEdits` with project-only settings, Codex `workspace-write`, and Grok auto mode plus its `workspace` sandbox and write-capable tool list. Give every writer only a dedicated worktree or output directory. Never route a writer into the primary checkout.

Every concurrent external lane needs distinct prompt, output, and receipt paths. The launcher reserves output and receipt paths exclusively and refuses to overwrite them.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ describe("invocationCommand", () => {
"--reasoning-effort",
"xhigh",
"--permission-mode",
"plan",
"auto",
"--sandbox",
"read-only",
"--tools",
Expand Down Expand Up @@ -124,7 +124,7 @@ describe("invocationCommand", () => {
expect(grok.args).toEqual(
expect.arrayContaining([
"--permission-mode",
"acceptEdits",
"auto",
"--sandbox",
"workspace",
"--tools",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ export function invocationCommand(options: RunnerOptions): CommandSpec {
"--reasoning-effort",
options.effort,
"--permission-mode",
permissionMode(options.mode),
"auto",
"--sandbox",
grokSandbox(options.mode),
"--tools",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { describe, expect, it } from "bun:test";
import { parseProviderOutput, reportedModelMatches } from "./parse-output.ts";
import { parseProviderOutput, ProviderResultError, reportedModelMatches } from "./parse-output.ts";

describe("parseProviderOutput", () => {
it("extracts Claude text, model, usage, cost, and session", () => {
Expand Down Expand Up @@ -236,3 +236,52 @@ describe("parseProviderOutput", () => {
).toThrow("final agent message");
});
});


describe("Grok terminal failures (#39)", () => {
for (const marker of [
{ stop_reason: "cancelled" }, { stop_reason: "canceled" },
{ subtype: "cancelled" }, { subtype: "canceled" },
{ errors: ["PermissionCancelled: blocked command"] },
{ errors: ["User canceled the execution"] },
]) {
it(`recognizes cancellation ${JSON.stringify(marker)}`, () => {
const terminal = { type: "result", subtype: "error_during_execution", is_error: true, session_id: "s1", ...marker };
try {
parseProviderOutput("grok", JSON.stringify(terminal), "", "grok-4.6");
throw new Error("accepted cancellation");
} catch (error) {
expect(error).toBeInstanceOf(ProviderResultError);
if (!(error instanceof ProviderResultError)) throw error;
expect(error.status).toBe("cancelled");
expect(error.metadata.sessionId).toBe("s1");
}
});
}

for (const detail of [
{ errors: [null, "API unavailable", 42, "retry later"], expected: "API unavailable\nretry later" },
{ error: "API unavailable", expected: "API unavailable" },
{ error: { message: "API unavailable" }, expected: "API unavailable" },
{ result: "API unavailable", expected: "API unavailable" },
{ expected: "grok reported error_during_execution" },
]) {
it(`preserves error text: ${detail.expected}`, () => {
const { expected, ...fields } = detail;
expect(() => parseProviderOutput("grok", JSON.stringify({
type: "result", subtype: "error_during_execution", is_error: true, ...fields,
}), "", "grok-4.6")).toThrow(expected);
});
}

it("uses the last terminal event and rejects missing status fields", () => {
const success = { type: "result", subtype: "success", is_error: false, result: "earlier" };
const failure = { type: "result", subtype: "error_during_execution", is_error: true, errors: ["last error"] };
expect(() => parseProviderOutput("grok", [success, failure].map((event) => JSON.stringify(event)).join("\n"), "", "grok-4.6"))
.toThrow("last error");
for (const terminal of [{ type: "result", subtype: "success" }, { type: "result", is_error: true }]) {
expect(() => parseProviderOutput("grok", JSON.stringify(terminal), "", "grok-4.6"))
.toThrow("valid terminal status");
}
});
});
37 changes: 30 additions & 7 deletions plugins/pstack/skills/poteto-mode/scripts/runner/parse-output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,17 @@ import {
isRollingClaudeAlias,
} from "./model-aliases.ts";

export class ProviderResultError extends Error {
constructor(
message: string,
readonly status: "cancelled" | "child-failed",
readonly metadata: Omit<ParsedOutput, "text">
) {
super(message);
this.name = "ProviderResultError";
}
}

type JsonObject = Record<string, unknown>;

function object(value: unknown): JsonObject | null {
Expand Down Expand Up @@ -107,19 +118,31 @@ function parseGrok(stdout: string, requestedModel: string): ParsedOutput {
}

if (result === null) throw new Error("grok result did not contain a terminal event");
if (result.is_error === true || result.subtype !== "success") {
throw new Error("grok reported an error result");
if (nullableString(result.subtype) === null || typeof result.is_error !== "boolean") {
throw new Error("grok result did not contain a valid terminal status");
}
const text = nullableString(result.result);
if (text === null) throw new Error("grok result did not contain final text");

return {
text,
const metadata = {
reportedModel: modelFromUsage(result.modelUsage, "grok", requestedModel),
sessionId: nullableString(result.session_id),
usage: normalizedUsage(result.usage),
costUsd: finiteNumber(result.total_cost_usd) ?? null,
};
if (result.is_error || result.subtype !== "success") {
const errors = Array.isArray(result.errors)
? result.errors.filter((entry): entry is string => typeof entry === "string" && entry.length > 0)
: [];
const message = errors.join("\n") || nullableString(result.error)
|| nullableString(object(result.error)?.message) || nullableString(result.result)
|| `grok reported ${result.subtype}`;
const cancelled = result.stop_reason === "cancelled" || result.stop_reason === "canceled"
|| result.subtype === "cancelled" || result.subtype === "canceled"
|| /^(?:User cancel(?:led|ed) the execution|PermissionCancelled)/i.test(message);
throw new ProviderResultError(message, cancelled ? "cancelled" : "child-failed", metadata);
}
const text = nullableString(result.result);
if (text === null) throw new Error("grok result did not contain final text");

return { text, ...metadata };
}

// pstack-flex: `opencode run --format json` prints one event per line. The last
Expand Down
120 changes: 120 additions & 0 deletions plugins/pstack/skills/poteto-mode/scripts/runner/run.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,11 @@ if (name === "grok" && args[0] === "models") {
if (process.env.FAKE_GROK_PREFLIGHT_LOG_PATH) {
appendFileSync(process.env.FAKE_GROK_PREFLIGHT_LOG_PATH, "attempt\\n");
}
if (process.env.FAKE_GROK_PREFLIGHT_OUTPUT !== undefined) {
console.log(process.env.FAKE_GROK_PREFLIGHT_OUTPUT);
if (process.env.FAKE_GROK_PREFLIGHT_ERROR) console.error(process.env.FAKE_GROK_PREFLIGHT_ERROR);
process.exit(0);
}
const transientMarker = process.env.FAKE_GROK_TRANSIENT_UNAUTH_PATH;
if (transientMarker && !existsSync(transientMarker)) {
writeFileSync(transientMarker, String(process.pid));
Expand Down Expand Up @@ -282,6 +287,8 @@ beforeEach(() => {
delete process.env.FAKE_MODEL_EXITING_PATH;
delete process.env.FAKE_REMOVE_EXECUTABLE_AFTER_PREFLIGHT;
delete process.env.FAKE_GROK_UNAUTH;
delete process.env.FAKE_GROK_PREFLIGHT_OUTPUT;
delete process.env.FAKE_GROK_PREFLIGHT_ERROR;
delete process.env.FAKE_GROK_TRANSIENT_UNAUTH_PATH;
delete process.env.FAKE_GROK_PREFLIGHT_LOG_PATH;
delete process.env.FAKE_GROK_MISSING_MODEL;
Expand All @@ -307,6 +314,8 @@ afterEach(() => {
delete process.env.FAKE_MODEL_EXITING_PATH;
delete process.env.FAKE_REMOVE_EXECUTABLE_AFTER_PREFLIGHT;
delete process.env.FAKE_GROK_UNAUTH;
delete process.env.FAKE_GROK_PREFLIGHT_OUTPUT;
delete process.env.FAKE_GROK_PREFLIGHT_ERROR;
delete process.env.FAKE_GROK_TRANSIENT_UNAUTH_PATH;
delete process.env.FAKE_GROK_PREFLIGHT_LOG_PATH;
delete process.env.FAKE_GROK_MISSING_MODEL;
Expand Down Expand Up @@ -1501,3 +1510,114 @@ describe("childEnvironment", () => {
});
});
});


describe("Grok authentication and terminal receipts (#39)", () => {
for (const auth of ["You are logged in with grok.com.", "You are using XAI_API_KEY."]) {
it(`accepts ${auth} with an exact model token`, async () => {
process.env.FAKE_GROK_PREFLIGHT_OUTPUT = `${auth}\nAvailable models: [grok-4.6] (default)`;
const log = join(scratch, "preflight.log");
process.env.FAKE_GROK_PREFLIGHT_LOG_PATH = log;
const input = options("grok");
const result = await runLane(input);
expect(result.exitCode).toBe(0);
expect(readFileSync(log, "utf8")).toBe("attempt\n");
expect(readFileSync(input.outputPath, "utf8")).toBe("GROK_OK");
expect(result.receipt.preflight.status).toBe("passed");
});
}

for (const failure of ["unauthenticated", "You are not authenticated.", "Not logged in."]) {
it(`rejects contradictory API-key authentication: ${failure}`, async () => {
process.env.FAKE_GROK_PREFLIGHT_OUTPUT = "You are using XAI_API_KEY.\nAvailable models: grok-4.6";
process.env.FAKE_GROK_PREFLIGHT_ERROR = failure;
const log = join(scratch, "preflight.log");
const started = join(scratch, "model.started");
process.env.FAKE_GROK_PREFLIGHT_LOG_PATH = log;
process.env.FAKE_MODEL_STARTED_PATH = started;
const input = options("grok");
const result = await runLane(input);
expect(result.exitCode).toBe(77);
expect(result.receipt.status).toBe("unauthenticated");
expect(readFileSync(log, "utf8")).toBe("attempt\nattempt\n");
expect(result.receipt.preflight.evidence).toContain("attempt 2 failed");
expect(existsSync(started)).toBe(false);
expect(existsSync(input.outputPath)).toBe(false);
}, 10_000);
}

for (const listed of ["grok-4.60", "xgrok-4.6", "grok-4.6-build", "grok-4x6"]) {
it(`rejects neighboring model ${listed}`, async () => {
process.env.FAKE_GROK_PREFLIGHT_OUTPUT = `You are using XAI_API_KEY.\nAvailable models: ${listed}`;
const log = join(scratch, "preflight.log");
const started = join(scratch, "model.started");
process.env.FAKE_GROK_PREFLIGHT_LOG_PATH = log;
process.env.FAKE_MODEL_STARTED_PATH = started;
const result = await runLane(options("grok"));
expect(result.exitCode).toBe(69);
expect(result.receipt.status).toBe("unavailable-model");
expect(readFileSync(log, "utf8")).toBe("attempt\n");
expect(existsSync(started)).toBe(false);
});
}

it("rejects an explicit model error despite successful login and listing", async () => {
process.env.FAKE_GROK_PREFLIGHT_OUTPUT = "You are logged in.\nAvailable models: grok-4.6\n" + "startup info\n".repeat(500);
process.env.FAKE_GROK_PREFLIGHT_ERROR = "model grok-4.6 is not supported";
const log = join(scratch, "preflight.log");
process.env.FAKE_GROK_PREFLIGHT_LOG_PATH = log;
const result = await runLane(options("grok"));
expect(result.exitCode).toBe(69);
expect(result.receipt.status).toBe("unavailable-model");
expect(readFileSync(log, "utf8")).toBe("attempt\n");
});

for (const childExit of [0, 1]) {
for (const scenario of ["cancelled", "api-error", "auth-error", "model-error", "success"] as const) {
it(`preserves ${scenario} metadata at child exit ${childExit}`, async () => {
const message = scenario === "api-error" ? "API unavailable"
: scenario === "auth-error" ? "You are not authenticated."
: scenario === "model-error" ? "model grok-4.6 is not supported"
: "User cancelled the execution";
const terminal = {
type: "result", subtype: scenario === "success" ? "success" : "error_during_execution",
is_error: scenario !== "success", stop_reason: scenario === "cancelled" ? "cancelled" : "end_turn",
errors: scenario === "success" ? [] : [message], result: "partial or final text",
session_id: "grok-terminal-session", modelUsage: { "grok-4.6-build": {} },
usage: { input_tokens: 30, output_tokens: 4, total_tokens: 34 }, total_cost_usd: 0.02,
};
writeFileSync(join(bin, "grok"), `#!/usr/bin/env bun
if (process.argv[2] === "models") {
console.log("You are using XAI_API_KEY. Available models: grok-4.6");
process.exit(0);
}
console.log(JSON.stringify({type:"system",skills:Array(1000).fill("synthetic-skill")}));
console.error("startup warning\\n".repeat(500));
console.log(${JSON.stringify(JSON.stringify(terminal))});
process.exit(${childExit});
`);
const input = options("grok");
const result = await runLane(input);
const status = scenario === "success" ? (childExit === 0 ? "complete" : "child-failed")
: scenario === "cancelled" ? "cancelled"
: scenario === "auth-error" ? "unauthenticated"
: scenario === "model-error" ? "unavailable-model" : "child-failed";
expect(receipt(input.receiptPath)).toMatchObject({
status, exitCode: childExit, sessionId: "grok-terminal-session",
reportedModel: "grok-4.6-build", modelVerified: true, modelEvidence: "provider-report",
usage: { inputTokens: 30, outputTokens: 4, totalTokens: 34 }, costUsd: 0.02,
});
if (status === "complete") {
expect(result.exitCode).toBe(0);
expect(readFileSync(input.outputPath, "utf8")).toBe("partial or final text");
} else {
expect(result.exitCode).not.toBe(0);
expect(existsSync(input.outputPath)).toBe(false);
expect(result.receipt.error?.message).toBe(scenario === "success" ? "child exited with status 1" : message);
expect(result.receipt.error?.evidence).toContain('"session_id":"grok-terminal-session"');
expect(result.receipt.error?.evidence.length).toBeLessThanOrEqual(4_000);
}
});
}
}
});
Loading
Loading