From 98b78686c6f069849a3ea4fde4cd7d329cc69eeb Mon Sep 17 00:00:00 2001 From: Huishi Date: Tue, 22 Sep 2026 15:34:06 -0400 Subject: [PATCH 1/5] feat(runner): key a runner run to its flow with --flow-id A run on an interactive runner never saw QAWOLF_WORKFLOW_ID, so a flow that names and cleans up fixtures by it read undefined and Tester took the id for unavailable. `qawolf runner run --flow-id` names the flow, falling back to QAWOLF_WORKFLOW_ID from the shell, which the pod an AI Job runs on exports; the platform stamps it onto the run as QAWOLF_WORKFLOW_ID. The runner SDK's run verb takes the same flowId. Takes @qawolf/api-contracts 0.64.0, which carries the field. That version also adds runner.record and runner.recordings, listed with the rest of the hand-written runner group so the generator does not mint them. --- .changeset/runner-run-flow-id.md | 5 + skills/qawolf-cli/SKILL.md | 4 +- skills/qawolf-cli/references/runner.md | 14 +++ src/commands/__snapshots__/help.test.ts.snap | 5 + src/commands/qawolfCliSkill.template.md | 4 +- src/commands/runner/run.register.ts | 9 +- src/core/interactiveRunner/runFlowId.ts | 21 ++++ src/core/messages/interactiveRunner/run.ts | 2 + src/domains/interactiveRunner/prepareRun.ts | 8 ++ .../interactiveRunner/runFlow.delta.test.ts | 2 + .../runFlow.environment.test.ts | 1 + .../interactiveRunner/runFlow.files.test.ts | 3 + .../interactiveRunner/runFlow.flowId.test.ts | 96 +++++++++++++++++++ .../interactiveRunner/runFlow.follow.test.ts | 1 + .../runFlow.selection.test.ts | 1 + src/domains/interactiveRunner/runFlow.test.ts | 3 + src/domains/interactiveRunner/runFlow.ts | 3 + .../interactiveRunner/sendRunFlowRequest.ts | 2 + src/domains/interactiveRunner/submitRun.ts | 1 + src/runnerSdk/runVerbs.ts | 3 + src/runnerSdk/types.ts | 1 + 21 files changed, 186 insertions(+), 3 deletions(-) create mode 100644 .changeset/runner-run-flow-id.md create mode 100644 src/core/interactiveRunner/runFlowId.ts create mode 100644 src/domains/interactiveRunner/runFlow.flowId.test.ts diff --git a/.changeset/runner-run-flow-id.md b/.changeset/runner-run-flow-id.md new file mode 100644 index 000000000..0a4014444 --- /dev/null +++ b/.changeset/runner-run-flow-id.md @@ -0,0 +1,5 @@ +--- +"@qawolf/cli": minor +--- + +`qawolf runner run` takes `--flow-id`, the QA Wolf flow the run is for, and falls back to `QAWOLF_WORKFLOW_ID` when the flag is absent. The run then sees that id as `QAWOLF_WORKFLOW_ID`, the same value a platform run of the flow sees, so fixtures a flow names and cleans up by its workflow id match across both. A run that names no flow is unchanged. The `@qawolf/cli/runner-sdk` `run` verb takes the same `flowId`. diff --git a/skills/qawolf-cli/SKILL.md b/skills/qawolf-cli/SKILL.md index da6032fce..aa3ab4c20 100644 --- a/skills/qawolf-cli/SKILL.md +++ b/skills/qawolf-cli/SKILL.md @@ -40,7 +40,9 @@ exported variables, so pass the selected environment explicitly instead of rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` -when neither it nor `--env-file` is passed. +when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow +the run is for, falling back to `QAWOLF_WORKFLOW_ID`; the run reads that id as +`process.env.QAWOLF_WORKFLOW_ID`, and one that names no flow reads `undefined`. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/skills/qawolf-cli/references/runner.md b/skills/qawolf-cli/references/runner.md index 0c112b002..cccf15d89 100644 --- a/skills/qawolf-cli/references/runner.md +++ b/skills/qawolf-cli/references/runner.md @@ -477,6 +477,20 @@ The lines-file has to be one of the files that travel, so it lives under the directory you run from. A range whose file is not collected is refused before a runner is addressed, naming the path. +### Naming the flow the run is for + +`--flow-id` names the QA Wolf flow the run is for, and the run sees it as +`QAWOLF_WORKFLOW_ID`, the same value a platform run of that flow sees. A flow +that names fixtures by `process.env.QAWOLF_WORKFLOW_ID` and cleans them up by it +then behaves the same on a runner as on the platform. A run with no flag falls +back to `QAWOLF_WORKFLOW_ID` from the shell, which the pod an AI Job runs on +already exports; a run with neither is given no `QAWOLF_WORKFLOW_ID`, and the +flow reads it as `undefined`. That is not a sign the id is unavailable. + +```sh +qawolf runner run flows/checkout.flow.ts --flow-id +``` + ### Giving the run environment variables There are two ways, and a run takes one of them. `--env-id` names a QA Wolf diff --git a/src/commands/__snapshots__/help.test.ts.snap b/src/commands/__snapshots__/help.test.ts.snap index 879d1f4ef..2535bf346 100644 --- a/src/commands/__snapshots__/help.test.ts.snap +++ b/src/commands/__snapshots__/help.test.ts.snap @@ -369,6 +369,10 @@ Options: server and no size limit applies to them --env-file Dotenv file on this machine whose variables the run is given. Pass this or --env-id, not both + --flow-id QA Wolf flow this run is for. Defaults to + QAWOLF_WORKFLOW_ID. The run sees it as + QAWOLF_WORKFLOW_ID, the same value a platform run of that + flow sees, so fixtures named and cleaned up by it match --lines Run only these 1-indexed lines against the browser as it stands, instead of the whole flow from a fresh one --lines-file File the --lines range lives in. Defaults to ; @@ -388,6 +392,7 @@ Examples: $ qawolf runner run flows/checkout.flow.ts --lines 12-40 $ qawolf runner run flows/checkout.flow.ts --lines 4-9 --lines-file pages/login.ts $ qawolf runner run flows/checkout.flow.ts --env-id staging + $ qawolf runner run flows/checkout.flow.ts --flow-id " `; diff --git a/src/commands/qawolfCliSkill.template.md b/src/commands/qawolfCliSkill.template.md index f272e2f03..38e73c6bf 100644 --- a/src/commands/qawolfCliSkill.template.md +++ b/src/commands/qawolfCliSkill.template.md @@ -40,7 +40,9 @@ exported variables, so pass the selected environment explicitly instead of rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` -when neither it nor `--env-file` is passed. +when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow +the run is for, falling back to `QAWOLF_WORKFLOW_ID`; the run reads that id as +`process.env.QAWOLF_WORKFLOW_ID`, and one that names no flow reads `undefined`. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/src/commands/runner/run.register.ts b/src/commands/runner/run.register.ts index 3ed6a579b..38edd2db6 100644 --- a/src/commands/runner/run.register.ts +++ b/src/commands/runner/run.register.ts @@ -15,11 +15,13 @@ Examples: $ qawolf runner run flows/checkout.flow.ts --follow --logs $ qawolf runner run flows/checkout.flow.ts --lines 12-40 $ qawolf runner run flows/checkout.flow.ts --lines 4-9 --lines-file pages/login.ts - $ qawolf runner run flows/checkout.flow.ts --env-id staging`; + $ qawolf runner run flows/checkout.flow.ts --env-id staging + $ qawolf runner run flows/checkout.flow.ts --flow-id `; type RunFlags = { envFile?: string; envId?: string; + flowId?: string; follow: boolean; lines?: string; linesFile?: string; @@ -68,6 +70,10 @@ export function registerRunCommand( "--env-file ", "Dotenv file on this machine whose variables the run is given. Pass this or --env-id, not both", ) + .option( + "--flow-id ", + "QA Wolf flow this run is for. Defaults to QAWOLF_WORKFLOW_ID. The run sees it as QAWOLF_WORKFLOW_ID, the same value a platform run of that flow sees, so fixtures named and cleaned up by it match", + ) .option( "--lines ", "Run only these 1-indexed lines against the browser as it stands, instead of the whole flow from a fresh one", @@ -91,6 +97,7 @@ export function registerRunCommand( entryPoint: flowFile, envFile: opts.envFile, envId: opts.envId, + flowId: opts.flowId, follow: opts.follow, lines: opts.lines, linesFile: opts.linesFile, diff --git a/src/core/interactiveRunner/runFlowId.ts b/src/core/interactiveRunner/runFlowId.ts new file mode 100644 index 000000000..645d1ff13 --- /dev/null +++ b/src/core/interactiveRunner/runFlowId.ts @@ -0,0 +1,21 @@ +import { interactiveRunnerMessages } from "~/core/messages/index.js"; + +/** + * The flow a run is for: the flag when given, else `QAWOLF_WORKFLOW_ID` from + * the shell, which the pod an AI Job runs on exports. A blank flag is refused + * rather than read as "none", the way a blank `--env-id` is. + */ +export function resolveRunFlowId({ + env, + flag, +}: { + env: Readonly>; + flag: string | undefined; +}): { ok: true; flowId: string | undefined } | { ok: false; error: string } { + const explicit = flag?.trim(); + if (explicit === "") { + return { error: interactiveRunnerMessages.flowIdBlank, ok: false }; + } + const fromEnvVar = env["QAWOLF_WORKFLOW_ID"]?.trim(); + return { flowId: explicit ?? (fromEnvVar || undefined), ok: true }; +} diff --git a/src/core/messages/interactiveRunner/run.ts b/src/core/messages/interactiveRunner/run.ts index 69700dfcc..3cc01f922 100644 --- a/src/core/messages/interactiveRunner/run.ts +++ b/src/core/messages/interactiveRunner/run.ts @@ -10,6 +10,8 @@ export const runMessages = { "--env-id was given nothing. Pass the id or alias of a QA Wolf environment, or drop the flag to send no environment.", envIdWithEnvFile: "--env-id and --env-file both give the run its environment, so only one may be passed. Use --env-id for a QA Wolf environment, which QA Wolf reads itself, or --env-file for a dotenv file on this machine.", + flowIdBlank: + "--flow-id was given nothing. Pass the id of the QA Wolf flow this run is for, or drop the flag to key the run to no flow.", fileNotCollected: (path: string) => `"${path}" is not one of the files that travel to a runner. It must be inside the current directory and end in .ts, .tsx, .js, .mjs, .cjs or .json.`, fileUnreadable: (path: string, reason: string) => diff --git a/src/domains/interactiveRunner/prepareRun.ts b/src/domains/interactiveRunner/prepareRun.ts index f50032578..ad7ba6b23 100644 --- a/src/domains/interactiveRunner/prepareRun.ts +++ b/src/domains/interactiveRunner/prepareRun.ts @@ -6,6 +6,7 @@ import { type BuiltRunEnvironment, buildRunEnvironment, } from "~/core/interactiveRunner/runEnvironment.js"; +import { resolveRunFlowId } from "~/core/interactiveRunner/runFlowId.js"; import { checkRunFiles, describeRunFilesCheck, @@ -23,6 +24,7 @@ export type PreparedRun = environment: Record | undefined; environmentId: string | undefined; files: RunFiles; + flowId: string | undefined; selection: RunSelection | undefined; } | { ok: false; error: string; exitCode: number }; @@ -44,6 +46,7 @@ export async function prepareRun( entryPointPath: string; envFile: string | undefined; envId: string | undefined; + flowId: string | undefined; lines: string | undefined; linesFile: string | undefined; }, @@ -65,6 +68,9 @@ export async function prepareRun( explicitEnvId ?? (options.envFile === undefined && fromEnvVar ? fromEnvVar : undefined); + const flow = resolveRunFlowId({ env: deps.env, flag: options.flowId }); + if (!flow.ok) return refused(flow.error, exitCodes.invalidArgs); + const linesFilePath = options.linesFile === undefined ? undefined @@ -95,6 +101,7 @@ export async function prepareRun( environment: given, environmentId, files, + flowId: flow.flowId, ok: true, selection: undefined, } @@ -118,6 +125,7 @@ export async function prepareRun( environment: given, environmentId, files, + flowId: flow.flowId, ok: true, selection: built.selection, } diff --git a/src/domains/interactiveRunner/runFlow.delta.test.ts b/src/domains/interactiveRunner/runFlow.delta.test.ts index 81a21fa0e..4ad62ebc1 100644 --- a/src/domains/interactiveRunner/runFlow.delta.test.ts +++ b/src/domains/interactiveRunner/runFlow.delta.test.ts @@ -35,6 +35,7 @@ async function run( entryPoint: "flow.ts", envFile: undefined, envId: undefined, + flowId: undefined, follow: false, lines: undefined, linesFile: undefined, @@ -173,6 +174,7 @@ describe("handleRunnerRun file delta", () => { entryPoint: "flow.ts", envFile: undefined, envId: undefined, + flowId: undefined, follow: false, lines: undefined, linesFile: undefined, diff --git a/src/domains/interactiveRunner/runFlow.environment.test.ts b/src/domains/interactiveRunner/runFlow.environment.test.ts index 25c65eb02..32d7ee2ab 100644 --- a/src/domains/interactiveRunner/runFlow.environment.test.ts +++ b/src/domains/interactiveRunner/runFlow.environment.test.ts @@ -42,6 +42,7 @@ describe("handleRunnerRun with --env-file and --env-id", () => { entryPoint: "flow.ts", envFile, envId, + flowId: undefined, follow: false, lines: undefined, linesFile: undefined, diff --git a/src/domains/interactiveRunner/runFlow.files.test.ts b/src/domains/interactiveRunner/runFlow.files.test.ts index a58ce96f6..c610b2905 100644 --- a/src/domains/interactiveRunner/runFlow.files.test.ts +++ b/src/domains/interactiveRunner/runFlow.files.test.ts @@ -14,6 +14,7 @@ describe("handleRunnerRun file refusals", () => { follow: false, envFile: undefined, envId: undefined, + flowId: undefined, lines: undefined, linesFile: undefined, runner: "ci", @@ -40,6 +41,7 @@ describe("handleRunnerRun file refusals", () => { follow: false, envFile: undefined, envId: undefined, + flowId: undefined, lines: undefined, linesFile: undefined, runner: "ci", @@ -71,6 +73,7 @@ describe("handleRunnerRun file refusals", () => { follow: false, envFile: undefined, envId: undefined, + flowId: undefined, lines: undefined, linesFile: undefined, runner: undefined, diff --git a/src/domains/interactiveRunner/runFlow.flowId.test.ts b/src/domains/interactiveRunner/runFlow.flowId.test.ts new file mode 100644 index 000000000..241dd2d11 --- /dev/null +++ b/src/domains/interactiveRunner/runFlow.flowId.test.ts @@ -0,0 +1,96 @@ +import { describe, expect, it } from "bun:test"; + +import { makeAuthCtx, makeTestDeps } from "./deps.testUtils.js"; +import { handleRunnerRun } from "./runFlow.js"; + +const submitted = { outcome: "success" as const, runId: "run-a" }; + +describe("handleRunnerRun with --flow-id", () => { + const run = async ({ + env = {}, + flowId, + }: { + env?: Record; + flowId?: string; + }) => { + const { callPublicApi, ctx } = makeAuthCtx(); + callPublicApi.mockResolvedValue({ ok: true, value: submitted }); + const result = await handleRunnerRun( + ctx, + { + entryPoint: "flow.ts", + envFile: undefined, + envId: undefined, + flowId, + follow: false, + lines: undefined, + linesFile: undefined, + logs: false, + recorderEvents: false, + runEvents: false, + runner: "ci", + timeout: undefined, + }, + makeTestDeps({ + env, + collectRunFiles: async () => ({ + files: { + "flow.ts": "export default {};", + "package.json": "{}", + }, + unresolvedImports: [], + }), + }), + ); + return { callPublicApi, result }; + }; + + const sentRequest = (callPublicApi: { mock: { calls: unknown[][] } }) => + callPublicApi.mock.calls[0]?.[1] ?? {}; + + it("names the flow the run is for", async () => { + const { callPublicApi, result } = await run({ flowId: "flow-7c2e" }); + + expect(result).toBeUndefined(); + expect(sentRequest(callPublicApi)).toMatchObject({ flowId: "flow-7c2e" }); + }); + + it("trims the id it was given", async () => { + const { callPublicApi } = await run({ flowId: " flow-7c2e " }); + + expect(sentRequest(callPublicApi)).toMatchObject({ flowId: "flow-7c2e" }); + }); + + it("names no flow when the flag is absent and nothing is exported", async () => { + const { callPublicApi } = await run({}); + + expect(Object.hasOwn(sentRequest(callPublicApi), "flowId")).toBe(false); + }); + + // A pod that runs an AI Job carries the job's flow this way, so a run started + // from its shell is keyed to the flow the way a platform run of it is. + it("falls back to QAWOLF_WORKFLOW_ID when the flag is absent", async () => { + const { callPublicApi } = await run({ + env: { QAWOLF_WORKFLOW_ID: "flow-7c2e" }, + }); + + expect(sentRequest(callPublicApi)).toMatchObject({ flowId: "flow-7c2e" }); + }); + + it("lets the flag win over QAWOLF_WORKFLOW_ID", async () => { + const { callPublicApi } = await run({ + env: { QAWOLF_WORKFLOW_ID: "flow-other" }, + flowId: "flow-7c2e", + }); + + expect(sentRequest(callPublicApi)).toMatchObject({ flowId: "flow-7c2e" }); + }); + + it("refuses a blank flag before addressing a runner", async () => { + const { callPublicApi, result } = await run({ flowId: " " }); + + expect(result?.error).toContain("--flow-id was given nothing"); + expect(result?.exitCode).toBe(2); + expect(callPublicApi).not.toHaveBeenCalled(); + }); +}); diff --git a/src/domains/interactiveRunner/runFlow.follow.test.ts b/src/domains/interactiveRunner/runFlow.follow.test.ts index c93253bca..1e68783cb 100644 --- a/src/domains/interactiveRunner/runFlow.follow.test.ts +++ b/src/domains/interactiveRunner/runFlow.follow.test.ts @@ -24,6 +24,7 @@ const runFollowing = ( follow: options.follow ?? true, envFile: undefined, envId: undefined, + flowId: undefined, lines: undefined, linesFile: undefined, logs: options.logs ?? false, diff --git a/src/domains/interactiveRunner/runFlow.selection.test.ts b/src/domains/interactiveRunner/runFlow.selection.test.ts index c5ac772b5..75c980c0b 100644 --- a/src/domains/interactiveRunner/runFlow.selection.test.ts +++ b/src/domains/interactiveRunner/runFlow.selection.test.ts @@ -42,6 +42,7 @@ async function runWith(options: { follow: false, envFile: undefined, envId: undefined, + flowId: undefined, lines: options.lines, linesFile: options.linesFile, logs: false, diff --git a/src/domains/interactiveRunner/runFlow.test.ts b/src/domains/interactiveRunner/runFlow.test.ts index 22c33098b..879d215ad 100644 --- a/src/domains/interactiveRunner/runFlow.test.ts +++ b/src/domains/interactiveRunner/runFlow.test.ts @@ -24,6 +24,7 @@ async function runWith( follow: false, envFile: undefined, envId: undefined, + flowId: undefined, lines: options.lines, linesFile: options.linesFile, runner: "ci", @@ -120,6 +121,7 @@ describe("handleRunnerRun", () => { follow: false, envFile: undefined, envId: undefined, + flowId: undefined, lines: undefined, linesFile: undefined, runner: undefined, @@ -158,6 +160,7 @@ describe("handleRunnerRun", () => { follow: false, envFile: undefined, envId: undefined, + flowId: undefined, lines: undefined, linesFile: undefined, runner: "ci", diff --git a/src/domains/interactiveRunner/runFlow.ts b/src/domains/interactiveRunner/runFlow.ts index 3e61fbcca..b081f9168 100644 --- a/src/domains/interactiveRunner/runFlow.ts +++ b/src/domains/interactiveRunner/runFlow.ts @@ -25,6 +25,7 @@ export async function handleRunnerRun( entryPoint: string; envFile: string | undefined; envId: string | undefined; + flowId: string | undefined; follow: boolean; lines: string | undefined; linesFile: string | undefined; @@ -53,6 +54,7 @@ export async function handleRunnerRun( entryPointPath, envFile: options.envFile, envId: options.envId, + flowId: options.flowId, lines: options.lines, linesFile: options.linesFile, }, @@ -90,6 +92,7 @@ export async function handleRunnerRun( environment: prepared.environment, environmentId: prepared.environmentId, files: prepared.files, + flowId: prepared.flowId, resolved, selection: prepared.selection, }, diff --git a/src/domains/interactiveRunner/sendRunFlowRequest.ts b/src/domains/interactiveRunner/sendRunFlowRequest.ts index b6d98ef1a..244b82f8e 100644 --- a/src/domains/interactiveRunner/sendRunFlowRequest.ts +++ b/src/domains/interactiveRunner/sendRunFlowRequest.ts @@ -29,6 +29,7 @@ export async function sendRunFlowRequest( entryPointPath: string; environment: Record | undefined; environmentId: string | undefined; + flowId: string | undefined; resolved: Runner; selection: RunSelection | undefined; }, @@ -49,6 +50,7 @@ export async function sendRunFlowRequest( ...(options.environmentId === undefined ? {} : { environmentId: options.environmentId }), + ...(options.flowId === undefined ? {} : { flowId: options.flowId }), ...(options.selection === undefined ? {} : { selection: options.selection }), diff --git a/src/domains/interactiveRunner/submitRun.ts b/src/domains/interactiveRunner/submitRun.ts index f503acace..c45f1a543 100644 --- a/src/domains/interactiveRunner/submitRun.ts +++ b/src/domains/interactiveRunner/submitRun.ts @@ -33,6 +33,7 @@ export async function submitRun( environment: Record | undefined; environmentId: string | undefined; files: RunFiles; + flowId: string | undefined; resolved: Runner; selection: RunSelection | undefined; }, diff --git a/src/runnerSdk/runVerbs.ts b/src/runnerSdk/runVerbs.ts index 974dc640b..29db32eb8 100644 --- a/src/runnerSdk/runVerbs.ts +++ b/src/runnerSdk/runVerbs.ts @@ -55,6 +55,7 @@ export function createRunVerbs({ deps, platformClient }: SdkContext) { async run({ entryPointPath, environment, + flowId, runnerId, selection, }: RunRequest): Promise> { @@ -63,6 +64,7 @@ export function createRunVerbs({ deps, platformClient }: SdkContext) { entryPointPath, envFile: undefined, envId: environment === "ambient" ? undefined : environment.id, + flowId, lines: toLineRange(selection), linesFile: toLinesFile(selection), }, @@ -77,6 +79,7 @@ export function createRunVerbs({ deps, platformClient }: SdkContext) { environment: prepared.environment, environmentId: prepared.environmentId, files: prepared.files, + flowId: prepared.flowId, resolved: { ...givenRunner(runnerId), type: "resolved" }, selection: prepared.selection, }, diff --git a/src/runnerSdk/types.ts b/src/runnerSdk/types.ts index dce6b7a86..65f31fcad 100644 --- a/src/runnerSdk/types.ts +++ b/src/runnerSdk/types.ts @@ -74,6 +74,7 @@ export type Recordings = PublicApiOutput; export type RunRequest = RunnerRequest & { entryPointPath: string; environment: RunEnvironment; + flowId?: string; // seen by the run as QAWOLF_WORKFLOW_ID; defaults to this process's selection: RunSelection; }; From 5c0f5bc945668a4f4dece03c52840c03fbc49acd Mon Sep 17 00:00:00 2001 From: Huishi Date: Wed, 23 Sep 2026 10:26:16 -0400 Subject: [PATCH 2/5] docs(runner): say who reads QAWOLF_WORKFLOW_ID for --flow-id The old wording said the run falls back to the variable, which reads as the run reading the id it is about to be given. The CLI reads it, from its own shell. Show the fixture-name convention the flag exists for instead of naming it in the abstract. --- skills/qawolf-cli/references/runner.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/skills/qawolf-cli/references/runner.md b/skills/qawolf-cli/references/runner.md index cccf15d89..fa01871df 100644 --- a/skills/qawolf-cli/references/runner.md +++ b/skills/qawolf-cli/references/runner.md @@ -479,18 +479,24 @@ runner is addressed, naming the path. ### Naming the flow the run is for -`--flow-id` names the QA Wolf flow the run is for, and the run sees it as -`QAWOLF_WORKFLOW_ID`, the same value a platform run of that flow sees. A flow -that names fixtures by `process.env.QAWOLF_WORKFLOW_ID` and cleans them up by it -then behaves the same on a runner as on the platform. A run with no flag falls -back to `QAWOLF_WORKFLOW_ID` from the shell, which the pod an AI Job runs on -already exports; a run with neither is given no `QAWOLF_WORKFLOW_ID`, and the -flow reads it as `undefined`. That is not a sign the id is unavailable. +`--flow-id` names the QA Wolf flow the run is for. QA Wolf stamps that id onto +the run as `QAWOLF_WORKFLOW_ID`, the same value a platform run of that flow sees. +A flow that builds fixture names from it, such as +`entity-${process.env.QAWOLF_WORKFLOW_ID}`, deletes its leftovers by searching +for that same id. Naming the flow makes those names identical on a runner and on +the platform, so a fixture one run strands is found by the next run of that flow +on either side. ```sh qawolf runner run flows/checkout.flow.ts --flow-id ``` +Without the flag, the CLI sends `QAWOLF_WORKFLOW_ID` from its own shell, which +the pod an AI Job runs on already exports. With neither, the run is given no +`QAWOLF_WORKFLOW_ID`: the name above becomes `entity-undefined`, and the run +shares no fixtures with the platform runs of its own flow. That is not a sign the +id is unavailable. + ### Giving the run environment variables There are two ways, and a run takes one of them. `--env-id` names a QA Wolf From c90fe778a64accb33d8fb66792be53155ba70748 Mon Sep 17 00:00:00 2001 From: Huishi Date: Wed, 23 Sep 2026 12:00:39 -0400 Subject: [PATCH 3/5] fix(runner): take the flow for a run from --flow-id only The fallback read QAWOLF_WORKFLOW_ID from the CLI's own environment. Only an AI Job's pod holds it, and there it is the id of that pod's own flow, not of whatever flow the run is for. An agent on such a pod routinely runs sibling flows, so the fallback would key those runs to the wrong flow and strand their fixtures in another flow's namespace. The flag is now the only source. A test pins that an ambient QAWOLF_WORKFLOW_ID is ignored, so the fallback is not reintroduced. --- .changeset/runner-run-flow-id.md | 2 +- skills/qawolf-cli/SKILL.md | 5 ++-- skills/qawolf-cli/references/runner.md | 11 ++++---- src/commands/__snapshots__/help.test.ts.snap | 3 +-- src/commands/qawolfCliSkill.template.md | 5 ++-- src/commands/runner/run.register.ts | 2 +- src/core/interactiveRunner/runFlowId.ts | 26 +++++++++---------- src/domains/interactiveRunner/prepareRun.ts | 2 +- .../interactiveRunner/runFlow.flowId.test.ts | 22 +++++----------- src/runnerSdk/types.ts | 2 +- 10 files changed, 36 insertions(+), 44 deletions(-) diff --git a/.changeset/runner-run-flow-id.md b/.changeset/runner-run-flow-id.md index 0a4014444..74da9617c 100644 --- a/.changeset/runner-run-flow-id.md +++ b/.changeset/runner-run-flow-id.md @@ -2,4 +2,4 @@ "@qawolf/cli": minor --- -`qawolf runner run` takes `--flow-id`, the QA Wolf flow the run is for, and falls back to `QAWOLF_WORKFLOW_ID` when the flag is absent. The run then sees that id as `QAWOLF_WORKFLOW_ID`, the same value a platform run of the flow sees, so fixtures a flow names and cleans up by its workflow id match across both. A run that names no flow is unchanged. The `@qawolf/cli/runner-sdk` `run` verb takes the same `flowId`. +`qawolf runner run` takes `--flow-id`, the QA Wolf flow the run is for. The run then sees that id as `QAWOLF_WORKFLOW_ID`, the same value a platform run of the flow sees, so fixtures a flow names and cleans up by its workflow id match across both. A run that names no flow is unchanged. The `@qawolf/cli/runner-sdk` `run` verb takes the same `flowId`. diff --git a/skills/qawolf-cli/SKILL.md b/skills/qawolf-cli/SKILL.md index aa3ab4c20..953aadf01 100644 --- a/skills/qawolf-cli/SKILL.md +++ b/skills/qawolf-cli/SKILL.md @@ -41,8 +41,9 @@ rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow -the run is for, falling back to `QAWOLF_WORKFLOW_ID`; the run reads that id as -`process.env.QAWOLF_WORKFLOW_ID`, and one that names no flow reads `undefined`. +the run is for, which the run reads as `process.env.QAWOLF_WORKFLOW_ID`. Pass it +whenever the run is for a flow QA Wolf knows; a run that names none reads +`undefined` there, which does not mean the id was unavailable. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/skills/qawolf-cli/references/runner.md b/skills/qawolf-cli/references/runner.md index fa01871df..986b4a82c 100644 --- a/skills/qawolf-cli/references/runner.md +++ b/skills/qawolf-cli/references/runner.md @@ -491,11 +491,12 @@ on either side. qawolf runner run flows/checkout.flow.ts --flow-id ``` -Without the flag, the CLI sends `QAWOLF_WORKFLOW_ID` from its own shell, which -the pod an AI Job runs on already exports. With neither, the run is given no -`QAWOLF_WORKFLOW_ID`: the name above becomes `entity-undefined`, and the run -shares no fixtures with the platform runs of its own flow. That is not a sign the -id is unavailable. +A run with no flag is given no `QAWOLF_WORKFLOW_ID`, so the name above becomes +`entity-undefined` and the run shares no fixtures with the platform runs of its +own flow. That is not a sign the id is unavailable. The flag is the only thing +that names the flow: a `QAWOLF_WORKFLOW_ID` exported in your own shell is not +read, because the one environment that holds it is an AI Job's pod, where it is +the id of that pod's own flow rather than of whatever flow the run is for. ### Giving the run environment variables diff --git a/src/commands/__snapshots__/help.test.ts.snap b/src/commands/__snapshots__/help.test.ts.snap index 2535bf346..93e9a61ed 100644 --- a/src/commands/__snapshots__/help.test.ts.snap +++ b/src/commands/__snapshots__/help.test.ts.snap @@ -369,8 +369,7 @@ Options: server and no size limit applies to them --env-file Dotenv file on this machine whose variables the run is given. Pass this or --env-id, not both - --flow-id QA Wolf flow this run is for. Defaults to - QAWOLF_WORKFLOW_ID. The run sees it as + --flow-id QA Wolf flow this run is for. The run sees it as QAWOLF_WORKFLOW_ID, the same value a platform run of that flow sees, so fixtures named and cleaned up by it match --lines Run only these 1-indexed lines against the browser as it diff --git a/src/commands/qawolfCliSkill.template.md b/src/commands/qawolfCliSkill.template.md index 38e73c6bf..499f9745b 100644 --- a/src/commands/qawolfCliSkill.template.md +++ b/src/commands/qawolfCliSkill.template.md @@ -41,8 +41,9 @@ rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow -the run is for, falling back to `QAWOLF_WORKFLOW_ID`; the run reads that id as -`process.env.QAWOLF_WORKFLOW_ID`, and one that names no flow reads `undefined`. +the run is for, which the run reads as `process.env.QAWOLF_WORKFLOW_ID`. Pass it +whenever the run is for a flow QA Wolf knows; a run that names none reads +`undefined` there, which does not mean the id was unavailable. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/src/commands/runner/run.register.ts b/src/commands/runner/run.register.ts index 38edd2db6..1d8cc182c 100644 --- a/src/commands/runner/run.register.ts +++ b/src/commands/runner/run.register.ts @@ -72,7 +72,7 @@ export function registerRunCommand( ) .option( "--flow-id ", - "QA Wolf flow this run is for. Defaults to QAWOLF_WORKFLOW_ID. The run sees it as QAWOLF_WORKFLOW_ID, the same value a platform run of that flow sees, so fixtures named and cleaned up by it match", + "QA Wolf flow this run is for. The run sees it as QAWOLF_WORKFLOW_ID, the same value a platform run of that flow sees, so fixtures named and cleaned up by it match", ) .option( "--lines ", diff --git a/src/core/interactiveRunner/runFlowId.ts b/src/core/interactiveRunner/runFlowId.ts index 645d1ff13..29f1940f7 100644 --- a/src/core/interactiveRunner/runFlowId.ts +++ b/src/core/interactiveRunner/runFlowId.ts @@ -1,21 +1,19 @@ import { interactiveRunnerMessages } from "~/core/messages/index.js"; /** - * The flow a run is for: the flag when given, else `QAWOLF_WORKFLOW_ID` from - * the shell, which the pod an AI Job runs on exports. A blank flag is refused - * rather than read as "none", the way a blank `--env-id` is. + * The flow a run is for, from the flag alone. A blank flag is refused rather + * than read as "none", the way a blank `--env-id` is. + * + * `QAWOLF_WORKFLOW_ID` in this process's own environment is deliberately not + * read. The only environment that holds it is an AI Job's pod, where it is the + * id of that pod's own flow, which is not the flow of every run started there. */ -export function resolveRunFlowId({ - env, - flag, -}: { - env: Readonly>; - flag: string | undefined; -}): { ok: true; flowId: string | undefined } | { ok: false; error: string } { - const explicit = flag?.trim(); - if (explicit === "") { +export function resolveRunFlowId( + flag: string | undefined, +): { ok: true; flowId: string | undefined } | { ok: false; error: string } { + const flowId = flag?.trim(); + if (flowId === "") { return { error: interactiveRunnerMessages.flowIdBlank, ok: false }; } - const fromEnvVar = env["QAWOLF_WORKFLOW_ID"]?.trim(); - return { flowId: explicit ?? (fromEnvVar || undefined), ok: true }; + return { flowId, ok: true }; } diff --git a/src/domains/interactiveRunner/prepareRun.ts b/src/domains/interactiveRunner/prepareRun.ts index ad7ba6b23..913d49443 100644 --- a/src/domains/interactiveRunner/prepareRun.ts +++ b/src/domains/interactiveRunner/prepareRun.ts @@ -68,7 +68,7 @@ export async function prepareRun( explicitEnvId ?? (options.envFile === undefined && fromEnvVar ? fromEnvVar : undefined); - const flow = resolveRunFlowId({ env: deps.env, flag: options.flowId }); + const flow = resolveRunFlowId(options.flowId); if (!flow.ok) return refused(flow.error, exitCodes.invalidArgs); const linesFilePath = diff --git a/src/domains/interactiveRunner/runFlow.flowId.test.ts b/src/domains/interactiveRunner/runFlow.flowId.test.ts index 241dd2d11..582ed23d7 100644 --- a/src/domains/interactiveRunner/runFlow.flowId.test.ts +++ b/src/domains/interactiveRunner/runFlow.flowId.test.ts @@ -61,29 +61,21 @@ describe("handleRunnerRun with --flow-id", () => { expect(sentRequest(callPublicApi)).toMatchObject({ flowId: "flow-7c2e" }); }); - it("names no flow when the flag is absent and nothing is exported", async () => { + it("names no flow when the flag is absent", async () => { const { callPublicApi } = await run({}); expect(Object.hasOwn(sentRequest(callPublicApi), "flowId")).toBe(false); }); - // A pod that runs an AI Job carries the job's flow this way, so a run started - // from its shell is keyed to the flow the way a platform run of it is. - it("falls back to QAWOLF_WORKFLOW_ID when the flag is absent", async () => { + // An AI Job's pod exports the id of its own flow, which is not the flow of + // every run started from that pod. Reading it here would key a run of one + // flow to another, so the flag is the only source. + it("ignores a QAWOLF_WORKFLOW_ID in its own environment", async () => { const { callPublicApi } = await run({ - env: { QAWOLF_WORKFLOW_ID: "flow-7c2e" }, + env: { QAWOLF_WORKFLOW_ID: "flow-of-this-pod" }, }); - expect(sentRequest(callPublicApi)).toMatchObject({ flowId: "flow-7c2e" }); - }); - - it("lets the flag win over QAWOLF_WORKFLOW_ID", async () => { - const { callPublicApi } = await run({ - env: { QAWOLF_WORKFLOW_ID: "flow-other" }, - flowId: "flow-7c2e", - }); - - expect(sentRequest(callPublicApi)).toMatchObject({ flowId: "flow-7c2e" }); + expect(Object.hasOwn(sentRequest(callPublicApi), "flowId")).toBe(false); }); it("refuses a blank flag before addressing a runner", async () => { diff --git a/src/runnerSdk/types.ts b/src/runnerSdk/types.ts index 65f31fcad..4ab2e923e 100644 --- a/src/runnerSdk/types.ts +++ b/src/runnerSdk/types.ts @@ -74,7 +74,7 @@ export type Recordings = PublicApiOutput; export type RunRequest = RunnerRequest & { entryPointPath: string; environment: RunEnvironment; - flowId?: string; // seen by the run as QAWOLF_WORKFLOW_ID; defaults to this process's + flowId?: string; // the flow the run is for; seen by the run as QAWOLF_WORKFLOW_ID selection: RunSelection; }; From 983f6bf4db54eb5297bd5b153ea5bb0e12325586 Mon Sep 17 00:00:00 2001 From: Huishi Date: Thu, 24 Sep 2026 10:25:35 -0400 Subject: [PATCH 4/5] docs(runner): shorten the --flow-id wording Same facts in fewer words across the help text, changeset, skill, reference and the helper's comment. --- .changeset/runner-run-flow-id.md | 2 +- skills/qawolf-cli/SKILL.md | 5 ++--- skills/qawolf-cli/references/runner.md | 21 ++++++++------------ src/commands/__snapshots__/help.test.ts.snap | 5 ++--- src/commands/qawolfCliSkill.template.md | 5 ++--- src/commands/runner/run.register.ts | 2 +- src/core/interactiveRunner/runFlowId.ts | 10 ++++------ src/core/messages/interactiveRunner/run.ts | 2 +- src/runnerSdk/types.ts | 2 +- 9 files changed, 22 insertions(+), 32 deletions(-) diff --git a/.changeset/runner-run-flow-id.md b/.changeset/runner-run-flow-id.md index 74da9617c..527501321 100644 --- a/.changeset/runner-run-flow-id.md +++ b/.changeset/runner-run-flow-id.md @@ -2,4 +2,4 @@ "@qawolf/cli": minor --- -`qawolf runner run` takes `--flow-id`, the QA Wolf flow the run is for. The run then sees that id as `QAWOLF_WORKFLOW_ID`, the same value a platform run of the flow sees, so fixtures a flow names and cleans up by its workflow id match across both. A run that names no flow is unchanged. The `@qawolf/cli/runner-sdk` `run` verb takes the same `flowId`. +`qawolf runner run --flow-id` names the flow a run is for. The run receives it as `QAWOLF_WORKFLOW_ID`, like a platform run of that flow does, so fixtures keyed by that id match across both. Runs that name no flow are unchanged. The runner SDK's `run` verb takes the same `flowId`. diff --git a/skills/qawolf-cli/SKILL.md b/skills/qawolf-cli/SKILL.md index 953aadf01..7ec974064 100644 --- a/skills/qawolf-cli/SKILL.md +++ b/skills/qawolf-cli/SKILL.md @@ -41,9 +41,8 @@ rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow -the run is for, which the run reads as `process.env.QAWOLF_WORKFLOW_ID`. Pass it -whenever the run is for a flow QA Wolf knows; a run that names none reads -`undefined` there, which does not mean the id was unavailable. +the run is for; the run receives it as `process.env.QAWOLF_WORKFLOW_ID`. Without +the flag that variable is `undefined`, which is expected, not a missing id. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/skills/qawolf-cli/references/runner.md b/skills/qawolf-cli/references/runner.md index 986b4a82c..28d9a757c 100644 --- a/skills/qawolf-cli/references/runner.md +++ b/skills/qawolf-cli/references/runner.md @@ -479,24 +479,19 @@ runner is addressed, naming the path. ### Naming the flow the run is for -`--flow-id` names the QA Wolf flow the run is for. QA Wolf stamps that id onto -the run as `QAWOLF_WORKFLOW_ID`, the same value a platform run of that flow sees. -A flow that builds fixture names from it, such as -`entity-${process.env.QAWOLF_WORKFLOW_ID}`, deletes its leftovers by searching -for that same id. Naming the flow makes those names identical on a runner and on -the platform, so a fixture one run strands is found by the next run of that flow -on either side. +`--flow-id` names the QA Wolf flow the run is for. The run receives it as +`QAWOLF_WORKFLOW_ID`, like a platform run of that flow does, so fixtures keyed by +that id, such as `entity-${process.env.QAWOLF_WORKFLOW_ID}`, and cleaned up by +it match across the runner and the platform. ```sh qawolf runner run flows/checkout.flow.ts --flow-id ``` -A run with no flag is given no `QAWOLF_WORKFLOW_ID`, so the name above becomes -`entity-undefined` and the run shares no fixtures with the platform runs of its -own flow. That is not a sign the id is unavailable. The flag is the only thing -that names the flow: a `QAWOLF_WORKFLOW_ID` exported in your own shell is not -read, because the one environment that holds it is an AI Job's pod, where it is -the id of that pod's own flow rather than of whatever flow the run is for. +Without the flag the run has no `QAWOLF_WORKFLOW_ID`, and the name above becomes +`entity-undefined`. That is expected, not a missing id. Your shell's own +`QAWOLF_WORKFLOW_ID` is never read: only an AI Job's pod sets one, and it names +the pod's flow, not every flow run from there. ### Giving the run environment variables diff --git a/src/commands/__snapshots__/help.test.ts.snap b/src/commands/__snapshots__/help.test.ts.snap index 93e9a61ed..f6e3d913d 100644 --- a/src/commands/__snapshots__/help.test.ts.snap +++ b/src/commands/__snapshots__/help.test.ts.snap @@ -369,9 +369,8 @@ Options: server and no size limit applies to them --env-file Dotenv file on this machine whose variables the run is given. Pass this or --env-id, not both - --flow-id QA Wolf flow this run is for. The run sees it as - QAWOLF_WORKFLOW_ID, the same value a platform run of that - flow sees, so fixtures named and cleaned up by it match + --flow-id The QA Wolf flow this run is for. The run receives it as + QAWOLF_WORKFLOW_ID, like a platform run of the flow does --lines Run only these 1-indexed lines against the browser as it stands, instead of the whole flow from a fresh one --lines-file File the --lines range lives in. Defaults to ; diff --git a/src/commands/qawolfCliSkill.template.md b/src/commands/qawolfCliSkill.template.md index 499f9745b..cdc2e4898 100644 --- a/src/commands/qawolfCliSkill.template.md +++ b/src/commands/qawolfCliSkill.template.md @@ -41,9 +41,8 @@ rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow -the run is for, which the run reads as `process.env.QAWOLF_WORKFLOW_ID`. Pass it -whenever the run is for a flow QA Wolf knows; a run that names none reads -`undefined` there, which does not mean the id was unavailable. +the run is for; the run receives it as `process.env.QAWOLF_WORKFLOW_ID`. Without +the flag that variable is `undefined`, which is expected, not a missing id. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/src/commands/runner/run.register.ts b/src/commands/runner/run.register.ts index 1d8cc182c..be4485d05 100644 --- a/src/commands/runner/run.register.ts +++ b/src/commands/runner/run.register.ts @@ -72,7 +72,7 @@ export function registerRunCommand( ) .option( "--flow-id ", - "QA Wolf flow this run is for. The run sees it as QAWOLF_WORKFLOW_ID, the same value a platform run of that flow sees, so fixtures named and cleaned up by it match", + "The QA Wolf flow this run is for. The run receives it as QAWOLF_WORKFLOW_ID, like a platform run of the flow does", ) .option( "--lines ", diff --git a/src/core/interactiveRunner/runFlowId.ts b/src/core/interactiveRunner/runFlowId.ts index 29f1940f7..401b7005c 100644 --- a/src/core/interactiveRunner/runFlowId.ts +++ b/src/core/interactiveRunner/runFlowId.ts @@ -1,12 +1,10 @@ import { interactiveRunnerMessages } from "~/core/messages/index.js"; /** - * The flow a run is for, from the flag alone. A blank flag is refused rather - * than read as "none", the way a blank `--env-id` is. - * - * `QAWOLF_WORKFLOW_ID` in this process's own environment is deliberately not - * read. The only environment that holds it is an AI Job's pod, where it is the - * id of that pod's own flow, which is not the flow of every run started there. + * The flow a run is for, from the flag alone. A blank flag is refused, like a + * blank `--env-id`. The process's own `QAWOLF_WORKFLOW_ID` is not read: only + * an AI Job's pod has one, and it names the pod's flow, not every flow run + * from there. */ export function resolveRunFlowId( flag: string | undefined, diff --git a/src/core/messages/interactiveRunner/run.ts b/src/core/messages/interactiveRunner/run.ts index 3cc01f922..5f8244a7e 100644 --- a/src/core/messages/interactiveRunner/run.ts +++ b/src/core/messages/interactiveRunner/run.ts @@ -11,7 +11,7 @@ export const runMessages = { envIdWithEnvFile: "--env-id and --env-file both give the run its environment, so only one may be passed. Use --env-id for a QA Wolf environment, which QA Wolf reads itself, or --env-file for a dotenv file on this machine.", flowIdBlank: - "--flow-id was given nothing. Pass the id of the QA Wolf flow this run is for, or drop the flag to key the run to no flow.", + "--flow-id was given nothing. Pass the id of the flow this run is for, or drop the flag.", fileNotCollected: (path: string) => `"${path}" is not one of the files that travel to a runner. It must be inside the current directory and end in .ts, .tsx, .js, .mjs, .cjs or .json.`, fileUnreadable: (path: string, reason: string) => diff --git a/src/runnerSdk/types.ts b/src/runnerSdk/types.ts index 4ab2e923e..a93b52313 100644 --- a/src/runnerSdk/types.ts +++ b/src/runnerSdk/types.ts @@ -74,7 +74,7 @@ export type Recordings = PublicApiOutput; export type RunRequest = RunnerRequest & { entryPointPath: string; environment: RunEnvironment; - flowId?: string; // the flow the run is for; seen by the run as QAWOLF_WORKFLOW_ID + flowId?: string; // the run receives it as QAWOLF_WORKFLOW_ID selection: RunSelection; }; From fbaa1adef792f64b84831dde53d1f8b561c4b162 Mon Sep 17 00:00:00 2001 From: Huishi Date: Thu, 1 Oct 2026 14:52:50 -0400 Subject: [PATCH 5/5] docs(runner): make --flow-id the override for the platform's entry-point lookup qawolf/platform#34164 finds the flow from the entry point on the environment's flow-code branch, so the flag is for runs that lookup cannot cover. --- .changeset/runner-run-flow-id.md | 2 +- skills/qawolf-cli/SKILL.md | 8 +++++--- skills/qawolf-cli/references/runner.md | 21 ++++++++++++-------- src/commands/__snapshots__/help.test.ts.snap | 5 +++-- src/commands/qawolfCliSkill.template.md | 8 +++++--- src/commands/runner/run.register.ts | 2 +- 6 files changed, 28 insertions(+), 18 deletions(-) diff --git a/.changeset/runner-run-flow-id.md b/.changeset/runner-run-flow-id.md index 527501321..5f807dfeb 100644 --- a/.changeset/runner-run-flow-id.md +++ b/.changeset/runner-run-flow-id.md @@ -2,4 +2,4 @@ "@qawolf/cli": minor --- -`qawolf runner run --flow-id` names the flow a run is for. The run receives it as `QAWOLF_WORKFLOW_ID`, like a platform run of that flow does, so fixtures keyed by that id match across both. Runs that name no flow are unchanged. The runner SDK's `run` verb takes the same `flowId`. +`qawolf runner run --flow-id` names the flow a run is for when the platform cannot tell from the entry point. The run receives it as `QAWOLF_WORKFLOW_ID`, like a platform run of that flow does, so fixtures keyed by that id match across both. The runner SDK's `run` verb takes the same `flowId`. diff --git a/skills/qawolf-cli/SKILL.md b/skills/qawolf-cli/SKILL.md index 7ec974064..9964b1138 100644 --- a/skills/qawolf-cli/SKILL.md +++ b/skills/qawolf-cli/SKILL.md @@ -40,9 +40,11 @@ exported variables, so pass the selected environment explicitly instead of rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` -when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow -the run is for; the run receives it as `process.env.QAWOLF_WORKFLOW_ID`. Without -the flag that variable is `undefined`, which is expected, not a missing id. +when neither it nor `--env-file` is passed. The run receives the flow it is for +as `process.env.QAWOLF_WORKFLOW_ID`, found from the entry point on the +environment's flow-code branch; `--flow-id` names the flow when that lookup +cannot. A run with neither has that variable `undefined`, which is expected, not +a missing id. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/skills/qawolf-cli/references/runner.md b/skills/qawolf-cli/references/runner.md index 28d9a757c..e0acf133f 100644 --- a/skills/qawolf-cli/references/runner.md +++ b/skills/qawolf-cli/references/runner.md @@ -479,19 +479,24 @@ runner is addressed, naming the path. ### Naming the flow the run is for -`--flow-id` names the QA Wolf flow the run is for. The run receives it as -`QAWOLF_WORKFLOW_ID`, like a platform run of that flow does, so fixtures keyed by -that id, such as `entity-${process.env.QAWOLF_WORKFLOW_ID}`, and cleaned up by -it match across the runner and the platform. +The run receives the id of the flow it is for as `QAWOLF_WORKFLOW_ID`, like a +platform run of that flow does, so fixtures keyed by that id, such as +`entity-${process.env.QAWOLF_WORKFLOW_ID}`, and cleaned up by it match across the +runner and the platform. When `--env-id` names an environment with a flow-code +branch, the platform finds the flow itself: the entry point is looked up on that +branch. `--flow-id` names the flow when that lookup cannot: the run uses +`--env-file`, the environment has no branch, or the entry point is a file the +branch does not know yet. ```sh qawolf runner run flows/checkout.flow.ts --flow-id ``` -Without the flag the run has no `QAWOLF_WORKFLOW_ID`, and the name above becomes -`entity-undefined`. That is expected, not a missing id. Your shell's own -`QAWOLF_WORKFLOW_ID` is never read: only an AI Job's pod sets one, and it names -the pod's flow, not every flow run from there. +A run whose flow neither the lookup nor the flag names has no +`QAWOLF_WORKFLOW_ID`, and the name above becomes `entity-undefined`. That is +expected, not a missing id. Your shell's own `QAWOLF_WORKFLOW_ID` is never read: +only an AI Job's pod sets one, and it names the pod's flow, not every flow run +from there. ### Giving the run environment variables diff --git a/src/commands/__snapshots__/help.test.ts.snap b/src/commands/__snapshots__/help.test.ts.snap index f6e3d913d..56dd5ceab 100644 --- a/src/commands/__snapshots__/help.test.ts.snap +++ b/src/commands/__snapshots__/help.test.ts.snap @@ -369,8 +369,9 @@ Options: server and no size limit applies to them --env-file Dotenv file on this machine whose variables the run is given. Pass this or --env-id, not both - --flow-id The QA Wolf flow this run is for. The run receives it as - QAWOLF_WORKFLOW_ID, like a platform run of the flow does + --flow-id The QA Wolf flow this run is for, when the platform + cannot tell from the entry point. The run receives it as + QAWOLF_WORKFLOW_ID --lines Run only these 1-indexed lines against the browser as it stands, instead of the whole flow from a fresh one --lines-file File the --lines range lives in. Defaults to ; diff --git a/src/commands/qawolfCliSkill.template.md b/src/commands/qawolfCliSkill.template.md index cdc2e4898..8ba5c4dd2 100644 --- a/src/commands/qawolfCliSkill.template.md +++ b/src/commands/qawolfCliSkill.template.md @@ -40,9 +40,11 @@ exported variables, so pass the selected environment explicitly instead of rediscovering it. Check the command's help for its environment flag: variable commands use `--environment-id`; `qawolf flows run` uses `--env`; and `qawolf runner run` uses `--env-id`, which falls back to `QAWOLF_ENVIRONMENT` -when neither it nor `--env-file` is passed. It also takes `--flow-id`, the flow -the run is for; the run receives it as `process.env.QAWOLF_WORKFLOW_ID`. Without -the flag that variable is `undefined`, which is expected, not a missing id. +when neither it nor `--env-file` is passed. The run receives the flow it is for +as `process.env.QAWOLF_WORKFLOW_ID`, found from the entry point on the +environment's flow-code branch; `--flow-id` names the flow when that lookup +cannot. A run with neither has that variable `undefined`, which is expected, not +a missing id. If multiple environments are returned and the task context does not identify the target, ask instead of guessing. Do not default to the newest environment. diff --git a/src/commands/runner/run.register.ts b/src/commands/runner/run.register.ts index be4485d05..6c2ff0d3d 100644 --- a/src/commands/runner/run.register.ts +++ b/src/commands/runner/run.register.ts @@ -72,7 +72,7 @@ export function registerRunCommand( ) .option( "--flow-id ", - "The QA Wolf flow this run is for. The run receives it as QAWOLF_WORKFLOW_ID, like a platform run of the flow does", + "The QA Wolf flow this run is for, when the platform cannot tell from the entry point. The run receives it as QAWOLF_WORKFLOW_ID", ) .option( "--lines ",