feat: fix target system live property

.changeset/ready-laws-pull.md

@@ -0,0 +1,5 @@
+---
+"@webiny/data-transfer": patch
+---
+
+Fix `addLiveField` in the OS lane and add `fix-live` reconciler command. `OsProcessor.querySourceRecord` now returns decompressed rows, so the published revision's `version` is readable — entries with a draft on top of a published revision correctly get `live: { version }` instead of `live: {}`. Add the `fix-live` command to reconcile already-migrated systems: scans DynamoDB and OpenSearch companion tables, reports changes in JSONL, writes only via conditional `UpdateItem`. Add a command menu (`yarn transfer` with no args), `@clack/prompts`-backed `Prompts`/`UI` abstraction, and `Command` registry. Remove `@inquirer/prompts` — all prompts now go through the abstraction. Update dependencies.

.gitignore

@@ -59,5 +59,3 @@ __tests__/fixtures/full-table-migrated.json
 __tests__/fixtures/es-table-migrated.json
 __tests__/fixtures/os-table-migrated.json
 .codegraph
-.mcp.json
-!templates/.mcp.json

.mcp.json

@@ -0,0 +1,13 @@
+{
+  "mcpServers": {
+    "stdlib": {
+      "command": "npx",
+      "args": ["-y", "@webiny/stdlib", "serve"]
+    },
+    "codegraph": {
+      "type": "stdio",
+      "command": "codegraph",
+      "args": ["serve", "--mcp"]
+    }
+  }
+}

AGENTS.md

@@ -21,14 +21,15 @@ Use the **codegraph MCP** as the first tool for browsing code. `codegraph_explor
 **Runtime flow (when deployed):**
 
 1. User writes a single `config.ts`: `createConfig({ source, target, pipeline })`. One file covers DDB, S3, and optional OpenSearch. **User-side custom DI:** `register` callback in `createConfig()` is the primary path (runs before preset loading). `setup.ts` next to the config file is the alternative for larger setups. Both are optional.
-2. CLI `transfer` command (no `--config`): the `TransferWizard` selects a project, writes `.env`, then on subsequent runs prompts for a preset and returns `WizardResult { configPath, preset }`. With `--config`: skips wizard, preset passed as `--preset` flag.
+2. CLI: `yarn transfer` with no arguments opens a menu over the `Command` registry (`src/commands/registry/`) — entries: `transfer`, `fix-live`. `yarn transfer transfer` (or the legacy `yarn transfer --config … --preset …`) runs the system-to-system transfer: without `--config` the `TransferWizard` selects a project, writes `.env`, then on subsequent runs prompts for a preset and returns `WizardResult { configPath, preset, dryRun }`. `yarn transfer <folder>` still scaffolds (`init`). Prompts go through the `Prompts` / `UI` abstractions (`src/commands/prompts/`, `@clack/prompts`); commands never import a prompt library. Cancel exits 130.
 3. Bootstrap loads the config, registers all features (DDB + S3 always; OS conditional on `config.target.opensearch != null`), loads the named preset, spawns worker processes per segment.
 4. Each worker runs one or more shards: scans source → for each record, first-match-wins pipeline runs: filters → transformers → each processor's `onEnd?` hook (sequential, array order) → commands accumulate in a pending buffer. Every `tuning.flushEvery` records (default 500) each processor's `execute()` drains its own keys from that buffer (sequential, array order) and the buffer resets — this bounds peak memory to `flushEvery × avg_record_size`. A final flush at shard end drains any remainder. `Commands.unclaimedKeys()` surfaces commands no processor claimed.
 
 **Read before big refactors:**
 
 - `docs/design/generic-pipeline-framework.md` — long-term design (pipeline-centric model, merge groups keyed by scanner, first-match-wins).
 - `docs/superpowers/specs/2026-04-18-*.md` — recent design docs (transformer-library, preset-migration).
+- `docs/superpowers/specs/2026-09-04-fix-live-field-and-command-menu-design.md` — `fix-live` reconciler + CLI command menu (what v6 actually maintains for `live`, certainty rules, UpdateItem-only writes).
 
 ---
 
@@ -44,7 +45,7 @@ Everything users import lives in `src/index.ts`: config builder (`createConfig`)
 
 ## 3. Project structure
 
-Source lives in `src/` with `cli.ts` entry point, `bootstrap.ts` DI setup, `index.ts` public API. Domain logic is in `src/features/` (one dir per feature), pipeline abstractions in `src/domain/pipeline/`, transform primitives in `src/domain/transform/`, ~30 built-in transformers in `src/transformers/`, and 5 built-in presets in `src/presets/`. Build scripts live in `scripts/features/BuildPackages/` (DI-based, mirrors `@webiny/stdlib`). Build tsconfigs in `config/`. Changeset config in `.changeset/`. CI/CD workflows in `.github/workflows/`.
+Source lives in `src/` with `cli.ts` entry point, `bootstrap.ts` DI setup, `index.ts` public API. CLI commands live in `src/commands/` as implementations of the `Command` token (`src/commands/registry/`); the entry `src/cli.ts` registers `registry.list()` with yargs plus a `$0 [folder]` default that preserves the two historical no-command invocations. Domain logic is in `src/features/` (one dir per feature), pipeline abstractions in `src/domain/pipeline/`, transform primitives in `src/domain/transform/`, ~30 built-in transformers in `src/transformers/`, and 5 built-in presets in `src/presets/`. Build scripts live in `scripts/features/BuildPackages/` (DI-based, mirrors `@webiny/stdlib`). Build tsconfigs in `config/`. Changeset config in `.changeset/`. CI/CD workflows in `.github/workflows/`.
 
 > Full reference: [Project structure](docs/project-structure.md)
 
@@ -105,10 +106,13 @@ These docs also ship in the published npm package and are referenced from the sc
 
 ### Open work
 
+0. **Fix live field + CLI command menu** — **implemented.** Transformer fix, reconciler, DDB/OS runners, clack-based command menu, `fix-live` guided command, all landed and tested. Spec: `docs/superpowers/specs/2026-09-04-fix-live-field-and-command-menu-design.md`.
 1. **First npm publish** — infrastructure is in place (changesets, CI, publish workflow, build scripts). Needs: `NPM_TOKEN` secret in GitHub, first `yarn changeset` to create a version bump, merge to main.
 2. **Init scaffolding smoke** — `init` scaffolds from `templates/`. Scaffold output: `config.ts`, `presets/example.ts`, optional `setup.ts`. Do a smoke run to verify a scaffolded project compiles + runs against a live sandbox.
 3. **End-to-end AWS smoke** — no test has ever run against real AWS. Day-long sandbox exercise. Catches real issues mocks can't.
 4. **Public API audit pass (post-refactor)** — `src/index.ts` grew organically. Re-audit before publish to confirm the surface matches user-authoring intent. `DdbCoreTransformContext` (= Base ∧ DdbProcessorSlice) was added as the narrower alternative to `DdbTransformContext`.
+5. **Inquirer removal** — `TransferWizard`, `init` and `initProject` still use `@inquirer/prompts`; migrate them to `Prompts` / `UI` and drop `@inquirer/*` from `package.json`.
+6. **`fix-live` OS propagation** — confirm v6's DynamoDB stream handler treats a `data`-only change on the OS companion table as an index update (spec 2026-09-04, open question 1).
 
 ---
 

__tests__/commands/dispatchDefault.test.ts

@@ -0,0 +1,66 @@
+import { describe, it, expect, vi } from "vitest";
+import type { Command } from "~/commands/registry/abstractions/Command.js";
+import type { CommandRegistry } from "~/commands/registry/abstractions/CommandRegistry.js";
+import { dispatchDefault } from "~/commands/dispatchDefault.js";
+
+function fakeRegistry(runs: Record<string, ReturnType<typeof vi.fn>>): CommandRegistry.Interface {
+    const commands = Object.entries(runs).map(
+        ([name, run]) => ({ name, description: name, configure: y => y, run }) as Command.Interface
+    );
+    return {
+        list: () => commands,
+        menu: () => commands,
+        get: (name: string) => commands.find(c => c.name === name)!
+    };
+}
+
+describe("dispatchDefault", () => {
+    it("`yarn transfer <folder>` runs init with the folder as project-name", async () => {
+        const init = vi.fn(async () => 0);
+        const openMenu = vi.fn(async () => 130);
+        const code = await dispatchDefault({
+            argv: { folder: "my-folder" },
+            registry: fakeRegistry({ init, transfer: vi.fn() }),
+            openMenu
+        });
+        expect(code).toBe(0);
+        expect(init).toHaveBeenCalledWith({
+            folder: "my-folder",
+            "project-name": "my-folder"
+        });
+        expect(openMenu).not.toHaveBeenCalled();
+    });
+
+    it("`yarn transfer --config --preset` runs the transfer command", async () => {
+        const transfer = vi.fn(async () => 0);
+        const argv = { config: "./c.ts", preset: "copy-ddb" };
+        const code = await dispatchDefault({
+            argv,
+            registry: fakeRegistry({ init: vi.fn(), transfer }),
+            openMenu: vi.fn(async () => 130)
+        });
+        expect(code).toBe(0);
+        expect(transfer).toHaveBeenCalledWith(argv);
+    });
+
+    it("`--config` alone still routes to transfer (wizard prompts for the rest)", async () => {
+        const transfer = vi.fn(async () => 0);
+        await dispatchDefault({
+            argv: { config: "./c.ts" },
+            registry: fakeRegistry({ init: vi.fn(), transfer }),
+            openMenu: vi.fn(async () => 130)
+        });
+        expect(transfer).toHaveBeenCalledOnce();
+    });
+
+    it("no arguments opens the menu and returns its exit code", async () => {
+        const openMenu = vi.fn(async () => 130);
+        const code = await dispatchDefault({
+            argv: {},
+            registry: fakeRegistry({ init: vi.fn(), transfer: vi.fn() }),
+            openMenu
+        });
+        expect(code).toBe(130);
+        expect(openMenu).toHaveBeenCalledOnce();
+    });
+});

__tests__/commands/fixLive/FixLiveCommand.test.ts

@@ -0,0 +1,220 @@
+import { describe, it, expect, vi, beforeEach } from "vitest";
+
+vi.mock("~/commands/transfer/wizard/projectDiscovery.ts", () => ({
+    discoverProjects: vi.fn(async () => ["acme"])
+}));
+vi.mock("~/commands/transfer/wizard/configDiscovery.ts", () => ({
+    discoverConfig: vi.fn(async () => "/w/projects/acme/config.ts")
+}));
+
+const CREDS = { accessKeyId: "a", secretAccessKey: "b" };
+const CONFIG = {
+    source: {
+        region: "eu-central-1",
+        credentials: CREDS,
+        dynamodb: { tableName: "acme-src-ddb" },
+        s3: { bucket: "acme-src-s3" }
+    },
+    target: {
+        region: "us-east-1",
+        credentials: CREDS,
+        accountId: "123456789012",
+        dynamodb: { tableName: "acme-prod-ddb" },
+        s3: { bucket: "acme-prod-s3" },
+        opensearch: {
+            endpoint: "https://os.example.com",
+            tableName: "acme-prod-os",
+            service: "opensearch" as const,
+            indexPrefix: ""
+        }
+    },
+    pipeline: { segments: 4 }
+};
+
+vi.mock("~/features/MigrationConfig/loadConfig.ts", () => ({
+    loadConfig: vi.fn(async () => CONFIG)
+}));
+
+const mockResolve = vi.fn();
+const mockRegisterInstance = vi.fn();
+
+vi.mock("~/bootstrap.ts", () => ({
+    bootstrap: vi.fn(() => ({
+        resolve: mockResolve,
+        registerInstance: mockRegisterInstance
+    }))
+}));
+
+import { FixLiveCommand } from "~/commands/fixLive/FixLiveCommand.js";
+import { StubPrompts } from "../prompts/StubPrompts.ts";
+import { StubUI } from "../prompts/StubUI.ts";
+import { MockDynamoDbClient } from "../../services/DynamoDbClient/MockDynamoDbClient.ts";
+import { MockChangeReport } from "../../features/FixLive/MockChangeReport.ts";
+import {
+    SourceDynamoDbClient,
+    TargetDynamoDbClient
+} from "~/services/DynamoDbClient/abstractions/DynamoDbClient.js";
+import { ChangeReport, DdbLiveFieldRunner, FixLiveState } from "~/features/FixLive/index.js";
+import type { LiveFieldRunner } from "~/features/FixLive/abstractions/LiveFieldRunner.js";
+import { createEmptyStats } from "~/features/FixLive/createEmptyStats.js";
+
+const v6Row = {
+    PK: "T#root#L#en-US#CMS#CME#abc",
+    SK: "L",
+    TYPE: "cms.entry.l",
+    _et: "CmsEntries",
+    _ct: "x",
+    _md: "x",
+    data: { modelId: "article", version: 1, status: "draft" }
+};
+
+const fakeRunner: LiveFieldRunner.Interface = {
+    async run(options) {
+        const stats = createEmptyStats();
+        stats.scanned = 100;
+        stats.entries = 10;
+        stats.changes["missing-live"] = 5;
+        options.onProgress(stats);
+        return stats;
+    }
+};
+
+const fakeState = {
+    read: vi.fn(() => null),
+    pathFor: vi.fn(() => ".transfer/state/fix-live/acme__target.json"),
+    recordDryRun: vi.fn(),
+    recordLiveRun: vi.fn()
+};
+
+beforeEach(() => {
+    vi.clearAllMocks();
+    const targetClient = new MockDynamoDbClient({
+        "acme-prod-ddb": [v6Row] as never
+    });
+    const sourceClient = new MockDynamoDbClient({
+        "acme-src-ddb": [v6Row] as never
+    });
+    mockResolve.mockImplementation((token: unknown) => {
+        if (token === TargetDynamoDbClient) {
+            return targetClient;
+        }
+        if (token === SourceDynamoDbClient) {
+            return sourceClient;
+        }
+        if (token === ChangeReport) {
+            return new MockChangeReport();
+        }
+        if (token === FixLiveState) {
+            return fakeState;
+        }
+        if (token === DdbLiveFieldRunner) {
+            return fakeRunner;
+        }
+        return {};
+    });
+});
+
+function command(
+    prompts: StubPrompts,
+    ui = new StubUI()
+): {
+    cmd: InstanceType<typeof FixLiveCommand>;
+    ui: StubUI;
+} {
+    const cmd = new FixLiveCommand(prompts, ui);
+    return { cmd, ui };
+}
+
+describe("FixLiveCommand", () => {
+    it("cancel at project select → 130", async () => {
+        const { cmd } = command(new StubPrompts());
+        expect(await cmd.run({})).toBe(130);
+    });
+
+    it("cancel at system select → 130", async () => {
+        const { cmd } = command(new StubPrompts({ select: ["acme"] }));
+        expect(await cmd.run({})).toBe(130);
+    });
+
+    it("cancel at system confirm → 130", async () => {
+        const { cmd } = command(new StubPrompts({ select: ["acme", "target"] }));
+        expect(await cmd.run({})).toBe(130);
+    });
+
+    it("cancel at mode select → 130", async () => {
+        const { cmd } = command(new StubPrompts({ select: ["acme", "target"], confirm: [true] }));
+        expect(await cmd.run({})).toBe(130);
+    });
+
+    it("--live without a dry run → 1", async () => {
+        const ui = new StubUI();
+        const { cmd } = command(new StubPrompts(), ui);
+        const code = await cmd.run({
+            project: "acme",
+            system: "target",
+            live: true,
+            yes: true
+        });
+        expect(code).toBe(1);
+        expect(ui.errors[0]).toMatch(/Run a dry run first/);
+    });
+
+    it("--yes --dry-run runs, records state, exits 0", async () => {
+        const prompts = new StubPrompts();
+        const ui = new StubUI();
+        const { cmd } = command(prompts, ui);
+        const code = await cmd.run({
+            project: "acme",
+            system: "target",
+            "dry-run": true,
+            yes: true,
+            table: "ddb"
+        });
+        expect(code).toBe(0);
+        expect(prompts.selectCalls).toHaveLength(0);
+        expect(prompts.confirmCalls).toHaveLength(0);
+        expect(fakeState.recordDryRun).toHaveBeenCalledWith(
+            { project: "acme", system: "target" },
+            expect.objectContaining({ changes: 5, skips: 0 })
+        );
+        expect(ui.outros).toEqual(["Done."]);
+    });
+
+    it("--table=os on a system without OpenSearch → 1", async () => {
+        const ui = new StubUI();
+        const { cmd } = command(new StubPrompts(), ui);
+        const code = await cmd.run({
+            project: "acme",
+            system: "source",
+            "dry-run": true,
+            yes: true,
+            table: "os"
+        });
+        expect(code).toBe(1);
+        expect(ui.errors[0]).toMatch(/no OpenSearch table/);
+    });
+
+    it("--live --yes with state records lastLiveRun", async () => {
+        fakeState.read.mockReturnValue({
+            lastDryRun: {
+                runId: "0",
+                at: "2026-09-04T09:12:00.000Z",
+                changes: 5,
+                skips: 0
+            }
+        } as never);
+        const { cmd } = command(new StubPrompts());
+        const code = await cmd.run({
+            project: "acme",
+            system: "target",
+            live: true,
+            yes: true,
+            table: "ddb"
+        });
+        expect(code).toBe(0);
+        expect(fakeState.recordLiveRun).toHaveBeenCalledWith(
+            { project: "acme", system: "target" },
+            expect.objectContaining({ written: 0, conditionFailed: 0 })
+        );
+    });
+});