diff --git a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md new file mode 100644 index 00000000..678f5c76 --- /dev/null +++ b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md @@ -0,0 +1,270 @@ +# MM CLI Command Reference for Mobile (iOS) + +Command reference for the prod-only MetaMask Mobile consumer. The generic core CLI exposes commands for multiple consumers; availability here is determined by the iOS platform driver and mobile session manager. + +## Contents + +- [Syntax Rules](#syntax-rules) +- [Lifecycle](#lifecycle) +- [Destructive Launch Flags](#destructive-launch-flags) +- [Installed State](#installed-state) +- [Interaction](#interaction) +- [Discovery](#discovery) +- [Knowledge Store](#knowledge-store) +- [Batching](#batching) +- [Hermes CDP](#hermes-cdp) +- [Simulator Selection](#simulator-selection) +- [Not Available on Mobile](#not-available-on-mobile) + +## Syntax Rules + +Flag names and argument positions are exact. Mistyped interaction flags may be parsed as literal targets and eventually time out. + +### Use exact lowercase interaction flags + +```bash +# Correct +yarn mm click --testid unlock-submit +yarn mm type --testid unlock-password "" + +# Incorrect +yarn mm click --testId unlock-submit +yarn mm click --test-id unlock-submit +``` + +The shared CLI parses `--testid`, `--selector`, `--timeout`, and `--within`, but the iOS driver only supports `--testid` (and positional a11y refs). `--selector` and `--within` are accepted by the parser and then **rejected at runtime by the mobile driver** — see [Targeting on iOS](#targeting-on-ios). + +### Accessibility references are positional + +```bash +# Correct +yarn mm click e5 +yarn mm type e2 "text" + +# Incorrect: there is no --ref flag +yarn mm click --ref e5 +``` + +Use exactly one targeting method per command: a positional accessibility reference or `--testid`. + +| Goal | Correct syntax | +| --- | --- | +| Click by test ID | `yarn mm click --testid X` | +| Click by accessibility reference | `yarn mm click e5` | +| Type by test ID | `yarn mm type --testid X "text"` | +| Type by accessibility reference | `yarn mm type e2 "text"` | + +### Targeting on iOS + +- **Supported:** `--testid ` and positional a11y refs (`e1`, `e2`, ...). +- **Not supported:** `--selector ` throws `CSS selectors are not supported on mobile`. `--within ` throws `Scoped element search (within) is not supported on mobile`. The parser accepts both flags, but the iOS driver rejects them at runtime. +- **Matching is fuzzy and case-insensitive.** idb matches accessibility label/identifier by substring, so `--testid Confirm` can match `Confirm Transaction`. Use exact, unique test IDs to disambiguate; there is no `--within` scoping fallback on mobile. +- **`--testid` is lowercase.** `--testId` is not recognized as a flag and its value is treated as a positional target. + +## Lifecycle + +| Command | Description | +| --- | --- | +| `yarn mm launch [options]` | Start the daemon and launch an iOS session using the installed app state | +| `yarn mm cleanup` | Close the app and clear the active session | +| `yarn mm cleanup --shutdown` | Clean up the session and stop the daemon | +| `yarn mm status` | Show daemon and session status | +| `yarn mm stop [--force]` | Stop the daemon; `--force` also clears stale daemon state | +| `yarn mm serve [--background]` | Start the daemon without launching a session | + +### Supported launch flags + +The locally installed core CLI parser exposes these launch flags that are meaningful for this consumer: + +| Flag | Description | +| --- | --- | +| `--platform ios` | Explicitly select the iOS platform (plain launch is preferred, as the mobile consumer automatically handles it) | +| `--device-id ` | Select an iOS Simulator by UDID | +| `--app-bundle ` | Install a specific `.app` bundle before launching (for example `ios/build/MetaMask.app`). Required before any destructive flag | +| `--metro-port ` | Attach to a running Metro bundler on the given port. Equivalent to `MM_METRO_PORT`; the flag wins when both are set | +| `--goal ` | Record the session goal in knowledge metadata | +| `--flow-tags ` | Record comma-separated flow tags | +| `--force` | Replace an existing active session | + +Launch timeout is 120 seconds. The project CLI does not build MetaMask. Install the intended app on the simulator before launching, or pass `--app-bundle ` to install a specific build. + +The destructive flags `--reinstall`, `--reset-app-data`, and `--allow-fox-code-mismatch` are also parsed but guarded — see [Destructive Launch Flags](#destructive-launch-flags). For command-line Metro attachment, use `--metro-port ` or the `MM_METRO_PORT` environment variable as documented below. + +## Destructive Launch Flags + +These flags replace the installed app and destroy the wallet state it holds, so this prod-only consumer guards them. Do not use them on a wallet you need to preserve. + +| Flag | Description | +| --- | --- | +| `--reinstall` | Uninstall and reinstall the app before launching. Rejected unless `--app-bundle` is also supplied | +| `--reset-app-data` | Clear the app container/wallet state before launching. Rejected unless `--app-bundle` is also supplied | +| `--allow-fox-code-mismatch` | Bypass the app-identity guard so a bundle with a different `fox_code` can be installed. May make existing wallet/keychain data unreadable | + +Guardrails enforced at launch: + +- `--reinstall` and `--reset-app-data` are rejected unless `--app-bundle` is also supplied, because the installed app is otherwise the only copy and would be destroyed by the uninstall step. The rejection surfaces as `MM_INVALID_CONFIG`. +- Installing a bundle whose `fox_code` differs from the installed app is rejected (`MM_INVALID_CONFIG`) unless `--reinstall` or `--allow-fox-code-mismatch` is passed. +- A warning is printed to stderr whenever a destructive flag is honored. + +```bash +# Replace the installed app with a local build, discarding wallet state +yarn mm launch --app-bundle ios/build/MetaMask.app --reinstall +``` + +## Installed State + +`yarn mm launch` opens the MetaMask app already installed on the selected simulator and preserves its current wallet state. The workflow does not guarantee: + +- an onboarding or unlocked screen +- a password +- a particular account, network, token, or balance + +Always run `yarn mm describe-screen` and inspect the current app before interacting. Obtain credentials from the user or approved environment rather than assuming them. + +The generic core package may display environment-selection and test-state commands because it supports other consumers. This mobile consumer always reports prod, cannot switch environments, and does not provide local test-state capabilities. Do not use those generic commands in a mobile visual-testing flow. + +## Interaction + +| Command | Description | +| --- | --- | +| `yarn mm click ` | Click by accessibility reference or test ID | +| `yarn mm type ` | Clear and type into an editable element | +| `yarn mm get-text ` | Read an element's text | +| `yarn mm wait-for ` | Wait for an element to become visible | + +All interaction commands accept on iOS: + +- `--timeout `: one deadline for visibility and action +- `--testid `: target by test ID + +`--selector` and `--within` are parsed but rejected by the iOS driver (see [Targeting on iOS](#targeting-on-ios)). To disambiguate duplicate targets, use a unique test ID or the exact element's fresh a11y ref — there is no scoped-search fallback on mobile. + +## Discovery + +| Command | Description | +| --- | --- | +| `yarn mm describe-screen` | Return app state, visible test IDs, accessibility tree, and prior knowledge | +| `yarn mm screenshot [--name ]` | Capture the current simulator screen | +| `yarn mm accessibility-snapshot` | Return a trimmed accessibility tree (the shared `--root ` flag is ignored on iOS; the full tree is always returned) | +| `yarn mm list-testids [--limit ]` | List visible test IDs | +| `yarn mm get-state` | Return the current app-state snapshot | +| `yarn mm get-context` | Report the static prod environment and available mobile capabilities | + +Mutating tools return observations with fresh references. Later actions may return compact differences from the prior observation. Use `describe-screen` whenever a complete tree or refreshed baseline is needed. + +## Knowledge Store + +| Command | Description | +| --- | --- | +| `yarn mm knowledge-search ` | Search recorded steps | +| `yarn mm knowledge-last` | Return recent steps from the current session | +| `yarn mm knowledge-sessions` | List recorded sessions and metadata | +| `yarn mm knowledge-summarize [--session ]` | Generate a session recipe | + +## Batching + +### `yarn mm run-steps ''` + +The argument must be a JSON object containing a `steps` array: + +```bash +yarn mm run-steps '{"steps":[ + {"tool":"click","args":{"a11yRef":"e3"}}, + {"tool":"wait_for","args":{"testId":"home-screen","timeoutMs":10000}} +]}' +``` + +| Parameter | Description | +| --- | --- | +| `steps` | Required array of `{tool, args}` objects | +| `stopOnError` | Stop after the first failure; defaults to `false` (remaining steps still run). Set `true` to abort on the first error | +| `includeObservations` | `'all'`, `'none'`, or `'failures'` | +| `batchTimeoutMs` | Overall deadline; remaining steps are skipped after expiry | + +Each step is independently checked for mobile platform support. Do not put browser-only commands in a mobile batch. + +## Hermes CDP + +### `yarn mm cdp [params-json] [flags]` + +On mobile, `cdp` connects to the Hermes runtime through Metro's inspector proxy. It does not expose browser DOM, Page, or Network domains. + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify(globalThis.__DEV__)"}' +yarn mm cdp Runtime.evaluate '{"expression":"1+1"}' --timeout 60000 +``` + +| Flag | Description | +| --- | --- | +| `--timeout ` | Per-command timeout | +| `--metro-port ` | Override the Metro inspector port for this command | +| `--app-id ` | Require a matching app identity for target selection | + +### `yarn mm hermes-targets [flags]` + +Read-only, mobile-only. Lists and diagnoses the debuggable React Native Hermes targets Metro exposes, reporting which target would be chosen or why selection is ambiguous. Use it to confirm Metro is running, the app is registered, and to discover the real app identity. + +| Flag | Description | +| --- | --- | +| `--all` | List every discovered target instead of only the selected one | +| `--metro-port ` | Override the Metro inspector port | +| `--app-id ` | Require a matching app identity for target selection | + +The driver verifies app identity and device pinning before executing commands. + +Start Metro attachment before runtime inspection, using either the `--metro-port` flag or the `MM_METRO_PORT` environment variable (the flag wins when both are set): + +```bash +yarn watch:clean +yarn mm launch --metro-port 8081 + +# Equivalent, still supported +MM_METRO_PORT=8081 yarn mm launch +``` + +On Node 20, add `NODE_OPTIONS="--experimental-websocket"` when launching. Run `yarn mm describe-screen` after runtime mutation to resynchronize observations. + +## Simulator Selection + +```bash +xcrun simctl list devices +xcrun simctl boot +yarn mm launch --device-id +``` + +Prefer one booted simulator and one Metro process per worktree, especially during Hermes inspection. + +## Not Available on Mobile + +The core CLI is shared across consumers, so its help lists commands that this prod-only mobile consumer does not support. Two groups are unavailable here: browser-only commands, and e2e-context commands (fixtures, seeding, environment switching). Do not use any of these in a mobile visual-testing flow. + +### Browser-only + +| Command | Mobile behavior or alternative | +| --- | --- | +| `yarn mm navigate ` | Browser-only; navigate through visible UI elements | +| `yarn mm navigate-home` | Not implemented; click the Wallet tab in the UI | +| `yarn mm navigate-settings` | Not implemented; click the Settings tab in the UI | +| `yarn mm switch-to-tab` | Browser tabs do not exist in the mobile session | +| `yarn mm close-tab` | Browser tabs do not exist in the mobile session | +| `yarn mm wait-for-notification` | Browser notification-page command | +| `yarn mm clipboard` | Browser command in the current CLI surface | +| `yarn mm mock-network` | Browser-only network interception | +| `yarn mm build` | No mobile build capability; build and install separately | + +### E2E-context only (not applicable to prod-only mobile) + +This consumer always reports the static `prod` environment and provides no local test infrastructure, so these commands are meaningless here even though the shared CLI still lists them. + +| Command | Mobile behavior or alternative | +| --- | --- | +| `yarn mm set-context` | Environment switching is unavailable; the mobile consumer is always `prod` | +| `yarn mm get-state` | E2E fixture/state snapshot; use `yarn mm describe-screen` for live UI state | +| `yarn mm seed-contract` | Contract seeding needs the e2e chain, which this consumer does not run | +| `yarn mm seed-contracts` | Contract seeding needs the e2e chain, which this consumer does not run | +| `yarn mm get-contract-address` | Depends on seeded e2e contracts that do not exist here | +| `yarn mm list-contracts` | Depends on seeded e2e contracts that do not exist here | + +`yarn mm get-context` is available and always reports the static `prod` environment plus the mobile capabilities on offer. + +`yarn mm cdp` and `yarn mm hermes-targets` are available for Metro-attached mobile development sessions. diff --git a/domains/testing/skills/mobile-visual-testing/references/error-recovery.md b/domains/testing/skills/mobile-visual-testing/references/error-recovery.md new file mode 100644 index 00000000..95c34f9d --- /dev/null +++ b/domains/testing/skills/mobile-visual-testing/references/error-recovery.md @@ -0,0 +1,141 @@ +# Error Recovery and Troubleshooting for Mobile (iOS) + +## Contents + +- [On Failure](#on-failure) +- [Error Codes](#error-codes) +- [Common Failures and Solutions](#common-failures-and-solutions) +- [Safe App Resolution](#safe-app-resolution) +- [Daemon Issues](#daemon-issues) +- [Metro and Hermes Failures](#metro-and-hermes-failures) + +## On Failure + +If launch or the iOS toolchain is the problem (not an in-app screen issue), run the environment doctor first: + +```bash +yarn mm:doctor +``` + +It prints a PASS/FAIL report for Xcode, `idb`, `idb_companion`, and a booted simulator, with install commands for anything missing, and exits non-zero when a prerequisite is absent. This is the fastest way to resolve `MM_DEPENDENCIES_MISSING` and `MM_DEVICE_NOT_AVAILABLE`. + +For in-app failures: + +1. Run `yarn mm describe-screen`. +2. Identify the current screen and visible blockers. If the screen is unknown, capture a screenshot. +3. Query prior successful runs: + + ```bash + yarn mm knowledge-search "" + yarn mm knowledge-sessions + yarn mm knowledge-last + ``` + +4. Capture evidence with `yarn mm screenshot --name "debug"`. +5. Retry only after obtaining fresh accessibility references. + +## Error Codes + +Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only preserves a consumer-thrown code when it is a known core code and otherwise collapses it into `MM_LAUNCH_FAILED`, so the iOS driver reports launch and prerequisite failures with core codes (`MM_DEPENDENCIES_MISSING`, `MM_DEVICE_NOT_AVAILABLE`, `MM_INVALID_CONFIG`, `MM_LAUNCH_FAILED`). The iOS-specific detail is carried in the message and remediation text, not in a dedicated `MM_IOS_*` code. + +### Interaction Errors + +| Code | Meaning and recovery | +| --- | --- | +| `MM_TARGET_NOT_FOUND` | The element is not visible or the reference is stale. Run `yarn mm describe-screen` and target again. | +| `MM_WAIT_TIMEOUT` | The element did not appear before the deadline. Verify the screen and increase `--timeout` if the app is still transitioning. | +| `MM_CLICK_FAILED` | The element was found but could not be clicked. Check for overlays, alerts, or disabled state. | +| `MM_CLICK_TIMEOUT` | The click stalled and may have completed. Describe the screen before retrying. | +| `MM_TYPE_FAILED` | The target may not be editable. Verify the selected element and keyboard state. | +| `MM_TYPE_TIMEOUT` | Input stalled. Describe the screen, obtain a fresh target, and retry. | +| `MM_GETTEXT_FAILED` | The target detached or does not expose text. Re-describe and re-target. | +| `MM_GETTEXT_TIMEOUT` | Text retrieval exceeded the deadline. Retry with a fresh target or larger `--timeout`. | +| `MM_PAGE_CLOSED` | The target closed during the action. This can be expected for transitions; inspect current state. | +| `MM_BATCH_TIMEOUT` | `run-steps` exceeded `batchTimeoutMs`. Reduce the batch or increase its deadline. | + +### Session and Launch Errors + +| Code | Meaning and recovery | +| --- | --- | +| `MM_SESSION_ALREADY_RUNNING` | A session or launch already exists. Run `yarn mm cleanup`, then launch again. | +| `MM_NO_ACTIVE_SESSION` | No app session exists. Run `yarn mm launch`. | +| `MM_LAUNCH_FAILED` | The app or platform driver failed to start. Run `yarn mm cleanup` and retry; inspect the simulator, installed app, and `.mm-daemon.log`. | +| `MM_DEPENDENCIES_MISSING` | Xcode command-line tools or `idb` (Facebook iOS Debug Bridge) are missing. Run `yarn mm:doctor`, then install `idb` with `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb`. | +| `MM_DEVICE_NOT_AVAILABLE` | No simulator is booted, the given UDID does not exist, or `simctl` failed. Run `xcrun simctl list devices` and boot one; verify `--device-id`. | +| `MM_INVALID_CONFIG` | The launch options are not usable: no app installed and no `--app-bundle`, a destructive flag without `--app-bundle`, a `fox_code` mismatch, an unreachable Metro port, or an e2e-only option in this prod-only workflow. Read the remediation text in the error. | +| `MM_PORT_IN_USE` | The daemon port is already bound by stale state. Run `yarn mm stop --force`, then launch again. | +| `MM_INVALID_INPUT` | A command or flag value is malformed. Correct it before retrying. | + +### Discovery and Capture Errors + +| Code | Meaning and recovery | +| --- | --- | +| `MM_DISCOVERY_FAILED` | A `describe-screen` / accessibility snapshot failed. Wait for transitions to settle, capture a screenshot, and verify the app has not crashed, then retry. | +| `MM_SCREENSHOT_FAILED` | The simulator screen capture failed. Verify the simulator is booted and the session is active. | + +### Hermes and Platform Errors + +`cdp` and `hermes-targets` run against the Hermes runtime through Metro. The idb workflow surfaces a small set of codes here — do not expect granular per-phase Hermes codes. + +| Code | Meaning and recovery | +| --- | --- | +| `MM_HERMES_NOT_AVAILABLE` | `hermes-targets` was run on a session with no mobile driver, or Metro is not attached. Launch with `--metro-port ` (or `MM_METRO_PORT`) set and retry. | +| `MM_HERMES_FAILED` | Hermes target discovery failed. Verify Metro is running, the app is a development build, and one simulator/one Metro process per worktree. | +| `MM_CDP_BLOCKED` | The requested `cdp` method is blocked as destructive. Use a safe inspection method instead. | +| `MM_CDP_FAILED` | `cdp` execution failed or timed out. On Node 20 confirm `--experimental-websocket` was set at launch; inspect Metro and `.mm-daemon.log`. | +| `MM_TOOL_NOT_SUPPORTED_ON_PLATFORM` | The command is browser-only (or a mobile-only command was run without a mobile session). Use visible mobile UI interactions instead. | + +## Common Failures and Solutions + +| Symptom | Likely cause | Safe solution | +| --- | --- | --- | +| Previous session blocks launch | Session was not cleaned up | `yarn mm cleanup`, then `yarn mm launch` | +| No active session | The app has not been launched through the daemon | `yarn mm launch` | +| Launch cannot locate MetaMask | MetaMask is not installed on the selected simulator | Install the intended app build on that simulator, then launch again | +| Simulator is unavailable | No booted simulator or incorrect UDID | Check `xcrun simctl list devices`, boot the intended device, and use `--device-id ` | +| App identity mismatch (`MM_INVALID_CONFIG`, `different fox_code`) | Installed and requested builds have different `fox_code` values | Reuse the existing installed app, or install a matching build with `--app-bundle` plus `--reinstall` / `--allow-fox-code-mismatch` (destructive to wallet state) | +| `idb` not installed | Missing iOS Debug Bridge dependency (`MM_DEPENDENCIES_MISSING`) | Run `yarn mm:doctor`, then `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb` | +| Empty or failed screen snapshot | Splash screen, animation, app transition, or crash | Wait briefly, describe again, and capture a screenshot | +| Stale accessibility references | Screen changed after the references were generated | Run `yarn mm describe-screen` and use fresh references | +| Interaction timeout | Animation, overlay, or stale/ambiguous target | Re-`describe-screen`, target by a unique test ID or fresh a11y ref (no `--within` on mobile), and increase `--timeout` only when appropriate | +| `--testId` times out | Incorrect capitalization | Use `--testid` in lowercase | +| Daemon address already in use | Stale daemon state (`MM_PORT_IN_USE`) | Run `yarn mm stop --force`, then `yarn mm launch` | + +## Safe App Resolution + +The mobile workflow preserves the installed app and its wallet state by default. + +For a `fox_code` mismatch (`MM_INVALID_CONFIG`, message contains `different fox_code`): + +1. Prefer launching the already-installed app. +2. If replacing it, build and install a matching app outside of the `mm` workflow (such as through Xcode or standard repository setup scripts), or install it through the guarded `mm` path with `--app-bundle --reinstall`. Both `--reinstall` and `--allow-fox-code-mismatch` are destructive to existing wallet state. + +## Daemon Issues + +If the CLI hangs or returns connection errors: + +1. Check status: `yarn mm status`. +2. Stop stale state: `yarn mm stop --force`. +3. Inspect `.mm-daemon.log`. +4. Restart with `yarn mm launch`. + +The daemon shuts down after 30 minutes of inactivity. Its state is stored in `.mm-server` at the project root, isolated per worktree. + +## Metro and Hermes Failures + +If Metro attachment or Hermes inspection fails: + +1. Start Metro with `yarn watch:clean`. +2. Verify its status endpoint, normally `http://localhost:8081/status`. +3. Ensure `--metro-port ` (or `MM_METRO_PORT`) matches the running Metro process. +4. Confirm the installed app is a compatible development build and belongs to the selected simulator. +5. Use one Metro process and one active simulator per worktree to avoid target ambiguity. +6. Inspect `.mm-daemon.log` for attachment and target-selection details. +7. Restart safely: + + ```bash + yarn mm cleanup + yarn mm launch --metro-port 8081 + ``` + +On Node 20, launch the daemon with `NODE_OPTIONS="--experimental-websocket"` when Hermes WebSocket support is required. Node 22 and later provide WebSocket support directly. diff --git a/domains/testing/skills/mobile-visual-testing/references/runtime-monitoring.md b/domains/testing/skills/mobile-visual-testing/references/runtime-monitoring.md new file mode 100644 index 00000000..3f4ce7f4 --- /dev/null +++ b/domains/testing/skills/mobile-visual-testing/references/runtime-monitoring.md @@ -0,0 +1,285 @@ +# Runtime Monitoring — Network & Console Capture + +Inject interceptors into the Hermes runtime to capture `fetch` requests (including JSON-RPC method names and errors), and `console` output during visual testing. Read them back at any point to detect silent failures, slow API calls, RPC errors, or unexpected behavior that doesn't surface in the UI. + +## Contents + +- [Prerequisites](#prerequisites) +- [Install Interceptors](#install-interceptors) +- [Read Captured Data](#read-captured-data) +- [Check Interceptor Health](#check-interceptor-health) +- [Dismissing Dev Error Overlays](#dismissing-dev-error-overlays) +- [Anomaly Detection](#anomaly-detection) +- [Daemon & Metro Logs](#daemon--metro-logs) +- [Gotchas](#gotchas) + +## Prerequisites + +- Metro must be running (`MM_METRO_PORT` set at launch time) +- Same CDP requirements as [state-manipulation.md](state-manipulation.md) — Node 20 needs `--experimental-websocket` +- Interceptors operate in the JS thread only — native-layer network calls (e.g., iOS URLSession) are not captured + +## Install Interceptors + +Run once after launch. Installs both fetch and console interceptors in a single call. Idempotent — safe to re-run. + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"(function(){if(globalThis.__mmNet)return JSON.stringify(\"already installed\");var of=globalThis.fetch;globalThis.__mmNet=[];globalThis.fetch=function(){var u=arguments[0],o=arguments[1]||{},e={t:Date.now(),method:o.method||\"GET\",url:typeof u===\"string\"?u:(u&&u.url)||String(u)};if(o.body){try{var b=typeof o.body===\"string\"?JSON.parse(o.body):o.body;if(b&&b.method)e.rpcMethod=b.method}catch(x){}}globalThis.__mmNet.push(e);if(globalThis.__mmNet.length>2000)globalThis.__mmNet=globalThis.__mmNet.slice(-1000);return of.apply(globalThis,arguments).then(function(r){e.status=r.status;e.ms=Date.now()-e.t;if(e.rpcMethod){try{r.clone().text().then(function(t){try{var j=JSON.parse(t);if(j.error){e.rpcError=(j.error.message||JSON.stringify(j.error)).substring(0,200);e.rpcErrorCode=j.error.code}}catch(x){}}).catch(function(){})}catch(x){}}return r}).catch(function(err){e.err=String(err);e.ms=Date.now()-e.t;throw err})};globalThis.__mmCon=[];[\"log\",\"warn\",\"error\"].forEach(function(l){var orig=console[l];console[l]=function(){var a=Array.prototype.slice.call(arguments);globalThis.__mmCon.push({t:Date.now(),level:l,msg:a.map(function(x){try{return String(x)}catch(e){return\"[unstringifiable]\"}}).join(\" \")});if(globalThis.__mmCon.length>1000)globalThis.__mmCon=globalThis.__mmCon.slice(-500);return orig.apply(console,arguments)}});return JSON.stringify(\"interceptors installed\")})()","returnByValue":true}' +``` + +**Readable version** of the expression: + +```javascript +(function() { + if (globalThis.__mmNet) return JSON.stringify("already installed"); + + // --- Fetch interceptor (with JSON-RPC capture) --- + var origFetch = globalThis.fetch; + globalThis.__mmNet = []; + globalThis.fetch = function() { + var url = arguments[0]; + var opts = arguments[1] || {}; + var entry = { + t: Date.now(), + method: opts.method || "GET", + url: typeof url === "string" ? url : (url && url.url) || String(url), + }; + // Extract JSON-RPC method from request body + if (opts.body) { + try { + var b = typeof opts.body === "string" ? JSON.parse(opts.body) : opts.body; + if (b && b.method) entry.rpcMethod = b.method; + } catch(x) {} + } + globalThis.__mmNet.push(entry); + // Auto-trim: keep last 1000 when buffer exceeds 2000 + if (globalThis.__mmNet.length > 2000) { + globalThis.__mmNet = globalThis.__mmNet.slice(-1000); + } + return origFetch.apply(globalThis, arguments) + .then(function(resp) { + entry.status = resp.status; + entry.ms = Date.now() - entry.t; + // For RPC requests, clone response to extract JSON-RPC errors. + // Without this, RPC errors are invisible — Infura returns HTTP 200 + // with {"error": {"message": "..."}} in the body. + if (entry.rpcMethod) { + try { + resp.clone().text().then(function(body) { + try { + var j = JSON.parse(body); + if (j.error) { + entry.rpcError = (j.error.message || JSON.stringify(j.error)).substring(0, 200); + entry.rpcErrorCode = j.error.code; + } + } catch(x) {} + }).catch(function(){}); + } catch(x) {} + } + return resp; + }) + .catch(function(err) { + entry.err = String(err); + entry.ms = Date.now() - entry.t; + throw err; + }); + }; + + // --- Console interceptor --- + globalThis.__mmCon = []; + ["log", "warn", "error"].forEach(function(level) { + var orig = console[level]; + console[level] = function() { + var args = Array.prototype.slice.call(arguments); + globalThis.__mmCon.push({ + t: Date.now(), + level: level, + msg: args.map(function(a) { + try { return String(a); } catch(e) { return "[unstringifiable]"; } + }).join(" "), + }); + // Auto-trim: keep last 500 when buffer exceeds 1000 + if (globalThis.__mmCon.length > 1000) { + globalThis.__mmCon = globalThis.__mmCon.slice(-500); + } + return orig.apply(console, arguments); + }; + }); + + return JSON.stringify("interceptors installed"); +})() +``` + +## Read Captured Data + +### Drain both buffers (read and clear) + +Returns all captured data and resets the buffers. Use this between test steps or after a flow completes. + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"(function(){var r=JSON.stringify({net:globalThis.__mmNet||[],con:globalThis.__mmCon||[]});globalThis.__mmNet=[];globalThis.__mmCon=[];return r})()","returnByValue":true}' +``` + +### Read without clearing + +Peek at the buffers without resetting. Useful when you want to keep accumulating across steps. + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify({net:globalThis.__mmNet||[],con:globalThis.__mmCon||[]})","returnByValue":true}' +``` + +### Response shape + +```json +{ + "net": [ + { "t": 1718000000000, "method": "GET", "url": "https://api.example.com/data", "status": 200, "ms": 142 }, + { "t": 1718000001000, "method": "POST", "url": "https://mainnet.infura.io/v3/KEY", "status": 200, "ms": 85, "rpcMethod": "eth_blockNumber" }, + { "t": 1718000002000, "method": "POST", "url": "https://mainnet.infura.io/v3/KEY", "status": 200, "ms": 3021, "rpcMethod": "eth_sendRawTransaction", "rpcError": "Signer had insufficient balance", "rpcErrorCode": -32000 }, + { "t": 1718000003000, "method": "GET", "url": "https://api.example.com/fail", "err": "TypeError: Network request failed", "ms": 15000 } + ], + "con": [ + { "t": 1718000000500, "level": "warn", "msg": "Deprecated API call: use v2 endpoint" }, + { "t": 1718000001200, "level": "error", "msg": "Unhandled promise rejection: RPC timeout" } + ] +} +``` + +| Field | Type | Description | +|-------|------|-------------| +| `t` | number | Unix timestamp (ms) | +| `method` | string | HTTP method | +| `url` | string | Request URL | +| `status` | number | HTTP response status (absent if request failed) | +| `ms` | number | Duration in milliseconds | +| `err` | string | Network-level error message (absent if fetch succeeded) | +| `rpcMethod` | string | JSON-RPC method from request body (absent for non-RPC requests) | +| `rpcError` | string | JSON-RPC error message from response body (absent if RPC succeeded) | +| `rpcErrorCode` | number | JSON-RPC error code (absent if RPC succeeded) | +| `level` | string | `log`, `warn`, or `error` | +| `msg` | string | Stringified console arguments | + +## Check Interceptor Health + +Verify interceptors are still installed. They are lost on app reload or Fast Refresh. + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify({installed:!!globalThis.__mmNet,netCount:(globalThis.__mmNet||[]).length,conCount:(globalThis.__mmCon||[]).length})","returnByValue":true}' +``` + +If `installed` is `false`, re-run the install command. + +## Dismissing Dev Error Overlays + +The Engine module walk (`globalThis.__r(id)` loop from [state-manipulation.md](state-manipulation.md)) triggers **3–4 dev error overlays** in development builds. These are caused by modules that throw on import (e.g., `SegmentFetcher` TurboModule not found). The overlays are purely cosmetic — interceptors and cached references survive them — but they block UI interaction until dismissed. + +**You must dismiss ALL overlays, not just the first one.** Each overlay is a separate error stacked on top of the previous. + +### Dismiss loop pattern + +```bash +for i in 1 2 3 4 5 6 7; do + RESULT=$(yarn mm describe-screen 2>&1 | python3 -c " +import json, sys +a = json.load(sys.stdin).get('a11y', {}).get('nodes', []) +dismiss = [n for n in a if n.get('name', '').lower() == 'dismiss'] +print(dismiss[0]['ref'] if dismiss else 'CLEAN') +" 2>&1) + if [ "$RESULT" = "CLEAN" ]; then + echo "All overlays dismissed after $((i-1)) dismissals" + break + fi + yarn mm click "$RESULT" 2>&1 | grep -q clicked && echo "Dismissed overlay $i" + sleep 0.5 +done +``` + +### Recommended instrumentation sequence + +1. Install fetch/console interceptors (safe, no overlays) +2. Run the Engine module walk (triggers overlays) +3. Run the dismiss loop above +4. Verify instrumentation survived: check `!!globalThis.__mmDebugNet` and `!!globalThis.__mmEngine` + +### Notes + +- Typically **3 overlays** appear, but the count can vary by build. The loop handles up to 7 as a safety margin. +- The overlays do **not** cause an app reload — interceptors and the cached Engine reference remain intact. +- If the app does reload (Fast Refresh, Metro reconnect), you must re-install everything from step 1. +- The "Dismiss" button's a11y ref changes between `describe-screen` calls, so the loop re-queries each iteration. + +## Anomaly Detection + +After draining the buffers, flag these patterns: + +### Network anomalies + +| Pattern | How to detect | Severity | +|---------|--------------|----------| +| Failed requests | `status >= 400` or `err` field present | High | +| **JSON-RPC errors** | **`rpcError` field present** (HTTP status will be 200) | **Critical** | +| Slow requests | `ms > 5000` | Medium | +| Network errors | `err` contains "Network request failed" or "aborted" | High | +| Repeated failures | Same `url` failing 3+ times | High | +| Unexpected RPC calls | `url` contains unfamiliar host (not infura, metamask, etc.) | Medium | +| Transaction failures | `rpcMethod` is `eth_sendRawTransaction` and `rpcError` present | Critical | + +### Console anomalies + +| Pattern | How to detect | Severity | +|---------|--------------|----------| +| Errors | `level === "error"` | High | +| Unhandled rejections | `msg` contains "Unhandled" or "rejection" | High | +| React render warnings | `msg` contains "Cannot update a component" or "Maximum update depth" | Medium | +| Deprecation warnings | `msg` contains "deprecated" or "Deprecated" | Low | + +### When to drain + +- **After each major flow step** (e.g., after completing a send transaction) +- **On unexpected UI state** (loading spinner stuck, missing data, blank screen) +- **On test failure** (before reporting — include anomalies in the failure context) +- **At session end** (final drain for the test report) + +## Daemon & Metro Logs + +Two additional log sources complement the JS interceptors: + +### Daemon log + +Records CLI events, session lifecycle, and tool execution timing. + +```bash +# Read last 50 lines +tail -50 .mm-daemon.log + +# Search for errors +grep -i "error\|fail\|crash" .mm-daemon.log | tail -20 +``` + +### Metro output + +Metro logs JS errors, warnings, and bundle events. When running Metro separately (`yarn watch:clean`), redirect output to a file: + +```bash +yarn watch:clean 2>&1 | tee .mm-metro.log & +``` + +Then check for errors during testing: + +```bash +grep -i "error\|warn\|exception" .mm-metro.log | tail -20 +``` + +## Gotchas + +- **JSON-RPC errors return HTTP 200.** Infura and relay services return `{"error": {"message": "..."}}` in the response body with a 200 status code. Without the `rpcError` field extraction (included in the default interceptor above), these errors are completely invisible — a failed `eth_sendRawTransaction` looks identical to a successful one at the HTTP level. Always check `rpcError`, not just `status`, when debugging transaction failures. +- **Many app errors never reach `console.error`.** Controllers like TransactionController catch RPC exceptions internally and update state (e.g., marking a transaction as "failed") without any console output. The console interceptor will not capture these. To detect on-chain failures, check the `rpcError` field in network entries or verify transaction status via the UI/activity list. +- **Response body cloning is async.** The `rpcError` field is populated asynchronously via `resp.clone().text().then(...)`. In rare cases, if you drain the buffer immediately after a request completes (< 1ms), the `rpcError` field may not yet be populated. A brief `sleep 1` before draining after a transaction submission avoids this. +- The Engine module walk triggers **multiple dev error overlays** (typically 3–4). You must dismiss all of them in a loop, not just the first one. See [Dismissing Dev Error Overlays](#dismissing-dev-error-overlays) for the pattern. +- Interceptors are **lost on Fast Refresh and app reload**. After any code change with Metro watch mode, re-run the install command. Use the health check to verify. +- The fetch interceptor captures **JS-level fetch only**. Native HTTP calls (iOS URLSession, image loading, WebSocket connections) are not captured. +- Buffer auto-trim is aggressive (1000-2000 entries) to prevent memory pressure. For long test sessions, drain periodically to avoid losing early entries. +- Console interceptor wraps `log`, `warn`, and `error`. It skips `debug` and `info` to reduce noise. If you need those, modify the `forEach` array in the install expression. +- The `url` field for `fetch(Request)` calls extracts `Request.url`. If the app uses a custom fetch wrapper that passes non-standard first arguments, the `url` may show `[object Object]`. +- `returnByValue: true` is required in the cdp params. Without it, you get a remote object reference instead of the actual JSON string. +- Interceptors add ~0.1ms overhead per fetch call (~0.2ms for RPC requests due to response cloning). No measurable impact on app performance during testing. diff --git a/domains/testing/skills/mobile-visual-testing/references/state-manipulation.md b/domains/testing/skills/mobile-visual-testing/references/state-manipulation.md new file mode 100644 index 00000000..024ad84d --- /dev/null +++ b/domains/testing/skills/mobile-visual-testing/references/state-manipulation.md @@ -0,0 +1,262 @@ +# Runtime State Manipulation (Mobile) + +Use `mm cdp` only for advanced runtime inspection or manipulation of the current installed app during a Metro-attached session. On mobile, `mm cdp` connects to the Hermes JavaScript runtime through Metro's inspector proxy. These operations use `Runtime.evaluate` and require Metro to be running (`MM_METRO_PORT`). + +## Contents + +- [CDP Basics (Mobile)](#cdp-basics-mobile) +- [Fiber Entry Point (React Native)](#fiber-entry-point-react-native) +- [Operations](#operations) +- [Finding the Engine Singleton](#finding-the-engine-singleton) +- [Verify State After Mutation](#verify-state-after-mutation) +- [When to Use CDP](#when-to-use-cdp) + +## CDP Basics (Mobile) + +`mm cdp` sends a Chrome DevTools Protocol command. On mobile, it connects to the Hermes runtime via Metro's inspector proxy, targeting the **React Native JS thread** (on the extension it targets the browser page instead). + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify(1+1)"}' +yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify(globalThis.__DEV__)"}' +``` + +**Requirements:** +- Metro must be running (`MM_METRO_PORT` set at launch time) +- Node 20 requires `--experimental-websocket` flag at daemon launch +- Node 22+ works natively + +CDP calls are **mutating**, so run `mm describe-screen` afterward to re-sync the a11y ref map. + +## Fiber Entry Point (React Native) + +The extension walks fibers from a DOM node (`document.getElementById("app-content").__reactFiber$...`). React Native has no DOM. Instead, use `__REACT_DEVTOOLS_GLOBAL_HOOK__`: + +```javascript +var hook = globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__; +var rid = hook.renderers.keys().next().value; +var root = hook.getFiberRoots(rid).values().next().value; +var fiber = root.current; // root fiber, start walking from here +``` + +From `fiber`, traverse the tree with `.child`, `.sibling`, and `.return` exactly like the extension pattern. The fiber shape (`memoizedState`, `memoizedProps`, `stateNode`) is identical. + +## Operations + +| Operation | Method | Scope | +|---|---|---| +| Read Redux state | Fiber walk, store, `getState()` | In-memory UI state | +| Write Redux state | Fiber walk, store, `dispatch()` | Instant UI update, lost on restart | +| Call controller methods | Fiber walk, Engine singleton, `context.SomeController.method()` | Triggers real controller logic and state propagation | + +**Preferred order:** Call controller methods (operation 3) first. This is the most correct approach because the controller manages its own state and propagates to Redux. Fall back to Redux dispatch (operation 2) only when you need to fake state that no controller API provides. + +### 1. Read Redux State + +Find the Redux store on the `` component's fiber props: + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"(function(){var hook=globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__;var rid=hook.renderers.keys().next().value;var root=hook.getFiberRoots(rid).values().next().value;function find(f){if(!f)return null;if(f.memoizedProps&&f.memoizedProps.store&&typeof f.memoizedProps.store.getState===\"function\")return f.memoizedProps.store;return find(f.child)||find(f.sibling)}var store=find(root.current);if(!store)return JSON.stringify(\"store not found\");var s=store.getState();return JSON.stringify({userRegion:s.engine.backgroundState.RampsController.userRegion,moneyEnabled:!!s.engine.backgroundState.RemoteFeatureFlagController})})()","returnByValue":true}' +``` + +**Readable version** of the expression: + +```javascript +(function() { + var hook = globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__; + var rid = hook.renderers.keys().next().value; + var root = hook.getFiberRoots(rid).values().next().value; + + function find(f) { + if (!f) return null; + if (f.memoizedProps && f.memoizedProps.store + && typeof f.memoizedProps.store.getState === "function") + return f.memoizedProps.store; + return find(f.child) || find(f.sibling); + } + + var store = find(root.current); + if (!store) return JSON.stringify("store not found"); + var s = store.getState(); + return JSON.stringify({ + userRegion: s.engine.backgroundState.RampsController.userRegion, + }); +})() +``` + +### 2. Write Redux State + +Dispatch an action to update the backgroundState slice. The UI re-renders immediately. + +```javascript +(function() { + // ... same fiber walk to find store ... + var s = store.getState(); + var bg = s.engine.backgroundState; + + // Patch the target controller state + var ramps = Object.assign({}, bg.RampsController, { + userRegion: { + regionCode: "BR", + country: { isoCode: "BR", name: "Brazil", supported: { buy: false } }, + state: null + } + }); + + // Dispatch backgroundState update + store.dispatch({ + type: "UPDATE_BG_STATE", + key: "RampsController", + payload: ramps + }); + return "ok"; +})() +``` + +> **Note:** The exact action type for backgroundState updates may differ. If `UPDATE_BG_STATE` doesn't work, inspect the Redux reducer to find the correct action type. Redux dispatch only updates what the UI reads via selectors, but it does NOT modify the live controller instance. + +### 3. Find Engine Singleton and Call Controller Methods (Preferred) + +The Engine singleton holds every controller at `Engine.context`. Components that use controllers (e.g., `useRampsProviders`, `useMoneyAccountDeposit`) import Engine as a module dependency. Walk the fiber tree looking for an object with the controller context shape: an object that has `RampsController`, `TransactionController`, and `NetworkController` as properties. + +**Strategy A: Shape-match Engine.context on fiber props/state:** + +```javascript +(function() { + var hook = globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__; + var rid = hook.renderers.keys().next().value; + var root = hook.getFiberRoots(rid).values().next().value; + var visited = 0; + + function isEngineContext(obj) { + return obj + && typeof obj.RampsController !== "undefined" + && typeof obj.TransactionController !== "undefined" + && typeof obj.NetworkController !== "undefined"; + } + + function searchObj(obj, depth) { + if (!obj || depth > 3 || typeof obj !== "object") return null; + if (isEngineContext(obj)) return obj; + for (var k in obj) { + try { + var v = obj[k]; + if (v && typeof v === "object") { + var found = searchObj(v, depth + 1); + if (found) return found; + } + } catch(e) {} + } + return null; + } + + function walk(f) { + if (!f || visited > 2000) return null; + visited++; + + // Check memoizedProps + var ctx = searchObj(f.memoizedProps, 0); + if (ctx) return ctx; + + // Check stateNode + if (f.stateNode && typeof f.stateNode === "object") { + ctx = searchObj(f.stateNode, 0); + if (ctx) return ctx; + } + + // Check hook state chain (memoizedState linked list) + var hookState = f.memoizedState; + while (hookState) { + if (hookState.memoizedState && typeof hookState.memoizedState === "object") { + ctx = searchObj(hookState.memoizedState, 0); + if (ctx) return ctx; + } + // useRef stores value in .current + if (hookState.memoizedState && hookState.memoizedState.current) { + ctx = searchObj(hookState.memoizedState.current, 0); + if (ctx) return ctx; + } + hookState = hookState.next; + } + + return walk(f.child) || walk(f.sibling); + } + + var ctx = walk(root.current); + if (!ctx) return JSON.stringify("Engine.context not found after " + visited + " fibers"); + + // Now call the controller method + ctx.RampsController.setUserRegion("BR"); + return JSON.stringify("setUserRegion called, visited " + visited + " fibers"); +})() +``` + +**Strategy B: Metro module registry (fallback):** + +In dev builds with Metro, `globalThis.__r` is Metro's module require function. Scan for the Engine module by checking exports: + +```javascript +(function() { + if (typeof globalThis.__r !== "function") return JSON.stringify("__r not available"); + + for (var id = 0; id < 80000; id++) { + try { + var m = globalThis.__r(id); + if (m && m.default && m.default.context + && m.default.context.RampsController + && m.default.context.NetworkController) { + // Found Engine default export + m.default.context.RampsController.setUserRegion("BR"); + return JSON.stringify("Engine found at module " + id); + } + } catch(e) {} + } + return JSON.stringify("Engine module not found"); +})() +``` + +> **Note:** Strategy B brute-forces module IDs. It works but is slow (~5-15s). Cache the module ID within a session once found. Strategy A (fiber walk) is faster and preferred. + +### Example: Simulate Unsupported Region for Fiat Deposits + +```bash +# 1. Set region to Brazil (unsupported for buy) +yarn mm cdp Runtime.evaluate '{"expression":"(function(){var hook=globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__;var rid=hook.renderers.keys().next().value;var root=hook.getFiberRoots(rid).values().next().value;var visited=0;function isCtx(o){return o&&typeof o.RampsController!==\"undefined\"&&typeof o.NetworkController!==\"undefined\"}function search(o,d){if(!o||d>3||typeof o!==\"object\")return null;if(isCtx(o))return o;for(var k in o){try{var v=o[k];if(v&&typeof v===\"object\"){var f=search(v,d+1);if(f)return f}}catch(e){}}return null}function walk(f){if(!f||visited>2000)return null;visited++;var c=search(f.memoizedProps,0)||search(f.stateNode,0);if(c)return c;var h=f.memoizedState;while(h){if(h.memoizedState&&typeof h.memoizedState===\"object\"){c=search(h.memoizedState,0);if(c)return c}h=h.next}return walk(f.child)||walk(f.sibling)}var ctx=walk(root.current);if(!ctx)return JSON.stringify(\"not found\");ctx.RampsController.setUserRegion(\"BR\");return JSON.stringify(\"region set to BR\")})()","returnByValue":true}' + +# 2. Wait for provider re-resolution +sleep 3 + +# 3. Navigate to Money, then Add Money sheet +yarn mm describe-screen +yarn mm click --testid money-action-button-row-add +yarn mm wait-for --testid money-add-money-sheet --timeout 10000 +yarn mm describe-screen +yarn mm screenshot --name "unsupported-region-no-deposit-funds" + +# 4. Verify: "Deposit Funds" option should be missing or disabled + +# 5. Restore to US +yarn mm cdp Runtime.evaluate '{"expression":"(function(){/* same walk */ctx.RampsController.setUserRegion(\"US\");return JSON.stringify(\"region restored\")})()","returnByValue":true}' +``` + +## Verify State After Mutation + +```bash +# Read current region from Redux +yarn mm cdp Runtime.evaluate '{"expression":"(function(){var hook=globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__;var rid=hook.renderers.keys().next().value;var root=hook.getFiberRoots(rid).values().next().value;function find(f){if(!f)return null;if(f.memoizedProps&&f.memoizedProps.store&&typeof f.memoizedProps.store.getState===\"function\")return f.memoizedProps.store;return find(f.child)||find(f.sibling)}var store=find(root.current);if(!store)return JSON.stringify(\"no store\");var r=store.getState().engine.backgroundState.RampsController;return JSON.stringify({regionCode:r.userRegion&&r.userRegion.regionCode,country:r.userRegion&&r.userRegion.country})})()","returnByValue":true}' +``` + +## When to Use CDP + +| Need | Approach | +|---|---| +| Read any Redux state value | Fiber walk, store, `getState()` | +| Change what the UI displays (fast, non-persistent) | Fiber walk, store, `dispatch()` | +| Trigger real controller logic (region change, provider refresh) | Fiber walk, Engine.context, controller method | +| Verify a JS global or Hermes flag | `Runtime.evaluate` with simple expression | +| Execute JS against the React Native runtime | `Runtime.evaluate` | + +| Symptom | Cause | Solution | +|---|---|---| +| Redux dispatch updated state but UI didn't change | Component reads from controller state, not Redux selector | Use Engine.context controller method instead | +| Fiber walk returns "not found" | DevTools hook not available (release build) or tree too deep | Try Metro `__r` fallback (Strategy B) | +| `setUserRegion` called but UI unchanged | Provider re-fetch is async; UI hasn't re-rendered yet | Wait 3-5 seconds, then `describe-screen` | diff --git a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md new file mode 100644 index 00000000..409833b8 --- /dev/null +++ b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md @@ -0,0 +1,205 @@ +--- +repo: metamask-mobile +parent: mobile-visual-testing +metadata: + location: test/llm-workflow/ + type: mobile-testing +--- + +# MetaMask Mobile Visual Testing — iOS + +Use this skill to visually validate MetaMask Mobile through the project-local `mm` CLI. + +## Architecture + +The `mm` CLI and a persistent local HTTP daemon come from `@metamask/client-mcp-core`. The device backend is `@metamask/device-mcp`, which drives the iOS Simulator through **`idb`** (`idb-companion` + `fb-idb`). Accessibility trees, taps, typing, and screenshots all flow through idb — there is no XCUITest runner in this workflow. + +For full architecture, component locations, and safety details, see the on-demand references below and the in-repo doc `tests/llm-workflow/README.md`. + +## Scope + +- iOS Simulator only; Android is unsupported. +- The workflow is prod-only and preserves installed app/wallet state by default. +- Launch reuses an installed MetaMask app on the target simulator. +- The workflow does not build the app, discover local build outputs, or initialize test state. + +## Prerequisites + +```bash +# Verify the iOS toolchain (Xcode, idb, idb_companion, booted simulator) +yarn mm:doctor + +# Install idb if MM_DEPENDENCIES_MISSING is reported +brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb + +# Build/install MetaMask separately if it is not already installed +yarn setup && yarn start:ios + +# Boot a simulator when needed +xcrun simctl boot +``` + +Run `yarn mm:doctor` before launching; it prints a PASS/FAIL report with install commands for anything missing and exits non-zero when a prerequisite is absent. If an app is not installed, install it separately on the simulator before launching. + +## Required Workflow + +### 1. Launch + +```bash +# Reuse the installed app and its current state +yarn mm launch + +# Pin a simulator if needed +yarn mm launch --device-id + +# Install a specific build before launching +yarn mm launch --app-bundle ios/build/MetaMask.app + +# Force-replace an existing active session (runs cleanup first) +yarn mm launch --force +``` + +There is only one supported environment: prod. Do not request or switch launch contexts. Supplying `--context e2e` is rejected. + +When a session is already active, `mm launch` rejects with `MM_SESSION_ALREADY_RUNNING` unless `--force` is passed (which cleans up the existing session then launches a new one). + +`--reinstall`, `--reset-app-data`, and `--allow-fox-code-mismatch` are destructive to the installed wallet state and are guarded (a destructive flag requires `--app-bundle`). See [references/cli-reference.md](references/cli-reference.md#destructive-launch-flags). + +When attaching to Metro (`--metro-port`), the workflow is **attach-only** — it never spawns Metro. If the app is already running and healthily attached to Metro (Hermes target found at `/json`), `mm launch` connects without relaunching. If the app is not healthily attached, it terminates and re-launches via the deep link. On a fresh-booted simulator, a one-time relaunch ensures the accessibility tree is valid. Release/prod builds have no Hermes inspector; Metro attach requires a dev build. + +### 2. Reuse Knowledge + +```bash +yarn mm knowledge-search "" +yarn mm knowledge-sessions +``` + +Reuse a known successful sequence when available. Otherwise discover the flow and let the session record it. + +### 3. Observe Before Acting + +```bash +yarn mm describe-screen +``` + +Use fresh output after navigation. Accessibility refs (`e1`, `e2`, ...) are ephemeral. + +### 4. Interact + +Only two targeting methods work on iOS: **test IDs** and **accessibility refs**. Use one per command; prefer a stable test ID, fall back to a fresh a11y ref from `describe-screen`. + +```bash +yarn mm click --testid unlock-submit +yarn mm type --testid unlock-password "" +yarn mm wait-for --testid account-overview --timeout 15000 +yarn mm get-text --testid balance-display + +yarn mm click e5 +yarn mm type e2 "text" +``` + +`--selector` (CSS) and `--within` (scoped search) are **rejected by the iOS driver** even though the shared CLI accepts the flags — see Gotchas. To disambiguate duplicate targets, use a unique test ID or the exact element's fresh a11y ref. + +The CLI flag is `--testid` (all lowercase); `--testId` is silently mis-parsed as a positional target and hits the wrong element. `mm type` clears the field before typing (idb does `cmd+a` → delete → type). + +### 5. Verify and Capture Evidence + +After a mutating sequence: + +1. Run `yarn mm describe-screen`. +2. Confirm the expected state. +3. Capture meaningful before/after evidence: + +```bash +yarn mm screenshot --name "after-action" +``` + +If the state is wrong, capture a debug screenshot, search knowledge, and retry from fresh refs. + +### 6. Cleanup + +```bash +yarn mm cleanup +yarn mm cleanup --shutdown +``` + +Always clean up when testing is complete. + +## Metro and Runtime Inspection + +For JS development, attach the installed development app to Metro. The workflow is attach-only — start Metro separately: + +```bash +yarn watch:clean +yarn mm launch --metro-port 8081 + +# Equivalent, still supported (the flag wins when both are set) +MM_METRO_PORT=8081 yarn mm launch +``` + +If Metro is not reachable on the given port, launch fails with `MM_INVALID_CONFIG`. If the app is already running and healthily attached to Metro, `mm launch` connects without relaunching (pure-attach). Release/prod builds have no Hermes inspector; Metro attach requires a dev build. + +Node 20 may require `NODE_OPTIONS="--experimental-websocket"` for `mm cdp`; Node 22+ includes WebSocket support. + +`mm cdp` evaluates JavaScript in the Hermes runtime through Metro's inspector proxy: + +```bash +yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify(globalThis.__DEV__)"}' +``` + +Prefer controller methods over raw Redux mutation when inspecting or changing runtime state. Runtime modifications affect the current installed app state; restore any state changed during testing. + +## Batching + +Use `run-steps` only for known deterministic sequences: + +```bash +yarn mm run-steps '{"steps":[ + {"tool":"type","args":{"testId":"login-password-input","text":""}}, + {"tool":"click","args":{"testId":"log-in-button"}}, + {"tool":"wait_for","args":{"testId":"wallet-screen","timeoutMs":15000}} +]}' +``` + +The input must be an object containing `steps`, not a bare array. + +## Mobile Limitations and Gotchas + +- **Only `--testid` and a11y refs target elements.** `--selector` throws (`CSS selectors are not supported on mobile`) and `--within` throws (`Scoped element search (within) is not supported on mobile`) at the driver, even though the shared CLI parses both flags. Do not use them on iOS. +- **`--testid` is case-sensitive and lowercase.** `--testId` is not recognized as a flag; the value is treated as a positional target and silently hits the wrong element (usually timing out). +- **Element matching is fuzzy and case-insensitive.** idb matches on accessibility label/identifier by substring, so `--testid Confirm` can match `Confirm Transaction`. Prefer exact, unique test IDs to avoid hitting the wrong element. +- **`mm type` clears first.** idb runs `cmd+a` → delete → type, so there is no need to clear the field manually. There is no trailing-newline submit trick; to submit, tap the on-screen keyboard action button (a fresh a11y ref) or the form's submit control. +- No `mm build`; build/install separately. +- No URL navigation, tab switching, browser notification pages, or browser clipboard APIs. +- `navigate-home` and `navigate-settings` are not implemented; navigate through visible UI elements. +- `mm cdp` requires Metro. +- One Metro process per worktree is recommended. +- Mutating commands can return compact observations; request a full `describe-screen` whenever refs or state are uncertain. +- Never assume wallet credentials, balances, networks, or onboarding state. Inspect the installed app and obtain needed credentials from the user/environment. + +## Error Recovery + +- `MM_DEPENDENCIES_MISSING`: Xcode command-line tools or `idb` are missing. Run `yarn mm:doctor`, then `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb`. +- `MM_WAIT_TIMEOUT`: target did not become visible; describe the screen and verify scope/test ID. +- `MM_CLICK_TIMEOUT`: click may have completed; describe before retrying. +- `MM_TYPE_TIMEOUT`: field interaction stalled; inspect focus and use a fresh target. +- `MM_DEVICE_NOT_AVAILABLE`: no simulator is booted, the UDID does not exist, or `simctl` failed. Run `xcrun simctl list devices` and boot one; verify MetaMask is installed. +- `MM_INVALID_CONFIG`: the launch options are unusable — no app and no `--app-bundle`, a destructive flag without `--app-bundle`, a `fox_code` mismatch, or an unreachable Metro port. Read the remediation text; reuse the installed app or install a matching build. + +Launch errors use core `ErrorCode`s (not `MM_IOS_*`): `@metamask/client-mcp-core` collapses unknown consumer codes into `MM_LAUNCH_FAILED`, so iOS detail is carried in the message and remediation. + +For the full error-code table and troubleshooting, see [references/error-recovery.md](references/error-recovery.md). + +## Reference Guides + +Load these on demand — not required for standard visual testing: + +- **[references/cli-reference.md](references/cli-reference.md)** — full command tables, syntax rules, targeting details, and commands not available on mobile. +- **[references/error-recovery.md](references/error-recovery.md)** — error codes, common failures, and troubleshooting. +- **[references/state-manipulation.md](references/state-manipulation.md)** — read/write runtime state and call controller methods via `mm cdp` (Hermes runtime). +- **[references/runtime-monitoring.md](references/runtime-monitoring.md)** — capture network requests and console logs via Hermes runtime interceptors. Load when testing flows that involve API calls or debugging silent failures. + +## References and Attribution + +- **In-repo workflow doc:** `tests/llm-workflow/README.md` — daemon/session architecture, installed-app safety, and the canonical prerequisites (`idb`, `yarn mm:doctor`). +- **Upstream packages:** `@metamask/client-mcp-core` (CLI + daemon) and `@metamask/device-mcp` (idb-based iOS device backend). diff --git a/domains/testing/skills/mobile-visual-testing/skill.md b/domains/testing/skills/mobile-visual-testing/skill.md new file mode 100644 index 00000000..6d7c523b --- /dev/null +++ b/domains/testing/skills/mobile-visual-testing/skill.md @@ -0,0 +1,4 @@ +--- +name: mobile-visual-testing +description: Drives the MetaMask Mobile via the mm CLI for visual testing in IOs simulator or Android emulator. Use when asked to visually verify UI changes, capture screenshots or debug Mobile UI state. Trigger phrases include "verify visually", "take a screenshot", "test the flow", and "check the UI". +---