From 1b43bac78450af75c603fa28f89119e1a6f7d430 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Tue, 22 Sep 2026 18:49:57 +0530 Subject: [PATCH 01/15] refactor(examples): one directory per adapter, and land the mobile ones --- .changeset/mobile-example-per-adapter.md | 18 + .gitignore | 7 +- ARCHITECTURE.md | 13 +- CLAUDE.md | 1 + examples/MOBILE.md | 379 ++++++++++++++++++ examples/README.md | 89 ++++ examples/mobile-preflight.cjs | 205 ++++++++++ .../nightwatch/{tests => bdd}/smoke-test.js | 0 examples/nightwatch/mobile/clock.js | 102 +++++ .../nightwatch/nightwatch.mobile.conf.cjs | 117 ++++++ examples/selenium-js/cucumber/cucumber.json | 12 + .../cucumber}/features/login.feature | 0 .../cucumber}/features/support/setup.js | 0 .../cucumber}/features/support/steps.js | 0 .../cucumber}/features/support/world.js | 0 .../test => selenium-js/jest}/example.js | 0 .../jest}/jest.config.json | 4 +- examples/selenium-js/mobile/clock.test.js | 198 +++++++++ .../test => selenium-js/mocha}/example.js | 0 .../{selenium => selenium-js}/package.json | 0 .../python-test => selenium-py}/.gitignore | 0 .../pytest}/pytest.ini | 0 .../pytest}/test_login_pytest.py | 0 examples/selenium-py/requirements-mobile.txt | 6 + examples/selenium-py/scripts/clock.py | 187 +++++++++ .../scripts}/login.py | 0 .../scripts}/web_form.py | 0 examples/selenium/cucumber-test/cucumber.json | 12 - examples/wdio/cucumber/wdio.conf.ts | 359 ++--------------- examples/wdio/cucumber/wdio.mobile.conf.ts | 78 ---- examples/wdio/cucumber/wdio.retention.conf.ts | 68 ---- examples/wdio/cucumber/wdio.trace.conf.ts | 66 --- examples/wdio/mobile/capabilities.ts | 81 ++++ examples/wdio/mobile/specs/clock.e2e.ts | 98 +++++ examples/wdio/mobile/wdio.conf.ts | 89 ++++ examples/wdio/mocha/wdio.conf.ts | 38 +- examples/wdio/mocha/wdio.trace.conf.ts | 60 --- package.json | 11 +- packages/nightwatch-devtools/package.json | 1 + packages/selenium-devtools-py/README.md | 6 +- packages/selenium-devtools/README.md | 10 +- packages/selenium-devtools/package.json | 11 +- 42 files changed, 1689 insertions(+), 637 deletions(-) create mode 100644 .changeset/mobile-example-per-adapter.md create mode 100644 examples/MOBILE.md create mode 100644 examples/README.md create mode 100644 examples/mobile-preflight.cjs rename examples/nightwatch/{tests => bdd}/smoke-test.js (100%) create mode 100644 examples/nightwatch/mobile/clock.js create mode 100644 examples/nightwatch/nightwatch.mobile.conf.cjs create mode 100644 examples/selenium-js/cucumber/cucumber.json rename examples/{selenium/cucumber-test => selenium-js/cucumber}/features/login.feature (100%) rename examples/{selenium/cucumber-test => selenium-js/cucumber}/features/support/setup.js (100%) rename examples/{selenium/cucumber-test => selenium-js/cucumber}/features/support/steps.js (100%) rename examples/{selenium/cucumber-test => selenium-js/cucumber}/features/support/world.js (100%) rename examples/{selenium/jest-test/test => selenium-js/jest}/example.js (100%) rename examples/{selenium/jest-test => selenium-js/jest}/jest.config.json (60%) create mode 100644 examples/selenium-js/mobile/clock.test.js rename examples/{selenium/mocha-test/test => selenium-js/mocha}/example.js (100%) rename examples/{selenium => selenium-js}/package.json (100%) rename examples/{selenium/python-test => selenium-py}/.gitignore (100%) rename examples/{selenium/python-test/trace-py-test => selenium-py/pytest}/pytest.ini (100%) rename examples/{selenium/python-test/trace-py-test => selenium-py/pytest}/test_login_pytest.py (100%) create mode 100644 examples/selenium-py/requirements-mobile.txt create mode 100644 examples/selenium-py/scripts/clock.py rename examples/{selenium/python-test => selenium-py/scripts}/login.py (100%) rename examples/{selenium/python-test => selenium-py/scripts}/web_form.py (100%) delete mode 100644 examples/selenium/cucumber-test/cucumber.json delete mode 100644 examples/wdio/cucumber/wdio.mobile.conf.ts delete mode 100644 examples/wdio/cucumber/wdio.retention.conf.ts delete mode 100644 examples/wdio/cucumber/wdio.trace.conf.ts create mode 100644 examples/wdio/mobile/capabilities.ts create mode 100644 examples/wdio/mobile/specs/clock.e2e.ts create mode 100644 examples/wdio/mobile/wdio.conf.ts delete mode 100644 examples/wdio/mocha/wdio.trace.conf.ts diff --git a/.changeset/mobile-example-per-adapter.md b/.changeset/mobile-example-per-adapter.md new file mode 100644 index 00000000..8b01d26d --- /dev/null +++ b/.changeset/mobile-example-per-adapter.md @@ -0,0 +1,18 @@ +--- +"@wdio/selenium-devtools": patch +"@wdio/nightwatch-devtools": patch +--- + +Add a mobile example per adapter — `pnpm demo:wdio:mobile`, `demo:selenium:mobile`, `demo:nightwatch:mobile`, `demo:python:mobile` — each honouring the same `DEVTOOLS_MODE=live|trace` switch the desktop demos already read. + +All four build the same capability bag — spelled per language, which is a duplication worth collapsing — so a difference in the dashboard between two adapters is a difference in the adapter rather than in the test. `examples/MOBILE.md` is the one place the prerequisites and switches are stated. + +The default target is the device's **own Clock app** (`com.google.android.deskclock`), which is what makes this landable at all: a native example previously needed an uploaded app and credentials in the environment, and that is the reason #354 closed without one. Nothing here needs an `.apk`. `APPIUM_APP` points it at a real app, and `DEVTOOLS_MOBILE=web` drives Chrome on the same device instead — worth running too, because a mobile browser session has a document and must keep every page-side call a native one skips. + +All four drive the same flow — start the 5-minute timer preset, pause it, clear it — and each clears any timer a previous run left behind, because a timer survives the session and while one exists the Timers tab shows its card instead of the presets. Verified on an Android 16 emulator with Clock 9.1; the Clock app updates independently of the Android version, so the resource-ids are re-read rather than assumed. + +Each one checks the toolchain before opening a session, through a shared `examples/mobile-preflight.cjs`, because none of the four frameworks reports a missing Appium in a way that names the cause: WDIO says "make sure browser driver is running", Nightwatch says it could not reach GeckoDriver, selenium-webdriver gives `ECONNREFUSED` and a stack trace. The check also distinguishes a missing Appium from a missing SDK from an emulator that is simply not started. + +Two framework details are worth recording, since both fail in ways that do not point at them. selenium-webdriver's `Builder.build()` throws unless `browserName` is a **string**, before it ever contacts the server, so a native session needs `browserName: ''` — which is the W3C signal for "no browser" anyway. And Nightwatch needs `selenium.use_appium`, not merely a `webdriver` block pointed at port 4723; without it it infers GeckoDriver, and even once connected it fills in `browserName: "firefox"` plus `moz:firefoxOptions` unless the capability is explicitly `null` — measured on the wire, and it made the adapters treat an app session as having a document. + +No adapter code changed: the dashboard already starts itself from `ensureBackendStarted()` and Python's `enable()`, so unlike the throwaway harnesses these replace, none of these examples needs a backend started by hand. diff --git a/.gitignore b/.gitignore index a36f454e..ad7b9c30 100644 --- a/.gitignore +++ b/.gitignore @@ -38,7 +38,10 @@ __pycache__/ *.tgz examples/wdio/wdio-*.json examples/wdio/wdio-*.webm +examples/wdio/*.log examples/nightwatch/logs/ +examples/**/__pycache__/ +.DS_Store # Adapter-encoded screencasts (written next to the project root by default) selenium-video-*.webm @@ -48,10 +51,6 @@ packages/nightwatch-devtools/nightwatch-video-*.webm # trace output (mode: 'trace') trace-*.zip examples/**/trace-*/ -# ...but this one is a committed example, not output. The rule above exists for -# unpacked archives (`trace-/`), and a folder named for what it -# demonstrates collides with it. -!examples/selenium/python-test/trace-py-test/ # test results examples/**/test-results*/ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index fd11018f..396bfb74 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -190,11 +190,16 @@ The DOM-walking scripts run in the page via `browser.execute`, so — like `scri ### `examples/` -Per-framework demo projects used for manual verification. +Per-adapter demo projects used for manual verification. One directory per adapter, and inside it one directory per test runner, named for the runner itself: -- `examples/wdio/` — WebdriverIO, split into `cucumber/` and `mocha/` (shared page objects in `pageobjects/`). Run via `pnpm demo:wdio` (Cucumber) or `pnpm demo:wdio:mocha`. -- `examples/nightwatch/` — Nightwatch (both vanilla and Cucumber). Run via `pnpm demo:nightwatch`. -- `examples/selenium/` — Selenium with subdirs for `mocha-test/`, `jest-test/`, `cucumber-test/`, `jasmine-test/`, `vitest-test/`. `pnpm demo:selenium` runs mocha; `pnpm --filter @wdio/selenium-devtools example:` runs the others. +- `examples/wdio/` — `mocha/`, `cucumber/`, `mobile/`, shared page objects in `pageobjects/`. Run via `pnpm demo:wdio` (Cucumber) or `pnpm demo:wdio:mocha`. +- `examples/nightwatch/` — `bdd/` (Nightwatch's `describe/it` interface), `cucumber/` and `mobile/`. Run via `pnpm demo:nightwatch`. +- `examples/selenium-js/` — `mocha/`, `jest/`, `cucumber/`, `mobile/`. `pnpm demo:selenium` runs Cucumber; `pnpm --filter @wdio/selenium-devtools example:` runs a specific one. +- `examples/selenium-py/` — the Python adapter: `scripts/` for the plain-script cases (no test runner) and `pytest/` for the pytest one. Run via `pnpm demo:python`, `:login` or `:pytest`. + +Every adapter also has a `mobile/` example driving the same Appium capability bag. A runner directory holds its own config and specs. `features/` inside a Cucumber directory is Cucumber's own convention, not ours. + +[`examples/README.md`](./examples/README.md) is the index: what each example demonstrates and how to run it. --- diff --git a/CLAUDE.md b/CLAUDE.md index 9196819f..4f4386a8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,6 +27,7 @@ Run from repo root unless noted. | `pnpm test:coverage` | Run vitest with v8 coverage. The thresholds in `vitest.config.ts` are aspirational, not a gate: that file states CI does not run this and the suite is currently below all four. CI runs `test`/`lint`/`test:ui`. | | `pnpm lint` | Lint all packages in parallel. Includes `eslint-plugin-security` for a subset of CodeQL findings; deeper taint-flow checks surface on the PR's CodeQL scan. | | `pnpm demo:wdio` / `pnpm demo:nightwatch` / `pnpm demo:selenium` | Run the per-framework example projects. Useful for manual verification of UI or runtime changes. | +| `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and default to the device's own Settings app, so a native example needs no `.apk` — `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches. `DEVTOOLS_MODE=live` flips any demo to live mode. | | `pnpm dev` | Run all packages in parallel dev mode. | `selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:mocha:allure` / `:jest` / `:cucumber`. diff --git a/examples/MOBILE.md b/examples/MOBILE.md new file mode 100644 index 00000000..82652717 --- /dev/null +++ b/examples/MOBILE.md @@ -0,0 +1,379 @@ +# Mobile examples + +One per adapter, all driving the same session so a difference in the dashboard +is a difference in the adapter and not in the test: + +```sh +pnpm demo:wdio:mobile +pnpm demo:selenium:mobile +pnpm demo:nightwatch:mobile +pnpm demo:python:mobile +``` + +`DEVTOOLS_MODE=live` flips any of them to live mode, exactly as the desktop +demos do. Trace is the default. + +```sh +DEVTOOLS_MODE=live pnpm demo:selenium:mobile +``` + +The dashboard starts itself — the adapter does it (`ensureBackendStarted` in the +JS adapters, `enable()` in Python). Nothing needs a backend started by hand. + +## What you need + +**This is an opt-in toolchain, and it is not small.** Between the Android SDK +and one system image, expect several gigabytes. Nothing here is bundled, and +none of it is needed for any other demo or test in this repo. + +1. **Android SDK and an emulator**, or a real device with USB debugging on. + Android Studio installs both, but it is not needed — Google's CLI installer + is far lighter and is the route these instructions assume: + + ```sh + # macOS arm64; see developer.android.com for the other builds + curl -fsSL https://dl.google.com/android/cli/latest/darwin_arm64/install.sh | bash + ``` + + That leaves `android-cli` in `~/.android/bin` (**not** on your PATH) and an + SDK root that it does _not_ necessarily report correctly: `android-cli info` + said `~/Library/Android/sdk` on the machine this was written on while the + actual `platform-tools`, `emulator` and `system-images` were in + `/opt/homebrew/share/android-commandlinetools`. **The directory holding + `platform-tools` is the one to export**, whatever `info` claims: + + ```sh + export ANDROID_HOME=/opt/homebrew/share/android-commandlinetools # check yours + export PATH="$PATH:$ANDROID_HOME/platform-tools:$ANDROID_HOME/emulator:$HOME/.android/bin" + ``` + + Appium's Android driver reads `ANDROID_HOME` and refuses the session without + it, _after_ accepting the connection — which is why an unset variable + surfaces as a 40-line driver stack trace rather than as a setup problem. + The preflight below catches that case and prints the export lines for + whichever SDK it can find. + + Then create a device once, and boot it: + + ```sh + android-cli --sdk="$ANDROID_HOME" emulator create medium_phone + # downloads an arm64 system image — a few minutes, ~3 GB + + emulator -avd medium_phone # leave this terminal open + adb wait-for-device + adb devices # must list emulator-5554 device + ``` + + The **emulator has to run in a terminal that stays alive** — a first boot + takes minutes, and anything that reaps the process group kills it mid-boot. + `android-cli emulator start medium_phone` also works and waits for boot, but + the raw `emulator` binary is the more predictable of the two. If a fresh API + 36 image fails to boot, its own log notes that "Guest Angle is still unstable + for API > 35" — add `-gpu swiftshader_indirect`. + +2. **Appium 2.x or 3.x** and the Android driver. **Export the SDK path first, + in this shell** — Appium's Android driver reads its OWN environment, so + exporting it where the tests run changes nothing and the session is refused + with `Neither ANDROID_HOME nor ANDROID_SDK_ROOT environment variable was +exported`, delivered through the client as a WebDriver failure that looks + like a test problem: + + ```sh + export ANDROID_HOME=/opt/homebrew/share/android-commandlinetools # yours may differ + export PATH="$PATH:$ANDROID_HOME/platform-tools" + + npm i -g appium + appium driver install uiautomator2 + appium --address 127.0.0.1 --port 4723 + ``` + + The preflight reads the running Appium's environment and says so when this + is what is wrong, because nothing else in the stack does. + +**You do not need an app.** The default target is the device's own Settings +app, so there is no `.apk` to build, upload, or keep credentials for — which is +what kept a native example from landing before (#354). + +Every one of the four checks this before it opens a session and tells you what +is missing, because the frameworks themselves do not: WDIO reports "make sure +browser driver is running", Nightwatch reports it could not reach _GeckoDriver_, +and selenium-webdriver reports `ECONNREFUSED` with a stack trace. All three mean +Appium is not up. `examples/mobile-preflight.cjs` is the shared check. It separates the five +states that look alike from the outside: Appium not up; Appium up but no SDK +exported here; **Appium up with no SDK in its own environment**, which is the +one that wastes the most time; SDK fine but no device attached; and a remote +Appium, for which none of the local checks apply. When an SDK is present but unexported it +prints the exact `export` lines for the path it found. + +## What the examples drive + +All four run the same flow against the **Clock app**, which ships with every +Android system image — so there is no `.apk` to supply, nothing to upload and no +credentials. Clock is used rather than Settings because it gives a native +session something deterministic to do: + +1. open the Timers tab +2. clear any timer a previous run left behind +3. start the 5-minute preset, and check the countdown is running +4. pause it, and check the control now offers **Start** +5. delete it, and check the timer is gone + +Step 2 is what makes them re-runnable. A timer **survives the session**, and +while one exists the Timers tab shows its card instead of the preset buttons — +so without it, one interrupted run breaks every later one. + +**VERIFIED ON:** Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), +Clock (`com.google.android.deskclock`) 9.1. The Clock app updates +**independently of the Android version**, so pinning a system image does not pin +these resource-ids. If a locator misses, re-read the tree rather than assuming +capture broke: + +```sh +adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml +``` + +A running countdown never reaches idle, and `uiautomator dump` fails outright +with `ERROR: could not get idle state` — so pause or clear the timer first. + +### Three per-adapter details worth knowing + +Each cost a debugging round when these examples were written, and none of them +points at itself: + +- **Selenium** — use `getDomAttribute()`, never `getAttribute()`. + selenium-webdriver implements the latter by executing a JavaScript atom, and a + native session has no JS to run it in; it fails with `Method is not + implemented`. It also does **no implicit wait**, so a tap that starts a screen + transition needs `driver.wait(until.elementLocated(...))` where WebdriverIO + auto-waits. +- **Nightwatch** — it rejects `-android uiautomator` with `InvalidSelectorError` + rather than forwarding it, so these use Appium's `id` strategy against a full + resource-id. And `findElements()` WAITS and then throws `NoSuchElementError` + when nothing matches, which Nightwatch reports as a run error even when + caught; the protocol-level `browser.elements()` returns an empty list instead. +- **Python** — the Appium client is an extra dependency the desktop examples do + not need; see below. + +## Choosing what to drive + +| variable | default | meaning | +| ------------------------------------------ | -------------------- | ----------------------------------------------------------------------------- | +| `DEVTOOLS_MODE` | `trace` | `live` opens the dashboard and streams; `trace` writes a zip | +| `DEVTOOLS_MOBILE` | `native` | `web` drives Chrome on the device instead of an app | +| `APPIUM_APP` | — | path to an `.apk`/`.app` to drive instead of Clock | +| `APPIUM_HOST` / `APPIUM_PORT` | `127.0.0.1` / `4723` | where Appium is listening | +| `DEVTOOLS_MOBILE_PLATFORM` | `android` | `ios` switches all four examples to XCUITest and the Clock app's bundle id | +| `IOS_DEVICE_NAME` / `IOS_PLATFORM_VERSION` | `iPhone 15` / — | which simulator; per-machine, see `xcrun simctl list devices` | + +```sh +DEVTOOLS_MOBILE=web pnpm demo:nightwatch:mobile # mobile web, not an app +APPIUM_APP=/tmp/my.apk pnpm demo:wdio:mobile # a real app +``` + +**`DEVTOOLS_MOBILE=web` needs one extra thing from the server.** Chrome on the +device needs a matching chromedriver, and the emulator's Chrome is usually +newer than anything installed — the session then fails with `No Chromedriver +found that can automate Chrome '133.0.6943'`. Appium can fetch one, but that is +a **server feature**, not a capability, so it goes on the `appium` command: + +```sh +appium --address 127.0.0.1 --port 4723 \ + --allow-insecure=uiautomator2:chromedriver_autodownload +``` + +There is no `appium:chromedriverAutodownload` capability, despite how often it +is written down — the driver only reads `chromedriverExecutable` and +`chromedriverExecutableDir`, so that name is silently ignored. Pointing at a +chromedriver you already have works too: + +```js +'appium:chromedriverExecutable': '/path/to/chromedriver' +``` + +**Both targets are worth running**, because they take different paths through +capture and the difference is deliberate. A native app has no document, so the +adapters skip every page-side call — the DOM drain, the collector injection, +the per-action element and accessibility scripts, `url` and `title`. A mobile +**browser** session runs on the same phone and does have a page, so it keeps +all of them. Getting that distinction wrong in either direction is a bug +(`shared/src/device.ts` `isNativeAppSession` is the one place that decides). + +## Python needs one extra package + +The desktop Python examples need nothing beyond Selenium; an Appium session is +built through the Appium client: + +```sh +pip install -r examples/selenium-py/requirements-mobile.txt +``` + +## What the `web` target exercises + +`DEVTOOLS_MOBILE=web` drives Chrome on the same device instead of an app, and it +is worth running: a mobile browser session **has a document**, so it must keep +every page-side call a native session skips. That contrast is the whole point of +the native guards, and only the web target proves the guards did not go too far. + +A mobile-web Appium session used to deadlock — the adapter issued page-side +calls from inside the hook wrapping the command being captured, and Appium +serialises commands per session, so the wrapped command never reached the +browser. That is fixed: per-action snapshots are now skipped only where Appium +has a document to probe, which is the webview half of a hybrid app. A mobile +browser and a native app are both captured normally. + +The traps below are about setup rather than the adapter, and you will hit them +on the way. + +## The `web` target has three separate traps + +All three report as the same thing — `No Chromedriver found that can automate +Chrome 'N'` — so they are worth telling apart. Measured against a real +emulator: + +1. **Autodownload is a server feature, not a capability.** Add + `--allow-insecure=uiautomator2:chromedriver_autodownload` to the `appium` + command. When the flag is missing the error carries the suffix "You could + also try to enable automated chromedrivers download"; when it is present the + suffix disappears, which is the only way to tell from the client. + +2. **`appium:chromedriverAutodownload` does not exist.** The driver reads only + `chromedriverExecutable` and `chromedriverExecutableDir`. The other spelling + is widely copied — this repo had it in `examples/wdio/cucumber/wdio.mobile.conf.ts` + too — and it is silently ignored. + +3. **A `sudo npm i -g appium` leaves the download target root-owned.** The + driver tree under `~/.appium` then belongs to root, autodownload fetches the + right chromedriver and fails to _unzip_ it with `EACCES`, and reports the + same "No Chromedriver found". Either fix the ownership: + + ```sh + sudo chown -R "$(whoami)" ~/.appium + ``` + + or give it somewhere writable, which needs no sudo: + + ```sh + CHROMEDRIVER_DIR=/tmp/chromedriver DEVTOOLS_MOBILE=web pnpm demo:wdio:mobile + ``` + +### And the emulator has to reach the page + +The default URL is public, and an emulator often cannot resolve public DNS +(corporate network or VPN). `10.0.2.2` is the emulator's alias for **this +machine's** localhost, so serving a page here is the reliable route: + +```sh +python3 -m http.server 8099 # in some directory with an index.html +DEVTOOLS_MOBILE_URL=http://10.0.2.2:8099/ DEVTOOLS_MOBILE=web pnpm demo:wdio:mobile +``` + +`DEVTOOLS_MOBILE_URL` skips the login assertions, which only exist on the +default page, and just navigates and captures. + +## Observed on a real emulator + +Measured on a `medium_phone` AVD, API 36 (Android 16) arm64, Appium 3.7.0 with +uiautomator2 7.6.1: + +- **The tests pass and the trace is correct.** `context-options` carried + `device: {platform: 'android', name: 'sdk_gphone64_arm64', version: '16'}` and + the device's real portrait viewport, `1080 × 2400` — not the 1280x720 + fallback — so the player takes the device-column layout. +- **The `web` target was NOT verified on a device.** With the flag and a + writable driver directory the session is created and chromedriver 133 is + fetched, but every command then timed out at 180 s, including the capture's + own `execute/sync`. That emulator's networking was independently unhealthy + (`ping 8.8.8.8` returned 50% loss with duplicate packets), so this is + unproven rather than broken. The **native** target on the same emulator works. +- **The UiAutomator2 instrumentation crashes partway through**, and it is not + this repo's doing: it happens with the filmstrip poller off, and once at + session creation (`The instrumentation process cannot be initialized`). After + it dies every `screenshot` and `source` probe returns `cannot be proxied … the +instrumentation process is not running`, so the trace ends up with fewer + per-action frames than commands. The assertions still pass and the zip is + still written. The emulator's own log notes "Guest Angle is still unstable for + API > 35", so a lower API image is the thing to try if this matters. + +## Expected noise + +Not failures, and not worth chasing: + +- **Selenium logs three BiDi warnings per run** (`BiDi LogInspector attach +failed`, `BiDi preload unavailable`, `BiDi NetworkInspector attach failed`). + The adapter requests `webSocketUrl` unconditionally and Appium serves no + BiDi, so the attach fails and capture falls back to per-document injection — + which is the correct path for a native session anyway. +- **An inherited `DEVTOOLS_MODE=live` changes what a run produces** — a + dashboard window and no zip, instead of a zip and no window. That is correct + behaviour for live mode and surprising when the variable is left over from an + earlier command, so each run now logs the mode it resolved. +- **Nightwatch's `describe/it` interface collapses per-test slicing** to one + session-scoped slice, so the config asks for `session` granularity rather + than pretending otherwise. See CLAUDE.md § Known debt. + +## iOS + +One switch, no hand-editing: + +```sh +DEVTOOLS_MOBILE_PLATFORM=ios IOS_DEVICE_NAME="iPhone 15" pnpm demo:wdio:mobile +``` + +It needs a Mac with **full Xcode** (Command Line Tools alone carry no +simulators), a booted simulator, and the XCUITest driver — whose first run also +builds WebDriverAgent: + +```sh +sudo xcode-select -s /Applications/Xcode.app/Contents/Developer +xcrun simctl list devices available | grep iPhone # the name is per-machine +xcrun simctl boot "iPhone 15" +appium driver install xcuitest +``` + +The target is the simulator's own Clock app (`com.apple.mobiletimer`), so +there is no `.app` to supply — the same trick as Android. + +**One iOS-specific thing to watch.** `getWindowSize()` returns **points** while +the screenshot is **pixels** — roughly a 3x factor, so a trace legitimately +declares `402 × 874` for a `1206 × 2622` image. That disagreement is why the +player fits a domless capture by the image's own decoded size and never by the +viewport. Android nearly hides it; iOS is where a regression there would show. + +## What to look for + +- The player uses the **device column** layout: the phone occupies the full + height at the right, with the dock between it and the Actions list. Drag the + divider between them to resize. +- The capture is framed in **device chrome** with the device's name above it, + rather than the mock browser window a desktop trace gets. +- The filmstrip thumbnails are **portrait**. +- **Metadata** names the device (`Pixel 7 (android 14)`) and the viewport the + device reported — not `1280 × 720`. +- On a native run **with the WDIO service**, the **A11y** tab is built from the + app's own view hierarchy, so it lists `android.widget.*` (or + `XCUIElementType*`) nodes rather than HTML roles. Selenium, Nightwatch and + Python do not derive one yet and show an empty tree — see Known gaps. + +## What has been verified, and what has not + +The three JS examples have been run end to end against a **stub Appium server** +— a local HTTP server implementing enough of the W3C protocol to complete a +session — so the client-side half is known to work: session creation, the +capability bag on the wire, the command path, and the guards. All three pass two +tests and make **zero page-script calls** on a native session. + +That is not the same as a device. It says nothing about whether the Clock app +has the views these examples look for, or whether `back()` behaves, or how a +real screenshot performs. Expect the first real run to need adjusting, and read +a failure as "the device disagreed" rather than "the example is broken". + +## Known gaps + +Both are tracked, and both are visible in these examples rather than hidden: + +- **Selenium and Nightwatch publish no viewport** (#373). The reader then falls + back to `1280 × 720`, which is landscape — and the player picks the stacked + layout for a landscape capture. So a native trace from those two adapters + does not get the device column until that is fixed. WDIO and Python do. +- **A native session gets no accessibility tree from Selenium, Nightwatch or + Python** (#372). Only the WDIO service derives one from the app's page source. diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 00000000..633e3bf8 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,89 @@ +# Examples + +Demo projects used to verify capture by hand. Type-checks and unit tests verify +code correctness; these verify that a feature actually works, which is what a UI +or runtime change has to be judged on. + +One directory per adapter, and inside it one per test runner, named for the +runner. A runner directory holds its own config and its specs — `features/` +inside a Cucumber directory is Cucumber's convention, not ours. + +``` +examples/ +├── wdio/ mocha/ cucumber/ mobile/ pageobjects/ +├── nightwatch/ bdd/ cucumber/ mobile/ +├── selenium-js/ mocha/ jest/ cucumber/ mobile/ +└── selenium-py/ scripts/ pytest/ +``` + +## Running them + +Every example drives the same login flow against the same site, so a trace from +one is comparable with a trace from another. + +| Command | What it runs | +| ---------------------------- | ---------------------------------------------------------------- | +| `pnpm demo:wdio` | WebdriverIO + Cucumber | +| `pnpm demo:wdio:mocha` | WebdriverIO + Mocha | +| `pnpm demo:wdio:retry` | WebdriverIO + Mocha, with a flaky test for retry-aware retention | +| `pnpm demo:nightwatch` | Nightwatch, BDD interface | +| `pnpm demo:nightwatch:retry` | The same, with `--retries 1` | +| `pnpm demo:selenium` | Selenium + Cucumber | +| `pnpm demo:wdio:retention` | WebdriverIO + Cucumber, a passing and a failing spec under a retention policy | +| `pnpm demo:python` | Python, plain script | +| `pnpm demo:python:login` | Python, plain script (login flow) | +| `pnpm demo:python:pytest` | Python + pytest | +| `pnpm demo:wdio:mobile` | WebdriverIO against Appium | +| `pnpm demo:nightwatch:mobile` | Nightwatch against Appium | +| `pnpm demo:selenium:mobile` | Selenium against Appium | +| `pnpm demo:python:mobile` | Python against Appium | + +`pnpm --filter @wdio/selenium-devtools example:` runs one Selenium +runner directly — `example:mocha`, `example:jest`, `example:cucumber`, or +`example:mocha:allure` for the Allure variant. + +`DEVTOOLS_MODE=trace` flips a demo to trace mode; the default is live. +`DEVTOOLS_TRACE_GRANULARITY` and `DEVTOOLS_TRACE_POLICY` walk the rest of the +ladder, so each runner needs only one config. + +## Mobile + +Every adapter has a `mobile/` example, and all four build the same capability +bag, so a difference in the dashboard between two of them is a difference in the +adapter rather than in the test. They drive the **Clock app**, which ships with every Android system image, so +they run without an `.apk` or credentials; +`APPIUM_APP` points one at a real app, and `DEVTOOLS_MOBILE=web` drives Chrome on +the same device instead — worth running too, because a mobile browser session +has a document and must keep every page-side call a native one skips. + +Each checks the toolchain before opening a session, through the shared +[`mobile-preflight.cjs`](./mobile-preflight.cjs), because none of the four +frameworks reports a missing Appium in a way that names the cause. + +[MOBILE.md](./MOBILE.md) has the prerequisites and the switches. + +## Per-adapter notes + +**`wdio/`** — `mocha/` carries the spec variants (`specs/`, and `retry/` for the +flaky test), each with its own `wdio.*.conf.ts`. `cucumber/` carries the feature +files. `pageobjects/` is shared by both, which is why it sits beside them rather +than inside either. + +**`nightwatch/`** — `bdd/` uses Nightwatch's `describe/it` interface, `cucumber/` +its Cucumber runner. The two interfaces capture differently: Cucumber exposes +per-scenario hooks, BDD `describe/it` does not, so per-test trace slicing +degrades to session scope there. + +**`selenium-js/`** — one directory per JS runner. The adapter is required via +`--require @wdio/selenium-devtools`, so each runner's own config is all that +differs. + +**`selenium-py/`** — `scripts/` holds the plain-script cases, which have no test +runner at all and produce a single synthetic suite; `pytest/` holds the pytest +case, where each test is a real node in the tree. + +## Output + +Runs write `test-results/` next to the example, and trace archives as +`trace-*.zip`. Both are ignored, and both are safe to delete — every one of them +is regenerated by the next run. diff --git a/examples/mobile-preflight.cjs b/examples/mobile-preflight.cjs new file mode 100644 index 00000000..489be95b --- /dev/null +++ b/examples/mobile-preflight.cjs @@ -0,0 +1,205 @@ +// Checks the mobile toolchain before a session is attempted, so a missing +// prerequisite reads as a prerequisite rather than as a driver failure. +// +// Without this the four frameworks each phrase the same missing Appium +// differently and none of them says what to do: WDIO reports "make sure browser +// driver is running", Nightwatch reports it could not reach GeckoDriver, +// selenium-webdriver reports ECONNREFUSED with a stack trace. +// +// CJS so the Nightwatch config can `require` it and the ESM examples can +// `import` it. + +const { execFileSync } = require('node:child_process') +const { existsSync } = require('node:fs') +const os = require('node:os') +const path = require('node:path') + +/** Hosts whose device this machine is expected to be able to see. A remote or + * cloud Appium needs no local SDK, so the local checks are skipped for it. */ +const LOCAL_HOSTS = new Set(['127.0.0.1', 'localhost', '::1', '0.0.0.0']) + +/** Where the SDK sits when Android Studio put it there, so a run that only + * lacks the export can be told the exact line rather than the whole setup. */ +function discoverSdk() { + const candidates = [ + path.join(os.homedir(), 'Library', 'Android', 'sdk'), + path.join(os.homedir(), 'Android', 'Sdk'), + '/usr/local/share/android-sdk', + '/opt/homebrew/share/android-sdk', + // Where Google's CLI installer and the homebrew cask land the SDK. Its own + // `android-cli info` may still report ~/Library/Android/sdk, which can be + // empty — so the directory holding platform-tools is what counts here. + '/usr/local/share/android-commandlinetools', + '/opt/homebrew/share/android-commandlinetools' + ] + return candidates.find((dir) => existsSync(path.join(dir, 'platform-tools'))) +} + +const SETUP = ` +Mobile examples need a device and Appium. Neither is bundled — between the +Android SDK and a system image this is a multi-gigabyte setup, so it is opt-in. + + 1. Android SDK + an emulator (or a real device with USB debugging on). + Android Studio installs both: https://developer.android.com/studio + Then check the device is visible: + + adb devices + + 2. Appium 2.x and the Android driver: + + npm i -g appium + appium driver install uiautomator2 + appium --address 127.0.0.1 --port 4723 + +To look at mobile capture WITHOUT any of that, the scratchpad harness builds +mobile traces and drives a live mobile session with no device at all — see +the mobile-adapters README. +` + +/** Whether Appium is answering. `/status` is unauthenticated and cheap. */ +async function appiumReady(host, port) { + try { + const res = await fetch(`http://${host}:${port}/status`, { + signal: AbortSignal.timeout(2500) + }) + return res.ok + } catch { + return false + } +} + +/** Devices `adb` can see, or null when adb itself is missing — the two are + * worth telling apart, because one is "install the SDK" and the other is + * "start the emulator". */ +function adbDevices() { + try { + const out = execFileSync('adb', ['devices'], { + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'ignore'] + }) + return out + .split('\n') + .slice(1) + .map((line) => line.trim()) + .filter((line) => line && !line.startsWith('*')) + } catch { + return null + } +} + +/** + * Whether the running Appium process has an SDK path in its own environment. + * `true` / `false` when that could be determined, `null` when it could not. + * + * This looks at ANOTHER process on purpose, and it is the single most + * confusing failure in the whole setup: the variable has to be exported in the + * shell that STARTED Appium, because Appium's Android driver reads its own + * environment. Exporting it where the test runs changes nothing, and the error + * comes back through the client as a WebDriver session failure, which points at + * the test rather than at the server. + */ +function appiumHasSdk() { + try { + const pids = execFileSync('pgrep', ['-f', 'appium'], { + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'ignore'] + }) + .split('\n') + .filter(Boolean) + if (!pids.length) { + return null + } + // `ps eww` prints a process's environment on macOS and Linux; it may be + // refused for a process owned by someone else, hence the null. + const env = execFileSync('ps', ['eww', '-p', pids[0]], { + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'ignore'] + }) + if (!/\bPATH=/.test(env)) { + return null + } + return /\bANDROID_(HOME|SDK_ROOT)=\S/.test(env) + } catch { + return null + } +} + +/** + * Print what is missing and exit, or return quietly. Exits rather than throws: + * a thrown error inside a framework hook is reported as a test failure, which + * is exactly the confusion this exists to remove. + */ +async function requireMobileToolchain({ + host = process.env.APPIUM_HOST ?? '127.0.0.1', + port = Number(process.env.APPIUM_PORT ?? 4723) +} = {}) { + if (await appiumReady(host, port)) { + // A remote or cloud Appium drives a device this machine knows nothing + // about, so none of the local checks below apply to it. + if (!LOCAL_HOSTS.has(host)) { + return + } + // Appium being up is NOT enough, and this is the case that made a 40-line + // stack trace: its Android driver reads ANDROID_HOME/ANDROID_SDK_ROOT and + // rejects the session with "Neither ANDROID_HOME nor ANDROID_SDK_ROOT + // environment variable was exported" long before any of our code runs. + const sdk = process.env.ANDROID_HOME || process.env.ANDROID_SDK_ROOT + if (!sdk) { + const found = discoverSdk() + console.error( + `\nAppium is up on ${host}:${port}, but neither ANDROID_HOME nor ` + + 'ANDROID_SDK_ROOT is set.\nIts Android driver needs one and will ' + + 'refuse the session without it.' + ) + console.error( + found + ? `\nAn SDK looks present at ${found}, so this is all that is missing:\n\n` + + ` export ANDROID_HOME=${found}\n` + + ` export PATH="$PATH:${found}/platform-tools"\n` + : SETUP + ) + process.exit(1) + } + // Ours is set — but Appium's is the one that counts. + if (appiumHasSdk() === false) { + console.error( + `\nAppium is up on ${host}:${port}, but its OWN environment has no\n` + + 'ANDROID_HOME or ANDROID_SDK_ROOT. Its Android driver reads that, so\n' + + 'exporting it here changes nothing — the session is refused and the\n' + + 'error arrives as a WebDriver failure that looks like a test problem.\n' + + '\nRestart Appium from a shell that has it:\n\n' + + ` export ANDROID_HOME=${sdk}\n` + + ` export PATH="$PATH:${sdk}/platform-tools"\n` + + ' appium --address 127.0.0.1 --port 4723\n' + ) + process.exit(1) + } + const devices = adbDevices() + if (devices && devices.length === 0) { + console.error( + `\nAppium is up on ${host}:${port}, but \`adb devices\` lists none.\n` + + 'Start an emulator or plug in a device, then run this again.\n' + ) + process.exit(1) + } + return + } + + const devices = adbDevices() + console.error( + `\nNothing is listening on ${host}:${port}, so Appium is not up.` + ) + if (devices === null) { + console.error( + '`adb` is not on PATH either, so the SDK is likely missing too.' + ) + } else if (devices.length) { + console.error( + `adb does see ${devices.length} device(s), so only Appium is missing.` + ) + } + console.error(SETUP) + process.exit(1) +} + +module.exports = { requireMobileToolchain, appiumReady, adbDevices } diff --git a/examples/nightwatch/tests/smoke-test.js b/examples/nightwatch/bdd/smoke-test.js similarity index 100% rename from examples/nightwatch/tests/smoke-test.js rename to examples/nightwatch/bdd/smoke-test.js diff --git a/examples/nightwatch/mobile/clock.js b/examples/nightwatch/mobile/clock.js new file mode 100644 index 00000000..4cb3e268 --- /dev/null +++ b/examples/nightwatch/mobile/clock.js @@ -0,0 +1,102 @@ +// Nightwatch against Appium. The WebdriverIO, Selenium and Python mobile +// examples drive the SAME flow, so a difference between two dashboards is a +// difference in the adapter rather than in the test. +// +// Drives the Clock app, which ships with every Android system image — no .apk, +// no upload, no credentials. Clock also gives a native session something +// deterministic to do: a timer can be started, paused and cleared, and each +// step changes the screen in a way the trace can be checked against. +// +// VERIFIED ON: Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), +// Clock (com.google.android.deskclock) 9.1. +// +// These are resource-ids read off that build, and the Clock app updates +// INDEPENDENTLY of the Android version — so pinning a system image does not pin +// them. If a locator misses, re-read the tree rather than assuming capture +// broke: `adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml` +// (a running countdown never reaches idle, so pause or clear the timer first). + +const APP_ID = 'com.google.android.deskclock' +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' + +/** A resource-id, in Nightwatch's element DEFINITION shape. The raw W3C + * `{using, value}` form is accepted syntactically and then issues no lookup + * at all, which reads as a missing element rather than a bad selector. + * + * Appium's `id` strategy matches a full resource-id, which keeps this inside + * Nightwatch's own strategy set — it rejects `-android uiautomator` with an + * InvalidSelectorError rather than forwarding it. */ +const byId = (id) => ({ + selector: `${APP_ID}:id/${id}`, + locateStrategy: 'id' +}) + +describe('Clock (native)', function () { + it('starts a preset timer, pauses it, and clears it', async function (browser) { + if (isWeb) { + // A mobile BROWSER session: it has a document, so every page-side call a + // native session skips must still happen. That contrast is the point. + await browser.url('https://the-internet.herokuapp.com/login') + await browser.assert.urlContains('the-internet') + return + } + + // Re-activated rather than relying on the launch capability alone, so the + // spec re-runs against a session left on another screen. + await browser.execute('mobile: activateApp', [{ appId: APP_ID }]) + + await browser.click(byId('tab_menu_timer')) + + // Clear anything a previous run left behind. A timer SURVIVES the session, + // and while one exists the Timers tab shows its card instead of the preset + // buttons — so without this, one interrupted run breaks every later one. + for (let i = 0; i < 5; i++) { + // `elements()` is the protocol-level lookup: it hands back a result with + // an empty list. The higher-level `findElements()` instead WAITS for a + // match and then throws NoSuchElementError, which Nightwatch reports as a + // run error even when caught — so "nothing to clear" would fail the test. + const found = await browser.elements('id', `${APP_ID}:id/delete_button`) + const count = Array.isArray(found) + ? found.length + : (found?.value?.length ?? 0) + if (!count) { + break + } + await browser.click(byId('delete_button')) + await browser.pause(300) + } + + // This build starts the timer straight from the preset, so the running + // countdown is the evidence the tap landed. + await browser.click(byId('timer_preset_2')) + await browser.assert.visible(byId('timer_text')) + + await browser.click(byId('play_pause_button')) + // The control's accessibility label flips with the timer's state, so + // asserting on it keeps this step off the countdown's own clock. + await browser.assert.attributeMatches( + byId('play_pause_button'), + 'content-desc', + /^Start/ + ) + + // Clearing the timer is what makes the spec re-runnable: it ends on the + // same screen it started from. + await browser.click(byId('delete_button')) + await browser.pause(500) + await browser.assert.not.elementPresent(byId('timer_text')) + }) + + it('captures a second action on the same session', async function (browser) { + // A second test, so `traceGranularity: 'test'` has two slices to key. + if (isWeb) { + await browser.url('https://the-internet.herokuapp.com/') + await browser.assert.urlContains('herokuapp.com') + return + } + + await browser.execute('mobile: activateApp', [{ appId: APP_ID }]) + await browser.click(byId('tab_menu_stopwatch')) + await browser.assert.visible(byId('tab_menu_stopwatch')) + }) +}) diff --git a/examples/nightwatch/nightwatch.mobile.conf.cjs b/examples/nightwatch/nightwatch.mobile.conf.cjs new file mode 100644 index 00000000..f270e40e --- /dev/null +++ b/examples/nightwatch/nightwatch.mobile.conf.cjs @@ -0,0 +1,117 @@ +// Mobile example for @wdio/nightwatch-devtools. Drives the device's own +// Settings app by default, so it needs no .apk. See examples/MOBILE.md for +// prerequisites and the DEVTOOLS_MOBILE / APPIUM_APP switches. +// +// pnpm demo:nightwatch:mobile +// DEVTOOLS_MODE=live pnpm demo:nightwatch:mobile + +const path = require('node:path') +const nightwatchDevtools = require('@wdio/nightwatch-devtools').default +const { requireMobileToolchain } = require('../mobile-preflight.cjs') + +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' + +/** The same bag the other three mobile examples build; see + * examples/wdio/mobile/capabilities.ts for the annotated original. */ +function mobileCapabilities() { + const base = { + platformName: IOS ? 'iOS' : 'Android', + 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', + // iOS needs the simulator named, and it is per-machine: + // `xcrun simctl list devices` shows yours. + ...(IOS + ? { + 'appium:deviceName': process.env.IOS_DEVICE_NAME ?? 'iPhone 15', + ...(process.env.IOS_PLATFORM_VERSION + ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } + : {}) + } + : {}), + 'appium:noReset': true, + 'appium:newCommandTimeout': 300 + } + if (isWeb) { + // Names a browser, so this session HAS a document and keeps its page-side + // capture — the distinction the native guards turn on. + return { + ...base, + // Chrome on the device needs a matching chromedriver. Appium can + // fetch one, but that is a SERVER feature, not a capability: + // `--allow-insecure=uiautomator2:chromedriver_autodownload`. + browserName: IOS ? 'Safari' : 'Chrome' + } + } + if (process.env.APPIUM_APP) { + return { ...base, browserName: null, 'appium:app': process.env.APPIUM_APP } + } + if (IOS) { + return { + ...base, + browserName: null, + 'appium:bundleId': 'com.apple.mobiletimer' + } + } + return { + ...base, + // Explicitly NO browser. Without this Nightwatch fills in its default and + // sends `browserName: "firefox"` plus `moz:firefoxOptions` — measured — and + // the adapters then correctly treat the session as having a document and + // run their page-side capture against an app. With `use_appium` set this + // returns early and does not warn as deprecated. + browserName: null, + 'appium:appPackage': 'com.google.android.deskclock', + 'appium:appActivity': 'com.android.deskclock.DeskClock' + } +} + +module.exports = { + src_folders: [path.resolve(__dirname, 'mobile')], + output_folder: false, + custom_commands_path: [], + custom_assertions_path: [], + + // Appium, not a browser driver Nightwatch would start itself. It has to be + // declared under `selenium` with `use_appium`, not merely pointed at port + // 4723: Nightwatch picks a transport from `browserName`, and with none it + // infers GeckoDriver and reports "Failed to connect to GeckoDriver". The + // older `browserName: null` route still works but warns as deprecated. + selenium: { + start_process: false, + use_appium: true, + host: process.env.APPIUM_HOST ?? '127.0.0.1', + port: Number(process.env.APPIUM_PORT ?? 4723), + // Appium 2 serves the W3C routes at the root; the default prefix is + // Selenium's `/wd/hub`. + default_path_prefix: '' + }, + webdriver: { + start_process: false + }, + + test_settings: { + default: { + skip_testcases_on_fail: false, + desiredCapabilities: mobileCapabilities(), + globals: Object.assign( + nightwatchDevtools({ + port: 3000, + // Trace by default, matching the desktop demos. + mode: process.env.DEVTOOLS_MODE === 'live' ? 'live' : 'trace', + // The BDD describe/it interface fires the plugin's beforeEach once per + // MODULE, so per-test slicing collapses to one session-scoped slice + // here regardless — see CLAUDE.md § Known debt. + traceGranularity: 'session', + // Appium serves no BiDi, so leave it off rather than have the attach + // fail once per run; the perf-log fallback covers a mobile-web session. + bidi: false + }), + { + // Nightwatch reports a missing Appium as a GeckoDriver failure, which + // names neither the cause nor the fix. + before: async () => requireMobileToolchain() + } + ) + } + } +} diff --git a/examples/selenium-js/cucumber/cucumber.json b/examples/selenium-js/cucumber/cucumber.json new file mode 100644 index 00000000..2100e3c0 --- /dev/null +++ b/examples/selenium-js/cucumber/cucumber.json @@ -0,0 +1,12 @@ +{ + "default": { + "import": [ + "../../examples/selenium-js/cucumber/features/support/setup.js", + "../../examples/selenium-js/cucumber/features/support/world.js", + "../../examples/selenium-js/cucumber/features/support/steps.js" + ], + "paths": ["../../examples/selenium-js/cucumber/features/*.feature"], + "publishQuiet": true, + "format": ["progress"] + } +} diff --git a/examples/selenium/cucumber-test/features/login.feature b/examples/selenium-js/cucumber/features/login.feature similarity index 100% rename from examples/selenium/cucumber-test/features/login.feature rename to examples/selenium-js/cucumber/features/login.feature diff --git a/examples/selenium/cucumber-test/features/support/setup.js b/examples/selenium-js/cucumber/features/support/setup.js similarity index 100% rename from examples/selenium/cucumber-test/features/support/setup.js rename to examples/selenium-js/cucumber/features/support/setup.js diff --git a/examples/selenium/cucumber-test/features/support/steps.js b/examples/selenium-js/cucumber/features/support/steps.js similarity index 100% rename from examples/selenium/cucumber-test/features/support/steps.js rename to examples/selenium-js/cucumber/features/support/steps.js diff --git a/examples/selenium/cucumber-test/features/support/world.js b/examples/selenium-js/cucumber/features/support/world.js similarity index 100% rename from examples/selenium/cucumber-test/features/support/world.js rename to examples/selenium-js/cucumber/features/support/world.js diff --git a/examples/selenium/jest-test/test/example.js b/examples/selenium-js/jest/example.js similarity index 100% rename from examples/selenium/jest-test/test/example.js rename to examples/selenium-js/jest/example.js diff --git a/examples/selenium/jest-test/jest.config.json b/examples/selenium-js/jest/jest.config.json similarity index 60% rename from examples/selenium/jest-test/jest.config.json rename to examples/selenium-js/jest/jest.config.json index 66853e25..30f5243c 100644 --- a/examples/selenium/jest-test/jest.config.json +++ b/examples/selenium-js/jest/jest.config.json @@ -1,6 +1,8 @@ { "testEnvironment": "node", - "testMatch": ["/test/example.js"], + "testMatch": [ + "/example.js" + ], "testTimeout": 60000, "transform": {} } diff --git a/examples/selenium-js/mobile/clock.test.js b/examples/selenium-js/mobile/clock.test.js new file mode 100644 index 00000000..72d8a2cc --- /dev/null +++ b/examples/selenium-js/mobile/clock.test.js @@ -0,0 +1,198 @@ +/** + * Mobile example for @wdio/selenium-devtools (Mocha runner). + * + * Drives the Clock app, which ships with every Android system image, so it + * needs no .apk. The WebdriverIO, Nightwatch and Python mobile examples drive + * the SAME flow, so a difference between two dashboards is a difference in the + * adapter rather than in the test. See examples/MOBILE.md for prerequisites and + * the DEVTOOLS_MOBILE / APPIUM_APP switches. + * + * VERIFIED ON: Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), + * Clock (com.google.android.deskclock) 9.1. The Clock app updates independently + * of the Android version, so pinning a system image does not pin these ids — + * re-read the tree with `adb shell uiautomator dump` if one misses. + * + * pnpm demo:selenium:mobile + * DEVTOOLS_MODE=live pnpm demo:selenium:mobile + * + * The dashboard starts itself — nothing needs a backend run by hand. + */ + +import { strict as assert } from 'node:assert' +import { Builder, until } from 'selenium-webdriver' +import { createRequire } from 'node:module' +import { DevTools } from '@wdio/selenium-devtools' + +const { requireMobileToolchain } = createRequire(import.meta.url)( + '../../mobile-preflight.cjs' +) + +DevTools.configure({ + // Trace by default, matching the desktop demos. + mode: process.env.DEVTOOLS_MODE === 'live' ? 'live' : 'trace', + traceGranularity: 'test' +}) + +const APP_ID = 'com.google.android.deskclock' +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' +const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ + process.env.APPIUM_PORT ?? 4723 +}` + +/** The same bag the other three mobile examples build; see + * examples/wdio/mobile/capabilities.ts for the annotated original. */ +function mobileCapabilities() { + const base = { + platformName: IOS ? 'iOS' : 'Android', + 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', + // iOS needs the simulator named, and it is per-machine: + // `xcrun simctl list devices` shows yours. + ...(IOS + ? { + 'appium:deviceName': process.env.IOS_DEVICE_NAME ?? 'iPhone 15', + ...(process.env.IOS_PLATFORM_VERSION + ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } + : {}) + } + : {}), + 'appium:noReset': true, + 'appium:newCommandTimeout': 300, + // selenium-webdriver's `Builder.build()` throws unless `browserName` is a + // STRING, before it ever contacts the server — and a native session has no + // browser. An empty string satisfies that check and is also the W3C signal + // for "no browser", which is what the adapters read to skip page-side + // capture. `withCapabilities` alone does not set it. + browserName: '' + } + if (isWeb) { + // Names a browser, so this session HAS a document and keeps its page-side + // capture — the distinction the native guards turn on. + return { + ...base, + // Chrome on the device needs a matching chromedriver. Appium can + // fetch one, but that is a SERVER feature, not a capability: + // `--allow-insecure=uiautomator2:chromedriver_autodownload`. + browserName: IOS ? 'safari' : 'chrome' + } + } + if (process.env.APPIUM_APP) { + return { ...base, 'appium:app': process.env.APPIUM_APP } + } + if (IOS) { + return { ...base, 'appium:bundleId': 'com.apple.mobiletimer' } + } + return { + ...base, + 'appium:appPackage': APP_ID, + 'appium:appActivity': 'com.android.deskclock.DeskClock' + } +} + +describe('Clock (native)', function () { + this.timeout(180000) + let driver + + /** A resource-id on the Clock app. UiAutomator2's own selector strategy, + * because selenium-webdriver's By.* helpers only speak web locators. */ + const uiSelector = (id) => ({ + using: '-android uiautomator', + value: `new UiSelector().resourceId("${APP_ID}:id/${id}")` + }) + /** Waited for, not just looked up: selenium-webdriver does no implicit wait, + * so a tap that starts a screen transition is followed by a findElement the + * destination has not rendered yet. WebdriverIO auto-waits and hides this. */ + const byId = async (id) => + driver.wait(until.elementLocated(uiSelector(id)), 15000) + + before(async function () { + // A missing emulator or Appium reads as a prerequisite here rather than as + // an ECONNREFUSED stack trace out of the driver. + await requireMobileToolchain() + driver = await new Builder() + .usingServer(APPIUM) + .withCapabilities(mobileCapabilities()) + .build() + }) + + after(async function () { + if (driver) { + await driver.quit() + } + }) + + it('starts a preset timer, pauses it, and clears it', async function () { + if (isWeb) { + // A mobile BROWSER session: it has a document, so every page-side call a + // native session skips must still happen. That contrast is the point. + await driver.get( + process.env.DEVTOOLS_MOBILE_URL ?? + 'https://the-internet.herokuapp.com/login' + ) + assert.ok((await driver.getCurrentUrl()).length > 0) + return + } + + // Re-activated rather than relying on the launch capability alone, so the + // spec re-runs against a session left on another screen. + await driver.executeScript('mobile: activateApp', { appId: APP_ID }) + + await (await byId('tab_menu_timer')).click() + + // Clear anything a previous run left behind. A timer SURVIVES the session, + // and while one exists the Timers tab shows its card instead of the preset + // buttons — so without this, one interrupted run breaks every later one. + for (let i = 0; i < 5; i++) { + const remaining = await driver.findElements(uiSelector('delete_button')) + if (!remaining.length) { + break + } + await remaining[0].click() + await driver.sleep(300) + } + + // This build starts the timer straight from the preset, so the running + // countdown is the evidence the tap landed. + await (await byId('timer_preset_2')).click() + const running = await (await byId('timer_text')).getText() + assert.match(running, /^\d{2}:\d{2}$/, `timer_text was "${running}"`) + + await (await byId('play_pause_button')).click() + // The control's accessibility label flips with the timer's state, so + // asserting on it keeps this step off the countdown's own clock. + // `getDomAttribute`, NOT `getAttribute`: selenium-webdriver implements the + // latter by executing a JavaScript atom, and a native session has no JS to + // run it in — it fails with "Method is not implemented". `getDomAttribute` + // is the plain W3C endpoint, which Appium serves. + const label = await ( + await byId('play_pause_button') + ).getDomAttribute('content-desc') + assert.match( + label, + /^Start/, + `expected the paused control to offer Start, got "${label}"` + ) + + // Clearing the timer is what makes the spec re-runnable: it ends on the + // same screen it started from. + await (await byId('delete_button')).click() + await driver.sleep(500) + assert.equal( + (await driver.findElements(uiSelector('timer_text'))).length, + 0, + 'the cleared timer is still on screen' + ) + }) + + it('captures a second action on the same session', async function () { + // A second test, so `traceGranularity: 'test'` has two slices to key. + if (isWeb) { + await driver.get('https://the-internet.herokuapp.com/') + assert.match(await driver.getCurrentUrl(), /herokuapp\.com/) + return + } + + await driver.executeScript('mobile: activateApp', { appId: APP_ID }) + await (await byId('tab_menu_stopwatch')).click() + }) +}) diff --git a/examples/selenium/mocha-test/test/example.js b/examples/selenium-js/mocha/example.js similarity index 100% rename from examples/selenium/mocha-test/test/example.js rename to examples/selenium-js/mocha/example.js diff --git a/examples/selenium/package.json b/examples/selenium-js/package.json similarity index 100% rename from examples/selenium/package.json rename to examples/selenium-js/package.json diff --git a/examples/selenium/python-test/.gitignore b/examples/selenium-py/.gitignore similarity index 100% rename from examples/selenium/python-test/.gitignore rename to examples/selenium-py/.gitignore diff --git a/examples/selenium/python-test/trace-py-test/pytest.ini b/examples/selenium-py/pytest/pytest.ini similarity index 100% rename from examples/selenium/python-test/trace-py-test/pytest.ini rename to examples/selenium-py/pytest/pytest.ini diff --git a/examples/selenium/python-test/trace-py-test/test_login_pytest.py b/examples/selenium-py/pytest/test_login_pytest.py similarity index 100% rename from examples/selenium/python-test/trace-py-test/test_login_pytest.py rename to examples/selenium-py/pytest/test_login_pytest.py diff --git a/examples/selenium-py/requirements-mobile.txt b/examples/selenium-py/requirements-mobile.txt new file mode 100644 index 00000000..d6227f9f --- /dev/null +++ b/examples/selenium-py/requirements-mobile.txt @@ -0,0 +1,6 @@ +# Extra dependency for `pnpm demo:python:mobile` only. The desktop Python +# examples need none of this — plain Selenium reaches a local browser, while an +# Appium session is built through the Appium client. +# +# pip install -r examples/selenium-py/requirements-mobile.txt +Appium-Python-Client>=4.0.0 diff --git a/examples/selenium-py/scripts/clock.py b/examples/selenium-py/scripts/clock.py new file mode 100644 index 00000000..b3b6b14a --- /dev/null +++ b/examples/selenium-py/scripts/clock.py @@ -0,0 +1,187 @@ +"""Mobile example for the Python adapter. + +The WebdriverIO, Selenium and Nightwatch mobile examples drive the SAME flow, +so a difference between two dashboards is a difference in the adapter rather +than in the test. + +Drives the Clock app, which ships with every Android system image, so it needs +no .apk. Clock also gives a native session something deterministic to do: a +timer can be started, paused and cleared, and each step changes the screen in a +way the trace can be checked against. See examples/MOBILE.md for prerequisites +and the DEVTOOLS_MOBILE / APPIUM_APP switches. + +VERIFIED ON: Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), Clock +(com.google.android.deskclock) 9.1. The Clock app updates independently of the +Android version, so pinning a system image does not pin these resource-ids -- +re-read the tree with `adb shell uiautomator dump` if one misses. + + pnpm demo:python:mobile + DEVTOOLS_MODE=live pnpm demo:python:mobile + +``enable()`` starts the dashboard itself, so nothing needs a backend run by +hand. Trace output lands in ``test-results/`` beside this file. + +Needs the Appium Python client, which the desktop examples do not: + + pip install -r examples/selenium-py/requirements-mobile.txt +""" + +import os +import re +import time + +import selenium_devtools as devtools + +try: + from appium import webdriver + from appium.options.android import UiAutomator2Options + from appium.options.ios import XCUITestOptions +except ImportError: # noqa: BLE001 — a missing optional dep, not a failure + raise SystemExit( + "this example needs the Appium client:\n" + " pip install -r examples/selenium-py/requirements-mobile.txt" + ) +from selenium.webdriver.common.by import By + +def require_appium(host: str, port: str) -> None: + """Appium up, or say what is missing and stop. The JS examples share + examples/mobile-preflight.cjs for this; ten lines is cheaper than reaching + across languages for it.""" + import urllib.error + import urllib.request + + try: + with urllib.request.urlopen( + "http://%s:%s/status" % (host, port), timeout=2.5 + ) as res: + if res.status == 200: + return + except (urllib.error.URLError, OSError): + pass + raise SystemExit( + "\nNothing is listening on %s:%s, so Appium is not up.\n\n" + "Mobile examples need a device and Appium; between the Android SDK and\n" + "a system image that is a multi-gigabyte setup, so it is opt-in:\n\n" + " 1. Android SDK + an emulator (Android Studio installs both), then\n" + " check it is visible: adb devices\n" + " 2. npm i -g appium && appium driver install uiautomator2\n" + " appium --address %s --port %s\n\n" + "See examples/MOBILE.md.\n" % (host, port, host, port) + ) + + +APP_ID = "com.google.android.deskclock" +IS_WEB = os.environ.get("DEVTOOLS_MOBILE") == "web" +IS_IOS = os.environ.get("DEVTOOLS_MOBILE_PLATFORM") == "ios" +APPIUM = "http://%s:%s" % ( + os.environ.get("APPIUM_HOST", "127.0.0.1"), + os.environ.get("APPIUM_PORT", "4723"), +) + + +def capabilities() -> dict: + """The same bag the other three mobile examples build; see + examples/wdio/mobile/capabilities.ts for the annotated original.""" + base = { + "platformName": "iOS" if IS_IOS else "Android", + "appium:automationName": "XCUITest" if IS_IOS else "UiAutomator2", + "appium:noReset": True, + "appium:newCommandTimeout": 300, + } + if IS_IOS: + # The simulator name is per-machine: `xcrun simctl list devices`. + base["appium:deviceName"] = os.environ.get("IOS_DEVICE_NAME", "iPhone 15") + version = os.environ.get("IOS_PLATFORM_VERSION") + if version: + base["appium:platformVersion"] = version + if IS_WEB: + # Names a browser, so this session HAS a document and keeps its + # page-side capture — the distinction the native guards turn on. + # Chrome on the device needs a matching chromedriver. Appium can + # fetch one, but that is a SERVER feature, not a capability: + # --allow-insecure=uiautomator2:chromedriver_autodownload + base["browserName"] = "Safari" if IS_IOS else "Chrome" + return base + app = os.environ.get("APPIUM_APP") + if app: + base["appium:app"] = app + return base + if IS_IOS: + base["appium:bundleId"] = "com.apple.mobiletimer" + return base + base["appium:appPackage"] = APP_ID + base["appium:appActivity"] = "com.android.deskclock.DeskClock" + return base + + +require_appium( + os.environ.get("APPIUM_HOST", "127.0.0.1"), + os.environ.get("APPIUM_PORT", "4723"), +) + +# Trace by default, matching the desktop demos. The Python adapter takes a +# boolean rather than a mode name; DEVTOOLS_MODE=live is the shared switch. +devtools.enable(trace=os.environ.get("DEVTOOLS_MODE") != "live") + +# The options class is per-platform: handing iOS capabilities to +# UiAutomator2Options builds an Android session request out of them. +_options = XCUITestOptions() if IS_IOS else UiAutomator2Options() +driver = webdriver.Remote(APPIUM, options=_options.load_capabilities(capabilities())) +try: + if IS_WEB: + driver.get("https://the-internet.herokuapp.com/login") + driver.find_element(By.ID, "username").send_keys("tomsmith") + driver.find_element(By.ID, "password").send_keys("SuperSecretPassword!") + driver.find_element(By.CSS_SELECTOR, 'button[type="submit"]').click() + print(driver.find_element(By.ID, "flash").text.strip()) + else: + # Re-activated rather than relying on the launch capability alone, so + # the script re-runs against a session left on another screen. + driver.execute_script("mobile: activateApp", {"appId": APP_ID}) + + def by_id(name): + return driver.find_element( + "-android uiautomator", + 'new UiSelector().resourceId("%s:id/%s")' % (APP_ID, name), + ) + + by_id("tab_menu_timer").click() + + # Clear anything a previous run left behind. A timer SURVIVES the + # session, and while one exists the Timers tab shows its card instead + # of the preset buttons -- so without this, one interrupted run breaks + # every later one. + for _ in range(5): + left = driver.find_elements( + "-android uiautomator", + 'new UiSelector().resourceId("%s:id/delete_button")' % APP_ID, + ) + if not left: + break + left[0].click() + time.sleep(0.3) + + # This build starts the timer straight from the preset, so the running + # countdown is the evidence the tap landed. + by_id("timer_preset_2").click() + running = by_id("timer_text").text + assert re.match(r"^\d{2}:\d{2}$", running), 'timer_text was "%s"' % running + + by_id("play_pause_button").click() + # The control's accessibility label flips with the timer's state, so + # asserting on it keeps this step off the countdown's own clock. + label = by_id("play_pause_button").get_dom_attribute("content-desc") + assert label.startswith("Start"), 'paused control read "%s"' % label + + # Clearing the timer is what makes the script re-runnable: it ends on + # the same screen it started from. + by_id("delete_button").click() + time.sleep(0.5) + assert not driver.find_elements( + "-android uiautomator", + 'new UiSelector().resourceId("%s:id/timer_text")' % APP_ID, + ), "the cleared timer is still on screen" + print("timer started, paused and cleared") +finally: + driver.quit() + devtools.wait_for_dashboard_close() # hold the UI open to inspect diff --git a/examples/selenium/python-test/login.py b/examples/selenium-py/scripts/login.py similarity index 100% rename from examples/selenium/python-test/login.py rename to examples/selenium-py/scripts/login.py diff --git a/examples/selenium/python-test/web_form.py b/examples/selenium-py/scripts/web_form.py similarity index 100% rename from examples/selenium/python-test/web_form.py rename to examples/selenium-py/scripts/web_form.py diff --git a/examples/selenium/cucumber-test/cucumber.json b/examples/selenium/cucumber-test/cucumber.json deleted file mode 100644 index d479ab56..00000000 --- a/examples/selenium/cucumber-test/cucumber.json +++ /dev/null @@ -1,12 +0,0 @@ -{ - "default": { - "import": [ - "../../examples/selenium/cucumber-test/features/support/setup.js", - "../../examples/selenium/cucumber-test/features/support/world.js", - "../../examples/selenium/cucumber-test/features/support/steps.js" - ], - "paths": ["../../examples/selenium/cucumber-test/features/*.feature"], - "publishQuiet": true, - "format": ["progress"] - } -} diff --git a/examples/wdio/cucumber/wdio.conf.ts b/examples/wdio/cucumber/wdio.conf.ts index e6fc82aa..27eeb0d0 100644 --- a/examples/wdio/cucumber/wdio.conf.ts +++ b/examples/wdio/cucumber/wdio.conf.ts @@ -1,163 +1,76 @@ +// WebdriverIO + Cucumber. The Mocha example beside this one uses the same +// capabilities and the same service block; only the framework and the spec +// layout differ. +// +// Every devtools option below reads from the environment, so ONE config walks +// the whole live→trace→per-test→retention ladder without being edited: +// +// pnpm demo:wdio live +// DEVTOOLS_MODE=trace pnpm demo:wdio one zip per run +// DEVTOOLS_MODE=trace DEVTOOLS_TRACE_GRANULARITY=test … one zip per test +// … DEVTOOLS_TRACE_POLICY=retain-on-failure keep only failures +// +// Retention is easiest to SEE with a failing spec beside a passing one, which +// is what `pnpm demo:wdio:retention` runs — same config, both features. import path from 'node:path' const __dirname = path.resolve(path.dirname(new URL(import.meta.url).pathname)) export const config: WebdriverIO.Config = { - // - // ==================== - // Runner Configuration - // ==================== - // WebdriverIO supports running e2e tests as well as unit and component tests. runner: 'local', - // - // ================== - // Specify Test Files - // ================== - // Define which test specs should run. The pattern is relative to the directory - // of the configuration file being run. - // - // The specs are defined as an array of spec files (optionally using wildcards - // that will be expanded). The test for each spec file will be run in a separate - // worker process. In order to have a group of spec files run in the same worker - // process simply enclose them in an array within the specs array. - // - // If you are calling `wdio` from an NPM script (see https://docs.npmjs.com/cli/run-script), - // then the current working directory is where your `package.json` resides, so `wdio` - // will be called from there. - // - specs: ['./features/**/*.feature'], - // Patterns to exclude. - exclude: [ - // 'path/to/excluded/files' - ], - // - // ============ - // Capabilities - // ============ - // Define your capabilities here. WebdriverIO can run multiple capabilities at the same - // time. Depending on the number of capabilities, WebdriverIO launches several test - // sessions. Within your capabilities you can overwrite the spec and exclude options in - // order to group specific specs to a specific capability. - // - // First, you can define how many instances should be started at the same time. Let's - // say you have 3 different capabilities (Chrome, Firefox, and Safari) and you have - // set maxInstances to 1; wdio will spawn 3 processes. Therefore, if you have 10 spec - // files and you set maxInstances to 10, all spec files will get tested at the same time - // and 30 processes will get spawned. The property handles how many capabilities - // from the same test should run tests. - // - // Live mode drives a single-session dashboard; >1 worker streams two feature - // files into it at once and neither renders cleanly. One instance = readable demo. + // DEVTOOLS_SPECS=all adds the deliberately failing feature, so a retention + // policy can be watched dropping one trace and keeping the other. + specs: + process.env.DEVTOOLS_SPECS === 'all' + ? ['./features/**/*.feature'] + : ['./features/login.feature'], + // Live mode drives a single-session dashboard; >1 worker streams two sessions + // into it at once and neither renders cleanly. One instance = readable demo. maxInstances: 1, - // - // If you have trouble getting all important capabilities together, check out the - // Sauce Labs platform configurator - a great tool to configure your capabilities: - // https://saucelabs.com/platform/platform-configurator - // capabilities: [ { browserName: 'chrome', - // browserVersion: '147.0.7727.56', // specify chromium browser version for testing 'goog:chromeOptions': { args: [ '--headless', + '--disable-gpu', '--remote-allow-origins=*', - '--window-size=1600,900' + '--window-size=1600,900', + // Chrome stops delivering synthesized input to a tab once a breached + // credential is submitted, which silently kills every later click. + // Pointing the check at localhost is what keeps the demo working. + '--host-resolver-rules=MAP passwordsleakcheck-pa.googleapis.com 127.0.0.1' ] } - // }, { - // browserName: 'firefox', - // 'moz:firefoxOptions': { - // args: ['-headless'] - // } } ], - - // - // =================== - // Test Configurations - // =================== - // Define all options that are relevant for the WebdriverIO instance here - // - // Level of logging verbosity: trace | debug | info | warn | error | silent logLevel: 'warn', - // - // Set specific log levels per logger - // loggers: - // - webdriver, webdriverio - // - @wdio/browserstack-service, @wdio/devtools-service, @wdio/sauce-service - // - @wdio/mocha-framework, @wdio/jasmine-framework - // - @wdio/local-runner - // - @wdio/sumologic-reporter - // - @wdio/cli, @wdio/config, @wdio/utils - // Level of logging verbosity: trace | debug | info | warn | error | silent - // logLevels: { - // webdriver: 'info', - // '@wdio/appium-service': 'info' - // }, - // - // If you only want to run your tests until a specific amount of tests have failed use - // bail (default is 0 - don't bail, run all tests). bail: 0, - // - // Set a base URL in order to shorten url command calls. If your `url` parameter starts - // with `/`, the base url gets prepended, not including the path portion of your baseUrl. - // If your `url` parameter starts without a scheme or `/` (like `some/path`), the base url - // gets prepended directly. baseUrl: 'http://localhost', - // - // Default timeout for all waitFor* commands. waitforTimeout: 10000, - // - // Default timeout in milliseconds for request - // if browser driver or grid doesn't send response connectionRetryTimeout: 120000, - // - // Default request retries count connectionRetryCount: 3, - // - // Test runner services - // Services take over a specific job you don't want to take care of. They enhance - // your test setup with almost no effort. Unlike plugins, they don't add new - // commands. Instead, they hook themselves up into the test process. services: [ [ 'devtools', { - mode: 'trace' as const, - filmstrip: true, - // tracePolicy: 'retain-on-failure', - traceGranularity: 'test' as const, - // NEW: per-test screenshot + video, attached inline to Allure. - // Both require traceGranularity:'test' (the uniform rule). - screenshot: 'only-on-failure' as const, // 'off' | 'on' | 'only-on-failure' - video: 'on' as const // 'off' | retention policy - // screencast: { enabled: true, pollIntervalMs: 200 } + mode: (process.env.DEVTOOLS_MODE === 'trace' ? 'trace' : 'live') as + 'live' | 'trace', + traceGranularity: (process.env.DEVTOOLS_TRACE_GRANULARITY ?? + 'session') as 'session' | 'spec' | 'test', + tracePolicy: (process.env.DEVTOOLS_TRACE_POLICY ?? 'on') as + | 'on' + | 'retain-on-failure' + | 'retain-on-first-failure' + | 'on-first-retry' + | 'on-all-retries' + | 'retain-on-failure-and-retries', + // Always emitted, so the artifact set is inspectable for any rung. + emitArtifactsManifest: true } ] ], - // - // Framework you want to run your specs with. - // The following are supported: Mocha, Jasmine, and Cucumber - // see also: https://webdriver.io/docs/frameworks - // - // Make sure you have the wdio adapter package for the specific framework installed - // before running any tests. framework: 'cucumber', - - // - // The number of times to retry the entire spec file when it fails as a whole - // specFileRetries: 1, - // - // Delay in seconds between the spec file retry attempts - // specFileRetriesDelay: 0, - // - // Whether or not retried spec files should be retried immediately or deferred to the end of the queue - // specFileRetriesDeferred: false, - // - // Test reporter for stdout. - // The only one supported by default is 'dot' - // see also: https://webdriver.io/docs/dot-reporter reporters: [ 'spec', [ @@ -174,198 +87,10 @@ export const config: WebdriverIO.Config = { } ] ], - - // If you are using Cucumber you need to specify the location of your step definitions. cucumberOpts: { - // (file/dir) require files before executing features require: [ path.resolve(__dirname, 'features', 'step-definitions', 'steps.ts') ], - // show full backtrace for errors - backtrace: false, - // ("extension:module") require files with the given EXTENSION after requiring MODULE (repeatable) - requireModule: [], - // invoke formatters without executing steps - dryRun: false, - // abort the run on first failure - failFast: false, - // hide step definition snippets for pending steps - snippets: true, - // hide source uris - source: true, - // fail if there are any undefined or pending steps - strict: false, - // (expression) only execute the features or scenarios with tags matching the expression - tagExpression: '', - // timeout for step definitions - timeout: 60000, - // Enable this config to treat undefined definitions as warnings. - ignoreUndefinedDefinitions: false - }, - - // - // ===== - // Hooks - // ===== - // WebdriverIO provides several hooks you can use to interfere with the test process in order to enhance - // it and to build services around it. You can either apply a single function or an array of - // methods to it. If one of them returns with a promise, WebdriverIO will wait until that promise got - // resolved to continue. - /** - * Gets executed once before all workers get launched. - * @param {object} config wdio configuration object - * @param {Array.} capabilities list of capabilities details - */ - // onPrepare: function (config, capabilities) { - // }, - /** - * Gets executed before a worker process is spawned and can be used to initialize specific service - * for that worker as well as modify runtime environments in an async fashion. - * @param {string} cid capability id (e.g 0-0) - * @param {object} caps object containing capabilities for session that will be spawn in the worker - * @param {object} specs specs to be run in the worker process - * @param {object} args object that will be merged with the main configuration once worker is initialized - * @param {object} execArgv list of string arguments passed to the worker process - */ - // onWorkerStart: function (cid, caps, specs, args, execArgv) { - // }, - /** - * Gets executed just after a worker process has exited. - * @param {string} cid capability id (e.g 0-0) - * @param {number} exitCode 0 - success, 1 - fail - * @param {object} specs specs to be run in the worker process - * @param {number} retries number of retries used - */ - // onWorkerEnd: function (cid, exitCode, specs, retries) { - // }, - /** - * Gets executed just before initializing the webdriver session and test framework. It allows you - * to manipulate configurations depending on the capability or spec. - * @param {object} config wdio configuration object - * @param {Array.} capabilities list of capabilities details - * @param {Array.} specs List of spec file paths that are to be run - * @param {string} cid worker id (e.g. 0-0) - */ - // beforeSession: function (config, capabilities) { - // }, - /** - * Gets executed before test execution begins. At this point you can access to all global - * variables like `browser`. It is the perfect place to define custom commands. - * @param {Array.} capabilities list of capabilities details - * @param {Array.} specs List of spec file paths that are to be run - * @param {object} browser instance of created browser/device session - */ - before: async function () { - await browser.pause(5000) + timeout: 60000 } - /** - * Runs before a WebdriverIO command gets executed. - * @param {string} commandName hook command name - * @param {Array} args arguments that command would receive - */ - // beforeCommand: function (commandName, args) { - // }, - /** - * Cucumber Hooks - * - * Runs before a Cucumber Feature. - * @param {string} uri path to feature file - * @param {GherkinDocument.IFeature} feature Cucumber feature object - */ - // beforeFeature: function (uri, feature) { - // }, - /** - * - * Runs before a Cucumber Scenario. - * @param {ITestCaseHookParameter} world world object containing information on pickle and test step - * @param {object} context Cucumber World object - */ - // beforeScenario: function (world, context) { - // }, - /** - * - * Runs before a Cucumber Step. - * @param {Pickle.IPickleStep} step step data - * @param {IPickle} scenario scenario pickle - * @param {object} context Cucumber World object - */ - // beforeStep: function (step, scenario, context) { - // }, - /** - * - * Runs after a Cucumber Step. - * @param {Pickle.IPickleStep} step step data - * @param {IPickle} scenario scenario pickle - * @param {object} result results object containing scenario results - * @param {boolean} result.passed true if scenario has passed - * @param {string} result.error error stack if scenario failed - * @param {number} result.duration duration of scenario in milliseconds - * @param {object} context Cucumber World object - */ - // afterStep: function (step, scenario, result, context) { - // }, - /** - * - * Runs after a Cucumber Scenario. - * @param {ITestCaseHookParameter} world world object containing information on pickle and test step - * @param {object} result results object containing scenario results - * @param {boolean} result.passed true if scenario has passed - * @param {string} result.error error stack if scenario failed - * @param {number} result.duration duration of scenario in milliseconds - * @param {object} context Cucumber World object - */ - // afterScenario: function (world, result, context) { - // }, - /** - * - * Runs after a Cucumber Feature. - * @param {string} uri path to feature file - * @param {GherkinDocument.IFeature} feature Cucumber feature object - */ - // afterFeature: function (uri, feature) { - // }, - - /** - * Runs after a WebdriverIO command gets executed - * @param {string} commandName hook command name - * @param {Array} args arguments that command would receive - * @param {number} result 0 - command success, 1 - command error - * @param {object} error error object if any - */ - // afterCommand: function (commandName, args, result, error) { - // }, - /** - * Gets executed after all tests are done. You still have access to all global variables from - * the test. - * @param {number} result 0 - test pass, 1 - test fail - * @param {Array.} capabilities list of capabilities details - * @param {Array.} specs List of spec file paths that ran - */ - // after: function (result, capabilities, specs) { - // }, - /** - * Gets executed right after terminating the webdriver session. - * @param {object} config wdio configuration object - * @param {Array.} capabilities list of capabilities details - * @param {Array.} specs List of spec file paths that ran - */ - // afterSession: function (config, capabilities, specs) { - // }, - /** - * Gets executed after all workers got shut down and the process is about to exit. An error - * thrown in the onComplete hook will result in the test run failing. - * @param {object} exitCode 0 - success, 1 - fail - * @param {object} config wdio configuration object - * @param {Array.} capabilities list of capabilities details - * @param {} results object containing test results - */ - // onComplete: function(exitCode, config, capabilities, results) { - // }, - /** - * Gets executed when a refresh happens. - * @param {string} oldSessionId session ID of the old session - * @param {string} newSessionId session ID of the new session - */ - // onReload: function(oldSessionId, newSessionId) { - // } } diff --git a/examples/wdio/cucumber/wdio.mobile.conf.ts b/examples/wdio/cucumber/wdio.mobile.conf.ts deleted file mode 100644 index 3fea1d5a..00000000 --- a/examples/wdio/cucumber/wdio.mobile.conf.ts +++ /dev/null @@ -1,78 +0,0 @@ -// Mobile-web (Android Chrome via Appium) variant of wdio.conf.ts. -// -// Prerequisites: -// 1. Appium 2.x running locally: `appium --address 127.0.0.1 --port 4723` -// 2. UiAutomator2 driver installed: `appium driver install uiautomator2` -// 3. An Android emulator running with Chrome installed -// (or a real device with USB debugging on and `adb devices` listing it). -// -// Run (from inside examples/wdio): -// pnpm mobile -// -// The DevTools service detects `platformName: Android|iOS` via shared -// capabilities and adjusts the action-snapshot probe (mobile XML element -// extraction) and the trace's context naming accordingly. - -import path from 'node:path' - -const __dirname = path.resolve(path.dirname(new URL(import.meta.url).pathname)) - -export const config: WebdriverIO.Config = { - runner: 'local', - - specs: ['./features/**/*.feature'], - exclude: [], - - hostname: '127.0.0.1', - port: 4723, - path: '/', - - maxInstances: 1, - // `wdio:enforceWebDriverClassic` isn't in @wdio/types yet but is honored - // at runtime — needed because Appium's BiDi shim for UiAutomator2 doesn't - // implement every BiDi command (e.g. script.addPreloadScript). - capabilities: [ - { - platformName: 'Android', - 'appium:automationName': 'UiAutomator2', - 'appium:deviceName': 'emulator-5554', - browserName: 'Chrome', - 'appium:chromedriverAutodownload': true, - 'wdio:enforceWebDriverClassic': true - } - // eslint-disable-next-line @typescript-eslint/no-explicit-any - ] as any, - - logLevel: 'warn', - bail: 0, - baseUrl: 'http://localhost', - waitforTimeout: 15000, - connectionRetryTimeout: 120000, - connectionRetryCount: 3, - services: [ - [ - 'devtools', - { - mode: 'trace' as const, - screencast: { enabled: true, pollIntervalMs: 250 } - } - ] - ], - framework: 'cucumber', - reporters: ['spec'], - cucumberOpts: { - require: [ - path.resolve(__dirname, 'features', 'step-definitions', 'steps.ts') - ], - backtrace: false, - requireModule: [], - dryRun: false, - failFast: false, - snippets: true, - source: true, - strict: false, - tagExpression: '', - timeout: 90000, - ignoreUndefinedDefinitions: false - } -} diff --git a/examples/wdio/cucumber/wdio.retention.conf.ts b/examples/wdio/cucumber/wdio.retention.conf.ts deleted file mode 100644 index a71b97bb..00000000 --- a/examples/wdio/cucumber/wdio.retention.conf.ts +++ /dev/null @@ -1,68 +0,0 @@ -import path from 'node:path' - -// Disposable harness for verifying tracePolicy end-to-end. Runs one passing -// spec (login.feature) and one failing spec (login-fail.feature) at spec -// granularity so retain-on-failure can be seen dropping the passing spec's -// trace while keeping the failing one. Change `tracePolicy` below to try each. - -const __dirname = path.resolve(path.dirname(new URL(import.meta.url).pathname)) - -export const config: WebdriverIO.Config = { - runner: 'local', - tsConfigPath: './tsconfig.json', - - specs: ['./features/login.feature', './features/login-fail.feature'], - - maxInstances: 1, - - capabilities: [ - { - browserName: 'chrome', - browserVersion: '149.0.7827.201', // specify chromium browser version for testing - 'goog:chromeOptions': { - args: [ - '--headless', - '--disable-gpu', - '--remote-allow-origins=*', - '--window-size=1600,1200' - ] - } - } - ], - - logLevel: 'warn', - baseUrl: 'http://localhost', - waitforTimeout: 10000, - connectionRetryTimeout: 120000, - connectionRetryCount: 3, - - services: [ - [ - 'devtools', - { - mode: 'trace' as const, - traceGranularity: 'spec' as const - // tracePolicy: 'retain-on-failure' as const - } - ] - ], - - framework: 'cucumber', - reporters: ['spec'], - - cucumberOpts: { - require: [ - path.resolve(__dirname, 'features', 'step-definitions', 'steps.ts') - ], - backtrace: false, - requireModule: [], - dryRun: false, - failFast: false, - snippets: true, - source: true, - strict: false, - tagExpression: '', - timeout: 60000, - ignoreUndefinedDefinitions: false - } -} diff --git a/examples/wdio/cucumber/wdio.trace.conf.ts b/examples/wdio/cucumber/wdio.trace.conf.ts deleted file mode 100644 index 3aa9e496..00000000 --- a/examples/wdio/cucumber/wdio.trace.conf.ts +++ /dev/null @@ -1,66 +0,0 @@ -import path from 'node:path' - -const __dirname = path.resolve(path.dirname(new URL(import.meta.url).pathname)) - -export const config: WebdriverIO.Config = { - runner: 'local', - tsConfigPath: './tsconfig.json', - - specs: ['./features/login.feature'], - - maxInstances: 1, - - capabilities: [ - { - browserName: 'chrome', - 'goog:chromeOptions': { - args: [ - '--headless', - '--disable-gpu', - '--remote-allow-origins=*', - '--window-size=1600,1200' - ] - } - } - ], - - logLevel: 'warn', - - baseUrl: 'http://localhost', - - waitforTimeout: 10000, - connectionRetryTimeout: 120000, - connectionRetryCount: 3, - - services: [ - [ - 'devtools', - { - // Trace by default (regen); DEVTOOLS_MODE=live flips to live for live-parity recording. - mode: (process.env.DEVTOOLS_MODE === 'live' ? 'live' : 'trace') as - 'live' | 'trace' - // traceFormat: 'ndjson-directory' - } - ] - ], - - framework: 'cucumber', - - reporters: ['spec'], - - cucumberOpts: { - require: [ - path.resolve(__dirname, 'features', 'step-definitions', 'steps.ts') - ], - backtrace: false, - requireModule: [], - dryRun: false, - failFast: false, - snippets: true, - source: true, - strict: false, - tagExpression: '', - timeout: 60000, - ignoreUndefinedDefinitions: false - } -} diff --git a/examples/wdio/mobile/capabilities.ts b/examples/wdio/mobile/capabilities.ts new file mode 100644 index 00000000..1c502aa8 --- /dev/null +++ b/examples/wdio/mobile/capabilities.ts @@ -0,0 +1,81 @@ +// The one capability set the mobile examples share, so a difference in the +// dashboard between two adapters is a difference in the adapter. The Selenium, +// Nightwatch and Python examples spell the same bag in their own syntax — +// keep the four in step, and see ../../MOBILE.md for what each switch means. + +/** The Clock app, which ships with every Android system image and every iOS + * device — so a native example needs no .apk or .app uploaded or kept in the + * repo. Chosen over Settings because it has a deterministic screen to drive: + * a timer can be started, paused and cleared, and each step is observable. + * + * The activity is named here so the session opens on Clock directly, but the + * specs still call `mobile: activateApp` — that is what makes them re-runnable + * against a session left on another screen, and it needs no adb_shell. */ +const CLOCK_APP = { + android: { + 'appium:appPackage': 'com.google.android.deskclock', + 'appium:appActivity': 'com.android.deskclock.DeskClock' + }, + ios: { 'appium:bundleId': 'com.apple.mobiletimer' } +} + +const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' + +export function mobileCapabilities(): Record { + const web = process.env.DEVTOOLS_MOBILE === 'web' + const app = process.env.APPIUM_APP + + return { + platformName: IOS ? 'iOS' : 'Android', + 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', + // iOS needs the simulator named explicitly and it is per-machine, so it + // comes from the environment: `xcrun simctl list devices` shows yours. + ...(IOS + ? { + 'appium:deviceName': process.env.IOS_DEVICE_NAME ?? 'iPhone 15', + ...(process.env.IOS_PLATFORM_VERSION + ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } + : {}) + } + : {}), + // Leave whatever is installed alone; a full reset costs minutes per run. + 'appium:noReset': true, + 'appium:newCommandTimeout': 300, + ...(web + ? { + // A mobile BROWSER session. It names a browser, which is exactly what + // tells the adapters it HAS a document and must keep its page-side + // capture — the native/web distinction this example exists to show. + // Chrome on the device needs a matching chromedriver. Appium can + // fetch one, but that is a SERVER feature, not a capability: + // `--allow-insecure=uiautomator2:chromedriver_autodownload`. There is + // no `appium:chromedriverAutodownload` capability — the driver only + // reads `chromedriverExecutable`/`chromedriverExecutableDir`, so + // passing the other name did nothing at all. + browserName: IOS ? 'Safari' : 'Chrome', + // Where autodownload may unpack. Appium's default is inside its own + // driver tree, which is root-owned after a `sudo npm i -g appium` + // and then the download succeeds and the UNZIP fails with EACCES — + // reported as the same "No Chromedriver found" as a missing version. + ...(process.env.CHROMEDRIVER_DIR + ? { + 'appium:chromedriverExecutableDir': process.env.CHROMEDRIVER_DIR + } + : {}), + // The exact binary, which skips autodownload entirely — the reliable + // route when the emulator's Chrome is older than anything the server + // will fetch. Must be a HOST build (it drives the device over adb). + ...(process.env.CHROMEDRIVER_EXECUTABLE + ? { + 'appium:chromedriverExecutable': + process.env.CHROMEDRIVER_EXECUTABLE + } + : {}) + } + : app + ? { 'appium:app': app } + : IOS + ? CLOCK_APP.ios + : CLOCK_APP.android) + } +} diff --git a/examples/wdio/mobile/specs/clock.e2e.ts b/examples/wdio/mobile/specs/clock.e2e.ts new file mode 100644 index 00000000..b77b7341 --- /dev/null +++ b/examples/wdio/mobile/specs/clock.e2e.ts @@ -0,0 +1,98 @@ +// WebdriverIO against Appium. The Selenium, Nightwatch and Python mobile +// examples drive the SAME flow, so a difference between two dashboards is a +// difference in the adapter rather than in the test. +// +// Drives the Clock app, which ships with every Android system image — no .apk, +// no upload, no credentials. Clock also gives a native session something +// deterministic to do: a timer can be started, paused and cleared, and each +// step changes the screen in a way the trace can be checked against. +// +// VERIFIED ON: Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), +// Clock (com.google.android.deskclock) 9.1. +// +// These are resource-ids read off that build, and the Clock app updates +// INDEPENDENTLY of the Android version — so pinning a system image does not pin +// them. If a locator misses, re-read the tree rather than assuming capture +// broke: `adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml` +// (a running countdown never reaches idle, so pause or clear the timer first). + +import { expect } from '@wdio/globals' + +const APP_ID = 'com.google.android.deskclock' + +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' + +// An emulator often cannot resolve public DNS (corporate network, VPN), and +// `10.0.2.2` is its alias for the HOST's localhost — so a page served on this +// machine is reachable when the internet is not. See examples/MOBILE.md. +const WEB_URL = + process.env.DEVTOOLS_MOBILE_URL ?? 'https://the-internet.herokuapp.com/login' + +const byId = (id: string) => + $(`android=new UiSelector().resourceId("${APP_ID}:id/${id}")`) + +/** Delete every timer already on the Timers tab, so the preset buttons are the + * ones on screen. Idempotent: no timers means nothing to click. */ +async function clearExistingTimers(): Promise { + for (let i = 0; i < 5; i++) { + const remaining = await $$( + `android=new UiSelector().resourceId("${APP_ID}:id/delete_button")` + ) + if (!remaining.length) { + return + } + await remaining[0]!.click() + await browser.pause(300) + } +} + +describe('Clock (native)', () => { + it('starts a preset timer, pauses it, and clears it', async () => { + if (isWeb) { + // A mobile BROWSER session: it has a document, so every page-side call a + // native session skips must still happen. That contrast is the point. + await browser.url(WEB_URL) + await expect(browser).toHaveUrl(expect.stringContaining('http')) + return + } + + // `mobile: activateApp` rather than relying on the launch capability alone: + // it re-runs against a session left on another screen, and needs no + // adb_shell, which Appium does not enable by default. + await browser.execute('mobile: activateApp', { appId: APP_ID }) + + await byId('tab_menu_timer').click() + + // Clear anything a previous run left behind. A timer SURVIVES the session, + // and while one exists the Timers tab shows its card instead of the preset + // buttons — so without this, one interrupted run breaks every later one. + await clearExistingTimers() + + // This build starts the timer straight from the preset, so the running + // countdown is the evidence the tap landed. + await byId('timer_preset_2').click() + await expect(byId('timer_text')).toHaveText(/^\d{2}:\d{2}$/) + + await byId('play_pause_button').click() + // The control's accessibility label flips with the timer's state, so + // asserting on it keeps this step off the countdown's own clock. + await expect($('~Start 5 minutes timer')).toBeDisplayed() + + // Clearing the timer is what makes the spec re-runnable: it ends on the + // same screen it started from. + await byId('delete_button').click() + await expect(byId('timer_text')).not.toBeDisplayed() + }) + + it('captures a second action on the same session', async () => { + // A second test, so `traceGranularity: 'test'` has two slices to key. + if (isWeb) { + await browser.url(WEB_URL) + return + } + + await browser.execute('mobile: activateApp', { appId: APP_ID }) + await expect(byId('tab_menu_stopwatch')).toBeDisplayed() + await byId('tab_menu_stopwatch').click() + }) +}) diff --git a/examples/wdio/mobile/wdio.conf.ts b/examples/wdio/mobile/wdio.conf.ts new file mode 100644 index 00000000..a78858a3 --- /dev/null +++ b/examples/wdio/mobile/wdio.conf.ts @@ -0,0 +1,89 @@ +// Mobile example for the WDIO service. Drives the device's own Settings app by +// default, so it needs no .apk — see ../../MOBILE.md for prerequisites and the +// DEVTOOLS_MOBILE / APPIUM_APP switches. +// +// pnpm demo:wdio:mobile +// DEVTOOLS_MODE=live pnpm demo:wdio:mobile + +import path from 'node:path' + +import { createRequire } from 'node:module' + +import { mobileCapabilities } from './capabilities.js' + +const { requireMobileToolchain } = createRequire(import.meta.url)( + '../../mobile-preflight.cjs' +) + +const MODE = process.env.DEVTOOLS_MODE === 'live' ? 'live' : 'trace' + +const __dirname = path.resolve(path.dirname(new URL(import.meta.url).pathname)) + +export const config: WebdriverIO.Config = { + runner: 'local', + specs: [path.resolve(__dirname, 'specs', '*.e2e.ts')], + + // Appium, not a local browser driver. + hostname: process.env.APPIUM_HOST ?? '127.0.0.1', + port: Number(process.env.APPIUM_PORT ?? 4723), + path: '/', + + maxInstances: 1, + // `wdio:enforceWebDriverClassic` is honored at runtime but not yet in + // @wdio/types: Appium's BiDi shim for UiAutomator2 does not implement every + // BiDi command (`script.addPreloadScript` among them), and the service falls + // back cleanly only if it is not told BiDi is available. + capabilities: [ + { + ...mobileCapabilities(), + 'wdio:enforceWebDriverClassic': true + } + // eslint-disable-next-line @typescript-eslint/no-explicit-any + ] as any, + + logLevel: 'warn', + bail: 0, + waitforTimeout: 20000, + connectionRetryTimeout: 180000, + connectionRetryCount: 3, + + services: [ + [ + 'devtools', + { + // Trace by default, matching the desktop demos. An inherited + // DEVTOOLS_MODE=live silently changes what a run produces — a UI window + // and no zip instead of a zip and no window — so the mode is logged. + mode: MODE, + traceGranularity: 'test', + // Filmstrip is OPT-IN on mobile, unlike the desktop demos, purely on + // cost: the poller takes a screenshot every interval and a real device + // screenshot is slow — a 3-command run against an emulator took 45 s. + // + // NOT because it crashes the UiAutomator2 instrumentation. That crash + // does happen on an API 36 emulator, but it happens with the poller off + // and at session creation too, so it is the image or the driver rather + // than anything here — see examples/MOBILE.md. + ...(process.env.DEVTOOLS_FILMSTRIP === '1' + ? { filmstrip: true, screencast: { pollIntervalMs: 2000 } } + : {}) + } + ] + ], + + // Before any worker opens a session, so a missing emulator or Appium reads as + // a prerequisite rather than as "make sure browser driver is running". + onPrepare: async () => { + console.log( + `devtools mobile: ${MODE} mode` + + (MODE === 'live' + ? ' — opens the dashboard, writes no trace zip' + : ' — writes a trace zip to test-results/, opens no window') + ) + await requireMobileToolchain() + }, + + framework: 'mocha', + reporters: ['spec'], + mochaOpts: { ui: 'bdd', timeout: 180000 } +} diff --git a/examples/wdio/mocha/wdio.conf.ts b/examples/wdio/mocha/wdio.conf.ts index e6724f21..3f7af781 100644 --- a/examples/wdio/mocha/wdio.conf.ts +++ b/examples/wdio/mocha/wdio.conf.ts @@ -1,5 +1,17 @@ -// Mocha counterpart to wdio.conf.ts (which runs the Cucumber example). Same -// capabilities and devtools service; only the framework + spec layout differ. +// WebdriverIO + Mocha. The Cucumber example beside this one uses the same +// capabilities and the same service block; only the framework and the spec +// layout differ. +// +// Every devtools option below reads from the environment, so ONE config walks +// the whole live→trace→per-test→retention ladder without being edited: +// +// pnpm demo:wdio:mocha live +// DEVTOOLS_MODE=trace pnpm demo:wdio:mocha one zip per run +// DEVTOOLS_MODE=trace DEVTOOLS_TRACE_GRANULARITY=test … one zip per test +// … DEVTOOLS_TRACE_POLICY=retain-on-failure keep only failures +// +// Retries are the one rung that needs its own config, because they change the +// runner and not just the service: `pnpm demo:wdio:retry`. export const config: WebdriverIO.Config = { runner: 'local', specs: ['./specs/**/*.e2e.ts'], @@ -30,15 +42,19 @@ export const config: WebdriverIO.Config = { [ 'devtools', { - // ── Config ladder — change ONLY this block per rung ────────────── - // 1 live: mode: 'live' - // 2 trace: mode: 'trace' - // 3 per-test: mode: 'trace', traceGranularity: 'test' - // 4 fail: mode: 'trace', traceGranularity: 'test', tracePolicy: 'retain-on-failure' - // 5 retry: use `pnpm demo:wdio:retry` (adds retries:1 + on-first-retry) - mode: 'live' as const, - traceGranularity: 'test' as const, - tracePolicy: 'retain-on-failure' as const + mode: (process.env.DEVTOOLS_MODE === 'trace' ? 'trace' : 'live') as + 'live' | 'trace', + traceGranularity: (process.env.DEVTOOLS_TRACE_GRANULARITY ?? + 'session') as 'session' | 'spec' | 'test', + tracePolicy: (process.env.DEVTOOLS_TRACE_POLICY ?? 'on') as + | 'on' + | 'retain-on-failure' + | 'retain-on-first-failure' + | 'on-first-retry' + | 'on-all-retries' + | 'retain-on-failure-and-retries', + // Always emitted, so the artifact set is inspectable for any rung. + emitArtifactsManifest: true } ] ], diff --git a/examples/wdio/mocha/wdio.trace.conf.ts b/examples/wdio/mocha/wdio.trace.conf.ts deleted file mode 100644 index b50b9961..00000000 --- a/examples/wdio/mocha/wdio.trace.conf.ts +++ /dev/null @@ -1,60 +0,0 @@ -// Trace-mode variant of wdio.conf.ts, for manually verifying trace output. Same -// capabilities and specs as the live config; session granularity writes one zip -// per run. Every knob reads from the environment so one config covers the -// live→trace→per-test→retain→retry ladder. Kept separate so the demo config's -// live default stays untouched. -export const config: WebdriverIO.Config = { - runner: 'local', - specs: ['./specs/**/*.e2e.ts'], - exclude: [], - maxInstances: 1, - capabilities: [ - { - browserName: 'chrome', - 'goog:chromeOptions': { - args: [ - '--headless', - '--disable-gpu', - '--remote-allow-origins=*', - '--window-size=1600,900' - ] - } - } - ], - logLevel: 'warn', - bail: 0, - baseUrl: 'http://localhost', - waitforTimeout: 10000, - connectionRetryTimeout: 120000, - connectionRetryCount: 3, - services: [ - [ - 'devtools', - { - // Trace by default (regen); DEVTOOLS_MODE=live flips to live for live-parity recording. - mode: (process.env.DEVTOOLS_MODE === 'live' ? 'live' : 'trace') as - 'live' | 'trace', - // Granularity/policy default to session/on and are env-overridable so - // one config can walk the whole grid, e.g. - // DEVTOOLS_TRACE_GRANULARITY=test DEVTOOLS_TRACE_POLICY=retain-on-failure - traceGranularity: (process.env.DEVTOOLS_TRACE_GRANULARITY ?? - 'session') as 'session' | 'spec' | 'test', - tracePolicy: (process.env.DEVTOOLS_TRACE_POLICY ?? 'on') as - | 'on' - | 'retain-on-failure' - | 'retain-on-first-failure' - | 'on-first-retry' - | 'on-all-retries' - | 'retain-on-failure-and-retries', - // Always emit the manifest so the artifact set is inspectable per run. - emitArtifactsManifest: true - } - ] - ], - framework: 'mocha', - reporters: ['spec'], - mochaOpts: { - ui: 'bdd', - timeout: 60000 - } -} diff --git a/package.json b/package.json index 0fd9355a..2bab2a3f 100644 --- a/package.json +++ b/package.json @@ -8,14 +8,19 @@ "show-trace": "node packages/backend/dist/show-trace.js", "backend": "node packages/backend/dist/server.js", "demo:wdio": "wdio run ./examples/wdio/cucumber/wdio.conf.ts", + "demo:wdio:mobile": "wdio run ./examples/wdio/mobile/wdio.conf.ts", + "demo:wdio:retention": "DEVTOOLS_MODE=trace DEVTOOLS_SPECS=all DEVTOOLS_TRACE_GRANULARITY=spec DEVTOOLS_TRACE_POLICY=retain-on-failure wdio run ./examples/wdio/cucumber/wdio.conf.ts", "demo:wdio:mocha": "wdio run ./examples/wdio/mocha/wdio.conf.ts", "demo:wdio:retry": "wdio run ./examples/wdio/mocha/wdio.retry.conf.ts", "demo:nightwatch": "pnpm --filter @wdio/nightwatch-devtools example", "demo:nightwatch:retry": "pnpm --filter @wdio/nightwatch-devtools example:retry", + "demo:nightwatch:mobile": "pnpm --filter @wdio/nightwatch-devtools example:mobile", "demo:selenium": "pnpm --filter @wdio/selenium-devtools example", - "demo:python": "python3 examples/selenium/python-test/web_form.py", - "demo:python:login": "python3 examples/selenium/python-test/login.py", - "demo:python:pytest": "python3 -m pytest examples/selenium/python-test/trace-py-test/", + "demo:selenium:mobile": "pnpm --filter @wdio/selenium-devtools example:mobile", + "demo:python": "python3 examples/selenium-py/scripts/web_form.py", + "demo:python:login": "python3 examples/selenium-py/scripts/login.py", + "demo:python:pytest": "python3 -m pytest examples/selenium-py/pytest/", + "demo:python:mobile": "python3 examples/selenium-py/scripts/clock.py", "dev": "pnpm --parallel dev", "preview": "pnpm --parallel preview", "test": "vitest run", diff --git a/packages/nightwatch-devtools/package.json b/packages/nightwatch-devtools/package.json index 36a41811..ee6176c7 100644 --- a/packages/nightwatch-devtools/package.json +++ b/packages/nightwatch-devtools/package.json @@ -35,6 +35,7 @@ "lint": "eslint .", "example": "nightwatch -c ../../examples/nightwatch/nightwatch.conf.cjs", "example:retry": "nightwatch -c ../../examples/nightwatch/nightwatch.conf.cjs --retries 1", + "example:mobile": "nightwatch -c ../../examples/nightwatch/nightwatch.mobile.conf.cjs", "example:cucumber": "nightwatch -c ../../examples/nightwatch/cucumber/nightwatch.cucumber.conf.cjs", "prepublishOnly": "pnpm build" }, diff --git a/packages/selenium-devtools-py/README.md b/packages/selenium-devtools-py/README.md index 283f0b88..d5aea049 100644 --- a/packages/selenium-devtools-py/README.md +++ b/packages/selenium-devtools-py/README.md @@ -126,7 +126,7 @@ devtools_trace_policy = "retain-on-failure" A plain script passes `devtools.enable(trace_granularity="test", trace_policy="retain-on-failure")`. A committed example of all of this is in -[`examples/selenium/python-test/trace-py-test/`](../../examples/selenium/python-test/trace-py-test/), +[`examples/selenium-py/pytest/`](../../examples/selenium-py/pytest/), whose `pytest.ini` documents every setting the adapter has. Naming a policy or a granularity **explicitly** selects trace mode — the CLI flag, the ini option @@ -174,7 +174,7 @@ devtools.wait_for_dashboard_close() # keep the UI open to inspect (no-op when devtools.disable() ``` -Runnable example: [`web_form.py`](../../examples/selenium/python-test/web_form.py), +Runnable example: [`web_form.py`](../../examples/selenium-py/scripts/web_form.py), the three-line version above. From the repo root, after `pip install -e` above and a `pnpm build` so the backend exists: @@ -441,7 +441,7 @@ scripts/gen_contract.py regenerate _contract.py from shared (dev-time; also a tests/ stdlib-unittest unit tests (no selenium/pytest needed) e2e_check.py real-Chrome smoke (plain script) e2e/test_smoke.py real-Chrome smoke (pytest + plugin) -(example lives at repo root: examples/selenium/python-test/web_form.py) +(example lives at repo root: examples/selenium-py/scripts/web_form.py) ``` ## Backend & publishing diff --git a/packages/selenium-devtools/README.md b/packages/selenium-devtools/README.md index 61dc3791..30829026 100644 --- a/packages/selenium-devtools/README.md +++ b/packages/selenium-devtools/README.md @@ -33,7 +33,7 @@ That's it. No other code changes required for Mocha / Jest / Cucumber. ## Setup per runner -Each block below is a **complete, copy-paste-ready example** including the `DevTools.configure(...)` call. Pick the runner you use, drop the snippet into your project, and run it. These mirror the working examples in [`example/`](./example). +Each block below is a **complete, copy-paste-ready example** including the `DevTools.configure(...)` call. Pick the runner you use, drop the snippet into your project, and run it. These mirror the working examples in [`examples/selenium-js/`](../../examples/selenium-js). ### Mocha @@ -132,7 +132,7 @@ NODE_OPTIONS=--experimental-vm-modules jest --config jest.config.json ### Cucumber -Cucumber's split layout means three small files — one to configure the plugin, one for World/hooks, and one for step definitions. They mirror [`example/cucumber-test/`](./example/cucumber-test). +Cucumber's split layout means three small files — one to configure the plugin, one for World/hooks, and one for step definitions. They mirror [`examples/selenium-js/cucumber/`](../../examples/selenium-js/cucumber). `features/support/setup.js` — load the plugin and configure once: @@ -440,9 +440,9 @@ Working smoke tests are included for each supported runner: | Directory | Runner | Command | |-----------|--------|---------| -| [`example/mocha-test/`](./example/mocha-test) | Mocha | `pnpm example:mocha` | -| [`example/jest-test/`](./example/jest-test) | Jest | `pnpm example:jest` | -| [`example/cucumber-test/`](./example/cucumber-test) | Cucumber | `pnpm example:cucumber` | +| [`examples/selenium-js/mocha/`](../../examples/selenium-js/mocha) | Mocha | `pnpm example:mocha` | +| [`examples/selenium-js/jest/`](../../examples/selenium-js/jest) | Jest | `pnpm example:jest` | +| [`examples/selenium-js/cucumber/`](../../examples/selenium-js/cucumber) | Cucumber | `pnpm example:cucumber` | Build the package first: diff --git a/packages/selenium-devtools/package.json b/packages/selenium-devtools/package.json index 43cc1375..5273d80c 100644 --- a/packages/selenium-devtools/package.json +++ b/packages/selenium-devtools/package.json @@ -1,7 +1,7 @@ { "name": "@wdio/selenium-devtools", "version": "1.5.1", - "description": "Selenium WebDriver adapter for WebdriverIO DevTools — runner-agnostic, reuses existing backend, UI, and capture infrastructure", + "description": "Selenium WebDriver adapter for WebdriverIO DevTools \u2014 runner-agnostic, reuses existing backend, UI, and capture infrastructure", "type": "module", "main": "dist/index.js", "types": "dist/index.d.ts", @@ -32,10 +32,11 @@ "lint": "eslint .", "prepublishOnly": "pnpm build", "example": "pnpm example:cucumber", - "example:mocha": "mocha --require @wdio/selenium-devtools --timeout 60000 ../../examples/selenium/mocha-test/test/example.js", - "example:mocha:allure": "mocha --require @wdio/selenium-devtools --reporter allure-mocha --reporter-options resultsDir=../../examples/selenium/mocha-test/allure-results --timeout 60000 ../../examples/selenium/mocha-test/test/example.js", - "example:jest": "NODE_OPTIONS=--experimental-vm-modules jest --config ../../examples/selenium/jest-test/jest.config.json", - "example:cucumber": "cucumber-js --config ../../examples/selenium/cucumber-test/cucumber.json" + "example:mocha": "mocha --require @wdio/selenium-devtools --timeout 60000 ../../examples/selenium-js/mocha/example.js", + "example:mobile": "mocha --require @wdio/selenium-devtools --timeout 180000 ../../examples/selenium-js/mobile/clock.test.js", + "example:mocha:allure": "mocha --require @wdio/selenium-devtools --reporter allure-mocha --reporter-options resultsDir=../../examples/selenium-js/mocha/allure-results --timeout 60000 ../../examples/selenium-js/mocha/example.js", + "example:jest": "NODE_OPTIONS=--experimental-vm-modules jest --config ../../examples/selenium-js/jest/jest.config.json", + "example:cucumber": "cucumber-js --config ../../examples/selenium-js/cucumber/cucumber.json" }, "keywords": [ "selenium", From efc919986b0f573513d4b089af9f483fb934ca07 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Tue, 22 Sep 2026 18:50:09 +0530 Subject: [PATCH 02/15] docs: describe mobile capture as it behaves today --- README.md | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9cec10dc..4d6c3e98 100644 --- a/README.md +++ b/README.md @@ -200,15 +200,35 @@ services: [[DevToolsHookService, { #### 📱 Mobile testing -Adapters detect mobile sessions via `platformName: 'android' | 'ios'` (case-insensitive) and adjust the per-action snapshot to extract elements from the mobile XML tree instead of the DOM. The trace's `context-options` records `title: 'android' — ` / `'ios' — ` so the viewer labels frames correctly. +Appium sessions are captured in both live and trace mode. What gets captured depends on one question — **does this session have a web document right now?** — because every DOM drain, page script and viewport read is a round trip that can only fail without one. -A reference WDIO config is at [examples/wdio/cucumber/wdio.mobile.conf.ts](examples/wdio/cucumber/wdio.mobile.conf.ts). Prereqs to run it end-to-end with a local emulator: +**Three kinds of session, and they behave differently:** + +| Session | Has a document | What is captured | +|---|---|---| +| **Mobile browser** (Chrome on Android, Safari on iOS) | always | the same as a desktop run — DOM, console, network, replay | +| **Native app** | never | commands, screenshots and element data from the platform's XML tree; no DOM | +| **Hybrid app** | only while in a webview context | native halves as an app, webview halves as a page | + +The discriminator is the **browser the capabilities name**, not the device: mobile web states one, a native app states none. A device is not required either — a Mac2 or tvOS session has no document and is treated the same way. + +A hybrid app is the case capabilities alone cannot answer, because the answer changes mid-run. Appium reports `NATIVE_APP` or a webview context, and anything that is not `NATIVE_APP` counts as a webview — the `WEBVIEW_` prefix is a convention, and a driver naming its webview otherwise would have its DOM capture skipped. Following the context costs no round trip: the switch command carries its destination in its own arguments. + +**Per-adapter:** all four adapters detect a native session and skip page-side capture on one. Only the **WebdriverIO** service additionally follows a hybrid app into its webview; Nightwatch, Selenium and the Python adapter answer from the startup capabilities, so a hybrid session's webview half is captured as if it were still native. + +**In the dashboard**, a phone capture is framed as the device — a full-height device column with the action list and dock beside it — in both live mode and the trace player. The trace's `context-options` records the platform, model and OS version, read back so the player labels frames without guessing. + +**What a mobile trace does not contain.** Per-action snapshots are issued from inside the command hook, and Appium serialises a probe behind the command it is observing — so where a session has a document to probe, the per-action snapshot is skipped. In practice that means **a hybrid app's webview actions carry no per-action element data, accessibility tree or settle screenshot**. Native sessions, mobile browsers and every desktop session are unaffected. Command rows and their screenshots, console, network and the archive itself are always captured. These absences are by design; they are not a capture failure. + +On a native session, element data and the A11y panel come from the platform's XML tree rather than the DOM, so locators read as `android=new UiSelector()…` / `-ios predicate string:…` rather than CSS or XPath over HTML. + +A runnable example per adapter lives under [examples/](examples/) — `pnpm demo:wdio:mobile`, `:selenium:mobile`, `:nightwatch:mobile`, `:python:mobile`. Each drives the Clock app that ships with every Android system image — starting a timer, pausing it and clearing it — so none needs an `.apk`; `DEVTOOLS_MOBILE=web` drives Chrome on the same device instead. [examples/MOBILE.md](examples/MOBILE.md) states the prerequisites and switches in one place. In short, for a local emulator: 1. **Java JDK** — `brew install --cask temurin` 2. **Android SDK** — `brew install --cask android-commandlinetools` then `yes | sdkmanager --licenses && sdkmanager "platform-tools" "emulator" "system-images;android-34;google_apis_playstore;arm64-v8a"`. The brew cask installs sdkmanager under `/opt/homebrew/share/android-commandlinetools/`, and sdkmanager downloads other SDK pieces alongside it — set `ANDROID_HOME` to that path (not `~/Library/Android/sdk/`). 3. **AVD + emulator** — `avdmanager create avd -n devtools-test -k "system-images;android-34;google_apis_playstore;arm64-v8a" -d "pixel_7"`, then `emulator -avd devtools-test &` + `adb wait-for-device`. 4. **Appium + UiAutomator2 driver** — `sudo npm i -g appium && appium driver install uiautomator2`. -5. **Chromedriver pinning** — Appium's autodownload doesn't reach back far enough for the Chrome version that ships with most Android system images (e.g. Chrome 113 on Android 14). Manually download the matching Chromedriver and start Appium with `--default-capabilities '{"appium:chromedriverExecutableDir": ""}'` plus `--allow-insecure=uiautomator2:chromedriver_autodownload`. +5. **Chromedriver pinning** — a webview is driven by Chromedriver, and Appium's autodownload frequently has no build matching the Chrome that ships with the system image, in either direction: too old on an older image, too new on a current one. It surfaces as `No Chromedriver found that can automate Chrome ''` when entering a webview context, which reads as a capture failure and is not one — it is why a mobile browser or hybrid run cannot start. Download the matching Chromedriver and start Appium with `--default-capabilities '{"appium:chromedriverExecutableDir": ""}'` plus `--allow-insecure=uiautomator2:chromedriver_autodownload`. A native-app run needs none of this. 6. **Classic WebDriver protocol** — Appium 3's BiDi shim for UiAutomator2 doesn't implement every BiDi command (e.g. `script.addPreloadScript`). Set `'wdio:enforceWebDriverClassic': true` in the capability block so WDIO doesn't attempt the BiDi handshake. These are emulator-specific issues; on a physical phone with USB debugging only steps 1, 4, 6 (and the Chromedriver pin if Chrome on the device is old) apply. @@ -361,6 +381,12 @@ pnpm lint # lint all packages # Run an example project for manual UI / runtime verification: pnpm demo:wdio # or: pnpm demo:nightwatch / pnpm demo:selenium + +# Mobile (Appium). Drives the Clock app that ships with every Android system +# image, so there is no .apk to supply. Needs Appium and an Android device or emulator — see +# examples/MOBILE.md. +pnpm demo:wdio:mobile # or :selenium:mobile / :nightwatch:mobile / :python:mobile +DEVTOOLS_MODE=trace pnpm demo:wdio:mobile # trace instead of live ``` See **[CONTRIBUTING.md](./CONTRIBUTING.md)** for the full contributor workflow and **[ARCHITECTURE.md](./ARCHITECTURE.md)** for where each piece lives. From 2fb22006d13ec4c1071a4edce19d1bbb514a710e Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Tue, 22 Sep 2026 20:01:00 +0530 Subject: [PATCH 03/15] fix(examples): repair nightwatch discovery, and stop advertising iOS --- CLAUDE.md | 2 +- examples/MOBILE.md | 49 ++++++++----------- examples/mobile-preflight.cjs | 17 +++++++ examples/nightwatch/mobile/clock.js | 21 ++++++++ examples/nightwatch/nightwatch.conf.cjs | 2 +- .../nightwatch/nightwatch.mobile.conf.cjs | 3 +- examples/selenium-js/mobile/clock.test.js | 14 ++++++ examples/selenium-py/scripts/clock.py | 9 ++++ examples/wdio/mobile/specs/clock.e2e.ts | 14 ++++++ examples/wdio/mobile/wdio.conf.ts | 5 +- 10 files changed, 103 insertions(+), 33 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4f4386a8..8feca60a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,7 +27,7 @@ Run from repo root unless noted. | `pnpm test:coverage` | Run vitest with v8 coverage. The thresholds in `vitest.config.ts` are aspirational, not a gate: that file states CI does not run this and the suite is currently below all four. CI runs `test`/`lint`/`test:ui`. | | `pnpm lint` | Lint all packages in parallel. Includes `eslint-plugin-security` for a subset of CodeQL findings; deeper taint-flow checks surface on the PR's CodeQL scan. | | `pnpm demo:wdio` / `pnpm demo:nightwatch` / `pnpm demo:selenium` | Run the per-framework example projects. Useful for manual verification of UI or runtime changes. | -| `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and default to the device's own Settings app, so a native example needs no `.apk` — `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches. `DEVTOOLS_MODE=live` flips any demo to live mode. | +| `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and drive the Clock app that ships with every Android system image — starting a timer, pausing it, clearing it — so a native example needs no `.apk`. Android only. `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches; `DEVTOOLS_MODE=trace` flips any demo to trace mode. | | `pnpm dev` | Run all packages in parallel dev mode. | `selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:mocha:allure` / `:jest` / `:cucumber`. diff --git a/examples/MOBILE.md b/examples/MOBILE.md index 82652717..86eeeccb 100644 --- a/examples/MOBILE.md +++ b/examples/MOBILE.md @@ -31,8 +31,13 @@ none of it is needed for any other demo or test in this repo. is far lighter and is the route these instructions assume: ```sh - # macOS arm64; see developer.android.com for the other builds - curl -fsSL https://dl.google.com/android/cli/latest/darwin_arm64/install.sh | bash + # macOS arm64; see developer.android.com for the other builds. Downloaded + # and read before it runs, rather than piped into a shell: `latest` is a + # mutable URL, so piping executes whatever it returns at that moment. + curl -fsSL -o /tmp/android-cli-install.sh \ + https://dl.google.com/android/cli/latest/darwin_arm64/install.sh + less /tmp/android-cli-install.sh # read it + bash /tmp/android-cli-install.sh ``` That leaves `android-cli` in `~/.android/bin` (**not** on your PATH) and an @@ -263,7 +268,10 @@ The default URL is public, and an emulator often cannot resolve public DNS machine's** localhost, so serving a page here is the reliable route: ```sh -python3 -m http.server 8099 # in some directory with an index.html +# --bind and --directory are both deliberate: the default serves the CURRENT +# directory on EVERY interface, so a checkout's contents would be readable by +# anything on the LAN or VPN. The emulator only needs this machine's loopback. +python3 -m http.server 8099 --bind 127.0.0.1 --directory /tmp/mobile-page DEVTOOLS_MOBILE_URL=http://10.0.2.2:8099/ DEVTOOLS_MOBILE=web pnpm demo:wdio:mobile ``` @@ -311,33 +319,18 @@ failed`, `BiDi preload unavailable`, `BiDi NetworkInspector attach failed`). session-scoped slice, so the config asks for `session` granularity rather than pretending otherwise. See CLAUDE.md § Known debt. -## iOS +## iOS is not supported -One switch, no hand-editing: +`DEVTOOLS_MOBILE_PLATFORM=ios` exits immediately, with the reason. The +capability builders can shape an XCUITest session against the simulator's Clock +app (`com.apple.mobiletimer`), and that part is real — but **no example has an +iOS body**. Every flow drives Android's Clock through UiAutomator resource-ids, +which XCUITest cannot resolve, so an iOS run would build a session and then fail +on its first lookup. -```sh -DEVTOOLS_MOBILE_PLATFORM=ios IOS_DEVICE_NAME="iPhone 15" pnpm demo:wdio:mobile -``` - -It needs a Mac with **full Xcode** (Command Line Tools alone carry no -simulators), a booted simulator, and the XCUITest driver — whose first run also -builds WebDriverAgent: - -```sh -sudo xcode-select -s /Applications/Xcode.app/Contents/Developer -xcrun simctl list devices available | grep iPhone # the name is per-machine -xcrun simctl boot "iPhone 15" -appium driver install xcuitest -``` - -The target is the simulator's own Clock app (`com.apple.mobiletimer`), so -there is no `.app` to supply — the same trick as Android. - -**One iOS-specific thing to watch.** `getWindowSize()` returns **points** while -the screenshot is **pixels** — roughly a 3x factor, so a trace legitimately -declares `402 × 874` for a `1206 × 2622` image. That disagreement is why the -player fits a domless capture by the image's own decoded size and never by the -viewport. Android nearly hides it; iOS is where a regression there would show. +Adding iOS means a flow and selectors per example, not new plumbing. It is left +out rather than half-advertised: a switch that builds a session and then cannot +find anything is worse than one that says no. ## What to look for diff --git a/examples/mobile-preflight.cjs b/examples/mobile-preflight.cjs index 489be95b..0dbf98e9 100644 --- a/examples/mobile-preflight.cjs +++ b/examples/mobile-preflight.cjs @@ -133,6 +133,23 @@ async function requireMobileToolchain({ host = process.env.APPIUM_HOST ?? '127.0.0.1', port = Number(process.env.APPIUM_PORT ?? 4723) } = {}) { + // Every check below is Android's, and so is every example flow: they drive + // the Clock app through UiAutomator resource-ids. The capability builders + // can still shape an XCUITest session, but no example has an iOS BODY — it + // would activate the Android package and then look for Android ids. Say so + // here rather than letting the run reach a lookup that cannot match. + if (process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios') { + console.error( + '\nDEVTOOLS_MOBILE_PLATFORM=ios is not supported by these examples.\n\n' + + 'They are Android-only: every flow drives the Clock app through\n' + + 'UiAutomator resource-ids, which XCUITest cannot resolve, and every\n' + + 'check below looks for the Android SDK.\n\n' + + 'The capability builders can still shape an XCUITest session, so an\n' + + 'iOS example needs a flow and selectors rather than new plumbing.\n' + + 'See examples/MOBILE.md.\n' + ) + process.exit(1) + } if (await appiumReady(host, port)) { // A remote or cloud Appium drives a device this machine knows nothing // about, so none of the local checks below apply to it. diff --git a/examples/nightwatch/mobile/clock.js b/examples/nightwatch/mobile/clock.js index 4cb3e268..4457e0a2 100644 --- a/examples/nightwatch/mobile/clock.js +++ b/examples/nightwatch/mobile/clock.js @@ -18,6 +18,8 @@ const APP_ID = 'com.google.android.deskclock' const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +/** APPIUM_APP replaces Clock, so the Clock flow does not apply to it. */ +const CUSTOM_APP = Boolean(process.env.APPIUM_APP) /** A resource-id, in Nightwatch's element DEFINITION shape. The raw W3C * `{using, value}` form is accepted syntactically and then issues no lookup @@ -43,6 +45,18 @@ describe('Clock (native)', function () { // Re-activated rather than relying on the launch capability alone, so the // spec re-runs against a session left on another screen. + if (CUSTOM_APP) { + // A supplied app has none of Clock's screens, so driving the Clock flow + // against it would look for ids that cannot exist. Capture its hierarchy + // instead — which is what a custom app is set here to exercise. + const source = await browser.source() + await browser.assert.ok( + Boolean(source), + 'the view hierarchy was readable' + ) + return + } + await browser.execute('mobile: activateApp', [{ appId: APP_ID }]) await browser.click(byId('tab_menu_timer')) @@ -95,6 +109,13 @@ describe('Clock (native)', function () { return } + if (CUSTOM_APP) { + await browser.assert.ok( + Boolean(await browser.source()), + 'the view hierarchy was readable' + ) + return + } await browser.execute('mobile: activateApp', [{ appId: APP_ID }]) await browser.click(byId('tab_menu_stopwatch')) await browser.assert.visible(byId('tab_menu_stopwatch')) diff --git a/examples/nightwatch/nightwatch.conf.cjs b/examples/nightwatch/nightwatch.conf.cjs index 7aa4bd94..6bbc2edd 100644 --- a/examples/nightwatch/nightwatch.conf.cjs +++ b/examples/nightwatch/nightwatch.conf.cjs @@ -4,7 +4,7 @@ const nightwatchDevtools = require('@wdio/nightwatch-devtools').default module.exports = { // Resolve relative to this config file so the path holds regardless of CWD. - src_folders: [path.resolve(__dirname, 'tests')], + src_folders: [path.resolve(__dirname, 'bdd')], output_folder: false, // Skip generating nightwatch reports for this example // Add custom reporter to capture commands custom_commands_path: [], diff --git a/examples/nightwatch/nightwatch.mobile.conf.cjs b/examples/nightwatch/nightwatch.mobile.conf.cjs index f270e40e..02f57ca5 100644 --- a/examples/nightwatch/nightwatch.mobile.conf.cjs +++ b/examples/nightwatch/nightwatch.mobile.conf.cjs @@ -1,5 +1,6 @@ // Mobile example for @wdio/nightwatch-devtools. Drives the device's own -// Settings app by default, so it needs no .apk. See examples/MOBILE.md for +// Clock app that ships with every Android system image, so it needs no .apk. +// See examples/MOBILE.md for // prerequisites and the DEVTOOLS_MOBILE / APPIUM_APP switches. // // pnpm demo:nightwatch:mobile diff --git a/examples/selenium-js/mobile/clock.test.js b/examples/selenium-js/mobile/clock.test.js index 72d8a2cc..7959a051 100644 --- a/examples/selenium-js/mobile/clock.test.js +++ b/examples/selenium-js/mobile/clock.test.js @@ -35,6 +35,8 @@ DevTools.configure({ const APP_ID = 'com.google.android.deskclock' const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +/** APPIUM_APP replaces Clock, so the Clock flow does not apply to it. */ +const CUSTOM_APP = Boolean(process.env.APPIUM_APP) const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ process.env.APPIUM_PORT ?? 4723 @@ -133,6 +135,14 @@ describe('Clock (native)', function () { return } + if (CUSTOM_APP) { + // A supplied app has none of Clock's screens, so driving the Clock flow + // against it would look for ids that cannot exist. Capture its hierarchy + // instead — which is what a custom app is set here to exercise. + assert.ok((await driver.getPageSource()).length > 0) + return + } + // Re-activated rather than relying on the launch capability alone, so the // spec re-runs against a session left on another screen. await driver.executeScript('mobile: activateApp', { appId: APP_ID }) @@ -192,6 +202,10 @@ describe('Clock (native)', function () { return } + if (CUSTOM_APP) { + assert.ok((await driver.getPageSource()).length > 0) + return + } await driver.executeScript('mobile: activateApp', { appId: APP_ID }) await (await byId('tab_menu_stopwatch')).click() }) diff --git a/examples/selenium-py/scripts/clock.py b/examples/selenium-py/scripts/clock.py index b3b6b14a..ee9d8212 100644 --- a/examples/selenium-py/scripts/clock.py +++ b/examples/selenium-py/scripts/clock.py @@ -71,6 +71,8 @@ def require_appium(host: str, port: str) -> None: APP_ID = "com.google.android.deskclock" +# APPIUM_APP replaces Clock, so the Clock flow does not apply to it. +CUSTOM_APP = bool(os.environ.get("APPIUM_APP")) IS_WEB = os.environ.get("DEVTOOLS_MOBILE") == "web" IS_IOS = os.environ.get("DEVTOOLS_MOBILE_PLATFORM") == "ios" APPIUM = "http://%s:%s" % ( @@ -134,6 +136,13 @@ def capabilities() -> dict: driver.find_element(By.ID, "password").send_keys("SuperSecretPassword!") driver.find_element(By.CSS_SELECTOR, 'button[type="submit"]').click() print(driver.find_element(By.ID, "flash").text.strip()) + elif CUSTOM_APP: + # A supplied app has none of Clock's screens, so driving the Clock flow + # against it would look for ids that cannot exist. Capture its + # hierarchy instead -- which is what a custom app is set here to + # exercise. + assert driver.page_source, "the view hierarchy was empty" + print("captured the supplied app's hierarchy") else: # Re-activated rather than relying on the launch capability alone, so # the script re-runs against a session left on another screen. diff --git a/examples/wdio/mobile/specs/clock.e2e.ts b/examples/wdio/mobile/specs/clock.e2e.ts index b77b7341..e2df9f5c 100644 --- a/examples/wdio/mobile/specs/clock.e2e.ts +++ b/examples/wdio/mobile/specs/clock.e2e.ts @@ -21,6 +21,8 @@ import { expect } from '@wdio/globals' const APP_ID = 'com.google.android.deskclock' const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +/** APPIUM_APP replaces Clock, so the Clock flow does not apply to it. */ +const CUSTOM_APP = Boolean(process.env.APPIUM_APP) // An emulator often cannot resolve public DNS (corporate network, VPN), and // `10.0.2.2` is its alias for the HOST's localhost — so a page served on this @@ -56,6 +58,14 @@ describe('Clock (native)', () => { return } + if (CUSTOM_APP) { + // A supplied app has none of Clock's screens, so driving the Clock flow against it would look for ids that cannot exist. + // Capture its hierarchy instead — which is what a custom app is set here to exercise. + const source = await browser.getPageSource() + expect(source.length).toBeGreaterThan(0) + return + } + // `mobile: activateApp` rather than relying on the launch capability alone: // it re-runs against a session left on another screen, and needs no // adb_shell, which Appium does not enable by default. @@ -91,6 +101,10 @@ describe('Clock (native)', () => { return } + if (CUSTOM_APP) { + expect((await browser.getPageSource()).length).toBeGreaterThan(0) + return + } await browser.execute('mobile: activateApp', { appId: APP_ID }) await expect(byId('tab_menu_stopwatch')).toBeDisplayed() await byId('tab_menu_stopwatch').click() diff --git a/examples/wdio/mobile/wdio.conf.ts b/examples/wdio/mobile/wdio.conf.ts index a78858a3..dbd76e34 100644 --- a/examples/wdio/mobile/wdio.conf.ts +++ b/examples/wdio/mobile/wdio.conf.ts @@ -1,5 +1,6 @@ -// Mobile example for the WDIO service. Drives the device's own Settings app by -// default, so it needs no .apk — see ../../MOBILE.md for prerequisites and the +// Mobile example for the WDIO service. Drives the Clock app that ships with +// every Android system image, so it needs no .apk — see ../../MOBILE.md for +// prerequisites and the // DEVTOOLS_MOBILE / APPIUM_APP switches. // // pnpm demo:wdio:mobile From f20059286f0852a2645220180adb3aac82ec765b Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Tue, 22 Sep 2026 21:19:49 +0530 Subject: [PATCH 04/15] fix(examples): refuse iOS in the Python example too, and correct the guide --- examples/MOBILE.md | 10 +++++----- examples/selenium-py/scripts/clock.py | 13 +++++++++++++ 2 files changed, 18 insertions(+), 5 deletions(-) diff --git a/examples/MOBILE.md b/examples/MOBILE.md index 86eeeccb..1311eced 100644 --- a/examples/MOBILE.md +++ b/examples/MOBILE.md @@ -95,9 +95,9 @@ exported`, delivered through the client as a WebDriver failure that looks The preflight reads the running Appium's environment and says so when this is what is wrong, because nothing else in the stack does. -**You do not need an app.** The default target is the device's own Settings -app, so there is no `.apk` to build, upload, or keep credentials for — which is -what kept a native example from landing before (#354). +**You do not need an app.** The default target is the device's own Clock app, +so there is no `.apk` to build, upload, or keep credentials for — which is what +kept a native example from landing before. Every one of the four checks this before it opens a session and tells you what is missing, because the frameworks themselves do not: WDIO reports "make sure @@ -167,8 +167,8 @@ points at itself: | `DEVTOOLS_MOBILE` | `native` | `web` drives Chrome on the device instead of an app | | `APPIUM_APP` | — | path to an `.apk`/`.app` to drive instead of Clock | | `APPIUM_HOST` / `APPIUM_PORT` | `127.0.0.1` / `4723` | where Appium is listening | -| `DEVTOOLS_MOBILE_PLATFORM` | `android` | `ios` switches all four examples to XCUITest and the Clock app's bundle id | -| `IOS_DEVICE_NAME` / `IOS_PLATFORM_VERSION` | `iPhone 15` / — | which simulator; per-machine, see `xcrun simctl list devices` | +| `DEVTOOLS_MOBILE_PLATFORM` | `android` | `ios` is **not supported** — the examples are Android-only, see below | +| `IOS_DEVICE_NAME` / `IOS_PLATFORM_VERSION` | `iPhone 15` / — | read by the capability builders only; no example has an iOS flow to use them | ```sh DEVTOOLS_MOBILE=web pnpm demo:nightwatch:mobile # mobile web, not an app diff --git a/examples/selenium-py/scripts/clock.py b/examples/selenium-py/scripts/clock.py index ee9d8212..a1ea89ca 100644 --- a/examples/selenium-py/scripts/clock.py +++ b/examples/selenium-py/scripts/clock.py @@ -116,6 +116,19 @@ def capabilities() -> dict: return base +# Android-only, for the same reason the JS examples are: the flow below drives +# Clock through UiAutomator resource-ids, which XCUITest cannot resolve. The +# capability builder above can still shape an XCUITest session, so an iOS +# example needs a flow and selectors rather than new plumbing. Refused here +# rather than in the shared JS preflight, which this script never reaches. +if IS_IOS: + raise SystemExit( + "\nDEVTOOLS_MOBILE_PLATFORM=ios is not supported by these examples.\n\n" + "They are Android-only: every flow drives the Clock app through\n" + "UiAutomator resource-ids, which XCUITest cannot resolve.\n\n" + "See examples/MOBILE.md.\n" + ) + require_appium( os.environ.get("APPIUM_HOST", "127.0.0.1"), os.environ.get("APPIUM_PORT", "4723"), From 21e710ed7d21409a43473b0d386eb5681f9416ca Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 00:25:20 +0530 Subject: [PATCH 05/15] fix(examples): clear leftover timers in the native Clock example --- examples/wdio/mocha/native/clock.e2e.ts | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/examples/wdio/mocha/native/clock.e2e.ts b/examples/wdio/mocha/native/clock.e2e.ts index 8d6ce80f..70787533 100644 --- a/examples/wdio/mocha/native/clock.e2e.ts +++ b/examples/wdio/mocha/native/clock.e2e.ts @@ -14,6 +14,23 @@ const byId = (id: string) => `android=new UiSelector().resourceId("com.google.android.deskclock:id/${id}")` ) +/** Delete every timer already on the Timers tab, so the preset buttons are the + * ones on screen. A timer SURVIVES the session, and while one exists the tab + * shows its card instead of the presets — so an interrupted run (which never + * reaches the delete below) would break every later one. Idempotent. */ +async function clearExistingTimers(): Promise { + for (let i = 0; i < 5; i++) { + const remaining = await $$( + `android=new UiSelector().resourceId("${APP_ID}:id/delete_button")` + ) + if (!remaining.length) { + return + } + await remaining[0]!.click() + await browser.pause(300) + } +} + describe('Clock (native)', () => { it('starts a preset timer, pauses it, and clears it', async () => { console.log('[TEST] launching the Clock app') @@ -25,6 +42,9 @@ describe('Clock (native)', () => { console.log('[TEST] opening the Timers tab') await byId('tab_menu_timer').click() + // Clear whatever a previous run left behind, before reading the presets. + await clearExistingTimers() + console.log('[TEST] starting the 5 minute preset') // This build starts the timer straight from the preset — verified on the // device — so the running countdown is the evidence the tap landed. From d6896f7259e33b3b9dd751e1a7ef85f9dc0632ca Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 00:47:07 +0530 Subject: [PATCH 06/15] fix(examples): bound timer cleanup by progress, and typecheck the mobile dir --- examples/nightwatch/mobile/clock.js | 13 +++++++++++- examples/selenium-js/mobile/clock.test.js | 11 +++++++++- examples/selenium-py/scripts/clock.py | 15 +++++++++++++- examples/wdio/mobile/specs/clock.e2e.ts | 25 +++++++++++++++++++++-- examples/wdio/mocha/native/clock.e2e.ts | 24 ++++++++++++++++++---- examples/wdio/tsconfig.json | 1 + 6 files changed, 80 insertions(+), 9 deletions(-) diff --git a/examples/nightwatch/mobile/clock.js b/examples/nightwatch/mobile/clock.js index 4457e0a2..542b19c5 100644 --- a/examples/nightwatch/mobile/clock.js +++ b/examples/nightwatch/mobile/clock.js @@ -64,7 +64,8 @@ describe('Clock (native)', function () { // Clear anything a previous run left behind. A timer SURVIVES the session, // and while one exists the Timers tab shows its card instead of the preset // buttons — so without this, one interrupted run breaks every later one. - for (let i = 0; i < 5; i++) { + let previous = Number.POSITIVE_INFINITY + for (;;) { // `elements()` is the protocol-level lookup: it hands back a result with // an empty list. The higher-level `findElements()` instead WAITS for a // match and then throws NoSuchElementError, which Nightwatch reports as a @@ -76,6 +77,16 @@ describe('Clock (native)', function () { if (!count) { break } + // Bounded by PROGRESS rather than by a count: any number of timers may + // have piled up, and a fixed cap leaves the presets unreachable past it. + if (count >= previous) { + // Deleting works card by card, and a long pile-up scrolls the + // earliest ones out of the viewport where a tap cannot reach them. + throw new Error( + `could not clear ${count} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` + ) + } + previous = count await browser.click(byId('delete_button')) await browser.pause(300) } diff --git a/examples/selenium-js/mobile/clock.test.js b/examples/selenium-js/mobile/clock.test.js index 7959a051..6dc6e450 100644 --- a/examples/selenium-js/mobile/clock.test.js +++ b/examples/selenium-js/mobile/clock.test.js @@ -152,11 +152,20 @@ describe('Clock (native)', function () { // Clear anything a previous run left behind. A timer SURVIVES the session, // and while one exists the Timers tab shows its card instead of the preset // buttons — so without this, one interrupted run breaks every later one. - for (let i = 0; i < 5; i++) { + let previous = Number.POSITIVE_INFINITY + for (;;) { const remaining = await driver.findElements(uiSelector('delete_button')) if (!remaining.length) { break } + if (remaining.length >= previous) { + // Deleting works card by card, and a long pile-up scrolls the + // earliest ones out of the viewport where a tap cannot reach them. + throw new Error( + `could not clear ${remaining.length} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` + ) + } + previous = remaining.length await remaining[0].click() await driver.sleep(300) } diff --git a/examples/selenium-py/scripts/clock.py b/examples/selenium-py/scripts/clock.py index a1ea89ca..19fa9c14 100644 --- a/examples/selenium-py/scripts/clock.py +++ b/examples/selenium-py/scripts/clock.py @@ -173,13 +173,26 @@ def by_id(name): # session, and while one exists the Timers tab shows its card instead # of the preset buttons -- so without this, one interrupted run breaks # every later one. - for _ in range(5): + # Bounded by PROGRESS rather than by a count: any number of timers may + # have piled up, and a fixed cap leaves the presets unreachable past + # it. A click that fails to reduce the count is the stuck case. + previous = None + while True: left = driver.find_elements( "-android uiautomator", 'new UiSelector().resourceId("%s:id/delete_button")' % APP_ID, ) if not left: break + if previous is not None and len(left) >= previous: + # Deleting works card by card, and a long pile-up scrolls the + # earliest ones out of the viewport where a tap cannot reach. + raise AssertionError( + "could not clear %d leftover timer(s) from the Timers tab. " + "Clear them by hand, or reset the app: " + "adb shell pm clear com.google.android.deskclock" % len(left) + ) + previous = len(left) left[0].click() time.sleep(0.3) diff --git a/examples/wdio/mobile/specs/clock.e2e.ts b/examples/wdio/mobile/specs/clock.e2e.ts index e2df9f5c..0ea40435 100644 --- a/examples/wdio/mobile/specs/clock.e2e.ts +++ b/examples/wdio/mobile/specs/clock.e2e.ts @@ -36,13 +36,34 @@ const byId = (id: string) => /** Delete every timer already on the Timers tab, so the preset buttons are the * ones on screen. Idempotent: no timers means nothing to click. */ async function clearExistingTimers(): Promise { - for (let i = 0; i < 5; i++) { + // Bounded by PROGRESS rather than by a count: any number of timers may have + // piled up, and a fixed cap leaves the presets unreachable past it. A click + // that fails to reduce the count is the stuck case, and says so instead of + // falling through to a preset that is not on screen. + let previous = Number.POSITIVE_INFINITY + for (;;) { + // `.getElements()`, not a bare await: `$$` returns a chainable whose + // `.length` is a Promise, so `remaining.length` would be a Promise — + // always truthy, and the "nothing left to clear" exit would never fire. const remaining = await $$( `android=new UiSelector().resourceId("${APP_ID}:id/delete_button")` - ) + ).getElements() if (!remaining.length) { return } + if (remaining.length >= previous) { + // Deleting works card by card, and a long pile-up scrolls the earliest + // ones out of the viewport where a tap cannot reach them. One timer is + // what an interrupted run leaves, so this is the unusual case — say how + // Deleting works card by card, and a long pile-up scrolls the earliest + // ones out of the viewport where a tap cannot reach them. One timer is + // all an interrupted run leaves, so this is the unusual case — say how + // to clear it rather than failing later on an absent preset. + throw new Error( + `could not clear ${remaining.length} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` + ) + } + previous = remaining.length await remaining[0]!.click() await browser.pause(300) } diff --git a/examples/wdio/mocha/native/clock.e2e.ts b/examples/wdio/mocha/native/clock.e2e.ts index 70787533..19e7304b 100644 --- a/examples/wdio/mocha/native/clock.e2e.ts +++ b/examples/wdio/mocha/native/clock.e2e.ts @@ -16,16 +16,32 @@ const byId = (id: string) => /** Delete every timer already on the Timers tab, so the preset buttons are the * ones on screen. A timer SURVIVES the session, and while one exists the tab - * shows its card instead of the presets — so an interrupted run (which never - * reaches the delete below) would break every later one. Idempotent. */ + * shows its card instead of the presets — so an interrupted run, which never + * reaches the delete at the end, would break every later one. + * + * Bounded by PROGRESS rather than by a count: a fixed cap would leave the + * presets unreachable past it. Deleting works card by card, and a long + * pile-up scrolls the earliest cards out of the viewport where a tap cannot + * reach them — one timer is all an interrupted run leaves, so that case says + * how to clear it rather than failing later on an absent preset. */ async function clearExistingTimers(): Promise { - for (let i = 0; i < 5; i++) { + let previous = Number.POSITIVE_INFINITY + for (;;) { + // `.getElements()`, not a bare await: `$$` returns a chainable whose + // `.length` is a Promise, so `remaining.length` would be a Promise — + // always truthy, and the "nothing left to clear" exit would never fire. const remaining = await $$( `android=new UiSelector().resourceId("${APP_ID}:id/delete_button")` - ) + ).getElements() if (!remaining.length) { return } + if (remaining.length >= previous) { + throw new Error( + `could not clear ${remaining.length} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` + ) + } + previous = remaining.length await remaining[0]!.click() await browser.pause(300) } diff --git a/examples/wdio/tsconfig.json b/examples/wdio/tsconfig.json index 8b8a6786..a5b163b6 100644 --- a/examples/wdio/tsconfig.json +++ b/examples/wdio/tsconfig.json @@ -27,6 +27,7 @@ "include": [ "cucumber", "mocha", + "mobile", "pageobjects" ] } From 0bdeae843b4611ca012af77a55d83377b7ad89af Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 02:59:01 +0530 Subject: [PATCH 07/15] fix(examples): drive the Clock keypad, and never start a timer --- .changeset/mobile-example-per-adapter.md | 2 +- examples/MOBILE.md | 26 ++-- examples/nightwatch/mobile/clock.js | 106 ++++++++------- examples/selenium-js/mobile/clock.test.js | 96 ++++++------- examples/selenium-py/scripts/clock.py | 98 ++++++-------- examples/wdio/mobile/specs/clock.e2e.ts | 127 ++++++++--------- examples/wdio/mobile/wdio.conf.ts | 2 +- examples/wdio/mocha/native/clock.e2e.ts | 158 +++++++++++++--------- 8 files changed, 313 insertions(+), 302 deletions(-) diff --git a/.changeset/mobile-example-per-adapter.md b/.changeset/mobile-example-per-adapter.md index 8b01d26d..b2bce3bd 100644 --- a/.changeset/mobile-example-per-adapter.md +++ b/.changeset/mobile-example-per-adapter.md @@ -9,7 +9,7 @@ All four build the same capability bag — spelled per language, which is a dupl The default target is the device's **own Clock app** (`com.google.android.deskclock`), which is what makes this landable at all: a native example previously needed an uploaded app and credentials in the environment, and that is the reason #354 closed without one. Nothing here needs an `.apk`. `APPIUM_APP` points it at a real app, and `DEVTOOLS_MOBILE=web` drives Chrome on the same device instead — worth running too, because a mobile browser session has a document and must keep every page-side call a native one skips. -All four drive the same flow — start the 5-minute timer preset, pause it, clear it — and each clears any timer a previous run left behind, because a timer survives the session and while one exists the Timers tab shows its card instead of the presets. Verified on an Android 16 emulator with Clock 9.1; the Clock app updates independently of the Android version, so the resource-ids are re-read rather than assumed. +All four drive the same flow on the timer SETUP screen — clear the entry, key a duration on the keypad, read it back, correct it with backspace — and none of them starts a timer. A running timer survives the session and replaces that screen with its card, so a spec that starts one is re-runnable only if it also finishes. They avoid the preset chips for the same class of reason: those are recently-used-duration suggestions, absent on a freshly reset Clock, so a preset-based flow fails on any device without timer history. Verified on an Android 16 emulator with Clock 9.1, from a reset app; the Clock app updates independently of the Android version, so the resource-ids are re-read rather than assumed. Each one checks the toolchain before opening a session, through a shared `examples/mobile-preflight.cjs`, because none of the four frameworks reports a missing Appium in a way that names the cause: WDIO says "make sure browser driver is running", Nightwatch says it could not reach GeckoDriver, selenium-webdriver gives `ECONNREFUSED` and a stack trace. The check also distinguishes a missing Appium from a missing SDK from an emulator that is simply not started. diff --git a/examples/MOBILE.md b/examples/MOBILE.md index 1311eced..33601cc2 100644 --- a/examples/MOBILE.md +++ b/examples/MOBILE.md @@ -118,14 +118,24 @@ credentials. Clock is used rather than Settings because it gives a native session something deterministic to do: 1. open the Timers tab -2. clear any timer a previous run left behind -3. start the 5-minute preset, and check the countdown is running -4. pause it, and check the control now offers **Start** -5. delete it, and check the timer is gone - -Step 2 is what makes them re-runnable. A timer **survives the session**, and -while one exists the Timers tab shows its card instead of the preset buttons — -so without it, one interrupted run breaks every later one. +2. backspace until the entry is empty, so the run starts from a known zero +3. key `1`, `0`, `0` on the keypad — it fills from the right, so that is one minute +4. read the duration back and check it changed, and that backspace is now enabled +5. press backspace once and check the duration changed again + +They deliberately never **start** a timer. A running timer survives the session +and replaces the setup screen with its card, so a spec that starts one is +re-runnable only if it also finishes — an interrupted run would break every +later one. Not starting one removes that whole class of failure, and the keypad +still exercises what an example is for: real input, real state change, captured. + +They also avoid the `timer_preset_*` chips. Those are recently-used-duration +suggestions rather than fixed controls: a freshly reset Clock offers only the +keypad, so a preset-based flow fails on any device without timer history — +including CI. Two Clock layouts exist on one app version, and the keypad is +common to both; the duration display is not, so the examples read +`timer_setup_time` when it is there and the separate hour/minute/second fields +otherwise. **VERIFIED ON:** Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), Clock (`com.google.android.deskclock`) 9.1. The Clock app updates diff --git a/examples/nightwatch/mobile/clock.js b/examples/nightwatch/mobile/clock.js index 542b19c5..11587e47 100644 --- a/examples/nightwatch/mobile/clock.js +++ b/examples/nightwatch/mobile/clock.js @@ -33,8 +33,28 @@ const byId = (id) => ({ locateStrategy: 'id' }) +/** The duration the setup screen shows, from whichever layout is live: a reset + * Clock renders it as one `timer_setup_time` field, a used one as separate + * hour/minute/second fields. */ +async function durationText(browser) { + // `isPresent`, not `elements()`: awaiting `browser.elements()` yields + // undefined in this Nightwatch version, which reads as "nothing matched" for + // every id. `findElements` would work but waits and then throws when nothing + // matches, which Nightwatch reports as a run error even when caught. + if (await browser.isPresent(byId('timer_setup_time'))) { + return (await browser.getText(byId('timer_setup_time'))).trim() + } + const parts = [] + for (const id of ['hour_text', 'minute_text', 'second_text']) { + parts.push( + (await browser.isPresent(byId(id))) ? await browser.getText(byId(id)) : '' + ) + } + return parts.join(':') +} + describe('Clock (native)', function () { - it('starts a preset timer, pauses it, and clears it', async function (browser) { + it('keys a duration into the timer and corrects it', async function (browser) { if (isWeb) { // A mobile BROWSER session: it has a document, so every page-side call a // native session skips must still happen. That contrast is the point. @@ -42,13 +62,9 @@ describe('Clock (native)', function () { await browser.assert.urlContains('the-internet') return } - - // Re-activated rather than relying on the launch capability alone, so the - // spec re-runs against a session left on another screen. if (CUSTOM_APP) { - // A supplied app has none of Clock's screens, so driving the Clock flow - // against it would look for ids that cannot exist. Capture its hierarchy - // instead — which is what a custom app is set here to exercise. + // A supplied app has none of Clock's screens, so capture its hierarchy + // rather than looking for ids that cannot exist. const source = await browser.source() await browser.assert.ok( Boolean(source), @@ -57,59 +73,47 @@ describe('Clock (native)', function () { return } + // Re-activated rather than relying on the launch capability alone, so the + // spec re-runs against a session left on another screen. await browser.execute('mobile: activateApp', [{ appId: APP_ID }]) - await browser.click(byId('tab_menu_timer')) - // Clear anything a previous run left behind. A timer SURVIVES the session, - // and while one exists the Timers tab shows its card instead of the preset - // buttons — so without this, one interrupted run breaks every later one. - let previous = Number.POSITIVE_INFINITY - for (;;) { - // `elements()` is the protocol-level lookup: it hands back a result with - // an empty list. The higher-level `findElements()` instead WAITS for a - // match and then throws NoSuchElementError, which Nightwatch reports as a - // run error even when caught — so "nothing to clear" would fail the test. - const found = await browser.elements('id', `${APP_ID}:id/delete_button`) - const count = Array.isArray(found) - ? found.length - : (found?.value?.length ?? 0) - if (!count) { + // Backspace until it disables itself, so the run starts from a known zero + // whatever the last one keyed in. WebdriverIO long-presses to clear in one + // go; Nightwatch has no portable long press, and a keyed duration is at + // most six digits. + for (let i = 0; i < 8; i++) { + const enabled = await browser + .isEnabled(byId('timer_setup_delete')) + .catch(() => false) + if (!enabled) { break } - // Bounded by PROGRESS rather than by a count: any number of timers may - // have piled up, and a fixed cap leaves the presets unreachable past it. - if (count >= previous) { - // Deleting works card by card, and a long pile-up scrolls the - // earliest ones out of the viewport where a tap cannot reach them. - throw new Error( - `could not clear ${count} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` - ) - } - previous = count - await browser.click(byId('delete_button')) - await browser.pause(300) + await browser.click(byId('timer_setup_delete')) + await browser.pause(200) } + const cleared = await durationText(browser) - // This build starts the timer straight from the preset, so the running - // countdown is the evidence the tap landed. - await browser.click(byId('timer_preset_2')) - await browser.assert.visible(byId('timer_text')) - - await browser.click(byId('play_pause_button')) - // The control's accessibility label flips with the timer's state, so - // asserting on it keeps this step off the countdown's own clock. - await browser.assert.attributeMatches( - byId('play_pause_button'), - 'content-desc', - /^Start/ + // The keypad fills from the right, so "1", "0", "0" reads as one minute. + for (const digit of ['1', '0', '0']) { + await browser.click(byId(`timer_setup_digit_${digit}`)) + } + const keyed = await durationText(browser) + await browser.assert.ok( + keyed !== cleared, + `the duration changed from "${cleared}" to "${keyed}"` ) + // Backspace is disabled at zero and enabled by an entry, so this reads the + // app's own state rather than the text the keypad just echoed. + await browser.assert.enabled(byId('timer_setup_delete')) - // Clearing the timer is what makes the spec re-runnable: it ends on the - // same screen it started from. - await browser.click(byId('delete_button')) - await browser.pause(500) - await browser.assert.not.elementPresent(byId('timer_text')) + await browser.click(byId('timer_setup_delete')) + await browser.pause(200) + const corrected = await durationText(browser) + await browser.assert.ok( + corrected !== keyed, + `backspace changed the duration from "${keyed}" to "${corrected}"` + ) }) it('captures a second action on the same session', async function (browser) { diff --git a/examples/selenium-js/mobile/clock.test.js b/examples/selenium-js/mobile/clock.test.js index 6dc6e450..e2195dfc 100644 --- a/examples/selenium-js/mobile/clock.test.js +++ b/examples/selenium-js/mobile/clock.test.js @@ -107,6 +107,22 @@ describe('Clock (native)', function () { const byId = async (id) => driver.wait(until.elementLocated(uiSelector(id)), 15000) + /** The duration the setup screen shows, from whichever layout is live: a + * reset Clock renders it as one `timer_setup_time` field, a used one as + * separate hour/minute/second fields. */ + const durationText = async () => { + const single = await driver.findElements(uiSelector('timer_setup_time')) + if (single.length) { + return (await single[0].getText()).trim() + } + const parts = [] + for (const id of ['hour_text', 'minute_text', 'second_text']) { + const field = await driver.findElements(uiSelector(id)) + parts.push(field.length ? await field[0].getText() : '') + } + return parts.join(':') + } + before(async function () { // A missing emulator or Appium reads as a prerequisite here rather than as // an ECONNREFUSED stack trace out of the driver. @@ -123,7 +139,7 @@ describe('Clock (native)', function () { } }) - it('starts a preset timer, pauses it, and clears it', async function () { + it('keys a duration into the timer and corrects it', async function () { if (isWeb) { // A mobile BROWSER session: it has a document, so every page-side call a // native session skips must still happen. That contrast is the point. @@ -134,11 +150,9 @@ describe('Clock (native)', function () { assert.ok((await driver.getCurrentUrl()).length > 0) return } - if (CUSTOM_APP) { - // A supplied app has none of Clock's screens, so driving the Clock flow - // against it would look for ids that cannot exist. Capture its hierarchy - // instead — which is what a custom app is set here to exercise. + // A supplied app has none of Clock's screens, so capture its hierarchy + // rather than looking for ids that cannot exist. assert.ok((await driver.getPageSource()).length > 0) return } @@ -146,61 +160,35 @@ describe('Clock (native)', function () { // Re-activated rather than relying on the launch capability alone, so the // spec re-runs against a session left on another screen. await driver.executeScript('mobile: activateApp', { appId: APP_ID }) - await (await byId('tab_menu_timer')).click() - // Clear anything a previous run left behind. A timer SURVIVES the session, - // and while one exists the Timers tab shows its card instead of the preset - // buttons — so without this, one interrupted run breaks every later one. - let previous = Number.POSITIVE_INFINITY - for (;;) { - const remaining = await driver.findElements(uiSelector('delete_button')) - if (!remaining.length) { + // Backspace until it disables itself, so the run starts from a known zero + // whatever the last one keyed in. WebdriverIO long-presses to clear in one + // go; selenium-webdriver has no portable long press, and a keyed duration + // is at most six digits. + for (let i = 0; i < 8; i++) { + const backspace = await byId('timer_setup_delete') + if (!(await backspace.isEnabled())) { break } - if (remaining.length >= previous) { - // Deleting works card by card, and a long pile-up scrolls the - // earliest ones out of the viewport where a tap cannot reach them. - throw new Error( - `could not clear ${remaining.length} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` - ) - } - previous = remaining.length - await remaining[0].click() - await driver.sleep(300) + await backspace.click() + await driver.sleep(200) } + const cleared = await durationText() - // This build starts the timer straight from the preset, so the running - // countdown is the evidence the tap landed. - await (await byId('timer_preset_2')).click() - const running = await (await byId('timer_text')).getText() - assert.match(running, /^\d{2}:\d{2}$/, `timer_text was "${running}"`) - - await (await byId('play_pause_button')).click() - // The control's accessibility label flips with the timer's state, so - // asserting on it keeps this step off the countdown's own clock. - // `getDomAttribute`, NOT `getAttribute`: selenium-webdriver implements the - // latter by executing a JavaScript atom, and a native session has no JS to - // run it in — it fails with "Method is not implemented". `getDomAttribute` - // is the plain W3C endpoint, which Appium serves. - const label = await ( - await byId('play_pause_button') - ).getDomAttribute('content-desc') - assert.match( - label, - /^Start/, - `expected the paused control to offer Start, got "${label}"` - ) - - // Clearing the timer is what makes the spec re-runnable: it ends on the - // same screen it started from. - await (await byId('delete_button')).click() - await driver.sleep(500) - assert.equal( - (await driver.findElements(uiSelector('timer_text'))).length, - 0, - 'the cleared timer is still on screen' - ) + // The keypad fills from the right, so "1", "0", "0" reads as one minute. + for (const digit of ['1', '0', '0']) { + await (await byId(`timer_setup_digit_${digit}`)).click() + } + const keyed = await durationText() + assert.notEqual(keyed, cleared, `the duration did not change: "${keyed}"`) + // Backspace is disabled at zero and enabled by an entry, so this reads the + // app's own state rather than the text the keypad just echoed. + assert.ok(await (await byId('timer_setup_delete')).isEnabled()) + + await (await byId('timer_setup_delete')).click() + await driver.sleep(200) + assert.notEqual(await durationText(), keyed) }) it('captures a second action on the same session', async function () { diff --git a/examples/selenium-py/scripts/clock.py b/examples/selenium-py/scripts/clock.py index 19fa9c14..49009c02 100644 --- a/examples/selenium-py/scripts/clock.py +++ b/examples/selenium-py/scripts/clock.py @@ -27,7 +27,6 @@ """ import os -import re import time import selenium_devtools as devtools @@ -150,10 +149,8 @@ def capabilities() -> dict: driver.find_element(By.CSS_SELECTOR, 'button[type="submit"]').click() print(driver.find_element(By.ID, "flash").text.strip()) elif CUSTOM_APP: - # A supplied app has none of Clock's screens, so driving the Clock flow - # against it would look for ids that cannot exist. Capture its - # hierarchy instead -- which is what a custom app is set here to - # exercise. + # A supplied app has none of Clock's screens, so capture its hierarchy + # rather than looking for ids that cannot exist. assert driver.page_source, "the view hierarchy was empty" print("captured the supplied app's hierarchy") else: @@ -161,62 +158,55 @@ def capabilities() -> dict: # the script re-runs against a session left on another screen. driver.execute_script("mobile: activateApp", {"appId": APP_ID}) - def by_id(name): - return driver.find_element( + def find_all(name): + return driver.find_elements( "-android uiautomator", 'new UiSelector().resourceId("%s:id/%s")' % (APP_ID, name), ) + def by_id(name): + return find_all(name)[0] + + def duration_text(): + """The duration the setup screen shows, from whichever layout is + live: a reset Clock renders it as one timer_setup_time field, a + used one as separate hour/minute/second fields.""" + single = find_all("timer_setup_time") + if single: + return single[0].text.strip() + parts = [] + for name in ("hour_text", "minute_text", "second_text"): + found = find_all(name) + parts.append(found[0].text if found else "") + return ":".join(parts) + by_id("tab_menu_timer").click() - # Clear anything a previous run left behind. A timer SURVIVES the - # session, and while one exists the Timers tab shows its card instead - # of the preset buttons -- so without this, one interrupted run breaks - # every later one. - # Bounded by PROGRESS rather than by a count: any number of timers may - # have piled up, and a fixed cap leaves the presets unreachable past - # it. A click that fails to reduce the count is the stuck case. - previous = None - while True: - left = driver.find_elements( - "-android uiautomator", - 'new UiSelector().resourceId("%s:id/delete_button")' % APP_ID, - ) - if not left: + # Backspace until it disables itself, so the run starts from a known + # zero whatever the last one keyed in. WebdriverIO long-presses to + # clear in one go; the Python client has no portable long press, and a + # keyed duration is at most six digits. + for _ in range(8): + backspace = by_id("timer_setup_delete") + if not backspace.is_enabled(): break - if previous is not None and len(left) >= previous: - # Deleting works card by card, and a long pile-up scrolls the - # earliest ones out of the viewport where a tap cannot reach. - raise AssertionError( - "could not clear %d leftover timer(s) from the Timers tab. " - "Clear them by hand, or reset the app: " - "adb shell pm clear com.google.android.deskclock" % len(left) - ) - previous = len(left) - left[0].click() - time.sleep(0.3) - - # This build starts the timer straight from the preset, so the running - # countdown is the evidence the tap landed. - by_id("timer_preset_2").click() - running = by_id("timer_text").text - assert re.match(r"^\d{2}:\d{2}$", running), 'timer_text was "%s"' % running - - by_id("play_pause_button").click() - # The control's accessibility label flips with the timer's state, so - # asserting on it keeps this step off the countdown's own clock. - label = by_id("play_pause_button").get_dom_attribute("content-desc") - assert label.startswith("Start"), 'paused control read "%s"' % label - - # Clearing the timer is what makes the script re-runnable: it ends on - # the same screen it started from. - by_id("delete_button").click() - time.sleep(0.5) - assert not driver.find_elements( - "-android uiautomator", - 'new UiSelector().resourceId("%s:id/timer_text")' % APP_ID, - ), "the cleared timer is still on screen" - print("timer started, paused and cleared") + backspace.click() + time.sleep(0.2) + cleared = duration_text() + + # The keypad fills from the right, so "1", "0", "0" is one minute. + for digit in ("1", "0", "0"): + by_id("timer_setup_digit_%s" % digit).click() + keyed = duration_text() + assert keyed != cleared, 'the duration did not change: "%s"' % keyed + # Backspace is disabled at zero and enabled by an entry, so this reads + # the app's own state rather than the text the keypad just echoed. + assert by_id("timer_setup_delete").is_enabled() + + by_id("timer_setup_delete").click() + time.sleep(0.2) + assert duration_text() != keyed, "backspace did not change the duration" + print('keyed a duration and corrected it: "%s" -> "%s"' % (cleared, keyed)) finally: driver.quit() devtools.wait_for_dashboard_close() # hold the UI open to inspect diff --git a/examples/wdio/mobile/specs/clock.e2e.ts b/examples/wdio/mobile/specs/clock.e2e.ts index 0ea40435..938b15da 100644 --- a/examples/wdio/mobile/specs/clock.e2e.ts +++ b/examples/wdio/mobile/specs/clock.e2e.ts @@ -3,18 +3,26 @@ // difference in the adapter rather than in the test. // // Drives the Clock app, which ships with every Android system image — no .apk, -// no upload, no credentials. Clock also gives a native session something -// deterministic to do: a timer can be started, paused and cleared, and each -// step changes the screen in a way the trace can be checked against. +// no upload, no credentials — and uses only the timer SETUP screen: tap the +// keypad, read the duration back, correct it with backspace. +// +// It deliberately never STARTS a timer. A running timer survives the session +// and replaces the setup screen with its card, so a spec that starts one is +// re-runnable only if it also finishes; an interrupted run would break every +// later one. Not starting one removes that whole class of failure, and the +// keypad still exercises what an example is for — real input, real state +// change, captured. // // VERIFIED ON: Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), -// Clock (com.google.android.deskclock) 9.1. +// Clock (com.google.android.deskclock) 9.1, from a freshly reset app. // -// These are resource-ids read off that build, and the Clock app updates -// INDEPENDENTLY of the Android version — so pinning a system image does not pin -// them. If a locator misses, re-read the tree rather than assuming capture -// broke: `adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml` -// (a running countdown never reaches idle, so pause or clear the timer first). +// Two Clock layouts exist on that one app version: a reset device renders the +// duration as a single `timer_setup_time` field, a used one as separate +// hour/minute/second fields, and only the latter offers the `timer_preset_*` +// suggestion chips — which is why those chips are not used here. The keypad is +// common to both. If a locator misses, re-read the tree rather than assuming +// capture broke: +// `adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml`. import { expect } from '@wdio/globals' @@ -33,44 +41,25 @@ const WEB_URL = const byId = (id: string) => $(`android=new UiSelector().resourceId("${APP_ID}:id/${id}")`) -/** Delete every timer already on the Timers tab, so the preset buttons are the - * ones on screen. Idempotent: no timers means nothing to click. */ -async function clearExistingTimers(): Promise { - // Bounded by PROGRESS rather than by a count: any number of timers may have - // piled up, and a fixed cap leaves the presets unreachable past it. A click - // that fails to reduce the count is the stuck case, and says so instead of - // falling through to a preset that is not on screen. - let previous = Number.POSITIVE_INFINITY - for (;;) { - // `.getElements()`, not a bare await: `$$` returns a chainable whose - // `.length` is a Promise, so `remaining.length` would be a Promise — - // always truthy, and the "nothing left to clear" exit would never fire. - const remaining = await $$( - `android=new UiSelector().resourceId("${APP_ID}:id/delete_button")` - ).getElements() - if (!remaining.length) { - return - } - if (remaining.length >= previous) { - // Deleting works card by card, and a long pile-up scrolls the earliest - // ones out of the viewport where a tap cannot reach them. One timer is - // what an interrupted run leaves, so this is the unusual case — say how - // Deleting works card by card, and a long pile-up scrolls the earliest - // ones out of the viewport where a tap cannot reach them. One timer is - // all an interrupted run leaves, so this is the unusual case — say how - // to clear it rather than failing later on an absent preset. - throw new Error( - `could not clear ${remaining.length} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` - ) - } - previous = remaining.length - await remaining[0]!.click() - await browser.pause(300) +const allById = (id: string) => + $$(`android=new UiSelector().resourceId("${APP_ID}:id/${id}")`).getElements() + +/** The duration the setup screen shows, from whichever layout is live. */ +async function durationText(): Promise { + const single = await allById('timer_setup_time') + if (single.length) { + return (await single[0]!.getText()).trim() + } + const parts: string[] = [] + for (const id of ['hour_text', 'minute_text', 'second_text']) { + const field = await allById(id) + parts.push(field.length ? await field[0]!.getText() : '') } + return parts.join(':') } describe('Clock (native)', () => { - it('starts a preset timer, pauses it, and clears it', async () => { + it('keys a duration into the timer and corrects it', async () => { if (isWeb) { // A mobile BROWSER session: it has a document, so every page-side call a // native session skips must still happen. That contrast is the point. @@ -78,41 +67,35 @@ describe('Clock (native)', () => { await expect(browser).toHaveUrl(expect.stringContaining('http')) return } - if (CUSTOM_APP) { - // A supplied app has none of Clock's screens, so driving the Clock flow against it would look for ids that cannot exist. - // Capture its hierarchy instead — which is what a custom app is set here to exercise. - const source = await browser.getPageSource() - expect(source.length).toBeGreaterThan(0) + // A supplied app has none of Clock's screens, so capture its hierarchy + // rather than looking for ids that cannot exist. + expect((await browser.getPageSource()).length).toBeGreaterThan(0) return } - // `mobile: activateApp` rather than relying on the launch capability alone: - // it re-runs against a session left on another screen, and needs no - // adb_shell, which Appium does not enable by default. + // Re-activated rather than relying on the launch capability alone, so the + // spec re-runs against a session left on another screen. await browser.execute('mobile: activateApp', { appId: APP_ID }) - await byId('tab_menu_timer').click() - // Clear anything a previous run left behind. A timer SURVIVES the session, - // and while one exists the Timers tab shows its card instead of the preset - // buttons — so without this, one interrupted run breaks every later one. - await clearExistingTimers() - - // This build starts the timer straight from the preset, so the running - // countdown is the evidence the tap landed. - await byId('timer_preset_2').click() - await expect(byId('timer_text')).toHaveText(/^\d{2}:\d{2}$/) + // Long press clears the whole entry where a tap removes one digit, so the + // run starts from a known zero whatever the last one keyed in. + await byId('timer_setup_delete').longPress() + const cleared = await durationText() - await byId('play_pause_button').click() - // The control's accessibility label flips with the timer's state, so - // asserting on it keeps this step off the countdown's own clock. - await expect($('~Start 5 minutes timer')).toBeDisplayed() - - // Clearing the timer is what makes the spec re-runnable: it ends on the - // same screen it started from. - await byId('delete_button').click() - await expect(byId('timer_text')).not.toBeDisplayed() + // The keypad fills from the right, so "1", "0", "0" reads as one minute. + for (const digit of ['1', '0', '0']) { + await byId(`timer_setup_digit_${digit}`).click() + } + const keyed = await durationText() + expect(keyed).not.toBe(cleared) + // Backspace is disabled at zero and enabled by an entry, so this reads the + // app's own state rather than the text the keypad just echoed. + await expect(byId('timer_setup_delete')).toBeEnabled() + + await byId('timer_setup_delete').click() + expect(await durationText()).not.toBe(keyed) }) it('captures a second action on the same session', async () => { @@ -121,13 +104,13 @@ describe('Clock (native)', () => { await browser.url(WEB_URL) return } - if (CUSTOM_APP) { expect((await browser.getPageSource()).length).toBeGreaterThan(0) return } + await browser.execute('mobile: activateApp', { appId: APP_ID }) - await expect(byId('tab_menu_stopwatch')).toBeDisplayed() await byId('tab_menu_stopwatch').click() + await expect(byId('tab_menu_stopwatch')).toBeDisplayed() }) }) diff --git a/examples/wdio/mobile/wdio.conf.ts b/examples/wdio/mobile/wdio.conf.ts index dbd76e34..fbdb5b00 100644 --- a/examples/wdio/mobile/wdio.conf.ts +++ b/examples/wdio/mobile/wdio.conf.ts @@ -56,7 +56,7 @@ export const config: WebdriverIO.Config = { // DEVTOOLS_MODE=live silently changes what a run produces — a UI window // and no zip instead of a zip and no window — so the mode is logged. mode: MODE, - traceGranularity: 'test', + traceGranularity: 'session', // Filmstrip is OPT-IN on mobile, unlike the desktop demos, purely on // cost: the poller takes a screenshot every interval and a real device // screenshot is slow — a 3-command run against an emulator took 45 s. diff --git a/examples/wdio/mocha/native/clock.e2e.ts b/examples/wdio/mocha/native/clock.e2e.ts index 19e7304b..aea290af 100644 --- a/examples/wdio/mocha/native/clock.e2e.ts +++ b/examples/wdio/mocha/native/clock.e2e.ts @@ -1,80 +1,116 @@ -// A native Android spec: no document, no URL, no DOM — the capture path a -// browser session never exercises. Drives Clock, which ships with every Android -// system image, so the example needs no APK and no app upload. +// A native Android spec run on its own via `pnpm demo:wdio:native`: no +// document, no URL, no DOM — the capture path a browser session never +// exercises. Same flow as the four per-adapter mobile examples. // -// Every selector below was read off an emulator (API 37, Clock from -// com.google.android.deskclock); resource-ids are used over text because the -// countdown text changes every second. +// Drives the Clock app, which ships with every Android system image — no .apk, +// no upload, no credentials — and uses only the timer SETUP screen: tap the +// keypad, read the duration back, correct it with backspace. +// +// It deliberately never STARTS a timer. A running timer survives the session +// and replaces the setup screen with its card, so a spec that starts one is +// re-runnable only if it also finishes; an interrupted run would break every +// later one. Not starting one removes that whole class of failure, and the +// keypad still exercises what an example is for — real input, real state +// change, captured. +// +// VERIFIED ON: Android emulator `sdk_gphone64_arm64`, Android 16 (API 36), +// Clock (com.google.android.deskclock) 9.1, from a freshly reset app. +// +// Two Clock layouts exist on that one app version: a reset device renders the +// duration as a single `timer_setup_time` field, a used one as separate +// hour/minute/second fields, and only the latter offers the `timer_preset_*` +// suggestion chips — which is why those chips are not used here. The keypad is +// common to both. If a locator misses, re-read the tree rather than assuming +// capture broke: +// `adb shell uiautomator dump /sdcard/ui.xml && adb shell cat /sdcard/ui.xml`. + import { expect } from '@wdio/globals' const APP_ID = 'com.google.android.deskclock' +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +/** APPIUM_APP replaces Clock, so the Clock flow does not apply to it. */ +const CUSTOM_APP = Boolean(process.env.APPIUM_APP) + +// An emulator often cannot resolve public DNS (corporate network, VPN), and +// `10.0.2.2` is its alias for the HOST's localhost — so a page served on this +// machine is reachable when the internet is not. See examples/MOBILE.md. +const WEB_URL = + process.env.DEVTOOLS_MOBILE_URL ?? 'https://the-internet.herokuapp.com/login' + const byId = (id: string) => - $( - `android=new UiSelector().resourceId("com.google.android.deskclock:id/${id}")` - ) + $(`android=new UiSelector().resourceId("${APP_ID}:id/${id}")`) -/** Delete every timer already on the Timers tab, so the preset buttons are the - * ones on screen. A timer SURVIVES the session, and while one exists the tab - * shows its card instead of the presets — so an interrupted run, which never - * reaches the delete at the end, would break every later one. - * - * Bounded by PROGRESS rather than by a count: a fixed cap would leave the - * presets unreachable past it. Deleting works card by card, and a long - * pile-up scrolls the earliest cards out of the viewport where a tap cannot - * reach them — one timer is all an interrupted run leaves, so that case says - * how to clear it rather than failing later on an absent preset. */ -async function clearExistingTimers(): Promise { - let previous = Number.POSITIVE_INFINITY - for (;;) { - // `.getElements()`, not a bare await: `$$` returns a chainable whose - // `.length` is a Promise, so `remaining.length` would be a Promise — - // always truthy, and the "nothing left to clear" exit would never fire. - const remaining = await $$( - `android=new UiSelector().resourceId("${APP_ID}:id/delete_button")` - ).getElements() - if (!remaining.length) { - return - } - if (remaining.length >= previous) { - throw new Error( - `could not clear ${remaining.length} leftover timer(s) from the Timers tab. Clear them by hand, or reset the app: adb shell pm clear com.google.android.deskclock` - ) - } - previous = remaining.length - await remaining[0]!.click() - await browser.pause(300) +const allById = (id: string) => + $$(`android=new UiSelector().resourceId("${APP_ID}:id/${id}")`).getElements() + +/** The duration the setup screen shows, from whichever layout is live. */ +async function durationText(): Promise { + const single = await allById('timer_setup_time') + if (single.length) { + return (await single[0]!.getText()).trim() } + const parts: string[] = [] + for (const id of ['hour_text', 'minute_text', 'second_text']) { + const field = await allById(id) + parts.push(field.length ? await field[0]!.getText() : '') + } + return parts.join(':') } describe('Clock (native)', () => { - it('starts a preset timer, pauses it, and clears it', async () => { - console.log('[TEST] launching the Clock app') - // `mobile: activateApp` rather than an `appium:app`/`appActivity` - // capability: the activity name is build-specific and this needs no - // adb_shell, which Appium does not enable by default. - await browser.execute('mobile: activateApp', { appId: APP_ID }) + it('keys a duration into the timer and corrects it', async () => { + if (isWeb) { + // A mobile BROWSER session: it has a document, so every page-side call a + // native session skips must still happen. That contrast is the point. + await browser.url(WEB_URL) + await expect(browser).toHaveUrl(expect.stringContaining('http')) + return + } + if (CUSTOM_APP) { + // A supplied app has none of Clock's screens, so capture its hierarchy + // rather than looking for ids that cannot exist. + expect((await browser.getPageSource()).length).toBeGreaterThan(0) + return + } - console.log('[TEST] opening the Timers tab') + // Re-activated rather than relying on the launch capability alone, so the + // spec re-runs against a session left on another screen. + await browser.execute('mobile: activateApp', { appId: APP_ID }) await byId('tab_menu_timer').click() - // Clear whatever a previous run left behind, before reading the presets. - await clearExistingTimers() + // Long press clears the whole entry where a tap removes one digit, so the + // run starts from a known zero whatever the last one keyed in. + await byId('timer_setup_delete').longPress() + const cleared = await durationText() + + // The keypad fills from the right, so "1", "0", "0" reads as one minute. + for (const digit of ['1', '0', '0']) { + await byId(`timer_setup_digit_${digit}`).click() + } + const keyed = await durationText() + expect(keyed).not.toBe(cleared) + // Backspace is disabled at zero and enabled by an entry, so this reads the + // app's own state rather than the text the keypad just echoed. + await expect(byId('timer_setup_delete')).toBeEnabled() - console.log('[TEST] starting the 5 minute preset') - // This build starts the timer straight from the preset — verified on the - // device — so the running countdown is the evidence the tap landed. - await byId('timer_preset_2').click() - await expect(byId('timer_text')).toHaveText(/^\d{2}:\d{2}$/) + await byId('timer_setup_delete').click() + expect(await durationText()).not.toBe(keyed) + }) - console.log('[TEST] pausing the timer') - await byId('play_pause_button').click() - // The button's accessibility label flips with the timer's state; asserting - // on it keeps this step off the countdown's own clock. - await expect($('~Start 5 minutes timer')).toBeDisplayed() + it('captures a second action on the same session', async () => { + // A second test, so `traceGranularity: 'test'` has two slices to key. + if (isWeb) { + await browser.url(WEB_URL) + return + } + if (CUSTOM_APP) { + expect((await browser.getPageSource()).length).toBeGreaterThan(0) + return + } - console.log('[TEST] clearing the timer') - await byId('delete_button').click() - await expect(byId('timer_text')).not.toBeDisplayed() + await browser.execute('mobile: activateApp', { appId: APP_ID }) + await byId('tab_menu_stopwatch').click() + await expect(byId('tab_menu_stopwatch')).toBeDisplayed() }) }) From d8256a526ea983b5ad601b6e61edb5a3e91f9d97 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 17:32:18 +0530 Subject: [PATCH 08/15] fix(examples): attach to the booted simulator instead of creating one --- examples/mobile-preflight.cjs | 78 ++++++++++++++++++++++++---- examples/wdio/mobile/capabilities.ts | 41 +++++++++++++-- 2 files changed, 104 insertions(+), 15 deletions(-) diff --git a/examples/mobile-preflight.cjs b/examples/mobile-preflight.cjs index 0dbf98e9..2a10ef2b 100644 --- a/examples/mobile-preflight.cjs +++ b/examples/mobile-preflight.cjs @@ -124,6 +124,38 @@ function appiumHasSdk() { } } +/** Booted iOS simulators as `{ name, udid }`, or null when `xcrun simctl` is + * absent — the two are different failures and get different advice. + * + * The udid matters: naming a simulator that does not exist does NOT fail, it + * makes the XCUITest driver CREATE one (`appiumTest--`) and boot + * it, every run, beside the one already running. Identifying the device by + * udid is what makes a run attach to it instead. */ +function bootedSimulators() { + try { + const out = execFileSync('xcrun', ['simctl', 'list', 'devices', 'booted'], { + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'ignore'] + }) + return out + .split('\n') + .filter((line) => line.includes('(Booted)')) + .map((line) => { + const match = line.match(/^\s*(.+?)\s+\(([0-9A-F-]{36})\)\s+\(Booted\)/i) + return match ? { name: match[1], udid: match[2] } : null + }) + .filter(Boolean) + } catch { + return null + } +} + +/** Appium is not listening. Shared, because the iOS and Android paths report + * it identically and only diverge on what they check next. */ +const APPIUM_DOWN = (host, port) => + `\nNothing is listening on ${host}:${port}, so Appium is not up.\n\n` + + ` appium --address ${host} --port ${port}\n` + /** * Print what is missing and exit, or return quietly. Exits rather than throws: * a thrown error inside a framework hook is reported as a test failure, which @@ -139,16 +171,35 @@ async function requireMobileToolchain({ // would activate the Android package and then look for Android ids. Say so // here rather than letting the run reach a lookup that cannot match. if (process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios') { - console.error( - '\nDEVTOOLS_MOBILE_PLATFORM=ios is not supported by these examples.\n\n' + - 'They are Android-only: every flow drives the Clock app through\n' + - 'UiAutomator resource-ids, which XCUITest cannot resolve, and every\n' + - 'check below looks for the Android SDK.\n\n' + - 'The capability builders can still shape an XCUITest session, so an\n' + - 'iOS example needs a flow and selectors rather than new plumbing.\n' + - 'See examples/MOBILE.md.\n' - ) - process.exit(1) + // iOS has its own toolchain, so none of the Android checks below apply: + // they look for ANDROID_HOME and an adb device, and a correctly set up + // XCUITest machine has neither. + if (!(await appiumReady(host, port))) { + console.error(APPIUM_DOWN(host, port)) + process.exit(1) + } + if (!LOCAL_HOSTS.has(host)) { + return + } + const booted = bootedSimulators() + if (booted === null) { + console.error( + '\n`xcrun simctl` is not available, so there is no iOS toolchain here.\n\n' + + 'Xcode is needed — the Command Line Tools alone ship no simulators:\n\n' + + ' sudo xcode-select -s /Applications/Xcode.app/Contents/Developer\n' + + ' xcodebuild -downloadPlatform iOS\n' + ) + process.exit(1) + } + if (!booted.length) { + console.error( + '\nNo iOS simulator is booted, so there is nothing to drive.\n\n' + + ' xcrun simctl list devices available\n' + + ' xcrun simctl boot ""\n' + ) + process.exit(1) + } + return } if (await appiumReady(host, port)) { // A remote or cloud Appium drives a device this machine knows nothing @@ -219,4 +270,9 @@ async function requireMobileToolchain({ process.exit(1) } -module.exports = { requireMobileToolchain, appiumReady, adbDevices } +module.exports = { + requireMobileToolchain, + appiumReady, + adbDevices, + bootedSimulators +} diff --git a/examples/wdio/mobile/capabilities.ts b/examples/wdio/mobile/capabilities.ts index 1c502aa8..09e64aa2 100644 --- a/examples/wdio/mobile/capabilities.ts +++ b/examples/wdio/mobile/capabilities.ts @@ -1,3 +1,5 @@ +import { createRequire } from 'node:module' + // The one capability set the mobile examples share, so a difference in the // dashboard between two adapters is a difference in the adapter. The Selenium, // Nightwatch and Python examples spell the same bag in their own syntax — @@ -16,11 +18,43 @@ const CLOCK_APP = { 'appium:appPackage': 'com.google.android.deskclock', 'appium:appActivity': 'com.android.deskclock.DeskClock' }, - ios: { 'appium:bundleId': 'com.apple.mobiletimer' } + ios: { 'appium:bundleId': 'com.apple.Preferences' } } +const { bootedSimulators } = createRequire(import.meta.url)( + '../../mobile-preflight.cjs' +) as { bootedSimulators: () => { name: string; udid: string }[] | null } + const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' +/** The simulator to drive, as a udid. + * + * By udid rather than by name, because naming one that does not exist does + * NOT fail: the XCUITest driver CREATES it (`appiumTest--`) and + * boots it, every run, beside the simulator already running. The default name + * used to be a device Xcode no longer ships, so every iOS run left another + * simulator behind. + * + * Defaults to whatever is already booted — the iOS counterpart of attaching + * to the running emulator on Android. `IOS_DEVICE_NAME` picks among several, + * and `IOS_UDID` names one outright. */ +function iosDevice(): Record { + if (process.env.IOS_UDID) { + return { 'appium:udid': process.env.IOS_UDID } + } + const booted = bootedSimulators() ?? [] + const wanted = process.env.IOS_DEVICE_NAME + const match = wanted + ? booted.find((device) => device.name === wanted) + : booted[0] + if (match) { + return { 'appium:udid': match.udid, 'appium:deviceName': match.name } + } + // Nothing booted: fall back to naming one, which is the only thing left — + // and the preflight has already said so before reaching here. + return { 'appium:deviceName': wanted ?? 'iPhone 17 Pro' } +} + export function mobileCapabilities(): Record { const web = process.env.DEVTOOLS_MOBILE === 'web' const app = process.env.APPIUM_APP @@ -28,11 +62,10 @@ export function mobileCapabilities(): Record { return { platformName: IOS ? 'iOS' : 'Android', 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', - // iOS needs the simulator named explicitly and it is per-machine, so it - // comes from the environment: `xcrun simctl list devices` shows yours. + // Which simulator, resolved to a udid — see `iosDevice`. ...(IOS ? { - 'appium:deviceName': process.env.IOS_DEVICE_NAME ?? 'iPhone 15', + ...iosDevice(), ...(process.env.IOS_PLATFORM_VERSION ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } : {}) From 92df2943f20103cb3f4a09cfb40e1daa0ffe6dcf Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 17:32:51 +0530 Subject: [PATCH 09/15] feat(examples): give every adapter an iOS mobile example --- .../nightwatch/mobile/{ => android}/clock.js | 4 + examples/nightwatch/mobile/ios/settings.js | 66 ++++++++ .../nightwatch/nightwatch.mobile.conf.cjs | 40 ++++- .../mobile/{ => android}/clock.test.js | 20 ++- .../selenium-js/mobile/ios/settings.test.js | 130 +++++++++++++++ .../{scripts => mobile/android}/clock.py | 37 +---- examples/selenium-py/mobile/ios/settings.py | 156 ++++++++++++++++++ .../mobile/specs/{ => android}/clock.e2e.ts | 9 +- .../wdio/mobile/specs/ios/settings.e2e.ts | 63 +++++++ examples/wdio/mobile/wdio.conf.ts | 12 +- package.json | 2 +- packages/selenium-devtools/package.json | 2 +- 12 files changed, 490 insertions(+), 51 deletions(-) rename examples/nightwatch/mobile/{ => android}/clock.js (96%) create mode 100644 examples/nightwatch/mobile/ios/settings.js rename examples/selenium-js/mobile/{ => android}/clock.test.js (92%) create mode 100644 examples/selenium-js/mobile/ios/settings.test.js rename examples/selenium-py/{scripts => mobile/android}/clock.py (83%) create mode 100644 examples/selenium-py/mobile/ios/settings.py rename examples/wdio/mobile/specs/{ => android}/clock.e2e.ts (92%) create mode 100644 examples/wdio/mobile/specs/ios/settings.e2e.ts diff --git a/examples/nightwatch/mobile/clock.js b/examples/nightwatch/mobile/android/clock.js similarity index 96% rename from examples/nightwatch/mobile/clock.js rename to examples/nightwatch/mobile/android/clock.js index 11587e47..05e257fb 100644 --- a/examples/nightwatch/mobile/clock.js +++ b/examples/nightwatch/mobile/android/clock.js @@ -1,3 +1,7 @@ +// Nightwatch against Appium, ANDROID. The iOS example is a sibling spec in +// ../ios — a separate file rather than a branch, because the two platforms +// ship different apps and share no selectors. +// // Nightwatch against Appium. The WebdriverIO, Selenium and Python mobile // examples drive the SAME flow, so a difference between two dashboards is a // difference in the adapter rather than in the test. diff --git a/examples/nightwatch/mobile/ios/settings.js b/examples/nightwatch/mobile/ios/settings.js new file mode 100644 index 00000000..113b346a --- /dev/null +++ b/examples/nightwatch/mobile/ios/settings.js @@ -0,0 +1,66 @@ +// Nightwatch against Appium, iOS. The Android example is a sibling spec in +// ../android — a separate file rather than a branch, because the two platforms +// ship different apps and share no selectors. +// +// Drives **Settings**, not Clock. Clock is simply not installed on the iOS +// simulator: `xcrun simctl listapps` lists Settings, Calendar, Reminders, Maps, +// Safari and a handful more, and no `com.apple.mobiletimer`. Settings is on +// every simulator and every real device, so this needs no .app, no upload and +// no credentials — the same property that makes Clock the Android choice. +// +// VERIFIED ON: iOS Simulator `iPhone 17 Pro`, iOS 26.5 (23F77), Xcode 26. + +const APP_ID = 'com.apple.Preferences' + +/** The navigation bar's title, which is how Settings says where it is. */ +async function navBarTitle(browser) { + const bars = await browser.findElements({ + selector: 'XCUIElementTypeNavigationBar', + locateStrategy: 'class name' + }) + if (!bars.length) { + return '' + } + return (await browser.elementIdAttribute(bars[0].getId(), 'name')) ?? '' +} + +describe('Settings (native)', function () { + it('navigates into a settings page and back', async function (browser) { + // Terminated before activating, not merely activated: Settings remembers + // the page the last run drilled into, so activating alone would start + // somewhere unpredictable. This is what makes the spec re-runnable. + await browser.execute('mobile: terminateApp', [{ bundleId: APP_ID }]) + await browser.execute('mobile: activateApp', [{ bundleId: APP_ID }]) + await browser.assert.ok( + (await navBarTitle(browser)).includes('Settings'), + 'Settings opened at its root' + ) + + await browser.click({ + selector: 'General', + locateStrategy: 'accessibility id' + }) + await browser.waitUntil( + async () => (await navBarTitle(browser)).includes('General'), + { timeout: 10000 } + ) + + // Back through the navigation stack rather than a tap on the back button: + // that button's accessibility id is the PARENT page's title, so tapping by + // name hits whichever row happens to share it — measured, it opened About. + await browser.back() + await browser.waitUntil( + async () => (await navBarTitle(browser)).includes('Settings'), + { timeout: 10000 } + ) + }) + + it('captures a second action on the same session', async function (browser) { + // A second test, so `traceGranularity: 'test'` has two slices to key. + await browser.execute('mobile: activateApp', [{ bundleId: APP_ID }]) + await browser.assert.ok( + (await navBarTitle(browser)).includes('Settings'), + 'Settings is foregrounded' + ) + }) +}) diff --git a/examples/nightwatch/nightwatch.mobile.conf.cjs b/examples/nightwatch/nightwatch.mobile.conf.cjs index 02f57ca5..3416730a 100644 --- a/examples/nightwatch/nightwatch.mobile.conf.cjs +++ b/examples/nightwatch/nightwatch.mobile.conf.cjs @@ -8,22 +8,48 @@ const path = require('node:path') const nightwatchDevtools = require('@wdio/nightwatch-devtools').default -const { requireMobileToolchain } = require('../mobile-preflight.cjs') +const { + requireMobileToolchain, + bootedSimulators +} = require('../mobile-preflight.cjs') const isWeb = process.env.DEVTOOLS_MOBILE === 'web' const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' +/** The simulator to drive, as a udid. + * + * By udid rather than by name, because naming one that does not exist does + * NOT fail: the XCUITest driver CREATES it (`appiumTest--`) and + * boots it, every run, beside the simulator already running. + * + * Defaults to whatever is already booted — the iOS counterpart of attaching + * to the running emulator on Android. `IOS_DEVICE_NAME` picks among several, + * and `IOS_UDID` names one outright. */ +function iosDevice() { + if (process.env.IOS_UDID) { + return { 'appium:udid': process.env.IOS_UDID } + } + const booted = bootedSimulators() ?? [] + const wanted = process.env.IOS_DEVICE_NAME + const match = wanted + ? booted.find((device) => device.name === wanted) + : booted[0] + if (match) { + return { 'appium:udid': match.udid, 'appium:deviceName': match.name } + } + return { 'appium:deviceName': wanted ?? 'iPhone 17 Pro' } +} + /** The same bag the other three mobile examples build; see * examples/wdio/mobile/capabilities.ts for the annotated original. */ function mobileCapabilities() { const base = { platformName: IOS ? 'iOS' : 'Android', 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', - // iOS needs the simulator named, and it is per-machine: - // `xcrun simctl list devices` shows yours. + // Which simulator, resolved to a udid — see `iosDevice`. ...(IOS ? { - 'appium:deviceName': process.env.IOS_DEVICE_NAME ?? 'iPhone 15', + ...iosDevice(), ...(process.env.IOS_PLATFORM_VERSION ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } : {}) @@ -50,7 +76,7 @@ function mobileCapabilities() { return { ...base, browserName: null, - 'appium:bundleId': 'com.apple.mobiletimer' + 'appium:bundleId': 'com.apple.Preferences' } } return { @@ -67,7 +93,9 @@ function mobileCapabilities() { } module.exports = { - src_folders: [path.resolve(__dirname, 'mobile')], + // One spec directory per platform, because Android and iOS ship different + // apps and share no selectors. + src_folders: [path.resolve(__dirname, 'mobile', IOS ? 'ios' : 'android')], output_folder: false, custom_commands_path: [], custom_assertions_path: [], diff --git a/examples/selenium-js/mobile/clock.test.js b/examples/selenium-js/mobile/android/clock.test.js similarity index 92% rename from examples/selenium-js/mobile/clock.test.js rename to examples/selenium-js/mobile/android/clock.test.js index e2195dfc..fc83e0b9 100644 --- a/examples/selenium-js/mobile/clock.test.js +++ b/examples/selenium-js/mobile/android/clock.test.js @@ -1,5 +1,7 @@ /** - * Mobile example for @wdio/selenium-devtools (Mocha runner). + * Mobile example for @wdio/selenium-devtools (Mocha runner), ANDROID. The iOS + * example is a sibling spec in ../ios — a separate file rather than a branch, + * because the two platforms ship different apps and share no selectors. * * Drives the Clock app, which ships with every Android system image, so it * needs no .apk. The WebdriverIO, Nightwatch and Python mobile examples drive @@ -24,7 +26,7 @@ import { createRequire } from 'node:module' import { DevTools } from '@wdio/selenium-devtools' const { requireMobileToolchain } = createRequire(import.meta.url)( - '../../mobile-preflight.cjs' + '../../../mobile-preflight.cjs' ) DevTools.configure({ @@ -37,7 +39,7 @@ const APP_ID = 'com.google.android.deskclock' const isWeb = process.env.DEVTOOLS_MOBILE === 'web' /** APPIUM_APP replaces Clock, so the Clock flow does not apply to it. */ const CUSTOM_APP = Boolean(process.env.APPIUM_APP) -const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' +const IS_IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ process.env.APPIUM_PORT ?? 4723 }` @@ -46,11 +48,11 @@ const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ * examples/wdio/mobile/capabilities.ts for the annotated original. */ function mobileCapabilities() { const base = { - platformName: IOS ? 'iOS' : 'Android', - 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', + platformName: IS_IOS ? 'iOS' : 'Android', + 'appium:automationName': IS_IOS ? 'XCUITest' : 'UiAutomator2', // iOS needs the simulator named, and it is per-machine: // `xcrun simctl list devices` shows yours. - ...(IOS + ...(IS_IOS ? { 'appium:deviceName': process.env.IOS_DEVICE_NAME ?? 'iPhone 15', ...(process.env.IOS_PLATFORM_VERSION @@ -75,14 +77,14 @@ function mobileCapabilities() { // Chrome on the device needs a matching chromedriver. Appium can // fetch one, but that is a SERVER feature, not a capability: // `--allow-insecure=uiautomator2:chromedriver_autodownload`. - browserName: IOS ? 'safari' : 'chrome' + browserName: IS_IOS ? 'safari' : 'chrome' } } if (process.env.APPIUM_APP) { return { ...base, 'appium:app': process.env.APPIUM_APP } } - if (IOS) { - return { ...base, 'appium:bundleId': 'com.apple.mobiletimer' } + if (IS_IOS) { + return { ...base, 'appium:bundleId': 'com.apple.Preferences' } } return { ...base, diff --git a/examples/selenium-js/mobile/ios/settings.test.js b/examples/selenium-js/mobile/ios/settings.test.js new file mode 100644 index 00000000..b5425502 --- /dev/null +++ b/examples/selenium-js/mobile/ios/settings.test.js @@ -0,0 +1,130 @@ +/** + * Mobile example for @wdio/selenium-devtools (Mocha runner), iOS. The Android + * example is a sibling spec in ../android — a separate file rather than a + * branch, because the two platforms ship different apps and share no selectors. + * + * Drives **Settings**, not Clock. Clock is simply not installed on the iOS + * simulator: `xcrun simctl listapps` lists Settings, Calendar, Reminders, Maps, + * Safari and a handful more, and no `com.apple.mobiletimer`. Settings is on + * every simulator and every real device, so this needs no .app, no upload and + * no credentials — the same property that makes Clock the Android choice. + * + * VERIFIED ON: iOS Simulator `iPhone 17 Pro`, iOS 26.5 (23F77), Xcode 26. + * + * DEVTOOLS_MOBILE_PLATFORM=ios pnpm demo:selenium:mobile + */ + +import { strict as assert } from 'node:assert' +import { Builder, until } from 'selenium-webdriver' +import { createRequire } from 'node:module' +import { DevTools } from '@wdio/selenium-devtools' + +const { requireMobileToolchain, bootedSimulators } = createRequire( + import.meta.url +)('../../../mobile-preflight.cjs') + +DevTools.configure({ + mode: process.env.DEVTOOLS_MODE === 'live' ? 'live' : 'trace', + traceGranularity: 'test' +}) + +const APP_ID = 'com.apple.Preferences' +const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ + process.env.APPIUM_PORT ?? 4723 +}` + +/** The simulator to drive, as a udid. Naming one that does not exist does NOT + * fail: the XCUITest driver CREATES it and boots it, every run, beside the + * simulator already running. Defaults to whatever is already booted. */ +function iosDevice() { + if (process.env.IOS_UDID) { + return { 'appium:udid': process.env.IOS_UDID } + } + const booted = bootedSimulators() ?? [] + const wanted = process.env.IOS_DEVICE_NAME + const match = wanted + ? booted.find((device) => device.name === wanted) + : booted[0] + if (match) { + return { 'appium:udid': match.udid, 'appium:deviceName': match.name } + } + return { 'appium:deviceName': wanted ?? 'iPhone 17 Pro' } +} + +function mobileCapabilities() { + return { + platformName: 'iOS', + 'appium:automationName': 'XCUITest', + ...iosDevice(), + ...(process.env.IOS_PLATFORM_VERSION + ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } + : {}), + 'appium:bundleId': APP_ID, + 'appium:noReset': true, + 'appium:newCommandTimeout': 300, + // selenium-webdriver's `Builder.build()` throws unless `browserName` is a + // STRING, before it ever contacts the server — and a native session has no + // browser. An empty string satisfies that check and is also the W3C signal + // for "no browser", which is what the adapters read to skip page-side + // capture. + browserName: '' + } +} + +describe('Settings (native)', function () { + this.timeout(180000) + let driver + + const navBarTitle = async () => { + // The raw `class name` strategy, not `By.className`: that compiles to a + // CSS selector (`.XCUIElementTypeNavigationBar`) and Appium rejects it. + const bars = await driver.findElements({ + using: 'class name', + value: 'XCUIElementTypeNavigationBar' + }) + return bars.length ? ((await bars[0].getDomAttribute('name')) ?? '') : '' + } + + before(async function () { + await requireMobileToolchain() + driver = await new Builder() + .usingServer(APPIUM) + .withCapabilities(mobileCapabilities()) + .build() + }) + + after(async function () { + if (driver) { + await driver.quit() + } + }) + + it('navigates into a settings page and back', async function () { + // Terminated before activating, not merely activated: Settings remembers + // the page the last run drilled into, so activating alone would start + // somewhere unpredictable. This is what makes the spec re-runnable. + await driver.executeScript('mobile: terminateApp', { bundleId: APP_ID }) + await driver.executeScript('mobile: activateApp', { bundleId: APP_ID }) + assert.match(await navBarTitle(), /Settings/) + + await driver + .wait( + until.elementLocated({ using: 'accessibility id', value: 'General' }), + 15000 + ) + .click() + await driver.wait(async () => /General/.test(await navBarTitle()), 15000) + + // Back through the navigation stack rather than a tap on the back button: + // that button's accessibility id is the PARENT page's title, so tapping by + // name hits whichever row happens to share it — measured, it opened About. + await driver.navigate().back() + await driver.wait(async () => /Settings/.test(await navBarTitle()), 15000) + }) + + it('captures a second action on the same session', async function () { + // A second test, so `traceGranularity: 'test'` has two slices to key. + await driver.executeScript('mobile: activateApp', { bundleId: APP_ID }) + assert.match(await navBarTitle(), /Settings/) + }) +}) diff --git a/examples/selenium-py/scripts/clock.py b/examples/selenium-py/mobile/android/clock.py similarity index 83% rename from examples/selenium-py/scripts/clock.py rename to examples/selenium-py/mobile/android/clock.py index 49009c02..887d1f16 100644 --- a/examples/selenium-py/scripts/clock.py +++ b/examples/selenium-py/mobile/android/clock.py @@ -1,4 +1,7 @@ -"""Mobile example for the Python adapter. +"""Mobile example for the Python adapter, ANDROID. + +The iOS example is a sibling script in ../ios -- a separate file rather than a +branch, because the two platforms ship different apps and share no selectors. The WebdriverIO, Selenium and Nightwatch mobile examples drive the SAME flow, so a difference between two dashboards is a difference in the adapter rather @@ -34,7 +37,6 @@ try: from appium import webdriver from appium.options.android import UiAutomator2Options - from appium.options.ios import XCUITestOptions except ImportError: # noqa: BLE001 — a missing optional dep, not a failure raise SystemExit( "this example needs the Appium client:\n" @@ -73,7 +75,6 @@ def require_appium(host: str, port: str) -> None: # APPIUM_APP replaces Clock, so the Clock flow does not apply to it. CUSTOM_APP = bool(os.environ.get("APPIUM_APP")) IS_WEB = os.environ.get("DEVTOOLS_MOBILE") == "web" -IS_IOS = os.environ.get("DEVTOOLS_MOBILE_PLATFORM") == "ios" APPIUM = "http://%s:%s" % ( os.environ.get("APPIUM_HOST", "127.0.0.1"), os.environ.get("APPIUM_PORT", "4723"), @@ -84,50 +85,28 @@ def capabilities() -> dict: """The same bag the other three mobile examples build; see examples/wdio/mobile/capabilities.ts for the annotated original.""" base = { - "platformName": "iOS" if IS_IOS else "Android", - "appium:automationName": "XCUITest" if IS_IOS else "UiAutomator2", + "platformName": "Android", + "appium:automationName": "UiAutomator2", "appium:noReset": True, "appium:newCommandTimeout": 300, } - if IS_IOS: - # The simulator name is per-machine: `xcrun simctl list devices`. - base["appium:deviceName"] = os.environ.get("IOS_DEVICE_NAME", "iPhone 15") - version = os.environ.get("IOS_PLATFORM_VERSION") - if version: - base["appium:platformVersion"] = version if IS_WEB: # Names a browser, so this session HAS a document and keeps its # page-side capture — the distinction the native guards turn on. # Chrome on the device needs a matching chromedriver. Appium can # fetch one, but that is a SERVER feature, not a capability: # --allow-insecure=uiautomator2:chromedriver_autodownload - base["browserName"] = "Safari" if IS_IOS else "Chrome" + base["browserName"] = "Chrome" return base app = os.environ.get("APPIUM_APP") if app: base["appium:app"] = app return base - if IS_IOS: - base["appium:bundleId"] = "com.apple.mobiletimer" - return base base["appium:appPackage"] = APP_ID base["appium:appActivity"] = "com.android.deskclock.DeskClock" return base -# Android-only, for the same reason the JS examples are: the flow below drives -# Clock through UiAutomator resource-ids, which XCUITest cannot resolve. The -# capability builder above can still shape an XCUITest session, so an iOS -# example needs a flow and selectors rather than new plumbing. Refused here -# rather than in the shared JS preflight, which this script never reaches. -if IS_IOS: - raise SystemExit( - "\nDEVTOOLS_MOBILE_PLATFORM=ios is not supported by these examples.\n\n" - "They are Android-only: every flow drives the Clock app through\n" - "UiAutomator resource-ids, which XCUITest cannot resolve.\n\n" - "See examples/MOBILE.md.\n" - ) - require_appium( os.environ.get("APPIUM_HOST", "127.0.0.1"), os.environ.get("APPIUM_PORT", "4723"), @@ -139,7 +118,7 @@ def capabilities() -> dict: # The options class is per-platform: handing iOS capabilities to # UiAutomator2Options builds an Android session request out of them. -_options = XCUITestOptions() if IS_IOS else UiAutomator2Options() +_options = UiAutomator2Options() driver = webdriver.Remote(APPIUM, options=_options.load_capabilities(capabilities())) try: if IS_WEB: diff --git a/examples/selenium-py/mobile/ios/settings.py b/examples/selenium-py/mobile/ios/settings.py new file mode 100644 index 00000000..d39ba9eb --- /dev/null +++ b/examples/selenium-py/mobile/ios/settings.py @@ -0,0 +1,156 @@ +"""Mobile example for the Python adapter, iOS. + +The Android example is a sibling script in ../android -- a separate file rather +than a branch, because the two platforms ship different apps and share no +selectors. + +Drives **Settings**, not Clock. Clock is simply not installed on the iOS +simulator: `xcrun simctl listapps` lists Settings, Calendar, Reminders, Maps, +Safari and a handful more, and no `com.apple.mobiletimer`. Settings is on every +simulator and every real device, so this needs no .app, no upload and no +credentials -- the same property that makes Clock the Android choice. + +VERIFIED ON: iOS Simulator `iPhone 17 Pro`, iOS 26.5 (23F77), Xcode 26. + + DEVTOOLS_MOBILE_PLATFORM=ios pnpm demo:python:mobile + +Needs the Appium Python client, which the desktop examples do not: + + pip install -r examples/selenium-py/requirements-mobile.txt +""" + +import json +import os +import re +import subprocess +import time + +import selenium_devtools as devtools + +try: + from appium import webdriver + from appium.options.ios import XCUITestOptions +except ImportError: # noqa: BLE001 -- a missing optional dep, not a failure + raise SystemExit( + "this example needs the Appium client:\n" + " pip install -r examples/selenium-py/requirements-mobile.txt" + ) + +APP_ID = "com.apple.Preferences" +APPIUM = "http://%s:%s" % ( + os.environ.get("APPIUM_HOST", "127.0.0.1"), + os.environ.get("APPIUM_PORT", "4723"), +) + + +def booted_simulators(): + """Booted simulators as (name, udid). The JS examples read this through + examples/mobile-preflight.cjs; reaching across languages for it costs more + than the six lines it takes here.""" + try: + out = subprocess.run( + ["xcrun", "simctl", "list", "devices", "booted"], + capture_output=True, text=True, timeout=15, + ).stdout + except (OSError, subprocess.SubprocessError): + return [] + pattern = re.compile(r"^\s*(.+?)\s+\(([0-9A-Fa-f-]{36})\)\s+\(Booted\)") + return [ + (m.group(1), m.group(2)) + for m in (pattern.match(line) for line in out.splitlines()) + if m + ] + + +def ios_device(): + """The simulator to drive, as a udid. + + By udid rather than by name, because naming one that does not exist does + NOT fail: the XCUITest driver CREATES it and boots it, every run, beside + the simulator already running. Defaults to whatever is already booted. + """ + if os.environ.get("IOS_UDID"): + return {"appium:udid": os.environ["IOS_UDID"]} + booted = booted_simulators() + wanted = os.environ.get("IOS_DEVICE_NAME") + match = None + if wanted: + match = next((d for d in booted if d[0] == wanted), None) + elif booted: + match = booted[0] + if match: + return {"appium:udid": match[1], "appium:deviceName": match[0]} + return {"appium:deviceName": wanted or "iPhone 17 Pro"} + + +def require_simulator(): + """A booted simulator, or say what is missing and stop.""" + if not booted_simulators(): + raise SystemExit( + "\nNo iOS simulator is booted, so there is nothing to drive.\n\n" + " xcrun simctl list devices available\n" + ' xcrun simctl boot ""\n' + ) + + +def capabilities(): + base = { + "platformName": "iOS", + "appium:automationName": "XCUITest", + "appium:bundleId": APP_ID, + "appium:noReset": True, + "appium:newCommandTimeout": 300, + } + base.update(ios_device()) + version = os.environ.get("IOS_PLATFORM_VERSION") + if version: + base["appium:platformVersion"] = version + return base + + +require_simulator() + +# Trace by default, matching the desktop demos. The Python adapter takes a +# boolean rather than a mode name; DEVTOOLS_MODE=live is the shared switch. +devtools.enable(trace=os.environ.get("DEVTOOLS_MODE") != "live") + +driver = webdriver.Remote( + APPIUM, options=XCUITestOptions().load_capabilities(capabilities()) +) + + +def nav_bar_title(): + """The navigation bar's title, which is how Settings says where it is.""" + bars = driver.find_elements("class name", "XCUIElementTypeNavigationBar") + return bars[0].get_dom_attribute("name") or "" if bars else "" + + +try: + # Terminated before activating, not merely activated: Settings remembers + # the page the last run drilled into, so activating alone would start + # somewhere unpredictable. This is what makes the script re-runnable. + driver.execute_script("mobile: terminateApp", {"bundleId": APP_ID}) + driver.execute_script("mobile: activateApp", {"bundleId": APP_ID}) + opened = nav_bar_title() + assert "Settings" in opened, 'Settings opened at "%s"' % opened + + driver.find_element("accessibility id", "General").click() + for _ in range(30): + if "General" in nav_bar_title(): + break + time.sleep(0.5) + assert "General" in nav_bar_title(), "Settings did not navigate to General" + + # Back through the navigation stack rather than a tap on the back button: + # that button's accessibility id is the PARENT page's title, so tapping by + # name hits whichever row happens to share it -- measured, it opened About. + driver.back() + for _ in range(30): + if "Settings" in nav_bar_title(): + break + time.sleep(0.5) + assert "Settings" in nav_bar_title(), "Settings did not navigate back" + print("navigated into General and back") +finally: + driver.quit() + devtools.wait_for_dashboard_close() # hold the UI open to inspect diff --git a/examples/wdio/mobile/specs/clock.e2e.ts b/examples/wdio/mobile/specs/android/clock.e2e.ts similarity index 92% rename from examples/wdio/mobile/specs/clock.e2e.ts rename to examples/wdio/mobile/specs/android/clock.e2e.ts index 938b15da..8bf8f4df 100644 --- a/examples/wdio/mobile/specs/clock.e2e.ts +++ b/examples/wdio/mobile/specs/android/clock.e2e.ts @@ -1,6 +1,8 @@ -// WebdriverIO against Appium. The Selenium, Nightwatch and Python mobile -// examples drive the SAME flow, so a difference between two dashboards is a -// difference in the adapter rather than in the test. +// WebdriverIO against Appium, ANDROID. The iOS example is a sibling spec in +// ../ios — a separate file rather than a branch, because the two platforms +// ship different apps and share no selectors. The Selenium, Nightwatch and +// Python examples drive this same flow, so a difference between two +// dashboards is a difference in the adapter rather than in the test. // // Drives the Clock app, which ships with every Android system image — no .apk, // no upload, no credentials — and uses only the timer SETUP screen: tap the @@ -73,7 +75,6 @@ describe('Clock (native)', () => { expect((await browser.getPageSource()).length).toBeGreaterThan(0) return } - // Re-activated rather than relying on the launch capability alone, so the // spec re-runs against a session left on another screen. await browser.execute('mobile: activateApp', { appId: APP_ID }) diff --git a/examples/wdio/mobile/specs/ios/settings.e2e.ts b/examples/wdio/mobile/specs/ios/settings.e2e.ts new file mode 100644 index 00000000..16895d70 --- /dev/null +++ b/examples/wdio/mobile/specs/ios/settings.e2e.ts @@ -0,0 +1,63 @@ +// WebdriverIO against Appium, iOS. The Android example is a sibling spec in +// ../android — a separate file rather than a branch, because the two platforms +// ship different apps and share no selectors. +// +// Drives **Settings**, not Clock. Clock is simply not installed on the iOS +// simulator: `xcrun simctl listapps` lists Settings, Calendar, Reminders, +// Maps, Safari and a handful more, and no `com.apple.mobiletimer`. Settings is +// on every simulator and every real device, so this needs no .app, no upload +// and no credentials — the same property that makes Clock the Android choice. +// +// The shape matches the Android spec even though the interactions cannot: +// put the app in a known state, change it, and read the change back. +// +// VERIFIED ON: iOS Simulator `iPhone 17 Pro`, iOS 26.5 (23F77), Xcode 26. +// +// iOS locators are accessibility ids and labels rather than resource-ids, so +// nothing here transfers from the Android spec. To re-read the tree, point +// Appium Inspector at the booted simulator, or dump it with a one-off session +// — an element's `name` is what `~` matches. + +import { expect } from '@wdio/globals' + +/** Settings. Present on every simulator and device, unlike Clock. */ +const APP_ID = 'com.apple.Preferences' + +/** The navigation bar's title, which is how Settings says where it is. */ +async function navBarTitle(): Promise { + const bars = await $$('XCUIElementTypeNavigationBar').getElements() + return bars.length ? ((await bars[0]!.getAttribute('name')) ?? '') : '' +} + +describe('Settings (native)', () => { + it('navigates into a settings page and back', async () => { + // Terminated before activating, not merely activated: Settings remembers + // the page the last run drilled into, so activating alone would start + // somewhere unpredictable. This is what makes the spec re-runnable — the + // iOS equivalent of the Android spec clearing its keypad entry. + await browser.execute('mobile: terminateApp', { bundleId: APP_ID }) + await browser.execute('mobile: activateApp', { bundleId: APP_ID }) + await expect(await navBarTitle()).toBe('Settings') + + await $('~General').click() + await browser.waitUntil(async () => (await navBarTitle()) === 'General', { + timeout: 10000, + timeoutMsg: 'Settings did not navigate to General' + }) + + // Back through the navigation stack rather than a tap on the back button: + // that button's accessibility id is the PARENT page's title, so tapping by + // name hits whichever row happens to share it — measured, it opened About. + await browser.back() + await browser.waitUntil(async () => (await navBarTitle()) === 'Settings', { + timeout: 10000, + timeoutMsg: 'Settings did not navigate back' + }) + }) + + it('captures a second action on the same session', async () => { + // A second test, so `traceGranularity: 'test'` has two slices to key. + await browser.execute('mobile: activateApp', { bundleId: APP_ID }) + await expect(await navBarTitle()).toBe('Settings') + }) +}) diff --git a/examples/wdio/mobile/wdio.conf.ts b/examples/wdio/mobile/wdio.conf.ts index fbdb5b00..5c182644 100644 --- a/examples/wdio/mobile/wdio.conf.ts +++ b/examples/wdio/mobile/wdio.conf.ts @@ -22,7 +22,17 @@ const __dirname = path.resolve(path.dirname(new URL(import.meta.url).pathname)) export const config: WebdriverIO.Config = { runner: 'local', - specs: [path.resolve(__dirname, 'specs', '*.e2e.ts')], + // One spec directory per platform, because Android and iOS ship different + // apps and share no selectors — a branch inside one spec would be two tests + // wearing one name. + specs: [ + path.resolve( + __dirname, + 'specs', + process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' ? 'ios' : 'android', + '*.e2e.ts' + ) + ], // Appium, not a local browser driver. hostname: process.env.APPIUM_HOST ?? '127.0.0.1', diff --git a/package.json b/package.json index 28eb77d5..f9604993 100644 --- a/package.json +++ b/package.json @@ -21,7 +21,7 @@ "demo:python": "python3 examples/selenium-py/scripts/web_form.py", "demo:python:login": "python3 examples/selenium-py/scripts/login.py", "demo:python:pytest": "python3 -m pytest examples/selenium-py/pytest/", - "demo:python:mobile": "python3 examples/selenium-py/scripts/clock.py", + "demo:python:mobile": "python3 examples/selenium-py/mobile/${DEVTOOLS_MOBILE_PLATFORM:-android}/*.py", "dev": "pnpm --parallel dev", "preview": "pnpm --parallel preview", "test": "vitest run", diff --git a/packages/selenium-devtools/package.json b/packages/selenium-devtools/package.json index 5273d80c..42136071 100644 --- a/packages/selenium-devtools/package.json +++ b/packages/selenium-devtools/package.json @@ -33,7 +33,7 @@ "prepublishOnly": "pnpm build", "example": "pnpm example:cucumber", "example:mocha": "mocha --require @wdio/selenium-devtools --timeout 60000 ../../examples/selenium-js/mocha/example.js", - "example:mobile": "mocha --require @wdio/selenium-devtools --timeout 180000 ../../examples/selenium-js/mobile/clock.test.js", + "example:mobile": "mocha --exit --require @wdio/selenium-devtools --timeout 180000 \"../../examples/selenium-js/mobile/${DEVTOOLS_MOBILE_PLATFORM:-android}/\"*.test.js", "example:mocha:allure": "mocha --require @wdio/selenium-devtools --reporter allure-mocha --reporter-options resultsDir=../../examples/selenium-js/mocha/allure-results --timeout 60000 ../../examples/selenium-js/mocha/example.js", "example:jest": "NODE_OPTIONS=--experimental-vm-modules jest --config ../../examples/selenium-js/jest/jest.config.json", "example:cucumber": "cucumber-js --config ../../examples/selenium-js/cucumber/cucumber.json" From 5307e1fbe74abbe583bef4b41f75648f9b3dffcd Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 17:33:34 +0530 Subject: [PATCH 10/15] docs: iOS is every adapter now, and say how the simulator is chosen --- CLAUDE.md | 2 +- README.md | 1 + examples/MOBILE.md | 70 ++++++++++++++++++++++++++++++++++++++-------- examples/README.md | 16 +++++++++++ 4 files changed, 76 insertions(+), 13 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index fc507d8f..2910f482 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,7 +27,7 @@ Run from repo root unless noted. | `pnpm test:coverage` | Run vitest with v8 coverage. The thresholds in `vitest.config.ts` are aspirational, not a gate: that file states CI does not run this and the suite is currently below all four. CI runs `test`/`lint`/`test:ui`. | | `pnpm lint` | Lint all packages in parallel. Includes `eslint-plugin-security` for a subset of CodeQL findings; deeper taint-flow checks surface on the PR's CodeQL scan. | | `pnpm demo:wdio` / `pnpm demo:nightwatch` / `pnpm demo:selenium` | Run the per-framework example projects. Useful for manual verification of UI or runtime changes. | -| `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and drive the Clock app that ships with every Android system image — starting a timer, pausing it, clearing it — so a native example needs no `.apk`. Android only. `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches; `DEVTOOLS_MODE=trace` flips any demo to trace mode. | +| `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and drive the Clock app that ships with every Android system image — starting a timer, pausing it, clearing it — so a native example needs no `.apk`. `DEVTOOLS_MOBILE_PLATFORM=ios` runs the iOS spec instead — all four adapters — which drives Settings because the simulator ships no Clock, from a separate spec per platform rather than a branch. The simulator is chosen by udid and defaults to whichever is already booted; naming one that does not exist makes the XCUITest driver create and boot it, every run. `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches; `DEVTOOLS_MODE=trace` flips any demo to trace mode. | | `pnpm dev` | Run all packages in parallel dev mode. | `selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:mocha:allure` / `:jest` / `:cucumber`. diff --git a/README.md b/README.md index 1461c50f..dbef4475 100644 --- a/README.md +++ b/README.md @@ -387,6 +387,7 @@ pnpm demo:wdio # or: pnpm demo:nightwatch / pnpm demo:selenium # examples/MOBILE.md. pnpm demo:wdio:mobile # or :selenium:mobile / :nightwatch:mobile / :python:mobile DEVTOOLS_MODE=trace pnpm demo:wdio:mobile # trace instead of live +DEVTOOLS_MOBILE_PLATFORM=ios pnpm demo:wdio:mobile # iOS simulator (any adapter) ``` See **[CONTRIBUTING.md](./CONTRIBUTING.md)** for the full contributor workflow and **[ARCHITECTURE.md](./ARCHITECTURE.md)** for where each piece lives. diff --git a/examples/MOBILE.md b/examples/MOBILE.md index 33601cc2..0c8173b4 100644 --- a/examples/MOBILE.md +++ b/examples/MOBILE.md @@ -177,8 +177,8 @@ points at itself: | `DEVTOOLS_MOBILE` | `native` | `web` drives Chrome on the device instead of an app | | `APPIUM_APP` | — | path to an `.apk`/`.app` to drive instead of Clock | | `APPIUM_HOST` / `APPIUM_PORT` | `127.0.0.1` / `4723` | where Appium is listening | -| `DEVTOOLS_MOBILE_PLATFORM` | `android` | `ios` is **not supported** — the examples are Android-only, see below | -| `IOS_DEVICE_NAME` / `IOS_PLATFORM_VERSION` | `iPhone 15` / — | read by the capability builders only; no example has an iOS flow to use them | +| `DEVTOOLS_MOBILE_PLATFORM` | `android` | `ios` runs the iOS spec, in every adapter — see below | +| `IOS_DEVICE_NAME` / `IOS_PLATFORM_VERSION` | `iPhone 15` / — | which simulator to drive; per-machine, see `xcrun simctl list devices` | ```sh DEVTOOLS_MOBILE=web pnpm demo:nightwatch:mobile # mobile web, not an app @@ -329,18 +329,64 @@ failed`, `BiDi preload unavailable`, `BiDi NetworkInspector attach failed`). session-scoped slice, so the config asks for `session` granularity rather than pretending otherwise. See CLAUDE.md § Known debt. -## iOS is not supported +## iOS -`DEVTOOLS_MOBILE_PLATFORM=ios` exits immediately, with the reason. The -capability builders can shape an XCUITest session against the simulator's Clock -app (`com.apple.mobiletimer`), and that part is real — but **no example has an -iOS body**. Every flow drives Android's Clock through UiAutomator resource-ids, -which XCUITest cannot resolve, so an iOS run would build a session and then fail -on its first lookup. +```sh +DEVTOOLS_MOBILE_PLATFORM=ios pnpm demo:wdio:mobile +``` + +**All four adapters.** Each has an `android/` and an `ios/` spec directory, and +the runner picks between them — no branch inside a spec, because the two +platforms share no selectors. + +iOS drives **Settings**, not Clock, because Clock is not installed on the +simulator at all — `xcrun simctl listapps` lists Settings, Calendar, Reminders, +Maps and Safari, and no `com.apple.mobiletimer`. Settings is on every simulator +and every device, which is the same property that makes Clock the Android +choice. The flow navigates into General and back, checking the navigation bar +title each way: the same shape as Android's keypad flow — change state, read it +back — with the app the platform actually ships. + +The two are **separate specs**, not one spec with a branch, because they share +no selectors: iOS locators are accessibility ids and labels rather than +resource-ids. + +``` +examples/wdio/mobile/specs/ +├── android/clock.e2e.ts +└── ios/settings.e2e.ts +``` + +**What you need**, beyond the Android prerequisites (none of which iOS uses): + +1. **Xcode** — the Command Line Tools alone ship no simulators: + ```sh + sudo xcode-select -s /Applications/Xcode.app/Contents/Developer + sudo xcodebuild -runFirstLaunch + ``` +2. **A simulator runtime**, a separate ~8 GB download even once Xcode is in: + ```sh + xcodebuild -downloadPlatform iOS + ``` +3. **A booted simulator** — the preflight checks for one and says so if none: + ```sh + xcrun simctl list devices available + xcrun simctl boot "iPhone 17 Pro" + ``` +4. **The XCUITest driver**; its first run also builds WebDriverAgent, once: + ```sh + appium driver install xcuitest + ``` + +Two things that cost a debugging round each. Appium loads drivers at **startup**, +so a server that was already running when you installed `xcuitest` reports +"Could not find a driver for automationName 'XCUITest'" — restart it. And the +navigation bar's back button carries the **parent page's** title as its +accessibility id, so tapping `~Settings` from General opens whichever row shares +that name (measured: it opened About). The spec navigates back through the stack +instead. -Adding iOS means a flow and selectors per example, not new plumbing. It is left -out rather than half-advertised: a switch that builds a session and then cannot -find anything is worse than one that says no. +**VERIFIED ON:** iOS Simulator `iPhone 17 Pro`, iOS 26.5 (23F77), Xcode 26. ## What to look for diff --git a/examples/README.md b/examples/README.md index 633e3bf8..b386d6ad 100644 --- a/examples/README.md +++ b/examples/README.md @@ -60,6 +60,22 @@ Each checks the toolchain before opening a session, through the shared [`mobile-preflight.cjs`](./mobile-preflight.cjs), because none of the four frameworks reports a missing Appium in a way that names the cause. +`DEVTOOLS_MOBILE_PLATFORM=ios` runs the iOS spec instead, in every adapter. iOS +drives **Settings**, because the simulator ships no Clock app at all, and lives +in its own spec rather than a branch: the two platforms share no selectors. + +``` +examples/wdio/mobile/specs/android|ios/ +examples/nightwatch/mobile/android|ios/ +examples/selenium-js/mobile/android|ios/ +examples/selenium-py/mobile/android|ios/ +``` + +The simulator is chosen by **udid**, defaulting to whichever is already booted. +Naming one that does not exist does not fail — the XCUITest driver creates and +boots it, every run — so `IOS_DEVICE_NAME` only picks among booted devices and +`IOS_UDID` names one outright. + [MOBILE.md](./MOBILE.md) has the prerequisites and the switches. ## Per-adapter notes From 816d308d334aa5be004accbe6c55bd9fc55be398 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 19:32:00 +0530 Subject: [PATCH 11/15] fix(nightwatch): don't fetch a perf log the session never asked for --- .../nightwatch-devtools/src/session-init.ts | 29 +++++++++++++++---- packages/nightwatch-devtools/src/session.ts | 11 ++++++- .../nightwatch-devtools/tests/session.test.ts | 15 ++++++++++ .../tests/sessionRearm.test.ts | 28 +++++++++++++++++- 4 files changed, 75 insertions(+), 8 deletions(-) diff --git a/packages/nightwatch-devtools/src/session-init.ts b/packages/nightwatch-devtools/src/session-init.ts index 0bd13200..d9a81c30 100644 --- a/packages/nightwatch-devtools/src/session-init.ts +++ b/packages/nightwatch-devtools/src/session-init.ts @@ -126,6 +126,20 @@ function readViewport(browser: NightwatchBrowser) { }) } +/** Whether the session was created asking for Chrome's `performance` log, from + * either side of the capability pair. A perf log exists only because a session + * asked for one, so this decides both the warning and whether the per-command + * fetch runs at all — one reading, two consumers. */ +export function wantsPerformanceLog( + capabilities: Record, + desiredCapabilities: Record +): boolean { + const loggingPrefs = (capabilities['goog:loggingPrefs'] || + desiredCapabilities['goog:loggingPrefs'] || + {}) as { performance?: string } + return Boolean(loggingPrefs.performance) +} + async function broadcastSessionMetadata( ctx: SessionInitCtx, browser: NightwatchBrowser @@ -167,12 +181,15 @@ async function broadcastSessionMetadata( `✓ Browser: ${browserName}${browserVersion ? ' ' + browserVersion : ''} (session: ${sessionId})` ) - const loggingPrefs = ((capabilities as Record)[ - 'goog:loggingPrefs' - ] || - (desiredCapabilities as Record)['goog:loggingPrefs'] || - {}) as { performance?: string } - if (!loggingPrefs.performance && !ctx.bidiEnabled) { + const wantsPerfLog = wantsPerformanceLog( + capabilities as Record, + desiredCapabilities as Record + ) + // The same reading decides whether the capture path runs at all: without the + // capability there is nothing to fetch, and asking anyway costs a failed + // round trip per command — see `SessionCapturer.perfLogsRequested`. + ctx.sessionCapturer.perfLogsRequested = wantsPerfLog + if (!wantsPerfLog && !ctx.bidiEnabled) { log.warn( "⚠ Network tab will be empty — add 'goog:loggingPrefs': { performance: 'ALL' } to your capabilities (or enable bidi:true)" ) diff --git a/packages/nightwatch-devtools/src/session.ts b/packages/nightwatch-devtools/src/session.ts index c8f459dd..e53ad503 100644 --- a/packages/nightwatch-devtools/src/session.ts +++ b/packages/nightwatch-devtools/src/session.ts @@ -56,6 +56,15 @@ export class SessionCapturer extends SessionCapturerBase { // capture path skips when set, so we don't double-emit network requests. bidiActive = false + /** Whether the session was created with `goog:loggingPrefs.performance`, set + * from the capabilities at session init. A perf log exists only because a + * session asked for one, so without it every fetch is a round trip that can + * only fail — and the driver's own transport prints each failure, so the + * cost is a wall of errors on a run that passed: measured on iOS Safari, + * 21 of them in a two-test run, where XCUITest lists `performance` among its + * supported types and then serves none of it. */ + perfLogsRequested = false + /** True once the collector is registered to run at document-start. Every * document then instruments and anchors itself, so the paths that exist to * notice a navigation after the fact — re-injection and the settle poll — have @@ -457,7 +466,7 @@ export class SessionCapturer extends SessionCapturerBase { */ async captureNetworkFromPerformanceLogs(browser: NightwatchBrowser) { // BiDi network inspector is the source of truth when attached. - if (this.bidiActive) { + if (this.bidiActive || !this.perfLogsRequested) { return } try { diff --git a/packages/nightwatch-devtools/tests/session.test.ts b/packages/nightwatch-devtools/tests/session.test.ts index 6f821018..085c4404 100644 --- a/packages/nightwatch-devtools/tests/session.test.ts +++ b/packages/nightwatch-devtools/tests/session.test.ts @@ -227,6 +227,7 @@ describe('SessionCapturer.captureNetworkFromPerformanceLogs', () => { getLog: vi.fn(async () => [perfMessage, finishMessage, loadingFinished]) }) const cap = makeCapturer(browser) + cap.perfLogsRequested = true await cap.captureNetworkFromPerformanceLogs(browser) expect(cap.networkRequests).toHaveLength(1) expect(cap.networkRequests[0]).toMatchObject({ @@ -243,10 +244,23 @@ describe('SessionCapturer.captureNetworkFromPerformanceLogs', () => { }) }) const cap = makeCapturer(browser) + cap.perfLogsRequested = true await expect( cap.captureNetworkFromPerformanceLogs(browser) ).resolves.toBeUndefined() }) + + // A perf log exists only because the session asked for one, so without the + // capability the fetch can only fail — once per command, each failure printed + // by the driver's own transport. Measured on iOS Safari: 21 error blocks in a + // two-test run that passed. + it('does not fetch when the session asked for no perf log', async () => { + const getLog = vi.fn() + const browser = makeMockBrowser({ getLog }) + const cap = makeCapturer(browser) + await cap.captureNetworkFromPerformanceLogs(browser) + expect(getLog).not.toHaveBeenCalled() + }) }) describe('SessionCapturer.takeScreenshotViaHttp', () => { @@ -395,6 +409,7 @@ describe('SessionCapturer.captureTrace', () => { transport: { settings: { webdriver: { host: '127.0.0.1', port } } } }) const cap = makeCapturer(browser) + cap.perfLogsRequested = true try { await cap.captureTrace(browser) // A `null` payload also sends the drain through its recovery probes, so diff --git a/packages/nightwatch-devtools/tests/sessionRearm.test.ts b/packages/nightwatch-devtools/tests/sessionRearm.test.ts index c6095bca..3530c033 100644 --- a/packages/nightwatch-devtools/tests/sessionRearm.test.ts +++ b/packages/nightwatch-devtools/tests/sessionRearm.test.ts @@ -2,7 +2,8 @@ import { describe, it, expect, vi, afterEach } from 'vitest' import type * as DevToolsCore from '@wdio/devtools-core' import { needsCaptureRearm, - rearmCaptureForSession + rearmCaptureForSession, + wantsPerformanceLog } from '../src/session-init.js' import type { SessionInitCtx } from '../src/session-init.js' import type { NightwatchBrowser } from '../src/types.js' @@ -252,3 +253,28 @@ describe('rearmCaptureForSession', () => { expect(registerCollectorPreload).not.toHaveBeenCalled() }) }) + +// The desktop examples pass `goog:loggingPrefs`, the mobile ones do not, and +// the answer decides whether every command pays for a fetch that cannot +// succeed. Read from either side of the pair because Nightwatch echoes the +// negotiated capabilities on one and the config's own on the other. +describe('wantsPerformanceLog', () => { + it('reads the capability from the negotiated side', () => { + expect( + wantsPerformanceLog({ 'goog:loggingPrefs': { performance: 'ALL' } }, {}) + ).toBe(true) + }) + + it('reads it from the desired side', () => { + expect( + wantsPerformanceLog({}, { 'goog:loggingPrefs': { performance: 'ALL' } }) + ).toBe(true) + }) + + it('is false when neither side asked — a mobile or Safari session', () => { + expect(wantsPerformanceLog({}, {})).toBe(false) + expect( + wantsPerformanceLog({ 'goog:loggingPrefs': { browser: 'ALL' } }, {}) + ).toBe(false) + }) +}) From 4dfaa909d698c19aa62e856aa74a6343e3d92a37 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 19:32:14 +0530 Subject: [PATCH 12/15] fix(examples): serve iOS web mode, and share one simulator resolver --- examples/mobile-preflight.cjs | 54 ++++++-- examples/nightwatch/mobile/android/clock.js | 14 ++- examples/nightwatch/mobile/ios/settings.js | 38 ++++++ .../nightwatch/nightwatch.mobile.conf.cjs | 30 +---- .../selenium-js/mobile/android/clock.test.js | 20 +-- .../selenium-js/mobile/ios/settings.test.js | 64 ++++++---- examples/selenium-py/mobile/android/clock.py | 21 +++- examples/selenium-py/mobile/ios/settings.py | 117 +++++++++++++----- examples/wdio/mobile/capabilities.ts | 36 +----- .../wdio/mobile/specs/ios/settings.e2e.ts | 33 +++++ 10 files changed, 280 insertions(+), 147 deletions(-) diff --git a/examples/mobile-preflight.cjs b/examples/mobile-preflight.cjs index 2a10ef2b..ce34c0c7 100644 --- a/examples/mobile-preflight.cjs +++ b/examples/mobile-preflight.cjs @@ -141,7 +141,9 @@ function bootedSimulators() { .split('\n') .filter((line) => line.includes('(Booted)')) .map((line) => { - const match = line.match(/^\s*(.+?)\s+\(([0-9A-F-]{36})\)\s+\(Booted\)/i) + const match = line.match( + /^\s*(.+?)\s+\(([0-9A-F-]{36})\)\s+\(Booted\)/i + ) return match ? { name: match[1], udid: match[2] } : null }) .filter(Boolean) @@ -156,6 +158,48 @@ const APPIUM_DOWN = (host, port) => `\nNothing is listening on ${host}:${port}, so Appium is not up.\n\n` + ` appium --address ${host} --port ${port}\n` +/** The simulator to drive, as capabilities. + * + * Always a udid, never a bare name. Naming a simulator that does not exist + * does NOT fail: the XCUITest driver CREATES it (`appiumTest--`) + * and boots it, every run, beside the one already running. So an unmatched + * `IOS_DEVICE_NAME` is refused here rather than passed through — a typo would + * otherwise pass the preflight (which only asks whether SOME simulator is + * booted) and quietly leave a new simulator behind on every run. + * + * Defaults to whatever is already booted, the iOS counterpart of attaching to + * the running Android emulator. `IOS_UDID` names one outright and is not + * checked against the booted list, so a remote or freshly created device can + * still be targeted deliberately. */ +function resolveIosDevice() { + if (process.env.IOS_UDID) { + return { 'appium:udid': process.env.IOS_UDID } + } + const booted = bootedSimulators() ?? [] + const wanted = process.env.IOS_DEVICE_NAME + if (!booted.length) { + throw new Error( + 'No iOS simulator is booted.\n' + + ' xcrun simctl list devices available\n' + + ' xcrun simctl boot ""' + ) + } + const names = booted.map((device) => device.name).join(', ') + if (wanted) { + const match = booted.find((device) => device.name === wanted) + if (!match) { + throw new Error( + `IOS_DEVICE_NAME="${wanted}" is not booted, and naming a simulator ` + + 'that does not exist makes Appium create one rather than fail.\n' + + ` booted now: ${names}\n` + + ' boot it first, or set IOS_UDID to target it deliberately.' + ) + } + return { 'appium:udid': match.udid, 'appium:deviceName': match.name } + } + return { 'appium:udid': booted[0].udid, 'appium:deviceName': booted[0].name } +} + /** * Print what is missing and exit, or return quietly. Exits rather than throws: * a thrown error inside a framework hook is reported as a test failure, which @@ -165,11 +209,6 @@ async function requireMobileToolchain({ host = process.env.APPIUM_HOST ?? '127.0.0.1', port = Number(process.env.APPIUM_PORT ?? 4723) } = {}) { - // Every check below is Android's, and so is every example flow: they drive - // the Clock app through UiAutomator resource-ids. The capability builders - // can still shape an XCUITest session, but no example has an iOS BODY — it - // would activate the Android package and then look for Android ids. Say so - // here rather than letting the run reach a lookup that cannot match. if (process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios') { // iOS has its own toolchain, so none of the Android checks below apply: // they look for ANDROID_HOME and an adb device, and a correctly set up @@ -274,5 +313,6 @@ module.exports = { requireMobileToolchain, appiumReady, adbDevices, - bootedSimulators + bootedSimulators, + resolveIosDevice } diff --git a/examples/nightwatch/mobile/android/clock.js b/examples/nightwatch/mobile/android/clock.js index 05e257fb..a5231283 100644 --- a/examples/nightwatch/mobile/android/clock.js +++ b/examples/nightwatch/mobile/android/clock.js @@ -22,6 +22,12 @@ const APP_ID = 'com.google.android.deskclock' const isWeb = process.env.DEVTOOLS_MOBILE === 'web' + +// An emulator often cannot resolve public DNS (corporate network, VPN), and +// `10.0.2.2` is its alias for the HOST's localhost — so a page served on this +// machine is reachable when the internet is not. See examples/MOBILE.md. +const WEB_URL = + process.env.DEVTOOLS_MOBILE_URL ?? 'https://the-internet.herokuapp.com/login' /** APPIUM_APP replaces Clock, so the Clock flow does not apply to it. */ const CUSTOM_APP = Boolean(process.env.APPIUM_APP) @@ -62,8 +68,8 @@ describe('Clock (native)', function () { if (isWeb) { // A mobile BROWSER session: it has a document, so every page-side call a // native session skips must still happen. That contrast is the point. - await browser.url('https://the-internet.herokuapp.com/login') - await browser.assert.urlContains('the-internet') + await browser.url(WEB_URL) + await browser.assert.urlContains('http') return } if (CUSTOM_APP) { @@ -123,8 +129,8 @@ describe('Clock (native)', function () { it('captures a second action on the same session', async function (browser) { // A second test, so `traceGranularity: 'test'` has two slices to key. if (isWeb) { - await browser.url('https://the-internet.herokuapp.com/') - await browser.assert.urlContains('herokuapp.com') + await browser.url(WEB_URL) + await browser.assert.urlContains('http') return } diff --git a/examples/nightwatch/mobile/ios/settings.js b/examples/nightwatch/mobile/ios/settings.js index 113b346a..a0d17412 100644 --- a/examples/nightwatch/mobile/ios/settings.js +++ b/examples/nightwatch/mobile/ios/settings.js @@ -11,6 +11,14 @@ // VERIFIED ON: iOS Simulator `iPhone 17 Pro`, iOS 26.5 (23F77), Xcode 26. const APP_ID = 'com.apple.Preferences' +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +/** APPIUM_APP replaces Settings, so the Settings flow does not apply to it. */ +const CUSTOM_APP = Boolean(process.env.APPIUM_APP) + +// A simulator shares the host's network stack, so `localhost` here is this +// machine — no `10.0.2.2` alias like the Android emulator needs. +const WEB_URL = + process.env.DEVTOOLS_MOBILE_URL ?? 'https://the-internet.herokuapp.com/login' /** The navigation bar's title, which is how Settings says where it is. */ async function navBarTitle(browser) { @@ -26,6 +34,24 @@ async function navBarTitle(browser) { describe('Settings (native)', function () { it('navigates into a settings page and back', async function (browser) { + if (isWeb) { + // A mobile BROWSER session: it has a document, so every page-side call a + // native session skips must still happen. That contrast is the point, + // and it costs nothing extra to set up here — Safari is driven by the + // XCUITest driver itself, where Chrome on Android needs a chromedriver. + await browser.url(WEB_URL) + await browser.assert.urlContains('http') + return + } + if (CUSTOM_APP) { + // A supplied app has none of Settings' screens, so capture its hierarchy + // rather than looking for ids that cannot exist. + await browser.assert.ok( + Boolean(await browser.source()), + 'the view hierarchy was readable' + ) + return + } // Terminated before activating, not merely activated: Settings remembers // the page the last run drilled into, so activating alone would start // somewhere unpredictable. This is what makes the spec re-runnable. @@ -57,6 +83,18 @@ describe('Settings (native)', function () { it('captures a second action on the same session', async function (browser) { // A second test, so `traceGranularity: 'test'` has two slices to key. + if (isWeb) { + await browser.url(WEB_URL) + await browser.assert.urlContains('http') + return + } + if (CUSTOM_APP) { + await browser.assert.ok( + Boolean(await browser.source()), + 'the view hierarchy was readable' + ) + return + } await browser.execute('mobile: activateApp', [{ bundleId: APP_ID }]) await browser.assert.ok( (await navBarTitle(browser)).includes('Settings'), diff --git a/examples/nightwatch/nightwatch.mobile.conf.cjs b/examples/nightwatch/nightwatch.mobile.conf.cjs index 3416730a..11bdc194 100644 --- a/examples/nightwatch/nightwatch.mobile.conf.cjs +++ b/examples/nightwatch/nightwatch.mobile.conf.cjs @@ -10,46 +10,22 @@ const path = require('node:path') const nightwatchDevtools = require('@wdio/nightwatch-devtools').default const { requireMobileToolchain, - bootedSimulators + resolveIosDevice } = require('../mobile-preflight.cjs') const isWeb = process.env.DEVTOOLS_MOBILE === 'web' const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' -/** The simulator to drive, as a udid. - * - * By udid rather than by name, because naming one that does not exist does - * NOT fail: the XCUITest driver CREATES it (`appiumTest--`) and - * boots it, every run, beside the simulator already running. - * - * Defaults to whatever is already booted — the iOS counterpart of attaching - * to the running emulator on Android. `IOS_DEVICE_NAME` picks among several, - * and `IOS_UDID` names one outright. */ -function iosDevice() { - if (process.env.IOS_UDID) { - return { 'appium:udid': process.env.IOS_UDID } - } - const booted = bootedSimulators() ?? [] - const wanted = process.env.IOS_DEVICE_NAME - const match = wanted - ? booted.find((device) => device.name === wanted) - : booted[0] - if (match) { - return { 'appium:udid': match.udid, 'appium:deviceName': match.name } - } - return { 'appium:deviceName': wanted ?? 'iPhone 17 Pro' } -} - /** The same bag the other three mobile examples build; see * examples/wdio/mobile/capabilities.ts for the annotated original. */ function mobileCapabilities() { const base = { platformName: IOS ? 'iOS' : 'Android', 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', - // Which simulator, resolved to a udid — see `iosDevice`. + // Which simulator, resolved to a udid — see `resolveIosDevice`. ...(IOS ? { - ...iosDevice(), + ...resolveIosDevice(), ...(process.env.IOS_PLATFORM_VERSION ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } : {}) diff --git a/examples/selenium-js/mobile/android/clock.test.js b/examples/selenium-js/mobile/android/clock.test.js index fc83e0b9..96f79e89 100644 --- a/examples/selenium-js/mobile/android/clock.test.js +++ b/examples/selenium-js/mobile/android/clock.test.js @@ -39,7 +39,6 @@ const APP_ID = 'com.google.android.deskclock' const isWeb = process.env.DEVTOOLS_MOBILE === 'web' /** APPIUM_APP replaces Clock, so the Clock flow does not apply to it. */ const CUSTOM_APP = Boolean(process.env.APPIUM_APP) -const IS_IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ process.env.APPIUM_PORT ?? 4723 }` @@ -48,18 +47,8 @@ const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ * examples/wdio/mobile/capabilities.ts for the annotated original. */ function mobileCapabilities() { const base = { - platformName: IS_IOS ? 'iOS' : 'Android', - 'appium:automationName': IS_IOS ? 'XCUITest' : 'UiAutomator2', - // iOS needs the simulator named, and it is per-machine: - // `xcrun simctl list devices` shows yours. - ...(IS_IOS - ? { - 'appium:deviceName': process.env.IOS_DEVICE_NAME ?? 'iPhone 15', - ...(process.env.IOS_PLATFORM_VERSION - ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } - : {}) - } - : {}), + platformName: 'Android', + 'appium:automationName': 'UiAutomator2', 'appium:noReset': true, 'appium:newCommandTimeout': 300, // selenium-webdriver's `Builder.build()` throws unless `browserName` is a @@ -77,15 +66,12 @@ function mobileCapabilities() { // Chrome on the device needs a matching chromedriver. Appium can // fetch one, but that is a SERVER feature, not a capability: // `--allow-insecure=uiautomator2:chromedriver_autodownload`. - browserName: IS_IOS ? 'safari' : 'chrome' + browserName: 'chrome' } } if (process.env.APPIUM_APP) { return { ...base, 'appium:app': process.env.APPIUM_APP } } - if (IS_IOS) { - return { ...base, 'appium:bundleId': 'com.apple.Preferences' } - } return { ...base, 'appium:appPackage': APP_ID, diff --git a/examples/selenium-js/mobile/ios/settings.test.js b/examples/selenium-js/mobile/ios/settings.test.js index b5425502..eb1d2c32 100644 --- a/examples/selenium-js/mobile/ios/settings.test.js +++ b/examples/selenium-js/mobile/ios/settings.test.js @@ -19,7 +19,7 @@ import { Builder, until } from 'selenium-webdriver' import { createRequire } from 'node:module' import { DevTools } from '@wdio/selenium-devtools' -const { requireMobileToolchain, bootedSimulators } = createRequire( +const { requireMobileToolchain, resolveIosDevice } = createRequire( import.meta.url )('../../../mobile-preflight.cjs') @@ -33,33 +33,24 @@ const APPIUM = `http://${process.env.APPIUM_HOST ?? '127.0.0.1'}:${ process.env.APPIUM_PORT ?? 4723 }` -/** The simulator to drive, as a udid. Naming one that does not exist does NOT - * fail: the XCUITest driver CREATES it and boots it, every run, beside the - * simulator already running. Defaults to whatever is already booted. */ -function iosDevice() { - if (process.env.IOS_UDID) { - return { 'appium:udid': process.env.IOS_UDID } - } - const booted = bootedSimulators() ?? [] - const wanted = process.env.IOS_DEVICE_NAME - const match = wanted - ? booted.find((device) => device.name === wanted) - : booted[0] - if (match) { - return { 'appium:udid': match.udid, 'appium:deviceName': match.name } - } - return { 'appium:deviceName': wanted ?? 'iPhone 17 Pro' } -} +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +/** APPIUM_APP replaces Settings, so the Settings flow does not apply to it. */ +const CUSTOM_APP = Boolean(process.env.APPIUM_APP) + +// A simulator shares the host's network stack, so `localhost` here is this +// machine — no `10.0.2.2` alias like the Android emulator needs. +const WEB_URL = + process.env.DEVTOOLS_MOBILE_URL ?? 'https://the-internet.herokuapp.com/login' function mobileCapabilities() { - return { + const base = { platformName: 'iOS', 'appium:automationName': 'XCUITest', - ...iosDevice(), + // Which simulator, resolved to a udid — see `resolveIosDevice`. + ...resolveIosDevice(), ...(process.env.IOS_PLATFORM_VERSION ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } : {}), - 'appium:bundleId': APP_ID, 'appium:noReset': true, 'appium:newCommandTimeout': 300, // selenium-webdriver's `Builder.build()` throws unless `browserName` is a @@ -69,6 +60,16 @@ function mobileCapabilities() { // capture. browserName: '' } + if (isWeb) { + // Names a browser, so this session HAS a document and keeps its page-side + // capture — the distinction the native guards turn on. Safari is driven by + // the XCUITest driver itself, where Chrome on Android needs a chromedriver. + return { ...base, browserName: 'safari' } + } + if (CUSTOM_APP) { + return { ...base, 'appium:app': process.env.APPIUM_APP } + } + return { ...base, 'appium:bundleId': APP_ID } } describe('Settings (native)', function () { @@ -100,6 +101,19 @@ describe('Settings (native)', function () { }) it('navigates into a settings page and back', async function () { + if (isWeb) { + // A mobile BROWSER session: it has a document, so every page-side call a + // native session skips must still happen. That contrast is the point. + await driver.get(WEB_URL) + assert.match(await driver.getCurrentUrl(), /^http/) + return + } + if (CUSTOM_APP) { + // A supplied app has none of Settings' screens, so capture its hierarchy + // rather than looking for ids that cannot exist. + assert.ok((await driver.getPageSource()).length > 0) + return + } // Terminated before activating, not merely activated: Settings remembers // the page the last run drilled into, so activating alone would start // somewhere unpredictable. This is what makes the spec re-runnable. @@ -124,6 +138,14 @@ describe('Settings (native)', function () { it('captures a second action on the same session', async function () { // A second test, so `traceGranularity: 'test'` has two slices to key. + if (isWeb) { + await driver.get(WEB_URL) + return + } + if (CUSTOM_APP) { + assert.ok((await driver.getPageSource()).length > 0) + return + } await driver.executeScript('mobile: activateApp', { bundleId: APP_ID }) assert.match(await navBarTitle(), /Settings/) }) diff --git a/examples/selenium-py/mobile/android/clock.py b/examples/selenium-py/mobile/android/clock.py index 887d1f16..dd43102b 100644 --- a/examples/selenium-py/mobile/android/clock.py +++ b/examples/selenium-py/mobile/android/clock.py @@ -75,6 +75,12 @@ def require_appium(host: str, port: str) -> None: # APPIUM_APP replaces Clock, so the Clock flow does not apply to it. CUSTOM_APP = bool(os.environ.get("APPIUM_APP")) IS_WEB = os.environ.get("DEVTOOLS_MOBILE") == "web" +# An emulator often cannot resolve public DNS (corporate network, VPN), and +# `10.0.2.2` is its alias for the HOST's localhost -- so a page served on this +# machine is reachable when the internet is not. See examples/MOBILE.md. +WEB_URL = os.environ.get( + "DEVTOOLS_MOBILE_URL", "https://the-internet.herokuapp.com/login" +) APPIUM = "http://%s:%s" % ( os.environ.get("APPIUM_HOST", "127.0.0.1"), os.environ.get("APPIUM_PORT", "4723"), @@ -122,11 +128,16 @@ def capabilities() -> dict: driver = webdriver.Remote(APPIUM, options=_options.load_capabilities(capabilities())) try: if IS_WEB: - driver.get("https://the-internet.herokuapp.com/login") - driver.find_element(By.ID, "username").send_keys("tomsmith") - driver.find_element(By.ID, "password").send_keys("SuperSecretPassword!") - driver.find_element(By.CSS_SELECTOR, 'button[type="submit"]').click() - print(driver.find_element(By.ID, "flash").text.strip()) + driver.get(WEB_URL) + if os.environ.get("DEVTOOLS_MOBILE_URL"): + # A supplied page has none of the login form, so navigating and + # capturing is all there is to do with it. + print("loaded %s" % driver.current_url) + else: + driver.find_element(By.ID, "username").send_keys("tomsmith") + driver.find_element(By.ID, "password").send_keys("SuperSecretPassword!") + driver.find_element(By.CSS_SELECTOR, 'button[type="submit"]').click() + print(driver.find_element(By.ID, "flash").text.strip()) elif CUSTOM_APP: # A supplied app has none of Clock's screens, so capture its hierarchy # rather than looking for ids that cannot exist. diff --git a/examples/selenium-py/mobile/ios/settings.py b/examples/selenium-py/mobile/ios/settings.py index d39ba9eb..f641b40b 100644 --- a/examples/selenium-py/mobile/ios/settings.py +++ b/examples/selenium-py/mobile/ios/settings.py @@ -37,6 +37,14 @@ ) APP_ID = "com.apple.Preferences" +# APPIUM_APP replaces Settings, so the Settings flow does not apply to it. +CUSTOM_APP = bool(os.environ.get("APPIUM_APP")) +IS_WEB = os.environ.get("DEVTOOLS_MOBILE") == "web" +# A simulator shares the host's network stack, so `localhost` here is this +# machine -- no `10.0.2.2` alias like the Android emulator needs. +WEB_URL = os.environ.get( + "DEVTOOLS_MOBILE_URL", "https://the-internet.herokuapp.com/login" +) APPIUM = "http://%s:%s" % ( os.environ.get("APPIUM_HOST", "127.0.0.1"), os.environ.get("APPIUM_PORT", "4723"), @@ -65,22 +73,40 @@ def booted_simulators(): def ios_device(): """The simulator to drive, as a udid. - By udid rather than by name, because naming one that does not exist does - NOT fail: the XCUITest driver CREATES it and boots it, every run, beside - the simulator already running. Defaults to whatever is already booted. + Always a udid, never a bare name. Naming a simulator that does not exist + does NOT fail: the XCUITest driver CREATES it and boots it, every run, + beside the one already running. So an unmatched IOS_DEVICE_NAME is refused + here rather than passed through -- a typo would otherwise pass the + preflight (which only asks whether SOME simulator is booted) and quietly + leave a new simulator behind on every run. + + Defaults to whatever is already booted. IOS_UDID names one outright and is + not checked against the booted list, so a remote or freshly created device + can still be targeted deliberately. This mirrors `resolveIosDevice` in + examples/mobile-preflight.cjs, which the three JS examples share. """ if os.environ.get("IOS_UDID"): return {"appium:udid": os.environ["IOS_UDID"]} booted = booted_simulators() wanted = os.environ.get("IOS_DEVICE_NAME") - match = None + if not booted: + raise SystemExit( + "\nNo iOS simulator is booted.\n" + " xcrun simctl list devices available\n" + ' xcrun simctl boot ""\n' + ) if wanted: match = next((d for d in booted if d[0] == wanted), None) - elif booted: - match = booted[0] - if match: + if not match: + raise SystemExit( + '\nIOS_DEVICE_NAME="%s" is not booted, and naming a simulator ' + "that does not exist makes Appium create one rather than " + "fail.\n booted now: %s\n" + " boot it first, or set IOS_UDID to target it deliberately.\n" + % (wanted, ", ".join(d[0] for d in booted)) + ) return {"appium:udid": match[1], "appium:deviceName": match[0]} - return {"appium:deviceName": wanted or "iPhone 17 Pro"} + return {"appium:udid": booted[0][1], "appium:deviceName": booted[0][0]} def require_simulator(): @@ -97,7 +123,6 @@ def capabilities(): base = { "platformName": "iOS", "appium:automationName": "XCUITest", - "appium:bundleId": APP_ID, "appium:noReset": True, "appium:newCommandTimeout": 300, } @@ -105,6 +130,16 @@ def capabilities(): version = os.environ.get("IOS_PLATFORM_VERSION") if version: base["appium:platformVersion"] = version + if IS_WEB: + # Names a browser, so this session HAS a document and keeps its + # page-side capture -- the distinction the native guards turn on. + # Safari is driven by the XCUITest driver itself, where Chrome on + # Android needs a matching chromedriver. + base["browserName"] = "Safari" + elif CUSTOM_APP: + base["appium:app"] = os.environ["APPIUM_APP"] + else: + base["appium:bundleId"] = APP_ID return base @@ -126,31 +161,45 @@ def nav_bar_title(): try: - # Terminated before activating, not merely activated: Settings remembers - # the page the last run drilled into, so activating alone would start - # somewhere unpredictable. This is what makes the script re-runnable. - driver.execute_script("mobile: terminateApp", {"bundleId": APP_ID}) - driver.execute_script("mobile: activateApp", {"bundleId": APP_ID}) - opened = nav_bar_title() - assert "Settings" in opened, 'Settings opened at "%s"' % opened - - driver.find_element("accessibility id", "General").click() - for _ in range(30): - if "General" in nav_bar_title(): - break - time.sleep(0.5) - assert "General" in nav_bar_title(), "Settings did not navigate to General" - - # Back through the navigation stack rather than a tap on the back button: - # that button's accessibility id is the PARENT page's title, so tapping by - # name hits whichever row happens to share it -- measured, it opened About. - driver.back() - for _ in range(30): - if "Settings" in nav_bar_title(): - break - time.sleep(0.5) - assert "Settings" in nav_bar_title(), "Settings did not navigate back" - print("navigated into General and back") + if IS_WEB: + # A mobile BROWSER session: it has a document, so every page-side call + # a native session skips must still happen. That contrast is the point. + driver.get(WEB_URL) + assert driver.current_url.startswith("http"), driver.current_url + print("loaded %s" % driver.current_url) + elif CUSTOM_APP: + # A supplied app has none of Settings' screens, so capture its + # hierarchy rather than looking for ids that cannot exist. + assert driver.page_source, "the view hierarchy was empty" + print("captured the supplied app's hierarchy") + else: + # Terminated before activating, not merely activated: Settings + # remembers the page the last run drilled into, so activating alone + # would start somewhere unpredictable. This makes the script + # re-runnable. + driver.execute_script("mobile: terminateApp", {"bundleId": APP_ID}) + driver.execute_script("mobile: activateApp", {"bundleId": APP_ID}) + opened = nav_bar_title() + assert "Settings" in opened, 'Settings opened at "%s"' % opened + + driver.find_element("accessibility id", "General").click() + for _ in range(30): + if "General" in nav_bar_title(): + break + time.sleep(0.5) + assert "General" in nav_bar_title(), "Settings did not reach General" + + # Back through the navigation stack rather than a tap on the back + # button: that button's accessibility id is the PARENT page's title, so + # tapping by name hits whichever row shares it -- measured, it opened + # About. + driver.back() + for _ in range(30): + if "Settings" in nav_bar_title(): + break + time.sleep(0.5) + assert "Settings" in nav_bar_title(), "Settings did not navigate back" + print("navigated into General and back") finally: driver.quit() devtools.wait_for_dashboard_close() # hold the UI open to inspect diff --git a/examples/wdio/mobile/capabilities.ts b/examples/wdio/mobile/capabilities.ts index 09e64aa2..fb38028c 100644 --- a/examples/wdio/mobile/capabilities.ts +++ b/examples/wdio/mobile/capabilities.ts @@ -21,40 +21,12 @@ const CLOCK_APP = { ios: { 'appium:bundleId': 'com.apple.Preferences' } } -const { bootedSimulators } = createRequire(import.meta.url)( +const { resolveIosDevice } = createRequire(import.meta.url)( '../../mobile-preflight.cjs' -) as { bootedSimulators: () => { name: string; udid: string }[] | null } +) as { resolveIosDevice: () => Record } const IOS = process.env.DEVTOOLS_MOBILE_PLATFORM === 'ios' -/** The simulator to drive, as a udid. - * - * By udid rather than by name, because naming one that does not exist does - * NOT fail: the XCUITest driver CREATES it (`appiumTest--`) and - * boots it, every run, beside the simulator already running. The default name - * used to be a device Xcode no longer ships, so every iOS run left another - * simulator behind. - * - * Defaults to whatever is already booted — the iOS counterpart of attaching - * to the running emulator on Android. `IOS_DEVICE_NAME` picks among several, - * and `IOS_UDID` names one outright. */ -function iosDevice(): Record { - if (process.env.IOS_UDID) { - return { 'appium:udid': process.env.IOS_UDID } - } - const booted = bootedSimulators() ?? [] - const wanted = process.env.IOS_DEVICE_NAME - const match = wanted - ? booted.find((device) => device.name === wanted) - : booted[0] - if (match) { - return { 'appium:udid': match.udid, 'appium:deviceName': match.name } - } - // Nothing booted: fall back to naming one, which is the only thing left — - // and the preflight has already said so before reaching here. - return { 'appium:deviceName': wanted ?? 'iPhone 17 Pro' } -} - export function mobileCapabilities(): Record { const web = process.env.DEVTOOLS_MOBILE === 'web' const app = process.env.APPIUM_APP @@ -62,10 +34,10 @@ export function mobileCapabilities(): Record { return { platformName: IOS ? 'iOS' : 'Android', 'appium:automationName': IOS ? 'XCUITest' : 'UiAutomator2', - // Which simulator, resolved to a udid — see `iosDevice`. + // Which simulator, resolved to a udid — see `resolveIosDevice`. ...(IOS ? { - ...iosDevice(), + ...resolveIosDevice(), ...(process.env.IOS_PLATFORM_VERSION ? { 'appium:platformVersion': process.env.IOS_PLATFORM_VERSION } : {}) diff --git a/examples/wdio/mobile/specs/ios/settings.e2e.ts b/examples/wdio/mobile/specs/ios/settings.e2e.ts index 16895d70..73a55122 100644 --- a/examples/wdio/mobile/specs/ios/settings.e2e.ts +++ b/examples/wdio/mobile/specs/ios/settings.e2e.ts @@ -23,6 +23,15 @@ import { expect } from '@wdio/globals' /** Settings. Present on every simulator and device, unlike Clock. */ const APP_ID = 'com.apple.Preferences' +const isWeb = process.env.DEVTOOLS_MOBILE === 'web' +/** APPIUM_APP replaces Settings, so the Settings flow does not apply to it. */ +const CUSTOM_APP = Boolean(process.env.APPIUM_APP) + +// A simulator shares the host's network stack, so `localhost` here is this +// machine — no `10.0.2.2` alias like the Android emulator needs. +const WEB_URL = + process.env.DEVTOOLS_MOBILE_URL ?? 'https://the-internet.herokuapp.com/login' + /** The navigation bar's title, which is how Settings says where it is. */ async function navBarTitle(): Promise { const bars = await $$('XCUIElementTypeNavigationBar').getElements() @@ -31,6 +40,21 @@ async function navBarTitle(): Promise { describe('Settings (native)', () => { it('navigates into a settings page and back', async () => { + if (isWeb) { + // A mobile BROWSER session: it has a document, so every page-side call a + // native session skips must still happen. That contrast is the point, + // and it costs nothing extra to set up here — Safari is driven by the + // XCUITest driver itself, where Chrome on Android needs a chromedriver. + await browser.url(WEB_URL) + await expect(browser).toHaveUrl(expect.stringContaining('http')) + return + } + if (CUSTOM_APP) { + // A supplied app has none of Settings' screens, so capture its hierarchy + // rather than looking for ids that cannot exist. + expect((await browser.getPageSource()).length).toBeGreaterThan(0) + return + } // Terminated before activating, not merely activated: Settings remembers // the page the last run drilled into, so activating alone would start // somewhere unpredictable. This is what makes the spec re-runnable — the @@ -57,6 +81,15 @@ describe('Settings (native)', () => { it('captures a second action on the same session', async () => { // A second test, so `traceGranularity: 'test'` has two slices to key. + if (isWeb) { + await browser.url(WEB_URL) + return + } + if (CUSTOM_APP) { + expect((await browser.getPageSource()).length).toBeGreaterThan(0) + return + } + await browser.execute('mobile: activateApp', { bundleId: APP_ID }) await expect(await navBarTitle()).toBe('Settings') }) From c8554f3a97bca43d4d46c64e29d82f790c5e7c88 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Wed, 23 Sep 2026 19:32:28 +0530 Subject: [PATCH 13/15] docs: iOS web mode, and how the simulator is chosen --- CLAUDE.md | 3 ++- README.md | 4 ++-- examples/MOBILE.md | 22 +++++++++++++++++++--- examples/README.md | 12 +++++++----- 4 files changed, 30 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 2910f482..310ed4b7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,7 +27,7 @@ Run from repo root unless noted. | `pnpm test:coverage` | Run vitest with v8 coverage. The thresholds in `vitest.config.ts` are aspirational, not a gate: that file states CI does not run this and the suite is currently below all four. CI runs `test`/`lint`/`test:ui`. | | `pnpm lint` | Lint all packages in parallel. Includes `eslint-plugin-security` for a subset of CodeQL findings; deeper taint-flow checks surface on the PR's CodeQL scan. | | `pnpm demo:wdio` / `pnpm demo:nightwatch` / `pnpm demo:selenium` | Run the per-framework example projects. Useful for manual verification of UI or runtime changes. | -| `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and drive the Clock app that ships with every Android system image — starting a timer, pausing it, clearing it — so a native example needs no `.apk`. `DEVTOOLS_MOBILE_PLATFORM=ios` runs the iOS spec instead — all four adapters — which drives Settings because the simulator ships no Clock, from a separate spec per platform rather than a branch. The simulator is chosen by udid and defaults to whichever is already booted; naming one that does not exist makes the XCUITest driver create and boot it, every run. `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches; `DEVTOOLS_MODE=trace` flips any demo to trace mode. | +| `pnpm demo:wdio:mobile` / `:selenium:mobile` / `:nightwatch:mobile` / `:python:mobile` | The same, against Appium. All four build the same capability bag and drive the Clock app that ships with every Android system image — starting a timer, pausing it, clearing it — so a native example needs no `.apk`. `DEVTOOLS_MOBILE_PLATFORM=ios` runs the iOS spec instead — all four adapters — which drives Settings because the simulator ships no Clock, from a separate spec per platform rather than a branch. The simulator is chosen by udid and defaults to whichever is already booted, and an unmatched `IOS_DEVICE_NAME` is refused: naming one that does not exist makes the XCUITest driver create and boot it, every run, rather than fail. `DEVTOOLS_MOBILE=web` drives the device's own browser on both platforms — Chrome on Android, Safari on iOS, which the XCUITest driver serves without a chromedriver. `examples/MOBILE.md` holds the prerequisites and the `DEVTOOLS_MOBILE` / `APPIUM_APP` switches; `DEVTOOLS_MODE=trace` flips any demo to trace mode. | | `pnpm dev` | Run all packages in parallel dev mode. | `selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:mocha:allure` / `:jest` / `:cucumber`. @@ -298,6 +298,7 @@ Documented divergences from the conventions above. They exist today as debt to b - **A preload dies with its session, and Nightwatch replaces sessions without saying so.** Registration used to run once at bringup, so a mid-run `browser.end()` left sessions 2..N with neither preload nor BiDi: measured 1 registration and 1 attach for a whole run, 4-6 `Collector missing … re-injecting` recoveries, and — because `bidiActive` stayed `true` from the dead session and gates the perf-log fallback off — **zero network capture after the rotation** (75 requests in session 1, 0 in session 2). `session-init.ts` `rearmCaptureForSession` re-arms both off the same command-hook detection the screencast rotation uses (`armedSessionId`, stamped before the first await so the command flood latches out), clearing `preloadRegistered`/`bidiActive` at detection so the fallbacks are open in the gap. After: 2 registrations, 2 attaches, 0-1 recoveries, 125 network entries (75 + 50), and the accumulated stream untouched — 21 action rows split 12/9 across the two sessions, identical to before. It deliberately does **not** rebuild the `SessionCapturer` — see the entry below, which made that the rule for every path rather than just this one. - The preload registers **before** the BiDi attach, sequentially: behind the attach it lost the race it exists to win (the network subscribe took 6.2 s while the triggering command navigated, so registration landed after its own document was born), and `Promise.all` races selenium-webdriver's unsynchronized `getBidi()` cache into two websockets of which `quit()` closes one. - Residual: detection is at command invocation and chromedriver serialises the handshake behind the in-flight command, so the **first navigation after a rotation** can still land before registration and falls back to `