iOS Simulator: Maestro engine and agent-device is/get/alert disagree depending on the bound snapshot backend (Simulator AX vs XCTest)

[vynci]

Summary

On the iOS Simulator, agent-device's own is / get / alert commands and the bundled Maestro engine (test --maestro) disagree about what is on screen, and which answer Maestro gives depends on which snapshot backend the session happens to bind. When the banner reads Simulator AX snapshot inapplicable (system-surface-host-lingering); used XCTest, which read app content, every flow passes. On a freshly booted simulator (Simulator AX bound) the same flows, same commit, fail on the same five kinds of element every run. The app is not involved: agent-device is visible 'id="members.count"' passes and agent-device alert prints Sign out? at the same second Maestro's assertVisible / tapOn against the same surfaces find nothing.

Filed as a concrete symptom for the convergence work in #2188 / #2274 (the "no fallback producer" note there matches what we see).

Environment

• agent-device 0.21.3 (npx agent-device@latest), headless, no OAuth
• macOS 26.6.2, Xcode 27.0, iOS 26.5 simulator (iPhone 17 Pro), also seen on iOS 26.3
• Expo SDK 57 development build (React Native 0.86, New Architecture, Hermes), @expo/ui (SwiftUI) lists and sheets, expo-router
• Flows run with agent-device test --maestro <dir> --platform ios; assertions/taps by testID only

What vanishes from the Simulator-AX tree (and not from XCTest)

1. A native Alert.alert's buttons. agent-device alert prints the alert (Sign out?), Maestro - tapOn: "Sign Out" matches nothing. Under XCTest the same step passes.
2. An out-of-process system surface (PHPickerViewController): the picker is plainly on screen ("Collections" and all), extendedWaitUntil visible: "Collections" times out.
3. Fields inside a pushed presentation: "formSheet" route (expo-router native stack) — intermittently.
4. A non-interactive leaf Text inside a FlatList ListHeaderComponent: agent-device get text 'id="members.count"' returns 4 members, Maestro assertVisible: id: "members.count" fails.
5. Anything inside a native map annotation (MapLibre).

A SwiftUI BottomSheet's contents are not affected — they resolve under both backends — so this is about surfaces presented outside the app's own host, not sheets as such.

Repro (minimal)

Any RN/Expo app with a screen that has a Text with testID="members.count" in a list header and a row that calls Alert.alert("Sign out?", ..., [{text: "Sign Out"}]).

1. xcrun simctl shutdown <udid>; xcrun simctl boot <udid> (a fresh boot binds Simulator AX).
2. agent-device open <bundle> --platform ios --foreground, navigate to the screen.
3. In one second, run:

   agent-device is visible 'id="members.count"' --platform ios     # passes
   agent-device get text 'id="members.count"' --platform ios       # prints the text
   agent-device alert --platform ios                                # prints "Sign out?" once the row was tapped
   agent-device test --maestro flow.yaml --platform ios             # assertVisible id=members.count / tapOn "Sign Out" FAIL
   

4. Use the simulator for a while (or let a long session drift) until the snapshot banner reports Simulator AX snapshot inapplicable (system-surface-host-lingering); used XCTest — the same flow now passes. agent-device prepare ios-runner does not change which backend is preferred.

Observed over 9 consecutive runs at 6–8/9 flows passing under XCTest, then 5 consecutive runs at 5–6/9 under Simulator AX, same commit, the same three flows failing each time.

Expected

Maestro's visibility predicate and agent-device is/get/alert should agree, or test --maestro should fall back to the XCTest producer for a selector the Simulator-AX tree cannot resolve (the same way is/get evidently do). Failing that, a way to pin the backend for a run.

Why it matters

Adapting flows means deleting exactly the assertions that matter most (the sign-out confirmation, the picker, the member count), and rebooting the simulator to "clean up" before a gate run makes the gate fail. Teams end up documenting "do not reboot the simulator", which is not a real fix.

[thymikee]

Triage

Reproduced and root-caused one class here; the out-of-process classes still need evidence.

Root cause (fixed in #2643). The interactive fold suppresses an Other wrapper that carries only
an identifier, delegating that identifier to the wrapper's content. A React Native testID wrapper
has no content in the accessibility tree — the text lands beside it, not inside it, on both producers
— so suppression left the identifier with no representative at all, and Maestro reads its visible set
from that presentation while is/get resolve from the capture. What made it backend-dependent is
that the suppression gate reads hittable === true: the Apple runner answers hittability for every
node (true on screen, false off screen), and the Simulator AX bridge answers it for none. Same
screen, same installed app, the two raw trees are node-for-node identical (165 nodes) and differ only
by the presence of hittable.

Suppression now needs the node's own declared hittable: false. After that change, snapshot -i over
both producers publishes the same 18 identifier hooks, and the Maestro flow passes where it previously
failed with Maestro visible condition did not match.

Still open, and what would unblock it. Classes you list as PHPickerViewController and MapLibre
annotation content are probably not this rule: that content may be owned by another process and
therefore absent from an app-process tree, which no presentation change can recover. To tell those
apart, please attach from one failing screen each, run twice (once bound to Simulator AX, once served
by XCTest):

agent-device snapshot --raw --json
agent-device snapshot -i --json

Alert.alert was re-checked against the same simulator today and resolves under Simulator AX, so if
that class still fails for you the captures above will say why. The intermittent formSheet fields
and the non-interactive leaf Text in a ListHeaderComponent may or may not be the fixed rule — the
same captures settle it.