Skip to content

fix(ios): Simulator AX bridge reports enabled from the NotEnabled trait - #2644

Merged
thymikee merged 4 commits into
callstack:mainfrom
okwasniewski:fix/ios-bridge-enabled-state
Sep 17, 2026
Merged

thymikee merged 4 commits into
callstack:mainfrom
okwasniewski:fix/ios-bridge-enabled-state

Conversation

@okwasniewski

@okwasniewski okwasniewski commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

The Simulator AX bridge requested no state attribute, so a bridge-served snapshot never carried enabled. A disabled control read as a plain button while the XCTest runner answered the same screen with enabled: false (see #2638).

The guest now requests XC_kAXXCAttributeTraits, and tree.ts derives enabled from UIAccessibilityTraitNotEnabled (bit 8). A missing word leaves enabled unknown; anything but a digit string is traits-invalid. Source version v1.6.0 rebuilds cached bridges.

Review follow-ups: foldedHittability presents a source-declared disabled node as hittable: false even without hittability evidence (runner parity; tap dispatch unchanged). The navigation title rule retracts that folded hittable from the Button it promotes, and the stored replacement is now authoritative so later patches cannot resurrect a retracted fact. The traits word travels as a decimal string and is parsed exactly. Both presentation effects are in the CHANGELOG.

13 files touched.

Validation

Tested at ee22c24.

  • vitest: snapshot-source, ios-snapshot-engine, the size ratchet, and test/integration/provider-scenarios: 376 passed. pnpm lint, pnpm typecheck, pnpm build clean.
  • pnpm check:affected --run: one failure in platform-web/src/agent-browser-lifecycle.test.ts, untouched here (its fake pid 101 is a live process on this host). CI Repo Guards failed once on the tmpdir-model timing test; it passes 24/24 locally and no script changed. Please rerun.
  • Live, iPhone 17 Pro simulator (iOS 26.5), React Native Pressable disabled: bridge-served raw snapshot reports enabled: false on both disabled controls, regular projection adds hittable: false, and a client toBeDisabled assertion that failed on 0.21.1 passes. Recording and snapshot JSON: https://github.com/okwasniewski/agent-device/releases/tag/evidence/pr-2644.

selected (trait bit 3) is a follow-up.

Copilot AI lite review requested due to automatic review settings September 16, 2026 09:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Disabled bridge controls may still be dispatched as taps.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds Simulator AX trait capture to expose disabled-state information consistently with XCTest.

Changes:

  • Requests and decodes UIAccessibilityTraitNotEnabled, including large trait values.
  • Updates protocol versioning, fixtures, tests, native bridge requests, and changelog.
File summaries
File Summary
packages/platform-apple/src/snapshot-source/tree.ts Decodes enabled; moderate (3 votes): the fact does not reach bridge tappability guards, so disabled controls may still be tapped.
packages/platform-apple/src/snapshot-source/tree.test.ts Adds trait validation tests; nit (1 vote): assert failureCode directly instead of matching error text.
packages/platform-apple/src/snapshot-source/protocol.ts Updates protocol attributes and source version.
packages/platform-apple/src/snapshot-source/protocol.test.ts Verifies native and TypeScript vocabulary alignment.
packages/platform-apple/src/snapshot-source/fixtures/wire-vocabulary.json Updates wire vocabulary fixtures.
CHANGELOG.md Documents the behavior change.
apple/snapshot-bridge/SnapshotBridgeRuntime.m Requests accessibility traits from the native bridge.
Review details

Suppressed comments (1)

packages/platform-apple/src/snapshot-source/tree.test.ts:174

  • These new assertions identify the failure by matching rendered error text (/traits-invalid/) instead of the SnapshotSourceError.failureCode field. That couples the regression test to message formatting; assert the typed failureCode === 'traits-invalid' as the neighboring protocol tests do, so wording can evolve independently.
  assert.throws(() => decode('256'), /traits-invalid/);
  assert.throws(() => decode(1.5), /traits-invalid/);
  assert.throws(() => decode(-1), /traits-invalid/);
  • Files reviewed: 7/7 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/platform-apple/src/snapshot-source/tree.ts
Copilot AI review requested due to automatic review settings September 16, 2026 09:21
@okwasniewski

Copy link
Copy Markdown
Contributor Author

Live evidence (4eba74c)

Recording of the patched build driving the React Native disabled-state repro on an iPhone 17 Pro simulator (iOS 26.5), simulator on the left, test runner log on the right:

Video: https://github.com/okwasniewski/agent-device/releases/download/evidence/pr-2644/proof-pr-2644.mp4

frame

What it shows:

  1. behavior.e2e.ts taps both disabled buttons and the enabled one. Counters end at 1 / 0 / 0, so the app really disables them.
  2. disabled.e2e.ts asserts toBeEnabled() / toBeDisabled() through a client of the snapshot. Both pass. On 0.21.1 the same two tests fail with observed: default states because the bridge tree carried no enabled.

Bridge-served snapshots from the same session (nodes carry role/subrole/pid, the bridge shape; a v1.6.0 bridge binary was compiled into ~/.agent-device/snapshot-source):

raw     disabled-button           enabled: false
raw     explicit-disabled-button  enabled: false
regular disabled-button           enabled: false, hittable: false
regular explicit-disabled-button  enabled: false, hittable: false
regular enabled-button            enabled: true   (no hittable key, evidence unavailable)

Full JSON: snap-raw-final.json, snap-regular.json. The hittable: false on the regular projection is the second commit answering the Copilot review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The native trait fixture is missing, and the summary still reports 7 instead of 9 reviewed files.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

packages/capture-kit/src/ios-snapshot-engine/geometry.ts:142

  • The PR summary says this change touches 7 files, but the supplied diff contains 9 changed files, including both protocol.test.ts and geometry.ts in addition to the listed groups. Please correct the touched-file count so the summary matches the actual scope.

apple/snapshot-bridge/SnapshotBridgeRuntime.m:295

  • The native fixture suite exercises this runtime, but none of its snapshots includes XC_kAXXCAttributeTraits; the vocabulary test only checks that the literal is present. A regression in the requested attribute's NSNumber-to-wire conversion could therefore drop enabled while the TypeScript tests still pass. Please add a native-runtime fixture case containing both 256 and the 2^53 trait word and assert they survive the bridge response.
    kAttributeTraits,

packages/capture-kit/src/ios-snapshot-engine/engine.test.ts:327

  • The PR summary says this change touches 7 files, but the diff also includes packages/capture-kit/src/ios-snapshot-engine/geometry.ts and engine.test.ts, for 9 files total. Please update the touched-file count so the summary and validation scope accurately describe the change.
test('a source-declared disabled node is not actionable without hittability evidence', () => {
  • Files reviewed: 9/9 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Copilot AI review requested due to automatic review settings September 16, 2026 09:35
@okwasniewski

Copy link
Copy Markdown
Contributor Author

Integration Tests on 4eba74c caught a real consequence of the fold change: the navigation title rule promotes a disabled title field to an enabled Button, and that Button inherited the folded hittable: false. Fixed in d1ffcb9: a replacement patch may retract a fact by setting it undefined, and the rule retracts hittable; presentation.test.ts covers the promoted row. test/integration/provider-scenarios passes locally (210/210) and the live repro still passes on the rebuilt CLI.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The trait bit mask is incorrect, and sequential replacement patches can restore retracted fields.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (2)

packages/capture-kit/src/ios-snapshot-engine/geometry.ts:142

  • This changes a pure geometry invariant, but the new assertion is only a single fixed engine case. Extend the existing fast-check/property coverage to generate unavailable-hittability acquisitions with both enabled states and assert that enabled: false never produces hittable: true; otherwise a regression to the old early-return behavior would still pass the property suite.
    return sourceHittable === false || !enabled ? { hittable: false } : {};

packages/platform-apple/src/snapshot-source/tree.test.ts:168

  • The new traits parser is a pure boundary with an unsafe-integer/BigInt edge, but coverage is limited to these hand-picked values. Add a fast-check or parser-fuzz property around 2^53 (and invalid numeric forms) so the bit-8 derivation and typed rejection cannot regress silently.
  const buttonTrait = 1;
  const notEnabledTrait = 256;
  const toggleButtonTrait = 2 ** 53;
  assert.equal(decode(buttonTrait)?.enabled, true);
  assert.equal(decode(buttonTrait + notEnabledTrait)?.enabled, false);
  • Files reviewed: 12/12 changed files
  • Comments generated: 2
  • Review effort level: Lite

* `UIAccessibilityTraitNotEnabled`, the trait UIKit sets on a disabled control. The runner path
* answers `enabled: false` for the same node, so the bridge derives the fact from this bit.
*/
const NOT_ENABLED_TRAIT = 1n << 8n;
Comment thread packages/capture-kit/src/ios-snapshot-engine/tree.ts
@thymikee

Copy link
Copy Markdown
Member

Reviewed at d1ffcb9.

patched deletes keys whose patch value is undefined from the stored replacement, but currentReplacement (tree.ts:216-221) rebuilds the node as {...node, ...replacements.get(index)}, so a later mergeReplacement/updateReplacement on the same node brings the source value back. Rules that run after web.ts and transitions.ts — scroll.ts:129 (viewport clip rect), rows.ts, noise-overlay.ts — patch the same nodes. Before this PR, web.ts:43's value: undefined retraction for a Heading survived because plain spread keeps an undefined-valued key; now it resurfaces once a scroll ancestor clips that Heading's rect. The new nav-title hittable: undefined has the same leak if a later rule patches the field's rect. Concretely: a clipped web Heading shows its source value again on the XCTest and bridge paths, changing get/is/Maestro text reads, and a retracted hittable: false on the promoted nav-title Button can come back under a clipping ancestor. Would currentReplacement returning replacements.get(node.index) ?? node (matching semantic-index.ts:70 and scroll.ts:21/70/198) fix this without touching patched, or should patched be dropped in favor of plain spread as web.ts already uses? Either way, a regression that applies two patches in sequence — a web Heading, then a later rect clip from scroll.ts — should end with no value.

packages/capture-kit/src/ios-snapshot-engine/presentation.test.ts:531 adds 54 lines to a file already over the size-ratchet limit (994 -> 1048), and that is the one test failing in Coverage. Could the new navigation-title test and the sequential-patch regression move into a new transitions.test.ts that mirrors transitions.ts, so presentation.test.ts stays at or below 994 lines?

The navigation-title rule in transitions.ts:36 also fires on the XCTest runner path, where the disabled field already folds to hittable: false. Retracting it there changes runner-path output: the promoted Button now carries no hittable fact, so maestro runtime-targets.ts:186 counts it as an atomic-dispatch candidate (0 -> 1) and snapshot-evidence.ts:101's interactiveNodeCount rises; on the bridge path disabled targets at runtime-targets.ts:186 drop from 1 to 0. The CHANGELOG only covers the bridge enabled fact, and the PR body says tap dispatch is unchanged, so the runner-path routing change and the Maestro eligibility shift aren't documented anywhere. Can the CHANGELOG state both effects explicitly, or should the retraction be restricted to captures where hittability is unavailable?

In platform-apple's tree.ts:309, the guest forwards the uint64 trait word as an NSNumber, and protocol.ts:183 decodes it with JSON.parse into a double; Number.isInteger accepts any double at or above 2^53, and BigInt(value) only stops the 32-bit & truncation — it can't restore precision the parse already lost. A private trait bit at 2^60 or higher would land in a double whose spacing is 256 or more, so bit 8 could round away or pick up a carry from a lower byte; the test's 2**53 value is exact and doesn't exercise this. I could not confirm any iOS trait actually sets a bit that high, so this may be moot. If it's a real risk, should the guest emit the traits word as a decimal string for exact BigInt parsing, or should the decoder treat values at or above 2^60 as enabled-unknown, with a test above that threshold?

Coverage fails on exactly one test, the test-file-size ratchet on presentation.test.ts (994 -> 1048), and the 54 lines this PR adds cause it.

The retraction leak comes from reading the rule order and currentReplacement; I did not run a test with a clipped web Heading. The trait-precision point depends on whether iOS ever sets private trait bits at 2^60 or higher, which I could not enumerate.

The replacement-retraction leak and moving the new test out of presentation.test.ts both need to be resolved before this is ready to merge.

Copilot AI review requested due to automatic review settings September 17, 2026 08:14
@okwasniewski

Copy link
Copy Markdown
Contributor Author

Thanks Michał, all four landed in 9d54c0b.

  1. Retraction leak. Confirmed by reading it the way you did: currentReplacement re-spread the source under the stored replacement, so a scroll clip after web.ts brought a Heading's value back, and a rect patch after transitions.ts would bring hittable back. Took your first option: the stored replacement is authoritative (replacements.get(index) ?? node), matching semantic-index and scroll. patched stays so retracted keys are absent rather than undefined. tree.test.ts applies a Heading retraction, then a rect clip, then a label update, and ends with no value.
  2. Ratchet. presentation.test.ts is back to 991 lines (base 994); the navigation title test lives in a new transitions.test.ts and the ratchet test passes locally.
  3. Runner-path effect. Kept the retraction on both paths, since the Button already claimed enabled: true and the row is the affordance, and documented both effects in the CHANGELOG: bridge disabled controls leave the interactive and atomic-dispatch counts; the runner-path title Button joins them.
  4. Trait precision. Took your first option: the guest emits the traits word as a decimal string (NSNumber.stringValue) and the decoder parses it with BigInt, rejecting anything but digits. Tests cover 2^53 and a word past 2^60 with and without bit 8, plus a low byte of 0xFF to show no carry. For the record I dumped every exported UIAccessibilityTrait* from the iOS 26.5 simulator UIKitCore: the highest are SupportsZoom (2^46) and ToggleButton (2^53), so nothing sits at 2^60 today; the string keeps it exact regardless.

Live: a third bridge binary compiled from the new source, and the bridge-served raw snapshot still reports enabled: false on both disabled controls; the customer's three tests pass on the rebuilt CLI.

The bridge requested no state attribute, so a disabled control read as a plain
button and clients could not observe it; the XCTest runner answered the same
screen with `enabled: false`. The guest now reads `XC_kAXXCAttributeTraits`
and the decoder derives `enabled` from `UIAccessibilityTraitNotEnabled`. The
word is masked as a BigInt because `UIAccessibilityTraitToggleButton` is 2^53,
so every switch carries a value past the safe integer range; a missing word
leaves `enabled` unknown and a non-integer or negative one is a malformed tree.
Source version v1.6.0 rebuilds the cached bridge.

Verified on an iPhone 17 Pro simulator (iOS 26.5) against a React Native app
with `Pressable disabled` and `accessibilityState={{ disabled: true }}`: the
bridge-served raw snapshot now carries `enabled: false` on both controls, and a
client `toBeDisabled` assertion that failed on 0.21.1 passes.
…ut hittability evidence

Review found the new `enabled` fact never reached presentation on the bridge
path: hittability evidence is unavailable there, so the fold returned before
consulting `enabled`, and a disabled control kept an open `hittable`. The fold
now marks a node the source declared disabled as `hittable: false` regardless
of evidence availability, which is what the runner path already presents.

The tree test asserts the typed `failureCode` instead of matching error text.
… hittability

The navigation title rule presents a disabled title field as an enabled
Button for the whole row, but the fold had already marked the field
`hittable: false` from its disabled state, so the Button carried a claim
it contradicted; the cloud iOS provider scenario caught it. A replacement
patch may now retract a fact by setting it `undefined`, and the rule
retracts `hittable`, leaving the row's actionability to the evidence the
acquisition actually has.
…aits word exactly

Review (thymikee): `currentReplacement` rebuilt a node from its source under
the stored replacement, so a later patch on the same node — a scroll clip, a
row label — brought back a fact an earlier rule had retracted: the web
Heading's `value`, and the promoted navigation title's `hittable`. The stored
replacement is now authoritative, as every other reader of the map already
treats it; a regression applies two patches in sequence.

The guest sends the accessibility traits word as a decimal string and the
decoder parses it with BigInt, so a bit set past 2^53 cannot round into or out
of the NotEnabled bit on the way through JSON; the test covers a word past
2^60. The navigation title test moves to transitions.test.ts, keeping
presentation.test.ts under the size ratchet. The CHANGELOG names the two
presentation effects: bridge disabled controls leave the interactive and
Maestro atomic-dispatch counts, and the runner-path title Button joins them.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Trait decoding has unresolved correctness issues, and native serialization lacks coverage for exact large trait values.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (3)

apple/snapshot-bridge/SnapshotBridgeRuntime.m:192

  • The native conversion added here is not exercised by the existing native snapshot fixture: its synthetic attribute dictionaries never contain the new traits attribute, and the runtime tests only check scenario exit codes. Add a fixture case that supplies an NSNumber traits value, including a value above 2^53, and assert the bridge emits the exact decimal string; otherwise a regression here could silently reintroduce precision loss while the current tests remain green.
  if ([name isEqualToString:kAttributeTraits] && [value isKindOfClass:NSNumber.class]) {
    return ((NSNumber *)value).stringValue;

packages/platform-apple/src/snapshot-source/tree.ts:118

  • UIAccessibilityTraitNotEnabled is bit 11 (1 << 11) in the UIKit traits word; bit 8 is UIAccessibilityTraitStaticText. Because the bridge is forwarding that UIKit UIAccessibilityTraits value, a normal disabled button will carry bit 11 and this mask will leave it as enabled: true, so the reported regression remains unfixed. Use the bit-11 mask and update the trait fixtures accordingly.
const NOT_ENABLED_TRAIT = 1n << 8n;

packages/platform-apple/src/snapshot-source/tree.ts:309

  • The decoder treats an explicitly present XC_kAXXCAttributeTraits: null as if the word were missing, returning an unknown enabled value. This conflicts with the documented contract that only a missing word is unknown and every non-digit value is traits-invalid; malformed bridge responses should not silently erase the state fact.
  if (value === undefined || value === null) return undefined;
  • Files reviewed: 13/13 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment on lines +310 to +313
if (typeof value !== 'string' || !/^\d{1,20}$/.test(value)) {
throw snapshotSourceError('malformed-tree', 'traits-invalid');
}
return (BigInt(value) & NOT_ENABLED_TRAIT) === 0n;
@okwasniewski
okwasniewski force-pushed the fix/ios-bridge-enabled-state branch from 9d54c0b to ee22c24 Compare September 17, 2026 08:21
Copilot AI review requested due to automatic review settings September 17, 2026 08:21
@okwasniewski

Copy link
Copy Markdown
Contributor Author

Rebased onto main (ef62c7f) as ee22c24; the only conflict was CHANGELOG ordering. Same four commits, suites and gates green locally (376 passed), live repro re-run on the rebuilt CLI.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The accessibility disabled-trait mask must use UIKit’s bit 10 instead of bit 8.

Review details

Suppressed comments (1)

packages/platform-apple/src/snapshot-source/tree.ts:118

  • UIAccessibilityTraitNotEnabled is not bit 8 (1 << 8 is the StaticText trait); UIKit defines the disabled trait at bit 10 (1 << 10, value 1024). With this mask, a disabled control carrying the real NotEnabled trait will still be decoded as enabled: true, so the bridge does not fix the reported mismatch. Update the mask and the regression fixture to use the platform constant/value.
const NOT_ENABLED_TRAIT = 1n << 8n;
  • Files reviewed: 13/13 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@okwasniewski

Copy link
Copy Markdown
Contributor Author

On the suppressed review note at tree.ts:118 claiming UIAccessibilityTraitNotEnabled is bit 10 and bit 8 is StaticText: that does not match UIKit. The constants are opaque externs in the SDK header, so I resolved every exported UIAccessibilityTrait* symbol with dlsym inside the iOS 26.5 simulator runtime (tool and full output attached to the evidence release: uikit-traits-ios-26.5.txt, uikit-traits-dump.swift):

UIAccessibilityTraitStaticText        = 64    (bit 6)
UIAccessibilityTraitNotEnabled        = 256   (bit 8)
UIAccessibilityTraitUpdatesFrequently = 512   (bit 9)
UIAccessibilityTraitSearchField       = 1024  (bit 10)
UIAccessibilityTraitToggleButton      = 2^53

Bit 10 is SearchField. The mask stays 1n << 8n, and the live bridge snapshot with this mask reports enabled: false on both disabled controls and true on the enabled ones, which a bit-10 mask could not produce. No change.

@thymikee

Copy link
Copy Markdown
Member

Reviewed at ee22c24. The new commit resolves the four points from the earlier review (d1ffcb9). Later patches no longer bring back retracted facts, presentation.test.ts is back under the size limit, the CHANGELOG names the runner-path and Maestro effects, and the traits word now crosses the wire as a decimal string that is parsed exactly.

CI is green, and the PR body reports a live Simulator run at this commit that covers the new string path.

Two optional notes: the decoder still accepts a 20-digit string above 2^64-1 instead of rejecting it as traits-invalid (tree.ts#L311), and the new regression calls the replacement helpers directly instead of running a full presentation pass with a clipped web Heading (tree.test.ts#L22).

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 17, 2026
@thymikee
thymikee merged commit 6df46c4 into callstack:main Sep 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants