Skip to content

1.0.7 — fix flaky browser setup, and make the adb port configurable - #20

Open
gautam-jain-dev wants to merge 12 commits into
AppiumTestDistribution:mainfrom
gautam-jain-dev:fix/browser-readiness-convergence
Open

1.0.7 — fix flaky browser setup, and make the adb port configurable#20
gautam-jain-dev wants to merge 12 commits into
AppiumTestDistribution:mainfrom
gautam-jain-dev:fix/browser-readiness-convergence

Conversation

@gautam-jain-dev

@gautam-jain-dev gautam-jain-dev commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #18, which fixed Brave's first-run race. Soaking all five browsers surfaced the same class of failure in the other scripts, plus three further defects. Also folds in a small, independent change: the adb server port is now configurable.

Problem

Sessions fail intermittently at creation with socket hang up or Debug list is empty or invalid on /json/list.

The skip-welcome-* scripts clear the browser profile — which guarantees first-run onboarding appears — then dismiss it by clicking a hard-coded sequence of controls. That choreography is racy in practice:

  • Dialog order and content vary per launch. Brave's first dialog was android:id/button2 on one run and com.brave.browser:id/btn_negative on the next.
  • Some stages are conditional. Opera unconditionally waited 30 s for the "Allow" notification prompt; when it does not appear the script threw with onboarding still on screen.
  • Edge's first-run screen is web-rendered and sometimes shows none of the expected controls ("Not now" missed 21 times in one run).
  • An Android system dialog can cover everything. A page-source dump during a failure showed "System UI isn't responding" / "Close app" / "Wait" — a SystemUI ANR over the welcome screen. No expected element is findable, and restarting the browser does not clear it.

When any of these happen the browser never opens its devtools socket. Worse, most of these scripts caught every error and still exited 0, so the caller reported success and started a session that could not work.

Fix

Converge on the end state the driver needs instead of requiring a fixed sequence. New scripts/browserReady.js:

  • dismisses whatever is on screen from a set of known-safe controls (ANR "Wait" first, so a system dialog is cleared before anything beneath it is touched);
  • relaunches the browser when nothing is dismissable;
  • treats the browser as ready only when onboarding is gone, the devtools endpoint belongs to this package, and /json/list reports an attachable page target. Ownership is checked through the Android-Package field of /json/version, since chrome, brave and edge share the chrome_devtools_remote name; the page check ignores worker targets, which also carry a webSocketDebuggerUrl. The list is polled, not slept on;
  • throws if the browser never becomes ready, so a failed setup is visible immediately instead of surfacing later as an opaque session error.
Browser Change
edge Optional FRE suppression, off by default (see below). Walkthrough and readiness pass handle it otherwise.
opera continue_button and the Allow dialog are optional; clicks no longer abort the walkthrough when the screen changes underneath them. Opera does not honour the Chromium flag (its onboarding is com.opera.android.startup.WelcomeActivity), so it relies on the readiness pass.
duckduckgo Readiness uses the pid-scoped webview_devtools_remote_<pid> socket — the bare prefix also matches other apps' WebViews.
samsung Readiness check added; walkthrough unchanged.
brave Shares the readiness helper. The flag written by #18 is now opt-in (see below).

src/driver.js

  1. Target selection. data.find((t) => t.url.includes('appium.io')) was a hard requirement; after a redirect, a new tab, or an onboarding page there is no such target and the session failed with Cannot read properties of undefined (reading 'webSocketDebuggerUrl'). Now falls back to any page target with a debugger URL, with a clear error if there is none. This reproduced independently on a second host after ~150 clean runs.
  2. Empty debug list. When /json/list returned [] the driver retried the fetch ten times, but nothing in that loop made a page appear. It now opens the start URL between attempts, via a new openStartUrl() — deliberately a plain VIEW intent rather than startApplication(), because the latter passes -S and force-stops the process; for WebView browsers the pid changes and the existing port forward is left pointing at a dead socket.

Configurable adb server port

The driver could only talk to adb on 5037: ADB.createADB() was called with no options in six places, and the appium:adbPort capability that callers already send was discarded (sessions logged it under "capabilities were provided, but are not recognized"). The port now resolves as appium:adbPortCDP_ADB_PORT → 5037.

The environment variable is not redundant: the skip-welcome-* scripts run as separate processes and never see the session's capabilities, so a capability-only fix would leave the browser setup on the requested port while the walkthrough silently used 5037. Each script also passes the resolved port into its UiAutomator2 session, so both halves address the same adb server.

No behaviour change when neither is setresolveAdbPort() returns 5037, and unusable values fall back to it rather than producing a broken -P. Covered by tests.

Verified on an Android emulator host, against a live Android 15 emulator:

Case Resolved adb argv Result
Neither set 5037 -P 5037 adb shell getpropsdk=35
CDP_ADB_PORT=5038, default server killed 5038 -P 5038 sdk=35, with 0 listeners on 5037 for the whole case
appium:adbPort=5038, default server killed 5038 -P 5038 sdk=35
CDP_ADB_PORT=5039 pointed at a non-adb listener, 5037 healthy 5039 -P 5039 fails — no silent fallback

The second case is the load-bearing one: with no adb server on 5037 at all, the 5038 server answered with a device entry (127.0.0.1:5555) that only it had. The last case is the inverse — a healthy 5037 was deliberately left running, and the driver still failed on the port it was told to use rather than quietly succeeding elsewhere.

Verification

A controlled A/B on Android 15 and 16 emulators (Appium 2.6.0), five browsers rotating, one session at a time, 400 tests in four builds of 100:

Group Driver Tests Passed Session-creation failures
host A this branch 100 100 0
host B this branch 100 100 0
long-lived machines current release 100 84 16 (16%)
freshly provisioned machines current release 100 100 0

Every failure was brave (13) or Edge (3) — exactly the two browsers whose first-run experience this change suppresses. Opera, DuckDuckGo and Samsung never failed anywhere. The freshly-provisioned control is the useful one: it ran the same current release and was clean, because a new machine has no stale first-run state — which isolates the cause to onboarding state rather than the driver, the machine, or the OS version.

FRE suppression was measured separately: 31 launches on freshly-cleared profiles across brave and Edge, 0 first-run screens. Note the 400-test run above had suppression enabled; the default path (readiness pass only, no file written) has not been measured at that scale for brave and edge, though it is the path the other three browsers used throughout.

Tests: test/browser-ready.spec.js (page-vs-worker target selection) and test/adb-port.spec.js (port resolution, including the 5037 default) — 10 cases, all passing.

Notes for the reviewer

  • First-run suppression is opt-in. /data/local/tmp/chrome-command-line is shared with Chrome, and chromedriver writes its own chromeOptions.args there, so writing it for brave or edge could change an unrelated Chrome session on the same device. Nothing is written unless CDP_SUPPRESS_FIRST_RUN=1 is set; by default onboarding is handled by the readiness pass, as it is for opera, duckduckgo and samsung. Note this differs from fix: flaky browser setup — converge on devtools readiness instead of onboarding choreography #18, which wrote the file unconditionally for brave.
  • Whether brave and edge honour a package-specific command-line file is unknown. It would avoid the shared path entirely, but I could not establish it: the only emulators available for testing are inside sessions that pin the foreground app, so a second browser cannot be launched in them. If you know the answer, this could become a per-package file and the opt-in would be unnecessary.
  • am start does not honour a bare trailing package. Both openStartUrl() and the readiness helper's page-open step used that form and were opening the start URL in whichever browser handles VIEW by default; they now pass -n <package>/<activity>. This was invisible in practice because a page was usually already present from the preceding launch.
  • The ownership check is best-effort. /json/version on some forks may omit Android-Package; when the field is absent the endpoint is accepted, since the socket name is browser-specific for everything except the chrome/brave/edge trio.
  • Samsung has no failure evidence behind it — it never failed in 400 tests. The readiness check was added for consistency, as its walkthrough has the same swallow-everything shape as the others.
  • Readiness costs a few adb calls and a short poll per setup on the happy path; the previous fixed 4 s sleep was removed in favour of polling.

🤖 Generated with Claude Code

gautam-jain-dev and others added 9 commits August 2, 2026 05:15
The skip-welcome-* scripts clear the browser profile and then click through
first-run onboarding in a fixed order. That choreography is racy in practice:

- dialog order and content vary per launch (Brave's first dialog was
  android:id/button2 on one run and btn_negative on the next);
- some stages are conditional — Opera's "Allow" notification prompt does not
  always appear, but the script required it and threw when it was missing;
- Edge's first-run screen is web-rendered and sometimes shows none of the
  expected controls;
- an Android system dialog ("System UI isn't responding") can cover the
  welcome screen entirely, so no expected element is ever findable, and
  restarting the browser does not clear it.

When any of these happen the browser never opens its devtools socket, and
session creation later fails with "socket hang up" or "Debug list is empty or
invalid". Worse, most of these scripts swallow their errors and exit 0, so the
caller reports success and starts a session that cannot work.

Rather than requiring a fixed sequence, converge on the end state the driver
needs. scripts/browserReady.js adds a shared helper that dismisses whatever is
on screen (including ANR dialogs, via "Wait"), relaunches the browser when
nothing is dismissable, and considers the browser ready only when its devtools
socket exists, onboarding is gone, and a page is open. It throws if the browser
never becomes ready, so a failed setup is visible immediately.

Per browser:
- brave, edge: suppress Chromium's FRE with --disable-fre --no-first-run via
  /data/local/tmp/chrome-command-line (honored on emulators and debuggable
  builds), so the walkthrough has nothing to do; keep it as a fallback.
- opera: continue_button and the Allow dialog are now optional, and clicks no
  longer abort the walkthrough when the screen changes underneath them.
- duckduckgo: readiness uses the pid-scoped webview_devtools_remote_<pid>
  socket — the bare prefix also matches other apps' WebViews.
- samsung: readiness check added; walkthrough unchanged.

src/driver.js additionally no longer requires a target whose URL contains
appium.io. After a redirect, a new tab, or an onboarding page there may be no
such target, and reading webSocketDebuggerUrl off undefined failed the session
with "Cannot read properties of undefined". It now falls back to any page
target with a debugger URL and raises a clear error if there is none.

Verified on Android 15 emulators across three hosts: ~470 consecutive sessions
across all five browsers with no setup failures, against a baseline where each
browser failed within the first few cycles.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Readiness was declared once the browser's devtools socket existed and
onboarding was gone. That is still not proof the driver can attach: a run on a
freshly cleared DuckDuckGo profile reached BrowserActivity with its pid-scoped
socket up and a page intent fired, and /json/list was nevertheless empty — the
exact "Debug list is empty or invalid" failure this helper exists to prevent.

Ask /json/list the same way the driver does — forward a port to the socket, GET
the target list, and require at least one target carrying a
webSocketDebuggerUrl — before reporting the browser ready. The target count is
logged either way, so a failure says whether the browser had no socket, no page,
or an unreachable debugger.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When /json/list returns an empty array the driver retries the fetch up to ten
times, but nothing in that loop makes a page appear, so every attempt sees the
same empty list and the session fails with "Debug list is empty or invalid".
WebView-based browsers hit this routinely: the devtools socket is up moments
after launch while no debuggable page exists yet.

Relaunch the browser (which carries the start URL) before each retry, so the
retry window is actually used to reach a usable state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The previous commit called startApplication() when /json/list came back empty,
but that passes -S and force-stops the process. For WebView-based browsers the
devtools socket name carries the pid, so the restart left the driver's port
forward pointing at a socket that no longer existed and the failure changed from
"Debug list is empty" to "socket hang up".

Add openStartUrl(), a plain VIEW intent that opens the start page in the
already-running browser, and use it for the retry nudge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bump the version for the browser-readiness fixes and declare node-fetch, which
src/driver.js already imports directly but only ever resolved transitively.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four defects in the readiness helper, all of which let it report a browser
ready when the driver could not have used it:

- Workers counted as pages. /json/list returns worker targets that also carry a
  webSocketDebuggerUrl (a real Brave response holds one page and one worker), so
  filtering on that field alone passed a browser whose only target was a worker
  — exactly the state this check exists to reject. Filter on type === 'page'.

- A shared socket name proved nothing. chrome, brave and edge all use
  chrome_devtools_remote, and abstract socket names are unique per owner: if
  Chrome held the name, Brave could not, and the check happily verified Chrome's
  page and reported Brave ready. Confirm the owner via the Android-Package field
  of /json/version before trusting the endpoint.

- The helper and the driver derived different socket names. pidof returns every
  process of the package; the helper took the first pid while the driver used
  the whole string, so with a multi-process browser the driver built an unusable
  name like webview_devtools_remote_1234 5678 while the helper verified
  ..._1234 and reported success. Both now take the first pid.

- A fixed four second sleep was charged to every successful setup. Poll
  /json/list instead, which normally settles in a few hundred milliseconds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The driver always talked to adb on 5037. ADB.createADB() was called without
options in six places, so appium-adb fell back to its default, and the
appium:adbPort capability that callers already send was discarded — sessions
logged it under "capabilities were provided, but are not recognized". Anyone
running an adb server on another port (multiple servers on one machine, a
sandboxed CI agent, a device farm host with per-device servers) could not use
this driver.

The port now comes from, in order: the appium:adbPort capability, the
CDP_ADB_PORT environment variable, then adb's default of 5037.

The environment variable is not redundant. The skip-welcome-* scripts run as
separate processes launched via `appium driver run`, so they never see the
session's capabilities; without it the browser setup would use the requested
port while the onboarding walkthrough silently used 5037. Each script now also
passes the resolved port into its UiAutomator2 session, so both halves address
the same adb server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Locks in the default: with no capability and no environment variable — every
existing caller — the port stays 5037, and any unusable value falls back to it
rather than producing a broken -P argument.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…kages

Brings the adb-port work into the 1.0.7 release: the port now comes from the
appium:adbPort capability, then CDP_ADB_PORT, then adb's default of 5037, so
nothing changes for callers that specify neither.

Also, from a review pass over this branch:

- pickPageTargets() is extracted and covered by tests. It is the check that
  decides a browser is usable, and the interesting case is negative: a worker
  target carries a webSocketDebuggerUrl too, so filtering on that field alone
  accepted browsers with no page — the state the check exists to reject.

- src/adb.js had the browser package list twice, once for openStartUrl and once
  inside startApplication. They are now one BROWSERS map holding package and
  launch activity, with identical values for all seven entries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gautam-jain-dev gautam-jain-dev changed the title DRAFT: 1.0.7 — converge on devtools readiness instead of onboarding choreography 1.0.7 — fix flaky browser setup, and make the adb port configurable Aug 6, 2026
@gautam-jain-dev
gautam-jain-dev marked this pull request as ready for review August 6, 2026 11:27
gautam-jain-dev and others added 3 commits August 6, 2026 17:27
…r when opening a page

Two problems found while testing the first-run suppression on a device.

/data/local/tmp/chrome-command-line is shared. Chromedriver writes its own
chromeOptions.args to that path, so writing it for brave or edge could change an
unrelated Chrome session on the same device or emulator. Suppression is now
opt-in through CDP_SUPPRESS_FIRST_RUN=1; with it unset nothing is written and
the readiness pass dismisses onboarding as it already does for the other
browsers. Whether brave and edge honour a package-specific command-line file
could not be established here — the only emulators available are inside sessions
that pin the foreground app, so a second browser cannot be launched in them.

am start does not honour a bare trailing package: the intent goes to whichever
browser handles VIEW by default. Both openStartUrl() and the readiness helper's
page-open step used that form, so they were opening the start URL in the wrong
browser; it went unnoticed because a page was usually already present from the
launch that preceded them. Both now pass -n <package>/<activity>.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Samsung Internet shows an "Updates to Samsung Internet Privacy Notice" alert on
SBrowserMainActivity, after the help intro rather than as part of it, so the
walkthrough — which only runs while the activity is HelpIntroActivity — never
sees it.

It does not block the devtools socket: the page loads underneath, so session
creation succeeds and the readiness pass has nothing to react to. The dialog then
sits over the page for the rest of the session, swallowing taps meant for the
content. Observed on a live session and confirmed dismissed by its Continue
button.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The walkthrough only looked for help_intro_legal_agree_button. Newer intros
label the button "Continue" and do not expose that id, so the click never landed
and the intro was left on screen for the readiness pass to clear.

Try the id first, then "Continue", then "Agree". Every selector matches by exact
text: the intro also carries a "Close app" button beside the one we want, and a
SystemUI ANR dialog ("Close app" / "Wait") can sit over the whole screen, so a
loose match would risk killing the browser being set up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant