Skip to content

Add an End-to-End test suite for the Flutter SDK - #31

Open
brionmario wants to merge 3 commits into
thunder-id:mainfrom
brionmario:mobile-e2e-tests
Open

Add an End-to-End test suite for the Flutter SDK#31
brionmario wants to merge 3 commits into
thunder-id:mainfrom
brionmario:mobile-e2e-tests

Conversation

@brionmario

@brionmario brionmario commented Sep 1, 2026

Copy link
Copy Markdown
Member

Purpose

The Flutter SDK had unit tests, but they mock the method channel, so they cover the Dart half in isolation and never exercise the bridge to the native SDKs. That leaves the most valuable failures uncovered: a flow step the SDK renders but cannot submit, a field the server renames, or an argument dropped on its way across the channel.

This adds a Maestro suite that drives the Quickstart sample against a real ThunderID server. It signs a user in, signs them out, registers a new account, and signs in as that new account. It is the only layer that tests the Dart and native halves together.

Two SDK changes were needed to make it possible, and both fix problems that affect users beyond the tests.

Approach

Maestro rather than integration_test. The sign-in and sign-up forms are not static widgets: the server returns a flow definition and the SDK renders it. All three mobile SDKs tag the resulting fields the same way, so one set of selectors and one set of flows works across iOS, Android and Flutter.

One script owns the run. tests/e2e/run-e2e.sh starts a server, provisions the test application and user, builds and installs the sample, and runs the flows. CI calls the same script through a composite action. run-e2e.sh drives an iOS Simulator and needs a Mac; run-e2e.ps1 drives an Android emulator and is the path for contributors on Windows.

Exposing identifiers to the platform accessibility tree

flow_form.dart tagged its fields and actions with a widget Key. A Key is internal to the Flutter tree and never reaches the platform accessibility tree, so nothing driving the app from outside could see them, and the tags looked correct in source while being unreachable in practice.

They now also carry Semantics.identifier, which maps to resource-id on Android and accessibilityIdentifier on iOS. The Key is kept so widget tests can keep finding these fields by key.

The identifier resolves the server's field identifier rather than its ref. That is deliberate: the iOS and Android SDKs both tag from the identifier, so preferring it here is what keeps one set of selectors working across all three. The ref remains the key used for form state and submission, so nothing about submission changes.

Semantics.identifier requires Flutter 3.19, so the declared minimum moves from 3.16 to 3.19 and the Dart constraint from 3.2 to 3.3. This is the one change reviewers may want to weigh: it affects consumers pinned to 3.16 through 3.18.

Worth noting separately that flow_form.dart previously contained no Semantics at all, which also left it short of the accessibility rule in this repository's own guidelines.

Reaching a development server

ThunderIDConfig had no way to accept the self-signed certificate ThunderID generates for localhost. The native Android SDK has allowInsecureConnections, but the plugin never forwarded it, so a Flutter app simply could not talk to a local server on Android. That blocked local development, not only these tests: anyone following the Flutter quickstart against https://localhost:8090 on Android hit the same wall.

ThunderIDConfig.allowInsecureConnections is now forwarded across the method channel. It is narrow by construction:

  • The native Android SDK honours it for loopback hosts only (localhost, 127.0.0.1, ::1, 10.0.2.2) and raises a configuration error otherwise. That change is in the companion Android PR.
  • The sample enables it only under kDebugMode, which is compiled out of release builds.
  • On iOS it is a no-op, since the same need is met by an NSAppTransportSecurity exemption at the app level.

Related Issues

Related PRs

Checklist

  • Followed the contribution guidelines.
  • Manual test round performed and verified.
  • Documentation provided. (Add links if there are any)
    • tests/e2e/README.md, plus the SDK Development section of the contributor docs
  • Tests provided. (Add links if there are any)
    • Unit Tests
    • Integration Tests
      • tests/e2e/flows/signin.yaml, tests/e2e/flows/signup.yaml
  • Breaking changes. (Fill if applicable)
    • Breaking changes section filled.
    • breaking change label added.

The minimum Flutter version moving from 3.16 to 3.19 is the only compatibility change. I have left the breaking changes section and label unticked because it does not alter any API, but if the project treats a minimum version bump as breaking, say so and I will fill that section in.

Security checks

  • Followed secure coding standards.
  • Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.

The credentials in this PR (e2e_mobile_user / TestPassword@123) belong to a throwaway account the suite creates on a local server. They are not secrets and grant nothing anywhere else.

Verification

flutter analyze reports no issues and flutter test passes, 66 tests. The suite was run end to end against a freshly downloaded ThunderID v1.0.1 on an iOS Simulator, 2 of 2 flows passing.

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 68826c40-b83b-4fd1-a840-695c1a98b90e


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@brionmario brionmario changed the title Add an end-to-end test suite for the Flutter Quickstart Add an end-to-end test suite for the Flutter SDK Sep 1, 2026
@brionmario brionmario changed the title Add an end-to-end test suite for the Flutter SDK Add an End-to-End test suite for the Flutter SDK Sep 1, 2026
Drive the Quickstart through real authentication against a real ThunderID
server using Maestro: sign in and sign out, then register a new account and
sign in as it. The suite lives in tests/e2e and one script owns the whole
run, so CI executes exactly what a contributor runs locally. run-e2e.sh
drives an iOS Simulator and needs a Mac; run-e2e.ps1 drives an Android
emulator and is the path for contributors on Windows.

This is the only layer that exercises the Dart and native halves together,
since the unit tests mock the method channel.

Expose flow field and action identifiers to the platform accessibility tree.
A widget Key never leaves the Flutter tree, so nothing driving the app from
outside could see these fields. They now also carry Semantics.identifier,
which maps to resource-id on Android and accessibilityIdentifier on iOS. The
identifier resolves the server's field identifier rather than its ref, which
is what keeps one set of selectors working across all three SDKs; the ref
remains the key used for form state and submission. Semantics.identifier
requires Flutter 3.19, so the declared minimum moves up from 3.16.

flow_form.dart previously had no Semantics at all, which also left it short
of the accessibility rule in the repository's own guidelines.

Add ThunderIDConfig.allowInsecureConnections and forward it across the
method channel. Without it there was no way for a Flutter app to reach a
development server over the self-signed certificate ThunderID generates for
localhost, which blocked local development on Android as much as it blocked
these tests. The native Android SDK honours it for loopback hosts only, and
the sample enables it for debug builds only.

Refs thunder-id/thunderid#5181

Signed-off-by: Brion <info@brionmario.com>
Every flow starts here, and a bare assertVisible gets Maestro's short
default lookup timeout rather than the generous budget the rest of the
suite uses. On a loaded CI emulator the landing screen does not always
render inside it, which failed the Android sign-up flow 19 seconds in,
before the flow had done anything. Wait for it explicitly instead.

Signed-off-by: Brion <info@brionmario.com>
CI installed whatever Maestro was newest at the time, so the test runner
changed under the suite between runs with nothing in the repository to
show it: CI has been on 2.10.0 since it shipped while a contributor
following the README gets whatever is current, which makes a CI-only
failure impossible to reproduce faithfully. Pin it, and bump
deliberately after checking the flows.

The JUnit report gives a failed run a machine-readable result instead of
a console log to scrape, and CI now collects it with the other debug
artifacts.

Signed-off-by: Brion <info@brionmario.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant