Skip to content

AppductCore unit tests are racy in a simulator (fixed Task.yield() draining) #61

Description

@V3RON

The AppductCore unit tests pass reliably on macOS (swift test, which is what CI runs) but fail readily in an iOS simulator. That's why AppductCore.podspec no longer declares a test spec (de3e32a): pod trunk push runs a pod's tests on iOS and tvOS simulators, and these made every CocoaPods publish a coin toss.

The tests are still fine as a macOS gate. They are not trustworthy anywhere else, and nothing currently runs them on iOS or tvOS.

Cause

drainPendingTasks() in packages/native/ios/Tests/AppductCoreTests/TestSupport.swift is 20 Task.yield() calls:

func drainPendingTasks(iterations: Int = 20) async {
  for _ in 0..<iterations {
    await Task.yield()
  }
}

Task.yield() is a scheduling hint, not a wait. It does not guarantee that work on another executor (the AppductClient actor) has run. It's called at ~60 sites in AppductAPITests.swift and AppductClientTests.swift, typically to "let the client catch up" before asserting on facade.state / sessionId, a synchronous snapshot the actor updates on its own executor.

Two distinct races come out of that, e.g. in testHandleRoutesAValidBootstrapLink:

  1. Before a simulated event. If the client hasn't reached transport.connect yet, simulateAck arrives with no pending attempt and is dropped. The state then stays connecting no matter how long the test waits.
  2. Before the assertion. The test reads the snapshot before the actor has applied the event.

Observed failures

  • AppductAPITests.testHandleRoutesAValidBootstrapLink: XCTAssertEqual failed: ("connecting") is not equal to ("active"), during pod spec lint (app-hosted, iOS simulator). The same commit passed all 132 tests in an earlier run.
  • AppductClientTests.testThrowingHandlerSendsToolExecutionError: failed on the first iteration of the repro below.

Reproduce

xcodebuild test -scheme AppductCore \
  -destination 'platform=iOS Simulator,name=iPhone 16 Pro' \
  -test-iterations 50 -run-tests-until-failure

Suggested fix

Replace fixed-yield draining with waits on the condition the test actually depends on, with a timeout that fails the test instead of hanging:

- await drainPendingTasks()
+ try await waitUntil { transport.connectCallCount == 1 }
  transport.simulateAck(sessionId: "session-9")
- await drainPendingTasks()
+ try await waitUntil { facade.state == .active }

FakeTransportSession already exposes lock-protected counters (connectCallCount, sentMessages, closeCallCount, …) to wait on.

Done when

  • The repro above passes 50 iterations on an iOS simulator
  • CI runs the suite on an iOS simulator (ideally with iterations), so a new race fails on the PR that adds it
  • Optionally: restore a test spec in AppductCore.podspec, once the suite is reliable in a simulator

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions