chore: Split Harness controller tests to avoid timeout#4013
Open
mrousavy wants to merge 1 commit into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The previous Harness Android AWS Device tests often failed due to timeouts - see this action run:
All other tests went through.
My first hunch was that the app crashed somewhere in the controller test and it therefore just timed out because nothing happened anymore - but we should have native crash detection set up now (unless that config is broken somehow @hannojg @V3RON?) - so I guess it might really be a timeout.
This makes me think - how come this test suite times out? Do individual tests reset the total test suite timeout counter? Or is it a static counter that kills tests, even if they are making progress? In that case timeout is probably the wrong idea since it might actually just take that long on our end to run all tests.
This CI kinda tests this by splitting the controller tests across 4 different files to see if that runs. If it does, we probably would want to fix this in Harness anyways to either bump the timeout on our end, or reset the timeout as long as individual tests are making progress.