feat(test): add retryStrategy option to defer retries#41297
Open
dgozman wants to merge 1 commit into
Open
Conversation
…he run Adds a `retryStrategy` test config option. With `'deferred'`, retries run only after all tests have had their first attempt, in parallel up to the configured number of workers. Defaults to `'immediate'` (current behavior). Fixes: microsoft#23354
Contributor
Test results for "MCP"1 failed 7340 passed, 1122 skipped Merge workflow run. |
Contributor
Test results for "tests 1"1 flaky39572 passed, 743 skipped Merge workflow run. |
yury-s
approved these changes
Jun 17, 2026
|
|
||
| Controls when failed tests are retried. Defaults to `'immediate'`. | ||
| * `'immediate'` - A failed test is retried as soon as a worker is available, interleaved with the rest of the run. This is the default. | ||
| * `'deferred'` - Retries are run only after all tests have had their first attempt, in parallel up to the configured number of [workers](#test-config-workers). |
Member
There was a problem hiding this comment.
attheend? I can imagine the same people could be unhappy with a cluster of retries at the end of the test run and wand to see a variation of deferred that would do some backoff of the test but not at the end.
| * `'immediate'` - A failed test is retried as soon as a worker is available, interleaved with the rest of the run. This is the default. | ||
| * `'deferred'` - Retries are run only after all tests have had their first attempt, in parallel up to the configured number of [workers](#test-config-workers). | ||
|
|
||
| Learn more about [test retries](../test-retries.md#retries). |
Member
There was a problem hiding this comment.
Should we also mention how it affects maxFailures?
| // 5. Possibly queue a new job with leftover tests and/or retries. | ||
| if (!this._isStopped && result.newJob) { | ||
| this._queue.unshift(result.newJob); | ||
| if (this._testRun.config.retryStrategy === 'deferred') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
retryStrategytest config option.'deferred'runs retries only after all tests have had their first attempt, in parallel up to the configured number of workers.'immediate'(current behavior).Fixes #23354