kokoro: Execute test cases concurrently#263
Conversation
arjan-bal
requested review from
eshitachandwani,
pawbhard and
sergiitk
and removed request for
eshitachandwani
pawbhard
left a comment
pawbhard
left a comment
There was a problem hiding this comment.
LGTM, Need One validation for failure case.
@sergiitk , DO you have any concerns as you were trying this earlier, else we can merge.
We will run for a week for these test suites and then enable for all with configurable parallelism
| psm::tools::log "Finished ${test_suite} suite test: ${test_name}" | ||
| echo | ||
| done | ||
| # TODO(b/526886100): Make the parallelism configurable using a function |
There was a problem hiding this comment.
Create a separate task for it
There was a problem hiding this comment.
Filed b/535109852.
| psm::tools::log "Running ${test_suite} suite test: ${test_name}" |& tee "${test_log}" | ||
| # Must be the last line. | ||
| "psm::${test_suite}::run_test" "${test_name}" |& tee -a "${test_log}" | ||
| local exit_code=0 |
There was a problem hiding this comment.
Can we have a verify run with one of the test failing so we can check if that is handled correctly by the logic
There was a problem hiding this comment.
I injected an error to cause the dualstack test to fail and verified that the failure is reported correctly, however the the counter in the final log is 0
Failed test suites: 0
I'm looking into this.
Run : https://source.cloud.google.com/results/invocations/922f81cd-5083-45b8-8784-8b5716559e37
This reverts commit 8a567b3.
3 participants
Related Issue: b/526886100
Context & Summary
This change enables concurrent execution of test cases within the Security, LB, and DualStack test suites. Shifting from sequential execution to a parallelized model drastically reduces overall execution times and resolves the timeouts observed in our scheduled test runs.
The CSM suite remains unaffected and will continue to execute sequentially to prevent any shared resource conflicts.
Performance Improvements
Concurrency in LB suite on master is slightly reduced due to a long bootstrap generator test that runs at the end.
Test Runs