test_runner: support test order randomization #3269

Job	Run time
Build slim tarball	2m 17s
aarch64-linux: with shared libraries	45m 31s
x86_64-linux: with shared libraries	45m 47s
aarch64-darwin: with shared libraries	45m 58s
x86_64-darwin: with shared libraries	46m 10s
	3h 5m 43s

Provide feedback