[PSA] Tast test can now repeat a given set of tests

[Soichiro Fujii]

TL;DR: New flag -repeats is now available for tast, that lets you repeat the same test multiple times. Hello everyone, Since R129-15999.0.0, you can now run the same set of tests multiple times by using the newly added -repeats N flag in Tast. What’s new? Tast (go/tast) now has a new flag -repeats N, that repeats the test multiple times when specified. When the flag is set to more than 0, the tast will repeat the given sequence of tests a number of times after the initial execution is done. This allows you to streamline the process of acquiring multiple samples for any measurement that requires statistical significance, such as A/B testing for performance profiling and detecting flaky tests caused by external force. Example of running  meta.LocalPass meta.RemotePass with -repeats=2. You can observe the test has been executed 2 times more, after the initial run. Actual command executed is displayed at the bottom for clarity. Using it via crosfleet The flag can also be specified from crosfleet when using tast as a harness, like so: -harness tast -test-args ’-repeats=N’. The test result for each invocation can be read from the CI output page. Example of running the same test as above, through crosfleet. Output of the crosfleet invocation above. Where can I learn more about? The design document can be read at go/tast-repeat-design-doc. Thank you for reading, Soichiro Fujii, on behalf of Tast Core Framework Team (Email: tast...@google.com)