Enabling Experimental Regression Test Selection (RTS) on CQ Dry Runs
Struan Shrimpton
Hi chromium-dev,
We are planning to enable Regression Test Selection (RTS) on 10% of all CLs that go through CQ for the next two weeks (2023-01-30 to 2023-02-10). We are gathering data that will help us decide if and how to enable RTS on all dry runs. If your CL is in the experiment, your dry run will be faster, but on submission, we will run the remaining tests that the dry run skipped.
What is Regression Test Selection (RTS)?
RTS is a selection strategy to run a minimal set of tests to validate a CL by deciding which tests are the most important to run based on which files have changed. If you have tried quick run 🚀 you’ve already used RTS.
Why are we doing this?
RTS-enabled builds use far fewer resources to determine if a patchset is likely safe to commit. To detect 95% of failures we only use about 65% of the testing resources. After we apply the current model to dry run, improving the quality of the selection model has huge potential for even more savings. We can reapply these saved resources to improve the cycle times of CQ as a whole.
Will this lead to more reverts?
We still run all tests before landing a CL so there should not be an increase in reverts. A subset of tests is run during CQ+1 while the rest of the tests are run at CQ+2. If CQ+1 is not used CQ+2 will continue to run all the tests at the same time.
What if RTS causes problems with my CL?
If RTS needs to be disabled, add this footer to the commit message:
Disable-Rts:True
This will prevent the experiment from affecting the CL. If you believe the build failed because of RTS please file a bug using the 'Infra>Client>Chrome' component.
On behalf of Chrome Operations,
Struan
Olamide bobby
Struan Shrimpton
On behalf of Chrome Operations,
Struan