🚀 CQ Quick Run General Availability: Test changes faster

[Gregory Guterman]

A Faster CQ Run Mode

Quick Run is here.

Tldr: Quick Run is available in Gerrit. It’s like a dry run, plus it uses algorithms to skip tests. 

After launching Quick Run Dogfood last August, we’ve worked on incorporating feedback and making Quick Run better. With our improvements in place, we’re happy to finally announce our launch. For those not already familiar with Quick Run, here is some more information.

 

How do I get it/use it?

Quick Run is very similar to dry run. You can use the new Quick Run button in the Gerrit UI.

Or if you prefer the CLI:

$ git cl try -q or $ git cl upload -q

 

Quick Runs toggle the CQ+1 and Quick-Run+1 Gerrit labels at the same time. Successful Quick Runs can’t be reused in CQ+2 submits.

Performance

Test selection is dynamic and based on your changed files list. Some CLs will be tested more thoroughly, while others can skip many tests. Our per-build median savings for the Dogfood group is 33% (17 hours) in CPU hours, and 25% (9 minutes) in wall time savings over dry runs, all while finding 97% of the failures dry run did. This figure only takes failed runs into account, and the correctness would be much higher if we included passes.

We do want to stress that different CLs will skip different amounts of tests, and many tests run in parallel, so the variance is high with respect to wall time savings. Quick Runs will never be slower than dry runs.

The main algorithm powering the selection uses an affinity graph built from Git history. We have many exciting plans, such as speedups and correctness improvements. Come to Brew and Bites tomorrow to learn more.

Brews and Bites tomorrow

If you’d like to see a more detailed presentation of Quick Run and underlying algorithms, join us at tomorrow’s Brews and Bites.

Thanks,

Greg on behalf of the Chrome EngProd team