Test Retry Policy

Skip to first unread message

En Low

Feb 18, 2021, 8:06:35 PMFeb 18
to infra-dev
Hi! I read the test retry policy in the CQ run here. Looking at past CLs, I don't see many test failures where it " Recompile each failing test suite without the CL. Rerun each failing test suite in isolation." When clicking into failures, I mainly see that it was retried with the patch and failed, not retrying without the patch. 
  1. Is the documentation was still accurate?
  2. Are there exceptions to the test retry logic?

Stephen Martinis

Feb 19, 2021, 1:50:44 PMFeb 19
to En Low, infra-dev
The documentation is mostly accurate. I believe the only inaccuracy is that if a bunch of tests fail with the patch, it assumes the patch is just bad and gives up retrying, to save capacity. 

If you have a specific question about a build you can link to it and I could take a look at it.

You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/7bf9d709-92b7-4663-80c9-0899d8cb623an%40chromium.org.

Dirk Pranke

Feb 19, 2021, 4:12:35 PMFeb 19
to En Low, infra-dev, Stephen Martinis
As an example, just picking just the most recent build that failed due to test failures on the `linux-rel` trybot:

you can see that the 'blink_platform_unittests' test suite failed with patch, failed on the retry of the shard, and then passed without patch.

Other builds may show different results, of course, but I think we will still always retry a test failure without the patch unless too many tests failed, as Stephen says.

-- Dirk

Reply all
Reply to author
0 new messages