Test Retry Policy

[En Low]

Hi! I read the test retry policy in the CQ run here. Looking at past CLs, I don't see many test failures where it " Recompile each failing test suite without the CL. Rerun each failing test suite in isolation." When clicking into failures, I mainly see that it was retried with the patch and failed, not retrying without the patch. 

1. Is the documentation was still accurate?
2. Are there exceptions to the test retry logic?

Thanks, 

En

[Stephen Martinis]

The documentation is mostly accurate. I believe the only inaccuracy is that if a bunch of tests fail with the patch, it assumes the patch is just bad and gives up retrying, to save capacity. 

If you have a specific question about a build you can link to it and I could take a look at it.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/7bf9d709-92b7-4663-80c9-0899d8cb623an%40chromium.org.

[Dirk Pranke]

As an example, just picking just the most recent build that failed due to test failures on the `linux-rel` trybot:

https://ci.chromium.org/p/chromium/builders/try/linux-rel/619898

you can see that the 'blink_platform_unittests' test suite failed with patch, failed on the retry of the shard, and then passed without patch.

Other builds may show different results, of course, but I think we will still always retry a test failure without the patch unless too many tests failed, as Stephen says.

-- Dirk

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CANheWB%3D1%2BppEJecHU%3DnZiM%3Dw7BDS2PaTw19dU9K%2ByK43xWkELA%40mail.gmail.com.