PSA Turning off retry for Android instrumentation tests

[Bo Liu]

Skip if you don’t care about android tests.

Hi chromium-dev,

The retry-on-failure behavior is going to be removed for non-flaky Android instrumentation test cases over the weekend, when [0] lands. Rationale [1] and tracking bug [2]. We believe all flaky tests are annotated with @RetryOnFailure so this change should not make bots more flaky.

Chromium devs: Please let us know either here or on the bug if you notice extra flakiness from android bots.

Chromium sheriffs: If an Android instrumentation test becomes flaky, add the @RetryOnFailure annotation first instead of directly disabling the test.

Android sheriffs: Some bots on chromium.android are expected to become more flaky/red. These bots are not on cq, so we did not check test results from these bots when annotating flaky tests.

Thanks!

[0] https://codereview.chromium.org/2348693004/

[1] https://docs.google.com/document/d/1u3PIlYZ8Op4k9jsVG-WNZ5UvBpbEjNEqDCoWG257Krs/edit?usp=sharing

[2] https://bugs.chromium.org/p/chromium/issues/detail?id=619055

[Paweł Hajdan, Jr.]

+infra-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Erik Staab]

Thanks for doing this! It sounds like a promising approach for driving down flakiness and I hope we can do more of this across projects.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.
To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

[Mike Stipicevic]

Awesome, thank you for tackling flakiness head-on!

On Fri, Sep 16, 2016 at 1:45 PM Erik Staab <est...@chromium.org> wrote:

Thanks for doing this! It sounds like a promising approach for driving down flakiness and I hope we can do more of this across projects.

On Fri, Sep 16, 2016 at 11:25 AM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

+infra-dev

On Fri, Sep 16, 2016 at 9:52 AM, Bo Liu <bo...@chromium.org> wrote:

Skip if you don’t care about android tests.

Hi chromium-dev,

The retry-on-failure behavior is going to be removed for non-flaky Android instrumentation test cases over the weekend, when [0] lands. Rationale [1] and tracking bug [2]. We believe all flaky tests are annotated with @RetryOnFailure so this change should not make bots more flaky.

Chromium devs: Please let us know either here or on the bug if you notice extra flakiness from android bots.

Chromium sheriffs: If an Android instrumentation test becomes flaky, add the @RetryOnFailure annotation first instead of directly disabling the test.

Android sheriffs: Some bots on chromium.android are expected to become more flaky/red. These bots are not on cq, so we did not check test results from these bots when annotating flaky tests.

Thanks!

[0] https://codereview.chromium.org/2348693004/

[1] https://docs.google.com/document/d/1u3PIlYZ8Op4k9jsVG-WNZ5UvBpbEjNEqDCoWG257Krs/edit?usp=sharing

[2] https://bugs.chromium.org/p/chromium/issues/detail?id=619055

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

[John Abd-El-Malek]

Will someone be tracking failure rates of this test suite on the CQ to make sure it doesn't regress?

On Fri, Sep 16, 2016 at 3:56 PM, Mike Stipicevic <st...@chromium.org> wrote:

Awesome, thank you for tackling flakiness head-on!

On Fri, Sep 16, 2016 at 1:45 PM Erik Staab <est...@chromium.org> wrote:

Thanks for doing this! It sounds like a promising approach for driving down flakiness and I hope we can do more of this across projects.

On Fri, Sep 16, 2016 at 11:25 AM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

+infra-dev

On Fri, Sep 16, 2016 at 9:52 AM, Bo Liu <bo...@chromium.org> wrote:

Skip if you don’t care about android tests.

Hi chromium-dev,

The retry-on-failure behavior is going to be removed for non-flaky Android instrumentation test cases over the weekend, when [0] lands. Rationale [1] and tracking bug [2]. We believe all flaky tests are annotated with @RetryOnFailure so this change should not make bots more flaky.

Chromium devs: Please let us know either here or on the bug if you notice extra flakiness from android bots.

Chromium sheriffs: If an Android instrumentation test becomes flaky, add the @RetryOnFailure annotation first instead of directly disabling the test.

Android sheriffs: Some bots on chromium.android are expected to become more flaky/red. These bots are not on cq, so we did not check test results from these bots when annotating flaky tests.

Thanks!

[0] https://codereview.chromium.org/2348693004/

[1] https://docs.google.com/document/d/1u3PIlYZ8Op4k9jsVG-WNZ5UvBpbEjNEqDCoWG257Krs/edit?usp=sharing

[2] https://bugs.chromium.org/p/chromium/issues/detail?id=619055

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

[John Budorick]

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.

[Bo Liu]

Specifically, we'll monitor flakiness dashboard and new Via-TryFlakes bugs.

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.

[Ojan Vafai]

Did this monitoring happen? The main risk of this change is that someone commits a flake and it causes the CQ to be flaky for N days before the flake is identified and addressed. 

+Sergiy Byelozyorov +Katie Thomas is it hard to see if there were flakiness spikes due to android instrumentation tests in the past quarter?

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

[Bo Liu]

John and I set up crbug filters for android Via-TryFlakes bugs. John is more diligent than I am at taking action though. Those bugs also have the sheriff label, so sheriffs also look at them pretty promptly.

Looking at recent weekly reports of trybot flakiness, android usually isn't at the very top. So I think in practice this change was not very bad for flakiness?

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

[Anita Woodruff]

If adding @RetryOnFailure is the new recommended first step for dealing with flaky instrumentation tests, please can this be added to build sheriff instructions at http://dev.chromium.org/developers/tree-sheriffs/sheriff-details-chromium ? 

Cheers

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Anthony Berent]

If a test is marked @RetryOnFailure, should there also be an associated bug? Tests should not generally be flaky, although there may be a few special cases where some flakiness is unavoidable.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAFObWWYfnhJkeU0Cfjv358um3wiNxeTY_AQ5j6XrmwaefU8kPQ%40mail.gmail.com.

[Bo Liu]

I added @RetryOnFailure as an option sheriff may take on the sheriffs page.

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

[Ojan Vafai]

I'm sheriff today and I'm seeing this test suite be very flaky. While this may have worked initially, it doesn't seem to be working well anymore.

Do you have ideas for how to more reliably stay on top of the flakes?

https://luci-milo.appspot.com/buildbot/chromium.linux/Android%20Tests/?limit=200

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

[John Budorick]

We made some changes this week to how these tests run, and bpastene@ noticed yesterday that they've had some difficulties since then. We're collecting more information about which tests are failing and the failure modes and will take action from there.

https://luci-milo.appspot.com/buildbot/chromium.linux/Android%20Tests/?limit=200

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

[Ojan Vafai]

This continues to be extremely flaky. Is there a bug to follow about this getting fixed? Is it possible to revert the changes from this week and reland after the flakiness issues are fixed?

BTW, this doesn't run on the CQ, does it? If so, I expect it's causing a lot of CQ flakiness and increase in cycle time. If not, it's just a nuisance to sheriffs.

https://luci-milo.appspot.com/buildbot/chromium.linux/Android%20Tests/?limit=200

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

[bpas...@chromium.org]

I'm tracking the effort in https://bugs.chromium.org/p/chromium/issues/detail?id=748145

The path to reverting isn't the easiest thing in the world, so I'd like to hold off on that if possible. That said, if I can't come up with anything in the next couple days that improves the situation, I plan on reverting the heavier suites.

And this bot is indeed on the CQ at https://build.chromium.org/p/tryserver.chromium.android/builders/linux_android_rel_ng?numbuilds=200

https://luci-milo.appspot.com/buildbot/chromium.linux/Android%20Tests/?limit=200

Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAATLsPZaKxr%2BNOetySQtrc0bxuwW-DBfX%2B3m%3DMXewOzhqAmfbw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAN60yaJ8LFAcJfcE2pgZgMHU%3D%2BocAtmoT7gF8HopEFnep1i0wQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.

To post to this group, send email to infr...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAP8mwOAfXi%2BTdkw-x0sbxTXS2LaCuqDX2XzRNDwcHbchfURRQQ%40mail.gmail.com.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

[John Budorick]

On Mon, Jul 24, 2017 at 2:44 PM Ojan Vafai <oj...@chromium.org> wrote:

This continues to be extremely flaky. Is there a bug to follow about this getting fixed? Is it possible to revert the changes from this week and reland after the flakiness issues are fixed?

https://bugs.chromium.org/p/chromium/issues/detail?id=748145 tracks this directly; https://bugs.chromium.org/p/chromium/issues/detail?id=732205 is also related. I believe reverting is nontrivial but not impossible.

 

BTW, this doesn't run on the CQ, does it? If so, I expect it's causing a lot of CQ flakiness and increase in cycle time. If not, it's just a nuisance to sheriffs.

It does. One of the reasons we made this change was to improve cycle time, and our monitoring bears that out. Success rate does appear to be down over the last few days, though, and that obviously has an impact on overall CQ time to commit.