interactive_ui_tests flakiness

[Fred Akalin]

Hey guys,

I've noticed that interactive_ui_tests is probably the flakiest test executable recently. I almost always have one or two trybots fail on it, and it closes the tree semi-regularly.

I don't think it was always this way. Is someone looking into why this got so flaky recently?

-- Fred

[Ryan Hamilton]

+1  

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
 
 
 

[Fady Samuel]

+1, we're seeing a lot of WebViewInteractiveTest flakiness, but we can't repro locally and haven't been able to explain the flakiness. What's going on?

[Scott Violet]

Is it at all related to 177163?

[Yoyo Zhou]

177163 is believed only to affect Win debug bots, no? I've been
noticing interactive_ui_tests flaking on other platforms.

[James Cook]

Unscientific, but win_rel seems to be flaking for my try jobs on WebViewInteractiveTest.NewWindow

mac_rel likes to choke on instant extended tests, like:

InstantExtendedTest.ExtendedModeIsOn

InstantExtendedTest.MiddleClickOnSuggestionOpensInNewTab: 

InstantExtendedTest.OverlayDoesntSupportInstant: 

etc.

[Charlie Reis]

+1 to seeing lots of interactive_ui_test flakiness.

As I was sheriffing last week, it was particularly bad in the ExtensionCrashRecoveryTests across various Windows and Linux bots.  I found that https://code.google.com/p/chromium/issues/detail?id=174705 was filed for it, but no one seems to be working on it.

We saw lots of other interactive_ui_tests fail as well, though, so the problem does feel widespread (for both the tree and trybots).

Charlie

[Paweł Hajdan, Jr.]

interactive_ui_tests do a lot of things that are flakiness-prone, like focus, OS-level mouse and keyboard events, and so on.

Please keep filing bugs and disabling flakes you see.

http://chromium-build-logs.appspot.com/ can be useful to see the history of past test failures.

Paweł

[Fady Samuel]

Something very odd is happening with WebViewInteractiveTest.NewWindow. It's failing here on a NOTREACHED() here:

https://code.google.com/p/chromium/codesearch#chromium/src/content/browser/browser_plugin/browser_plugin_guest.cc&q=browser_plugin_guest.cc&sq=package:chromium&type=cs&l=666

This indicates that a shared memory buffer was not mapped successfully. Is Windows running out of memory? Should I simply remove the NOTREACHED() here?

Fady

[Paweł Hajdan, Jr.]

On Thu, May 16, 2013 at 8:27 PM, Fady Samuel <fsa...@google.com> wrote:

Something very odd is happening with WebViewInteractiveTest.NewWindow. It's failing here on a NOTREACHED() here:

https://code.google.com/p/chromium/codesearch#chromium/src/content/browser/browser_plugin/browser_plugin_guest.cc&q=browser_plugin_guest.cc&sq=package:chromium&type=cs&l=666

This indicates that a shared memory buffer was not mapped successfully. Is Windows running out of memory? Should I simply remove the NOTREACHED() here?

Generally I wouldn't be so eager to remove a NOTREACHED() - after all, it was put there for a reason, to alert us about issues the developer putting it there thought should not happen.

[Fady Samuel]

I know. I put it there. :-)

I have no idea why mapping a shared memory buffer is failing. It shouldn't fail. BrowserPlugin expects this to work...

Did anything change about the Windows sandbox recently?

Fady

[Paweł Hajdan, Jr.]

On Fri, May 17, 2013 at 10:36 AM, Fady Samuel <fsa...@google.com> wrote:

I have no idea why mapping a shared memory buffer is failing. It shouldn't fail. BrowserPlugin expects this to work...

I don't know enough about BrowserPlugin to make a judgment here.

Note that SharedMemory::Map can fail, so NOTREACHING() on that seems wrong (you cannot be sure it will never fail - for example the system may legitimately run out of resources).

I can recommend adding more logging to help debug this - chromium-build-logs.appspot.com will collect the failing logs so you'll be able to easily get to them later.

Paweł