Waiting for a Page to Fully Load in Browser Tests

[Christopher Sharp]

I've been looking into a flaky Autofill Test (AutofillTest.ProfilesNotMergedWhenNoMinAddressData) and I seem to be having trouble executing some JavaScript before a page is fully loaded which is causing some flakiness.

The flake occurs in FillFormAndSubmit, which loads a page via ui_test_utils::NavigateToURL then uses some JavaScript to fill the form and submit it. When the test fails it is because the JavaScript fails to find any of the document elements (getElementById returns null). I added some logs and when the test fails, NavigateToURL seems to have returned before the onload function in the html page is executed. This seems to suggest that ui_test_utils::NavigateToURL is returning before the page is fully done loading.

Looking at NavigateToURL, it seems to support returning only once the page has finished loading (I'm calling ui_test_utils::NavigateToURL(browser(), url)), but since the JavaScript is failing to find an element it seems like this isn't the case.

Looking around the code base, it seems to be assumed that NavigateToURL will wait until the page is fully loaded, like I need. Is there something else I need to listen to if I want to ensure all the elements will be visible to my JavaScript, am I using NavigateToURL incorrectly, or is there some weird bug I might be hitting in it?

Thanks in advance for any help,

C#

[Charlie Reis]

In general, there's often confusion about what it means for a page to "fully load" in tests.  Some tests just wait for the navigation to commit, which isn't the same as when the NOTIFICATION_LOAD_STOP occurs (i.e., when all frames have reached their onload event).  Even after that, a page could have timers that continue to modify the page indefinitely.

It looks like NavigateToURL is using TestNavigationObserver, which watches for NOTIFICATION_LOAD_STOP.  That seems like the right thing to me, so you might look at what the page is doing or what state it's in when the test proceeds.

Charlie

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Christopher Sharp]

What do you mean by look at what the page is doing or it state? The page is just an html form, it doesn't contains any JavaScript code at all, so it should always have the same state on load (I believe). I had added the onload function myself just to log when it got there, but the test is flaky without that.

C#

[Patrik Höglund]

Hi!

I think I'm hitting the same problem here. Sometimes javascript functions are randomly missing (and sometimes different functions are missing), so it seems the parsing of the javascript races with the browser test. This only happens on Mac for now though. Could just be that the timing is different there. Note: we do have an window.onload() defined, which according to the definition should run precisely when all scripts and content have been loaded. That event runs before I hit my missing function.

Either there's a bug so onload runs too early, or there's a bug that NavigateToURL returns too early. Perhaps the latter is more likely.

[1417:1799:0829/035052:2221800663418263:INFO:CONSOLE(217)] "Once again trying to open web socket", source: http://127.0.0.1:59559/files/webrtc/video_extraction.js (217)

[1417:1799:0829/035052:2221800665623615:INFO:CONSOLE(232)] "Opened WebSocket connection", source: http://127.0.0.1:59559/files/webrtc/video_extraction.js (232)

[1417:1799:0829/035052:2221800671231594:INFO:CONSOLE(63)] "Uncaught ReferenceError: debug is not defined", source: http://127.0.0.1:59559/files/webrtc/getusermedia.js (63)   <----- !?!?!?!?!?!?

/ P

[Paweł Hajdan, Jr.]

Right, I think there is a problem somewhere (how helpful is that?).

I don't really understand navigation logic in Blink well enough to help here. I'm adding people that understand it better, and can point you in the right direction.

I'd be happy to help with the test side - once we know where the problem is and what to wait for, I can make our testing infrastructure do the right thing.

Paweł

[Garrett Casto]

Is there reason why TestNavigationObserver/NavigateToURL is waiting on NOTIFICATION_LOAD_STOP instead of WebContentsObserver::DidFinishLoad()? I just tracked down a related issue where my expectation of "done" was different from NavigateToURL() for this reason. I know basically nothing about Blink, but looking at WebContentsObserver it calls out the fact that onload has been dispatched by DidFinishLoad() but makes no such claims about DidStopLoading().

[Paweł Hajdan, Jr.]

On Fri, Sep 6, 2013 at 2:49 PM, Garrett Casto <gca...@chromium.org> wrote:

Is there reason why TestNavigationObserver/NavigateToURL is waiting on NOTIFICATION_LOAD_STOP instead of WebContentsObserver::DidFinishLoad()?

At least historical ones, I'm not aware of any others. Note that NavigateToURL code originates in times much earlier than WebContentsObserver.

 

I just tracked down a related issue where my expectation of "done" was different from NavigateToURL() for this reason. I know basically nothing about Blink, but looking at WebContentsObserver it calls out the fact that onload has been dispatched by DidFinishLoad() but makes no such claims about DidStopLoading().

Feel free to send changes my way. I also recommend adding at least one reviewer more deeply familiar with Blink.

[Patrik Höglund]

Hi!

I've investigated the issue with too-late javascript loading we've had on the bots. The problem isn't that the javascript loading happens too late; it's that the load gets starved out. I've filed a fresh issue with the findings. I think need help to understand what happens here, and why it only happens on certain machines. So basically the renderer process seems to get stuck halfway loading the page, and it will not finish loading until main test thread (is that the IO thread in browser_tests?) dies. Interestingly, it also works to do NavigateToURL twice. This will fully load the javascript and the test will succeed.

Umm, any ideas of what is happening here? This particular issue started happening on our Linux bots with 100% reproducibility around the 17th of November.

/ P

[Paweł Hajdan, Jr.]

On Thu, Dec 12, 2013 at 1:51 PM, Patrik Höglund <phog...@google.com> wrote:

I've investigated the issue with too-late javascript loading we've had on the bots. The problem isn't that the javascript loading happens too late; it's that the load gets starved out. I've filed a fresh issue with the findings.

This is weird. You're saying the load gets starved out - but since NavigateToURL returns, could you explain more what you mean by "starved out"?

 

So basically the renderer process seems to get stuck halfway loading the page, and it will not finish loading until main test thread (is that the IO thread in browser_tests?) dies. Interestingly, it also works to do NavigateToURL twice. This will fully load the javascript and the test will succeed.

Test codes executes on UI thread in browser tests.

Workarounds like Navigating to URL twice suggest there is a bug somewhere. Could be in the test infrastructure, might also be in the browser (or Blink).

Paweł

[Patrik Höglund]

Hi!

 

This is weird. You're saying the load gets starved out - but since NavigateToURL returns, could you explain more what you mean by "starved out"?

Indeed. I mean that NavigateToUrl returns, but the javascript in the page does not load. It seems it will not complete loading until 1) the IO process dies 2) we load the page twice (which restarts the renderer process, right?). Any ExecuteScript invocation from the test will not be able to see the loaded javascript. So it seems the javascript parsing and loading gets starved out by something. See the bug for more details.

 

 

So basically the renderer process seems to get stuck halfway loading the page, and it will not finish loading until main test thread (is that the IO thread in browser_tests?) dies. Interestingly, it also works to do NavigateToURL twice. This will fully load the javascript and the test will succeed.

Test codes executes on UI thread in browser tests.

Workarounds like Navigating to URL twice suggest there is a bug somewhere. Could be in the test infrastructure, might also be in the browser (or Blink).

It's also weird that this only happens on very particular machines, like chromeperf58.chrome. It reproes 100% there but doesn't on my workstation. Some things I thought about:

• The test launcher launches each test with --single_process.
• The bots do run stuff under xvfb/openbox.
• We have had other mysterious errors on the same linux bots, so perhaps they're set up in a particularly borked way. I know we had pulseaudio problems because the X environment didn't initialize correctly, but I think that has been fixed.

Since this is such a local problem I'm fine with going with the workaround. I'm not sure if any of the browser/blink guys should be worried though.

/ P

[Paweł Hajdan, Jr.]

On Fri, Dec 20, 2013 at 3:25 PM, Patrik Höglund <phog...@google.com> wrote:

Indeed. I mean that NavigateToUrl returns, but the javascript in the page does not load. It seems it will not complete loading until 1) the IO process dies 2) we load the page twice (which restarts the renderer process, right?).

What do you mean by "IO process"? I don't think reloading the page restarts the renderer.

It'd be great to get more comments about that from content/Blink experts on this thread.

 

It's also weird that this only happens on very particular machines, like chromeperf58.chrome. It reproes 100% there but doesn't on my workstation. Some things I thought about:

• The test launcher launches each test with --single_process.

To avoid a possible confusion, this is mostly internal to the test launcher, and tells the executable to execute the actual test instead of being the launcher. Note that this is different from --single-process (which affects the behavior of the browser). 

• The bots do run stuff under xvfb/openbox.

This should be easy to reproduce locally. Please let me know if I can help more with this. 

Paweł