Implicitly waiting for Ajax requests to complete

[Tim M]

Hi,

I am testing an application which makes heavy use of JavaScript and AJAX, using XMLHttpRequest.

Pages can take a while to fully load, and I am using FluentWait heavily.  However, I am also experiencing intermittent failures on page load where element clicks are not actioned (leading to the subsequent wait failing).

The issue seems to be related to the continuing AJAX requests and long running operations on the Event Queue.

I would like to implement a more implicit wait, waiting for the AJAX queries to complete before continuing following any clicks (at least for page load).

I have looked at overriding XMLHttpRequest.prototype.open to add a counter, using an EventFiringWebDriver.  However, by the time my javascript is executed the AJAX Requests have already been triggered.

Is there any way I can achieve this using standard WebDriver, or would I have to look at extending the Driver itself?  

Or is my approach fundamentally flawed?

Thanks

[Aniket Gadre]

Have a similar issue. I just read that ajaxStop() event handler can help notify all AJAX calls are completed. Can this event be injected or overridden after navigating to page to update some variable which can tell us all AJAX calls are completed? and then keep polling for this variable value?

just a thought. I am not sure even if this is a feasible solution

Best Regards,

Aniket Gadre

 

“Programming can be fun, so can cryptography; however they should not be combined.”

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/af15ac62-90fc-49ba-bc66-51435ba24825%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sudhansu Sekhar panda]

On Tuesday, 9 September 2014 15:20:54 UTC+5:30, Sudhansu Sekhar panda wrote:

Hi Tim,

In my application there are many places where AJAX has been used..When you click on some element the progress bar will load and you dont know when it will be disappeared...so for that reason i have used the below code and it worked for me....

Wait<WebDriver> wait = new WebDriverWait(driver, 120);

wait.until(ExpectedConditions.invisibilityOfElementLocated(locator));

Here locator: It is the id/xpath of the progress bar ... Please let me know if it helped...

There is one more way you can also try...after navigating to the browser(in the navigate method where you are launching the browser) just set the below code globally...

driver.manage().timeouts().setScriptTimeout(120, TimeUnit.SECONDS);

The above code will automatically wait if you are doing any AJAX operations...

Thanks, 

Sudhansu ,

Mindfire Solutions

http://www.mindfiresolutions.com/

[Tim M]

Hi Aniket,

ajaxStop is a jQuery specific feature.  This may work in your case if the AJAX requests are created using jQuery.  However, in my case they are created using the standard XMLHttpRequest API, which means that ajaxStop() always returns 0.

[Tim M]

Hi Sudansu,

I already make heavy use of FluentWait, waiting for various elements to appear/disappear.  However, there are situations where clicks are still not registered (intermittent).  It is either because I'm not waiting for everything to finish, or similarity between pages means that the wait conditions appear satisfied but are actually relating to the previous page (due to timing issues).

AFAIK, scriptTimeout() relates to scripts started by Selenium only.  The AJAX requests are all from the standard application so I don't think this will help in my case. 

[Xiang Dong]

HI Tim,

Using javascript to override XMLHttpRequest.prototype.send method via web driver, set XMLHttpRequest instance to a javascript 

variable in your send method. then, periodically using web driver to read the readyState from XMLHttpRequest instance. if 

readyState is 4,it means Ajax request done.

Best Regards,

--david

---

Date: Tue, 9 Sep 2014 17:19:05 -0700
From: tim.mye...@permeance.com.au
To: seleniu...@googlegroups.com
Subject: [selenium-users] Re: Implicitly waiting for Ajax requests to complete

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/435f0476-cfda-4c86-8ad5-76727b633f2f%40googlegroups.com.

[Tim M]

Hi David,

Wouldn't this be effectively the same as overriding XMLHttpRequest.prototype.open?  As mentioned in my original post, I have looked at doing that, but it isn't a reliable enough measure, since by the point my Javascript has executed the AJAX requests have already been fired.

Where is the appropriate point to run the Javascript?

At the moment I'm using WebDriverEventListener.afterNavigateTo() as a prototype (if it proves to be effective, I can expand it for clicks too).  However, by the point it fires, it's already too late, since the new page has started loading and AJAX requests have already fired.

Hence my original question: Is there any way I can achieve this using standard WebDriver, or would I have to look at extending the Driver itself?  

I'd prefer not to have to extend WebDriver itself, but perhaps that would be the only way.

[Xiang Dong]

I think it is almost the same override the open and send method, the core is delegate XMLHttpRequest instance to an external 

javascript variable. So, you webdriver code can access it.

My did is inject such javascript code into AUT web page after a new page loaded or before a ajax request triggered. such as, click a button will trigger an Ajax request, before it, you can inject the javascript to the web page, then click the button, then periodically retrieve readyState from AUT web page. 

--david

---

Date: Tue, 9 Sep 2014 18:56:36 -0700
From: tim.mye...@permeance.com.au
To: seleniu...@googlegroups.com
Subject: Re: [selenium-users] Re: Implicitly waiting for Ajax requests to complete

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/7b411a79-acb5-4996-a0b1-ccb46e0e513c%40googlegroups.com.

[Tim M]

Hi David.

Thanks for the reply.  It sounds like my approach *might* work then.

How did you inject it after a new page loads?  

Doing it before click should be more reliable, since I have much more control over the timing - I can be confident that the AJAX request is instrumented before I perform the click.  But the trouble I'm having is how to do the same for page load.  Especially since most of the AJAX I'm wanting to wait for is triggered very soon after the page loads.

[Xiang Dong]

if you inject before trigger ajax request, just remember does not inject twice or more in one same web page. I think it is easy for you to avoid it.

for the page load, it is a trouble because ,such as the ajax is triggered in javascript onload event, at this time, you almost no opportunity to inject your javascript codes into web page, if you can control your source code, you can add such javascript in your source code. If not, I guess waiting some condition is a possible solution.

How did you inject it after a new page loads?

I think you must want to know how to inject js before onload event, I am sorry it seams not easy. Add a http proxy may works but I never try it.

--david

---

Date: Tue, 9 Sep 2014 19:15:13 -0700

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/02d28b09-6437-479d-b4fa-2fe470412577%40googlegroups.com.

[Tim M]

for the page load, it is a trouble because ,such as the ajax is triggered in javascript onload event, at this time, you almost no opportunity to inject your javascript codes into web page, if you can control your source code, you can add such javascript in your source code. If not, I guess waiting some condition is a possible solution.

I was hoping to avoid modifying the source since I'm trying to focus on black box testing only.  But it's always a possibility. 

How did you inject it after a new page loads?

I think you must want to know how to inject js before onload event, I am sorry it seams not easy. Add a http proxy may works but I never try it.

I'd thought of that, but it's adding unnecessary complexity to the solution.  Perhaps it's the only viable solution though (or extending WebDriver).

[Xiang Dong]

sure, you are right, no matter extend web driver or set up a proxy to do it will increase your test complexity. but for a test purpose,

what we want to test is the result. like trigger ajax request from onload javascript method, what we need to know is what's the 

expect result. such as we make a criteria, a web element will be visible after 5 seconds, so, for this case, we can keep check the 

condition in 5 seconds, if we find it in the time, we can consider the test success, otherwise, we could consider the test failed.

If i am wrong, please correct me

--david

---

Date: Tue, 9 Sep 2014 19:33:22 -0700

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/b6ffcc04-ef40-4a91-bbb5-c5ee65ddb36e%40googlegroups.com.

[Krishnan Mahadevan]

If you introduce a proxy such as browser mob proxy then you could inject anything before the page even reaches the browser. Never tried doing it but I think it definitely is a possibility.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/b6ffcc04-ef40-4a91-bbb5-c5ee65ddb36e%40googlegroups.com.

[Tim M]

You're right, I'm concerned with expectations.  The path I've been looking at has all been to do with stability of the tests rather than checking my expectations per se.

I'm getting the situation that clicks aren't working sometimes, meaning my expectations aren't met.  However, the end result is that confidence in the tests is diminished, since they sometimes work and sometimes don't.  Perhaps my effort would be better spent creating a test case to clearly demonstrate the issue.

Perhaps it is actually a bug in the application I am testing, but performing the same operations manually does not display the same behaviour.

[Xiang Dong]

The biggest challenge for automated test is the race condition between your test program and AUT application. That's why selenium provide some implicit wait mechanism. But I think it may not enough for guarantee our test case health enough.  Sometimes we pursue the test speed to much, such as we always consider sleep for a while is a bad behavior in the test case,

but, consider human test, they also have such break. we must find a balance between them. I think you want to develop a Ajax request interceptor (monitor) is also comes from this thinking.

My did in my test is using interceptor to monitor the Ajax request status, page loading status. Those interceptors help me a lot in judge whether next step is ready. Below is my code in monitoring ajax request:

XMLHttpRequest.prototype.oldSend = XMLHttpRequest.prototype.send;

var watp_ajaxRequest = null;

var newSend = function(vdata) { 

watp_ajaxRequest = this;

this.oldSend(vdata);

} 

XMLHttpRequest.prototype.send = newSend;

function watp_getAjaxState(){

var watp_ajax_done = null;

if(watp_ajaxRequest == null ){

return null;

}

if (watp_ajaxRequest.readyState == 4) {

             if (watp_ajaxRequest.status == 200) {

watp_ajax_done = 'done';

}

}

return watp_ajax_done;

}

hope it help u.

---

Date: Tue, 9 Sep 2014 20:17:49 -0700

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/93cb2c5c-30e7-4567-80d0-9a261053075a%40googlegroups.com.