Internet Explorer is unstable

[Mercious]

Hey there,

i am really struggling hard trying to get my tests stable with InternetExplorer.

Google Chrome and Fire Fox both run 100% stable, they pass all 100 out of 100 tests.

But Internet Explorer fails around 30 times, randomly, uncalled for.

I have observed the issue and it seems that sometimes the browser doesn't understand that the page is loading. 

In WebDriver, as far as i know, it is automatically tested if the DOM's ready-state is set. If you submit something then Chrome and FireFox both automatically wait until the page has finished loading (at last its html content).

Internet Explorer does that too and it works, but not all of the times! It works in 7/10 cases ... .

I dont do any fancy with java-script. I would understand that this is complex and requires more insight, but it's really only the pages html content. 


I dont know why Internet Explorer randomly fails to realize if the page is still loading or not.

How do you guys handle the Internet Explorer ? Does it run absolutely stable for your tests? And what would you do in my case? Do i have to insert WebDriverWait's for every page-submit that i make? That's kind of stupid :/

[Mercious]

Anyone on this? 

We are supporting internet explorer for our web application so it is quite crucial to test the browser in selenium as well. 

Am i the only one having problems with it?

[sunny sachdeva]

Pls share the versions of IE and Selenium that you are/intent to use.

With IE 10 and IE 11, you might want to look at 

https://code.google.com/p/selenium/issues/detail?id=6511

Thanks

Sunny Sachdeva

[Sudhansu Sekhar panda]

Hi Mercious,

Yes the rendering process varies from browser to browser. Please let us know what are the particular act ivies failing consisntenly and what you code you are writing for that also. The IEDriver version also matters some extend in this regards.

Thanks,

Sudhansu

[Mercious]

Hello, thanks for your replies.

I have tried both IE9 and IE11, both are unstable by the same way.

I am using the selenium version found here : http://selenium-release.storage.googleapis.com/index.html?path=2.45/. And i use the IEWebDriver version found there as well (32 bit, as the 64bit is apparently buggy).

What i am doing code-wise is quite basic:

I fill some input fields with data and then .click() a button. The button is definitely found and click'ed at, because if it wasn't found then i would receive a different exception.

Clicking that button invokes a page load. And InternetExplorer sometimes waits for this page-load to return to ready and sometimes doesn't. 

Then, i try to locate a certain element and it's value. After a successfull login it should afterwards say ("LOGOUT").

If the InternetExplorer did not wait for this page to reload, then this check will fail because the value wont get be changed. 

But that isn't any JavaScript. It's plain HTML. And the Browser should be able to wait for HTML-DOM to be set to "ready". 

Both google chrome and firefox manage to do this in 100/100 cases without a single error. 

[Jim Evans]

Clicks in WebDriver are explicitly *not* guaranteed to be synchronous. There is no guarantee whatsoever that the browser will wait for a "complete page navigation" (whatever that means) after you click on an element. Most drivers will make a "good faith effort" to be synchronous, the Internet Explorer driver included, but there's no guarantee it will work. There are technical reasons for why this is so in the IE driver, boiling down to an unavoidable race condition when using so-called "native events" for user interactions, including mouse clicks and keyboard keystrokes. You can turn native events off with the IE driver by using a capability, which will force the driver to use events simulated using only JavaScript. Examples of how to set the capability are multitude, and finding one for your language of choice is left as an exercise to the reader. However, there is a chance that you lose some fidelity from what an actual user would do, and you run several risks, including, but not limited to, your WebDriver code hanging if the event you're triggering fires a JavaScript alert, or that the event generation done by the driver isn't exactly the same as that done by a native mouse click.

For further reading on how page loads are handled in the IE driver, I would refer you to this comment in an issue report in the Google Code Selenium issue tracker*: https://code.google.com/p/selenium/issues/detail?id=232#c11. If you find the approach of the IE driver lacking in this respect, pull requests are welcome.

--Jim

*NOTE: Link provided for historical value only. Issues in the Google Code issue tracker are no longer receiving any attention. All development on the project, including issue reporting, has migrated to GitHub.

[Mercious]

Hello and thanks for your post, interesting information you provide there!

So basically what you are saying is that the internet explorer, for whatever reasons, cannot properly handle an implicit or lets call it automatic detection of the pages ready-state? Just to make sure i got the problem right.

Taking from the way you describe the solution of turning off native events as well as various information on the internet this cannot be considered a solution. Despite that fact i will test if theoretically would fix the problem. 

However, i cannot eliminate one error and make myself vulnerable to many others. 

So what is a proper solution then? Do i really have to pollute my entire code with WebDriver-Waits, various ExpectedCondition-types, etc. ?

Sometimes .click() fail as well. I have found some SO-discussions about this issue and they have addressed various hacks around this problem, involving clicking twice, using JavaScript to click, obtaining focus and whatever. 
And then, many posts in the same thread, prompted that none of the suggested hacks worked consistently. 

If i really have to make my code a huge mess of various hacks and fixes just to end up with tests that are STILL not COMPLETELY stable for internet explorer, then i really see absolutely no appeal.

I understand that it might be complex to implement proper behavior for that IEWebDriver given the existing architecture ... but it's still quite non-neglect able that the other two browsers are doing so much better. 

And again i wonder ... Is anyone using Selenium featuring the Internet-Explorer and has at least 99% stable tests with it? 

[Jim Evans]

In order to detect a document navigation reliably, one must hook into the IWebBrowser2's DocumentComplete event. Then when one clicks on an element that causes navigation, one need only wait for the DocumentComplete event to fire to know that the navigation has completed, and one can reasonably assume that some definition of "document is loaded" is met. However, there are two problems with that.

First, not all element clicks cause document navigation. Yes, there's a corresponding NavigationStarted event, but one would have to wait some arbitrary amount for that to fire (on Every. Single. Click). So listening for a start to navigation on element clicks isn't practical.

The other issue is that native events happen outside IE's normal event handling mechanism. IE doesn't know about the mouse click or keyboard keystroke until it gets into it's processing queue. If driver's "check for document loaded" logic completes *before* IE has fully processed the click, then the driver has no reason to wait any further.

If you have another, more reliable mechanism to resolve these issues while remaining aligned with the other goals of the WebDriver project, I'm happy to hear them. Again, I put it to you, if you're unhappy with how the IE driver works, patches and pull requests are quite welcome. Failing that, a full, consistently reproducible test case that demonstrates the problem you experience would be an acceptable fallback.

To answer your questions, yes, using WebDriverWait at the appropriate places is the right approach. I would submit to you that there's very little practical difference in your code between

    myFoundElement.click();

and

    myFoundElement.click();
    waitForElement(byLocatorToFindTheNextElement);

where you define "waitForElement" as a helper method that hides the so-called "pollution" or "ugliness" of WebDriverWait in your code.

But please, spare me the discussion and complaints about how "the other two browsers are... so much better." The driver for Firefox, while open-source, is developed and maintained by employees of the Mozilla Corporation, which manufactures Firefox. The driver for Chrome, while open-source, is developed and maintained by members of the Chromium project and employees of Google, which manufactures Chrome. Microsoft has no material input into the development of the open-source IE driver. Their employees are even forbidden to browse the source code of the open-source project, in order to appear above reproach when it comes to copyright and patent legal issues. But complaining about how bad the driver is, or maligning the work of developers who provide it on a purely volunteer basis, particularly without any regard for the technical issues involved in producing that work, that's counterproductive and insulting.

[Mercious]

I wasn't aware of the fact that the development of the IE-Webdriver is community driven. I had assumed that it was, just like the for the other browsers, by Microsoft employees. I aplogize for striking the wrong tone regarding that fact, i didn't mean to insult anyone. 

Have you simply looked through the code or did you actually try and develop a solution to the problem that you had described?
Reading your short description of the architecture ... What speaks against offering a special "click" API that would definitely wait for such DocumentComplete-event to be returned before allowing further execution? It then would lay in the hands of the user of the API to determine when a click causes page navigation and when not. 

That would give me the ability to actually achieve what i want because as page owner i surely know what clicks causes a page-navigation and which don't. 

Then there is still the second problem you mentioned, which i, frankly, don't quite understand yet. I would have to look into the code to fully understand. But maybe there is a way of assuring that what you described doesn't happen. The fact that this solution might come with a lot of overhead would not matter, because you would provide this only to a special click-function that is KNOWN to be inefficient and ONLY for clicks that cause page-navigation. This way it would again be in the users hand and responsibility to use what is suited. 

As for the improvements to take in the code ... : Well, it's not always that simple. Consider the element that i am looking for already exists on the page and now i want to see if it's text has changed upon a finished page-navigation. 

I assume that the ExpectedConditions.textToBePresentInElement does not actually re-fetch the passed element again and again. 

So i would have to write custom code to achieve this, possibly a while loop that uses a custom timer to fetch the element again and again and check it's content.

That is definitely possible, but you can already assume that it's going to bloat up the code by a lot more than what you have given as an example. 

I would really love to craft a utility class, so that i wrap the drivers API with a more "secure" API, meaning i use .click() but internally i have waits and all. But there are many "custom" cases, so that i would have to provide a lot of methods. Like clickAndWaitForTextToBePresentInElement(WebElement, String, boolean), the boolean for re fetching the element or not, etc. 

There is a thing i wonder though ... Are the chrome and Firefox browsers designed better? Or why was it able for them to achieve this page-navigation thing securely? 

You said employees designed the drivers, but employees aren't necessarily better. They might have some more time on their hands, but i don't think that this is it.
Probably those two actually adjusted their browsers API to work with Selenium?

[Mercious]

Let me add another question regarding the Internet Explorer:

What is suggested for .click()'s that randomly fail? I have tried setting nativeEvents to false and it seems to have fixed the problem.

But it doesn't sound like it's the proper way of handling this case.

Is there a workaround that is really consistent? I read that it's supposed to be browser-focus specific, meaning the problem only occurs when the browser is not in the foreground. Then running this browser in a VM would fix the problem, as nothing else could steal focus.

But i have tested only this single browser and the error still occurs from time to time, so this is not a solution either. 

[Krishnan Mahadevan]

@Mercious

While I am in no way an SME on the WebDriver when compared to Jim Evans (Who is the owner of the IEDriverDriver component) am sharing some thoughts of mine

>>>Reading your short description of the architecture ... What speaks against offering a special "click" API that would definitely wait for such DocumentComplete-event to be returned before allowing further execution? I

I dont think this can be done unless and until the w3c specs for WebDriver are altered to accommodate a special API [ https://w3c.github.io/webdriver/webdriver-spec.html#clicking ].

 I don't think that only InternetExplorerDriver is going to be exposing this special API because that would basically be a violation of the WebDriver contract that all the other WebDriver implementations adhere to.

I still feel that you could basically build up a special ExpectedConditions utility class which is similar to existing one [ https://github.com/SeleniumHQ/selenium/blob/master/java/client/src/org/openqa/selenium/support/ui/ExpectedConditions.java] which would handle all these kind of special needs and then have just 1 special click method which basically accepts an ExpectedCondition as an argument. 

Since every test basically knows the type of clicks that it is going to be going through, the test could basically formulate the actual ExpectedCondition to use and then pass it on.

When we built SeLion [ http://selion.io ] we resorted to something like this. You can take a look at it here : https://github.com/paypal/SeLion/blob/develop/client/src/main/java/com/paypal/selion/platform/html/AbstractElement.java#L532

We basically started off by classifying WebElements to be TextBox, RadioButton etc.,. All these WebElements would have a common parent which would define the most common things that can be done with any html element and then the specific classes would basically have only the specific capabilities.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/60099dab-8bc5-4f06-a187-39413f4c7e83%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Mercious]

Oh well, i understand. 

But if it is only a problem of specs and it would technicallybe possible, then maybe it's time for whoever leads the Selenium project to allow some adjustments. I am sure they could come up with some agreement respecting this case.

--- Anways ---

So, i have further investigated the behavior of Internet Explorer.

I have come to the conclusion that it requires extreme code alterations to make it stable in a parallel universe.

Clicks fail randomly because focus might be stolen (i assume), page navigation is not detected and i am sure that a more complex case might come with more complex errors.

I now have only addressed the click and page-navigation in my test-case and it already adds so much complexity.

Consider that you cannot assume that a click worked in Internet Explorer. So EVERY click that i do has to be somehow be RETRIED. So i have to write a for-loop that tries clicking the element and then somehow evaluates if the click was successful. For example

• click some element
• see if another element appeared
• if not, try to click some element again
• rinse and repeat for x amount of seconds before failing

And that with every click that i do.

I wouldn't feel so bad about it if it was possible to write some generic use API. But consider the case: How can i write a generic API if there is no generic way to tell if the click succeeded?

So my API would have to take some kind of "wayToSeeIfClickSuccessfull", maybe some kind of Strategies.ELEMENT_APPEARED etc.

But you can already tell that this API would be really weird and not user friendly at all, because it tries to deal with such non generic problem. 

I can't believe that no one else has stumbled upon problems of the internet explorer yet. I am looking for users that actively support the Internet Explorer in parallel test enviroment. I want to know what exactly they wrapped and how exactly they made it safe. 

I am thinking that, maybe, for the .click() i could apply a lock on the thread, force browser foreground and the click. But i have also read that this still fails eventually. Sometimes stuff like a double-click is necessary.

I don't have time to try this all out, so i would be really glad if someone could share some experiences. 

I will also look into your project Krishnan, i am looking forward to find some hints. Thank you!

[Jim Evans]

The requireWindowFocus capability does something similar to what you propose. It attempts to bring the IE window to the foreground before interactions. Of course, this is another "best effort" approach, because Windows doesn't provide any absolutely reliable way to bring a window to the foreground programmatically. I've documented a fair amount of this in a couple of blog posts, which I'm sure you've already come across at http://jimevansmusic.blogspot.com.

It sounds to me like turning off native events may be your best bet. That *should* work to give you interactions while avoiding the race conditions inherent with native events. Unless, of course, you hit a situation where the simulated events aren't exactly what you want (note that there's every chance you may never encounter such a situation), but that's a legitimate trade-off you may be willing to make.

[Mercious]

Hey again!

Yea, i have come across your posts and they surely were an interesting read!

However i don't feel like migrating my code completely once more (going from WebElement.click() to Actions). 

Turning nativeEvents off seems to be no solution to the clicking-problem. I don't exactly know why, because from what the problem is described like, nativeEvents false should fix the problem but in reality it really doesn't. 

I feel like i will take the way of requesting a dedicated server that is solely used to host tests with InternetExplorer. I will have a single node with maxInstances of 1 running on that node and that's it.

I have tested the browser alone and it seems to be stable that way. The only thing i had to adjust was considering that page-loading events are not properly detected, thus i made my code bulletproof towards this mistake.

And i am STILL interested in knowing what others do. I cannot quite believe that noone tried running browsers parallel on the same machine without every realizing how often that makes the IE fail. 

[Jim Evans]

I'll leave you with one final thought, since you seem utterly disinterested in actually improving the driver to better handle your use case. The blog post describing the requireWindowFocus capability is over two years old. In that time, the base click() functionality has been migrated to respect the capability, so there should be no need to convert your code to use the advanced user interactions API (the Actions class in Java).

[Mercious]

Interesting, so that definitely changes the case and it obviously is worth a try, thank you. 

As for your supposing that i lack interest of improving the driver: Well, i will definitely look into the project once i have the available free time to invest into it, but quite frankly ... If it has been worked on since years and there hasn't been found any possible solution to it yet ... What are the chances that i will find one?

I am actually quite eager to learn something about it but i cannot make my companies test dependent on my success of changing the driver, thus i will have find a solution for the moment. 

I really just intended to shortly test what Selenium has to offer and how it could be integrated into a Jenkins build process. I investigated Selenium for a week, set up a server and then wrote a test. It's taking more and more time since i had began to learn that parallel executing challenges the Internet Explorer beyond its capabilities. 

At this point i just want to deliver a working setup that fulfill it's use. If that convinces, then i have all the time of the world to investigate further. 

[Without being deeply involved, this feels like a war of conventions anyways. I don't understand why such strict definitions of the behavior are made if these lead to those problems. It's absolutely senseless to limit a developer this way, since it ultimately causes troubles, which can be seen. I don't even know if there is any kind of communication between the two parties, but it should be addressed if people are interested in bettering the situation (I am not saying that such is terrible, but i think we all can agree that potential is wasted and it could probably be better, even without Microsoft getting involved to fix the driver)]

[Shawn McCarthy]

The way I run my IE tests (and Chrome and Firefox) in parallel is by having maxInstances = 1 for my grid nodes. Each node will only run 1 browser at a time. Won't this solve your focus issue?

[Krishnan Mahadevan]

We haven't seen any of our users complain about having problems with IE for the automation tests that we run.

We rely on the Grid execution environment to support our test execution and we configure each node such that it can run 

15 instances of Firefox

5 instances of Chrome and 

1 instance of IE

We have a concurrent execution limit of around 5, which means a test that runs on IE can very well be sharing the same machine with another test that is perhaps running on Firefox or Chrome for that matter.

In either ways, I believe it would be a good thing to just resort to running only 1 IE based test on a machine [ for which you dont need even a huge machine ]

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/b8f2e219-affe-4f11-899e-eb631935830a%40googlegroups.com.

[Mercious]

@Shawn Nope, that doesn't fix the problem. It's not only a second Internet Explorer instance that can fight for focus, its basically every other application running that could steal focus at the wrong time.

So running Chrome/Firefox tests while IE is testing on the same machine will cause trouble for the Internet Explorer, no matter what i do.

I investigated requireWindowFocus and it improves the situation by a lot but does not entirely fix the issue. 1/10 tests still fail.

A simple wrap around .click() that applies a lock to the thread if its the internet explorer that is supposed to perform it and then forcing the window to foreground, clicking and releasing the thread might actually work.

But that would require a little more knowledge of .click(). Sometimes IE does wait for page-navigation. And if the .click() is the one that is blocking (waiting for the end-event to fire) then this lock would be way too long and it would make multi threading useless, since the huge performance win comes from the I/O waiting. 

@Krishnan 

So what are the steps you have taken in order to make your Internet Explorer stable while other browsers are running on the system. Have you altered the way .click() works in any way? Have you set nativeEvents to false? Have you set requireWindowFocus to true? It would be cool if you could eloaborate a little, because to me it currently looks like there is a lot of sensitive tweaking required to making IE run stale parallel to other browsers. 

[Shawn McCarthy]

Hi Mercious, I am saying to not allow more than 1 browser total to run at once on a selenium grid node.

[Mercious]

I understand what you are saying but it is not a solution.

I run more than one grid node on the same system. So other browsers will run even if i only allow one internet explorer to run on one node.