Sporadic StaleElementReferenceException in IE

[Tester]

From time to time I always get a StaleElementReferenceException in IE
on different elements. This is very sporadic. The last time it
appeared was at the following example:

TestUtils.wait(driver, timeout, By.id("element_id"));
List<WebElement> elementList =
driver.findElements(By.id("element_id");
WebElement element = elementList.get(1);
element.getText();


From TestUtils:
public static WebElement wait(WebDriver driver, int timeout, final By
locator){
FluentWait<WebDriver> wait = new FluentWait<WebDriver>(driver)
.withTimeout(timeout, TimeUnit.SECONDS)
.pollingEvery(1, TimeUnit.SECONDS)
.ignoring(NoSuchElementException.class)

WebElement element = wait.until(new Function<WebDriver,
WebElement>() {
public WebElement apply(WebDriver driver) {
return driver.findElement(locator);
}
});
return element;
}


In every browser this works fine but in IE it crashed with the
following stack trace:
org.openqa.selenium.StaleElementReferenceException
Message: Element is no longer valid (WARNING: The server did not
provide any stacktrace information)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:
57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:
45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at
org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:
170)
at
org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:
123)
at
org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:
439)
at
org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:
231)
at
org.openqa.selenium.remote.RemoteWebElement.getText(RemoteWebElement.java:
135)

I mean, this is driving me crazy.. Why is it working in every other
browser, but not in IE? My wait-method doesn't crash, but a few
milliseconds later it crashes? And I can assure there is no JS or
anything else which is changing the DOM in the background! There are
only a few milliseconds between the wait and the getText-method. I
don't get it what I am doing wrong...
This is just an example, it happens on other examples too.

[Chema del Barco]

Hi!

Have you tried catching the StaleElementReferenceException? It usually
appears with Ajax content loading, so I'd suggest you to catch the
exception and try the interaction again (driver.findElement(locator)).
You can also add an implicit wait for the element
(driver.manage().timeouts().implicitlyWait...) instead of polling
every second, which can save you some precious and cumulative
milliseconds :)

[Tester]

Hi,

what exactly does this driver.manage().timeouts()implicitlyWait? I
already have a line like this in my test:
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

I saw it in a tutorial, and I just tried it, but I am not sure, what
it does exactly. So even with this line the IE crashes from time to
time.

There is no AJAX loading in the background.

[Chema del Barco]

I suggest you to check Selenium's documentation regarding explicit and
implicit waits:

http://seleniumhq.org/docs/04_webdriver_advanced.html#explicit-and-implicit-waits

Basically an implicit wait makes the driver to wait a set time for an
element to be found, meaning an statement such as
"driver.findElement(By...)" will poll the site every 250ms (this poll
time is also configurable) until the element is found before throwing
a "NoSuchElementException". The statement you point is setting up a 10
seconds implicit wait for all the "find" operations.

This known, what I see here is that you're implementing a
"waitForElementPresent" with a 1 second-poll explicit wait and a 250ms-
poll implicit wait of 10 seconds... Pretty messy, don't you think? Try
every option below separately to see if it fixes your problem:

- Comment the "pollingEvery..." line in your code
- Comment the "driver.manage().timeouts().implicitlyWait..." line in
your code
- Add ".ignoring(StaleElementReferenceException.class) " after
".ignoring(NoSuchElementException.class)"

Did you see any improvement?

[darrell]

A StaleElementReferenceException usually occurs because between
getting an element reference and using the reference the DOM changes
(usually due to Ajax or some sort of javascript). The problem probably
exists in the other browsers but the timing on the other browsers is
different and therefore the problem remains hidden.

Why are you using findElements() and By.id()? If there are more than
one tag with the same id then file a defect report. There should never
be two tags with the same id. If there is it is a violation of the
HTML standard.

You have the TestUtils.wait() method which appears to wait for a given
WebElement. It returns the WebElement once it is found. You don't
check the results of the TestUtil.wait() method. It is possible that
the element does not exists or the page never finishes load.
Essentially, you are assuming the element will be found and returned.

Why aren't you using the return value for the TestUtils.wait()? Can't
you just use:

WebElement element = TestUtils.wait(driver, timeout,
By.id("element_id"));
element.getText();

Your code seems overly complicated.

Darrell

> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcce ssorImpl.java:
> 57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstru ctorAccessorImpl.java:

[Tester]

I've tried the suggestions you gave, chema del Barco. But there are
still errors occuring, like sporadic WebDriverExceptions (why? there
is no error text), InvalidSelectorExceptions and still
StaleElementReferenceException, although I added
".ignoring(StaleElementReferenceException.class)" to the wait-method.
This StaleElementReferenceException now appears when I do the
following:

TestUtils.wait(driver, timeout, By.class("...));
List<WebElement> elements = driver.findElements(By.class("...));
WebElement element = elements.get(0);
return element.getText();

@darrell: There is no problem with double ids. I just provided a
source code I made up (it is not the original source code with the
right IDs and selectors and stuff). So i just made a mistake, but the
logic is the same like the original source code. you're right,
findElements and By.id can't work. I just change it to By.class in
this post.

[Tester]

Oh I forgot to say where the StaleElementReferenceException appears in
the code. It appears when I call getText() of the element. Two lines
before it got the elements and then it fails saying
staleelementreference? I can't believe it. I don't know what to do
there.

[Tester]

So the question here is: How can I implement a wait method which works
for that? Therefore I have to implement for every method (getText(),
getAttribute(), sendKeys(), findElement() etc) an extra wait method
which is ignoring all those exceptions until a certain amount of time.
Is this serious? Only for IE?

[darrell]

The best way for us to help you is to trim down the code to a bare
minimum and point it to a web site we can access. This is not always
possible. For example, the application I am currently testing is web
based but we deploy it internally and behind a firewall. So I trim the
code to a bare minimum and see if I can find another website which
exhibits the same problem. Often, as I'm doing this I find the problem
goes away. I then have to look at what is different from my
environment and the external website.

When something happens occasionally I know it is intermittent and
probably a timing issue. If it happens on IE but not Firefox (or vice
versa) then I'm pretty sure it is a timing problem. If for example you
have:

- get web element
- do something
- do another thing
- get text
- StaleElementReferenceException

I would change the code to:

- get web element
- print System.currentTimeMillis()
- do something
- do another thing
- print System.currentTimeMillis()
- get text
- StaleElementReferenceException

If I get the problem on IE but not Firefox I can look at the time
between the getting of the web element and the call to get text. If it
takes 1.2 second on Firefox and 2.1 seconds on IE I would suspect the
delay of 0.9 seconds is enough for the DOM to change and the web
element reference to go stale. If it wasn't my web code I'd talk to
the developers and see if there is something which alters the web
page. I can also see it in things like Chrome's inspect window. I have
one page which if I inspect an element using Chrome's built-in
equivalent to Friebug and leave it for a while, I can see the DOM
refresh in the inspect window.

I then need to either update the reference to the web element then use
it right away or talk to the developer and see if I can disable
whatever is refreshing the DOM.

If you post the actual code, even if we don't have access to the
website being tested (alter the URL and navigation code) we might be
able to point out ways to make the code more efficient.

Another possibly is you are causing the reference to go stale. For
example, you want to get all the anchors on a page, click them to make
sure they go to some valid page, i.e. not a 404 error. You might do
something like:

List<WebElement> anchors = driver.findElements(By.tagName("a"));
Iterator<WebElement> i = anchors.iterator();
while(i.hasNext()) {
WebElement a = i.next();
a.click();
// check for valid page
driver.navigate().back();
}

This will throw a StaleElementReferenceException. The moment you click
the first anchor all the other references become stale. So when you
navigate back and try clicking the second anchor it throws the
exception. You have to find some alternative solution, e.g.

List<WebElement> anchors = driver.findElements(By.tagName("a"));
int numAnchors = anchors.size();
for(int i = 1; i <= numAnchors; i++) {
WebElement a = driver.findElement(By.xpath("//a[" + i + "]");
a.click();
// check for valid page
driver.navigate().back();
}

The body of the loop is pretty much the same except I get each web
element as I need it.

Bottom line, you need to figure out what is changing the DOM
(javascript code, meta refresh, your code) and find some way to deal
with it. The most important thing to remember is if you give us what
you think is the equivalent code you are probably leaving out
something important. If you knew what you are leaving out, you
probably wouldn't need to post your question. That is what often
happens to me. In my attempt to create an equivalent code snippet I
realize what the real problem is (it is never what I thought it was).

Darrell

[Tester]

Hi,

wonderful answer darrell. It is very helpful. Okay, I have to test a
web shop based on Magento. Our shop is internal, so I can onyl give
you a test URL of the official Magento shop: http://enterprise-demo.magentocommerce.com/
But it is commonly the same. I have to test the checkout procedure, a
customer puts an article into cart and switches to the cart page. He
or she then changes the number of the article and the total price of
the article changes. And that is where I get problems with IE. Of
course some JS is changing the price in the background and I have to
wait until the price was changed. But how? To know if the price
changed I have to get the text of the price element and that gives me
of course a StaleElementReference. So I came up with a workaround as
follows:

private String doUnsecureAction(By location, int position){
long start = System.currentTimeMillis();
String result = null;
do{
try{
List<WebElement> prices = driver.findElements(location);
WebElement price = prices.get(position);
result = price.getText();
}catch(Exception e){/*ignoring any exception*/}
if(result != null) return result;
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}while((System.currentTimeMillis() - start) / 1000 < timeout);
return result;
}

This is only for IE (because it works in every other browser without
that workaround). This code tries to get the price text and gives it
back if it is not null. If a stale element exception appears it gets
ignored until a specified timeout is reached (in my case it was 30
sec). This works fine for me now, but this is the most dirtiest design
of code I have ever seen until now. Ignoring exceptions is not a good
choice (in my understanding of good programming) and imagine, this is
just a workaround for this special thing to get the prices of the
cart. What about all those other things to test? Do I always have to
make such a workaround only for IE? That is a lot of effort...
Don't get me wrong, this works, but it looks not right to me, if I am
honest. But thanks so far for your help. I am interested in your
opinion about that piece of code.

[darrell]

Interesting approach. Rather than preventing the reference from going
stale you retry whenever you get a stale reference. I'd keep the code
a little similar. Calls to system routines make you code run fairly
heavy. I'd just use:

private String doUnsecureAction(By location, int position) {

String result = null;
const long maxTime = 5000; // time out in milliseconds
const long waitTime = 500; // time to wait each loop
long i = 0;

do {
try {
List<WebElement> prices = driver.findElements(location);
WebElement price = prices.get(position);
result = price.getText();

break;
} catch(StaleElementReferenceException sere) {
// do nothing
}

// sleep for 'waitTime' milliseconds
try { Thread.sleep(waitTime); } catch(InterruptedException ie)
{ ie.printStackTrace(); }

} while((i += waitTime) < maxTime);

return result;
}

Having the variables at the top of the method will let you adjust time
out and wait times. There is no need to explicitly check the value of
'result'. There are two reasons to leave the do-while loop. One is
that you timed out (the while condition handles that) and the other is
that you used the element reference and it was not stale. If you make
it to the break statement, there was no
StaleElementReferenceException. If there is a stale element then the
loop will execute again. Additionally, it is bad practice to catch
Exception when you really are looking for a specific exception. If you
are trying to handle StaleElementReferenceException then handle
StaleElementReferenceException. If some other exception occurs you
want it to get thrown so you know about it and can deal with it.

This code assumes that Thread.sleep is going to be accurate enough but
being off by a few milliseconds should not matter for this purpose.

[Tester]

Okay, thank you very much.

But once again, do you think this is worth producing that much code
for IE-driver? In my test suite there are more problems like that one,
so I have to implement a work around every time for that. Is that the
approach of WebDriver/Selenium? I am not really sure about that.

[e-lena-s]

hi, I have a followup question on that. Running into a similar
problem, but in FF.
The solution above doesn't seem like is a great approach for a large
scale tests.
In my test I have implemented a util to wait for en element to be
present prior to doing anything further with it

public void waitForElementPresent(final By by, int timeout){
WebDriverWait wait = (WebDriverWait)new
WebDriverWait(driver,timeout).ignoring(StaleElementReferenceException.class);
wait.until(new ExpectedCondition<Boolean>(){
@Override
public Boolean apply(WebDriver webDriver) {
WebElement element = webDriver.findElement(by);
return element != null && element.isDisplayed();
}
});
}

This code sporadically passes (mind you the timeout is 15 seconds -
I'd say it's unreasonable for a modern UI app, but still I let it wait
for that long just in case) . Following this call I assert the text of
the element . Sometimes I can get passed all lines of code and have a
successful test, sometimes I can get passed the wait, but fail on the
getText() sometimes it times out waiting for the element - even though
I can see that my element is visible on the page.

Now, many of my tests are much more involved then just checking this
test, and they keep randomly failing. Is there any solution to this ?
It doesn't seem right to me to wrap every test and action in do/while
like you've suggested above.