webdriver.findElement is incredibly slow if the element is not found because of huge JSON in response

[john russell]

I've narrowed this down to line 492 of
HttpCommandExecutor.createResposne() which calls

response = new JsonToBeanConverter().convert(Response.class,
responseAsText);

When the element is found, the json it parses in responseAsText look
like this

{"status":0,"sessionId":"1320962366867","value":
{"ELEMENT":"0"},"class":"org.openqa.selenium.remote.Response","hCode":
1349374216}

No problem, takes a split second.

However, when it can't find it, the json returned is
Eeennnooouurrrmouss. Mostly because of the stack trace that it
contains with the NoSuchElementException. It takes about 2 full
seconds on my quad core macbook with 8GB of ram to churn through the
JsonToBeanConverter line above with that much json.

Is there any way to cut WAY down on the amount of json returned by the
selenium server when it can't find an element? I mean WAY down. Can I
pick and choose what comes back so I can throw out most of what I
don't want?

Thanks a lot

[Santiago Suarez Ordoñez]

I'm pretty sure automatic screenshots are the reason for this big response body. Here's a bug report requesting them to be optional:
http://code.google.com/p/selenium/issues/detail?id=185&colspec=ID%20Stars%20Type%20Status%20Priority%20Milestone%20Owner%20Summary

Best,
Santi

--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To post to this group, send email to selenium-...@googlegroups.com.
To unsubscribe from this group, send email to selenium-develo...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/selenium-developers?hl=en.

[Kristian Rosenvold]

Interesting; there are two levels of response to this problem:

A) 

You should not be using findElement to determine element non-presence; use findElements() and check for zero-length responses. As a consequence of this, your tests should not be throwing any exceptions that cross remoting boundaries for regular, non-failing runs. I updated the javadoc on findElement about this in r14821.

B) 

That sounds real slow, and I think we should look into it, at least consider upgrading our stone-age old version of the json library. Please file an issue. 

Kristian

2011/11/19 john russell <jjru...@gmail.com>

[Kristian Rosenvold]

Interesting; there are two levels of response to this problem:

A) 

You should not be using findElement to determine element non-presence; use findElements() and check for zero-length responses. As a consequence of this, your tests should not be throwing any exceptions that cross remoting boundaries for regular, non-failing runs. I updated the javadoc on findElement about this in r14821.

B) 

That sounds real slow, and I think we should look into it, at least consider upgrading our stone-age old version of the json library. Please file an issue. 

Kristian

2011/11/19 john russell <jjru...@gmail.com>

I've narrowed this down to line 492 of

[Paul Hammant]

Kristian,

   findElements().length[] == 0  

We need an explicit not_present equivalent in the API.

Without that I'm going to have to employ a number of hacks to get Sauce Labs to not paint lines red like so - http://screencast.com/t/y3W3dQbi

Not paint them red, because occasionally, shock horror, I'm wanting to confirm something's not in the page.

- Paul

[Jason Leyba]

Paul,

Sorry to be blunt, but I think that's a flaw on Sauce's end, not our wire protocol.

The two commands in question are:

1. Find an element on the page using the provided strategy; return an error if not found.
2. Find all elements on the page using the provided strategy; return an empty list if none are found.

Both can be used to test if an element is present in the DOM, so it would be redundant to add a special "can an element be found using X" command.  Furthermore, such a command wouldn't solve your problem unless it was an assert and not a simple query.  There are no other "assert" commands in the wire protocol, why introduce one now - especially when the functionality is offered through existing commands?

-- Jason

[Paul Hammant]

Perhaps you're right in that they are line coloring choices by SauceLabs.  

That said, there's a semantic difference fraught with pitfalls between wd.findElement an wd.findElements.  One throws exceptions, whereas the other does not. One returns quickly, whereas the other may not.

-ph

[Daniel Wagner-Hall]

On 21 November 2011 21:15, Paul Hammant <pa...@hammant.org> wrote:
> Perhaps you're right in that they are line coloring choices by SauceLabs.
> That said, there's a semantic difference fraught with pitfalls between
> wd.findElement an wd.findElements.  One throws exceptions, whereas the other
> does not. One returns quickly, whereas the other may not.

They should take the same time to return. Both respect implicit waits.

[Kristian Rosenvold]

I agree that the semantics involved in searching for not-present elements is a bit too subtle. In its current state it requires a little documentation, maybe something like a faq entry. It's a lot less intuitive than the rest of the api.

We run tests that do a lot of not_present asserts without throwing any exceptions, we simply always look for something that *is* present on the page before looking for something that is not (to assure we have the whole page).

Is it worth adding a whole new method simply to search for not-present ? Personally I'm no big fan of this, especially since "not present" has all sorts of halting semantics and dubious "wait-for" algorithms  attached to it. But that's just my 2C.

Kristian

2011/11/21 Paul Hammant <pa...@hammant.org>

[Mike]

+1

My tests also have some element not present" checks that they do.
They are used to verify that certain dynamic elements are not showing
up at the wrong time.

Perhaps the issue is that the lack of presence test is using something
designed to check for something being present. Adding a method that
would verify, via a boolean result, that something is not present
should provide a correct test that would not incur this type of
inadvertent overhead.

I am sure a lot of other people have the need to occasionally verify
that some element like a button or menu item is not showing up when it
shouldn't. Adding a method to do just that would be an improvement
IMHO.

Mike

On Nov 21, 12:03 pm, Kristian Rosenvold <kristian.rosenv...@gmail.com>
wrote:

[john russell]

Thanks a lot for the workaround. The findElements(locator).size() >
0 cut 10 seconds off of all 1500 of our tests.

I see the point that there might be a more obvious way to do this but
this certainly is a good workaround.
Adding something to the findElement javadoc saying that to do a
negative test to use findElements instead would also be helpful.

Thanks a lot everyone.