Blocking API or not?

Showing 1-11 of 11 messages
Blocking API or not? Jim Evans 7/13/10 4:59 AM
In several places where the question of a so-called "blocking API" for
the WebDriver API, we have positioned the toolset as being "mostly
blocking". I think that's a perfectly reasonable compromise for a
potentially sticky set of issues. However, I'd like to get a feel for
the group's opinion on what "mostly blocking" really means. I've been
thinking about doing some work on the IE driver core, but before I go
spelunking into that particular morass, I thought I'd see how people
felt about the issue. As a general design philosophy, should we be
leaning toward more blocking in our implementation or less? It seems
to me that the "principle of least surprise" that many of us value
highly would dictate striving toward more blocking where possible, but
I'd like to get a sense of how others see it.

I am particularly interested in the following use cases:
Page navigation (including framesets and opening new windows)
Clicks on elements (which may or may not trigger a navigation)
Submits of form and control elements

At the moment, the .NET unit tests are littered with Sleep()
statements in many of these use cases. We've done some good work to
provide mechanisms in the API to help with synchronization (implicit
waits, Java's wait class in the Support .jar, and so on), but mightn't
it be better to design our implementations so as to reduce the use of
such mechanisms to the exceptional case rather than the common one?
Note that I am explicitly omitting the AJAX case from this discussion,
as that, to me, is a slightly different problem.

I'm really interested in what the group has to say on this, and
whether there is consensus on it or not, so I'm looking forward to the
replies.

Regards,
--Jim
Re: Blocking API or not? QA_manager 7/13/10 6:19 AM
As a user, I greatly value a blocking API.  It makes the learning
curve much easier when things 'just work', as opposed to failing due
to timing.  It benefits new users and experienced users alike.
Re: [selenium-developers] Blocking API or not? fcohen 7/13/10 7:07 AM
Hi Jim:

I find it much easier to teach Selenium commands that have no blocking. If you write a test script to click an element and nothing appears to happen within the Web app, Selenium did its job of clicking the element.

I also strongly recommend against using sleep. Sleep means different things to different computers and operating environments. In an event-oriented design such as Selenium its better to use state sensing constructs like waitForElementPresent.

The Ajax conversation could be pretty simple: Selenium needs to define a callback mechanism to identify when a component is idling. This includes an API for components to register themselves. And, an API for the Selenium test to see if one or all of components are idling. If Selenium had this API it would become a standard among all the Ajax toolkits (GWT, YUI, Titanium, etc.)

-Frank


--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To post to this group, send email to selenium-...@googlegroups.com.
To unsubscribe from this group, send email to selenium-developers+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/selenium-developers?hl=en.


--
Frank Cohen, http://www.PushToTest.com, phone 408 871 0122
PushToTest, the open-source test automation company
Twitter: fcohen, LinkedIn: Frank Cohen


Re: Blocking API or not? Jim Evans 7/13/10 7:44 AM
Thanks for your input, Frank. This is exactly the kind of discussion
I'm wanting to spark.

I should have specified here that I'm talking explicitly about the
WebDriver API, not the "Selenium Classic" one. Outside of the implicit
wait timeout, there is no direct equivalent to "waitForElementPresent"
in the core WebDriver API [1]. Would you agree or disagree that we
should at least be waiting until the browser reports a navigation is
completed (whether initiated by an explicit navigation, an element
click, or a form submission) before moving to the next step? Why or
why not?

You'll get no argument from me that using any sort of sleep function
is a bad idea, whether the one provided by the Selenium API or an OS
API call.

Regards,
--Jim

[1] Yes, I'm aware of the Java language binding's wait class, which
may not be implemented in other language bindings. I'm also well
acquainted with the idea of polling over and over to check for the
existence of the element you next want to act upon. Popup browser
windows are a little more problematic, but manageable. Nevertheless, I
think these constructs make for less maintainable and less elegant
test code, but I'm not so religious about it that I'll argue the
point.
> Frank Cohen,http://www.PushToTest.com, phone 408 871 0122
> PushToTest, the open-source test automation company
> Twitter: fcohen, LinkedIn: Frank Cohen- Hide quoted text -
>
> - Show quoted text -
Re: [selenium-developers] Re: Blocking API or not? fcohen 7/13/10 1:51 PM
Hi Jim: Great discussion. It seems to me that WebDriver (as a product) needs a transformation utility to go from Selenium Classic (I like that name) to WebDriver-compliant scripts. It would be handy to have the wait commands implemented as some sort of helper class too. -Frank

--
Frank Cohen, http://www.PushToTest.com, phone 408 871 0122

PushToTest, the open-source test automation company
Twitter: fcohen, LinkedIn: Frank Cohen


Re: [selenium-developers] Blocking API or not? Aditya Ivaturi 7/13/10 4:05 PM
"Blocking" has to occur at some point - either at the API level or at
the end testscript level (if it isn't a blocking API). But if more &
more people are going to rely on IDEs, which seems to be the trend
(and I don't advocate it at all), it makes sense to have a more
blocking API. And as someone already said in this thread - it is more
user friendly.

--aditya

Re: [selenium-developers] Blocking API or not? fcohen 7/14/10 12:10 AM
Just curious about the reasons behind your avoidance of IDEs? Do you mean an IDE as in Selenium IDE and QTP?

-Frank



--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
To post to this group, send email to selenium-...@googlegroups.com.
To unsubscribe from this group, send email to selenium-developers+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/selenium-developers?hl=en.


--
Frank Cohen, http://www.PushToTest.com, phone 408 871 0122
PushToTest, the open-source test automation company
Twitter: fcohen, LinkedIn: Frank Cohen


Re: [selenium-developers] Blocking API or not? Simon Stewart 7/14/10 6:57 AM
I totally agree that the principle of least surprise suggests that a
blocking API is preferable. There's a few things that make this
difficult:

* The first thorny is issue is what do we mean when we say a page is
"loaded"? Is it when the "load" event has fired? Or that the page
quiesced? When there are no outstanding XmlHttpRequests left? When
there's no more javascript to left to be executed via "setTimeout"?
Any of these definitions can be argued to be "wrong". None of them are
universally "correct"

* Especially when using native events, it's hard to tell when a user
interaction will push the app into a state where blocking would be
required. If refreshes are decoupled from the flow of JS execution
(eg: using setTimeout) then things get even trickier.

* There are multiple race conditions between issuing an event and
figuring out that the page is reloading. Guaranteed, we'll lose that
race every odd now and again.

Given these constraints, the "best' we can probably do is to follow a
two-fold approach:

1) Return from commands as soon as is reasonable, but sniff for page
reloads before executing the _next_ command.

2) Educate users to be more explicit about what they're actually waiting for.

As an example of that second point, if I'm waiting for a page to be
ready for me to log in, all I need is for the username, password and
submit button to be present. I don't need the footer of the page or
ads to have been loaded. Waiting for the last of these elements in the
DOM (probably the submit button) is clearer than "wait until the page
has loaded".

WebDriver offers mechanisms to help with this. The implicit waits we
added recently allow you to wait for the presence of an element in the
DOM. It's possible that we could use the same timeout for checking for
visibility where that is a requirement too. To complement implicit
waits, we also have the "Wait" interface and "WebDriverWait"
implementation where we want to spell out the interactions, and the
"PageFactory" and the "ElementLocatorFactory" implementations for use
with Page Objects.

I think that we're doing a reasonable job balancing blocking and
non-blocking right now. It's certainly not perfect, and there's always
room for improvement (as some of the emails I see spell out clearly)
and I acknowledge that. If anyone's got practical advice on how to
improve things, I'm all ears :)

Simon

> --
> You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
> To post to this group, send email to selenium-...@googlegroups.com.
> To unsubscribe from this group, send email to selenium-developers+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/selenium-developers?hl=en.
>
>

Re: Blocking API or not? Wes Winham 7/14/10 7:53 AM
I'm currently very happy with the state of blockingness(?) of the api.
In our case, it gives us flexibility to choose when we do and don't
care about the state of certain javascript actions. We use a bit of
library code for our "blocking" whenever it's required that certain
things have happened (usually functions that get called via jQuery's $
(document).ready) and we've found that it's a nontrivial task figuring
out when a particular page is actually "loaded" in the sense that a
user would consider loaded. Sometimes we don't care if all of the Ajax
requests have completed, for example, but sometimes we do.

I think good libraries built on top of Selenium (conceivably as
separate projects in a thousand-flowers-bloom kind of way) can solve
the issue of non-blocking behavior being confusing for beginners. It
would be conceivable to write a library around selenium, for example,
that wait for the load event to fire, waits for all XmlHttpRequests to
finish and maybe even does something super fancy like taking advantage
of different JS library's ready-style functionality to ensure that all
of those events are completed. That would certainly make selenium
easier to pick up and run with, but that would be much too heavy-
weight for people who are already familiar with the tradeoffs.

-wes
Re: [selenium-developers] Blocking API or not? Daniel Wagner-Hall 7/14/10 9:53 AM

One issue I've found is the combination of native events and waiting
for elements to be present.  If the page has not fully "loaded",
elements may move as more elements are rendered, and so native clicks
may click at the wrong coordinates.  Certainly having hooks to detect
load-states could be handy, i.e. haveAllXhrsLoaded, isPageIdle, ...
methods.

Re: [selenium-developers] Blocking API or not? Simon Stewart 7/14/10 11:35 AM
((JavascriptExecutor) driver).excuteScript("return document.readyState
== 'complete';")

will indicate whether the document is complete. This works on (at
least) modern Firefox versions, IE and Chrome.

Simon

> --
> You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
> To post to this group, send email to selenium-...@googlegroups.com.
> To unsubscribe from this group, send email to selenium-developers+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/selenium-developers?hl=en.
>
>