API for checking for JavaScript errors on the page

[Marc Guillemot]

Hi list,

I've made some work that could become the base to answer the issue 148
"API for checking for JavaScript errors on the page"
(http://code.google.com/p/selenium/issues/detail?id=148#c7).
Daniel Wagner-Hall asked me to send an email here with the proposed API,
the capabilities it adds and the browser requirements. Before to start
such a discussion, I'd like to hear what would be the interest from the
project members for such features (beyond JavaScript errors, all
errors/warning displayed in Firefox or Chrome developer tools could be
interesting). My current feeling is that it is not particularly high but
I would be happy to be wrong ;-)

Cheers,
Marc.
--
HtmlUnit support & consulting from the source
Blog: http://mguillem.wordpress.com

[Jason Leyba]

This feature has been on my wish list for a while.  A JS error detection feature should have three main components:

1.) The ability to detect and record any global errors on the page.  While recording generic console messages would also be useful, that should be handled separately.

2.) Automatically throw an exception on the client when an error is detected.  99% of the time, an uncaught error is not an expected condition, so developers shouldn't have to keep requesting the recorded errors.  Instead, after executing each command, the driver should return an error if there were any uncaught errors on the page.  Since errors could occur between commands, before starting a new command, the driver should check for uncaught errors. If there were any, it should immediately fail, returning the errors to the user.  For backwards compatibility, as well as to support users that don't care about global errors, an API should be provided to disable automatic error reporting.

3.) Provide an API to query for recorded errors.  Ideally, errors would be returned as a list of {message:string, file:string, line:number} JSON objects.  Each time you request the recorded errors, the driver's internal state should be cleared.  This would be similar to alerts: if there's an alert present, you need to handle it before continuing with the test.  Likewise, if there were any uncaught errors, you should handle them before continuing.


The client API could be pretty straight forward:

   WebDriver driver = new FirefoxDriver();

   // Do we want auto-detection enabled by default?
   driver.manage().enableJsErrorDetection(true);

   driver.get(“http://www.example.com”);
   driver.findElement(By.id(“foo”)).click();

   // Make sure no new errors have occurred since click()
   // This is actually redundant since driver.quit() would auto-fail
   // if there were errors, but demonstrates how devs could
   // explicitly check for errors.
   assertTrue(driver.manage().getJsErrors().isEmpty());

   driver.quit();

Like I said, this feature has been on my wish list for a while.  I've given it a lot of thought, but it's never been a high enough priority to try to implement.  I think this would be pretty straight forward for the WebKit browsers and Firefox.  IE probably won't be so easy, but nothing ever is with IE :)

-- Jason

> --
> You received this message because you are subscribed to the Google Groups "Selenium Developers" group.
> To post to this group, send email to selenium-...@googlegroups.com.
> To unsubscribe from this group, send email to selenium-develo...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/selenium-developers?hl=en.
>
>

[Ross Patterson]

I’ve had a pure-JavaScript version of this for Selenium 1.0 for some time.  It mimics the existing getAlert/isAlertPresent interface, adds an ignoreJavaScriptErrors to enable or disable it, and resembles Jason’s proposal except for the JSON response, returning instead one error per getJavaScriptError call the way getAlert does.  It injects the necessary hook for window.onerror exactly the same way as the alert support does.  That probably means that it has the same can’t-intercept-during-onload-handler problem that the alert support does, but I’ve never tried to confirm that.    The code isn’t large (~150 lines of JS in user-extensions.js form, probably less when merged into the main codebase), and it’s hardly novel, but it does the job.

 

The first time I turned it on in a moderately large established test suite, it found a handful of bugs.  It also sped up the failure of the tests in question, because the error was detected nearly immediately, rather than waiting for an expected change to the page and timing out, and provided significantly better diagnostics for the developers to fix the bugs.

 

I know the Project is usually not interested in enhancements to the RC API and the 1.0 releases, but if desired, I could look into getting permission to hand it over.  I expect it wouldn’t be of any value to the WebDriver API.

 

Ross

[Adam Goucher]

I would argue that this sort of thing is valuable enough to consider adding to the RC side of the house. (Using the RC-isnt-going-away-anytime-soon-in-the-market logic.)

Barring that, if it was say put on Github or somewhere similar people who want it could just get it from there.

-adam

--

[Rostislav Matl]

Sounds useful. I'd invite such functionality, especially if it will be
possible to turn this behavior off.
Still, if I had to choose, I'd prefer accessing the console log
messages.

> On Tue, Oct 11, 2011 at 11:52 PM, Marc Guillemot <mguille...@yahoo.fr>

[Patrick Lightbody]

Don't get me wrong, I'd also like to see this. But if I put my Simon Stewart hat on, then isn't this out of scope for Selenium? Similarly, doing things a user can't do (clicking on an invisible element), detecting page performance, or having a more nuanced understanding of page load events has also been declared out of scope.

Simon? :)

--

Patrick Lightbody

+1 (415) 830-5488

[Andreas Tolf Tolfsen]

On Mon, 17 Oct 2011 16:17:14 +0200, Patrick Lightbody
<pat...@lightbody.net> wrote:

> On Oct 17, 2011, at 12:46 AM, Rostislav Matl wrote:
>
>> Sounds useful. I'd invite such functionality, especially if it will be
>> possible to turn this behavior off.
>>
>> Still, if I had to choose, I'd prefer accessing the console log
>> messages.
>

> Don't get me wrong, I'd also like to see this. But if I put my Simon
> Stewart hat on, then isn't this out of scope for Selenium? Similarly,
> doing things a user can't do (clicking on an invisible element),
> detecting page performance, or having a more nuanced understanding of
> page load events has also been declared out of scope.

Just a small heads-up: We'll (Opera) be adding support for inspecting the
console to OperaDriver soon.

I'm fine with that being a driver-specific extension, though.

[Jason Leyba]

Perhaps...except a user can view uncaught exceptions in the error console :)  Besides, being able to know that a WebElement.click() failed because the onclick handler threw an error would be immensely useful.

[Jason Leyba]

Sounds useful (and most browsers have a console these days anyway).  Will you be able to retrieve messages at different levels (info vs. error) or just a flat log?

 

--
You received this message because you are subscribed to the Google Groups "Selenium Developers" group.

To post to this group, send email to selenium-developers@googlegroups.com.
To unsubscribe from this group, send email to selenium-developers+unsub...@googlegroups.com.

[Paul Grandjean]

it's a great idea, I'm sure we would use it at my work.

To post to this group, send email to selenium-...@googlegroups.com.
To unsubscribe from this group, send email to selenium-develo...@googlegroups.com.

[Ross Patterson]

Access to the console log might be nice, and it would fulfill the original request ("I would like to be able to check if the browser indicates that an error occured when processing JavaScript."), but it wouldn't be in keeping with the rest of the RC and WebDriver APIs. I personally believe JavaScript errors should be regarded and handled similarly to alerts, confirmations, and prompts. I can, of course, see the counter-argument: the user doesn't usually see the errors, unlike alerts et al.

Ross