UI Testing Standardization

[Thomas Aylott]

Ahoy all,
I'm Thomas Aylott. Board member of MooTools and UI Developer @ Cloudera, Inc.

At the recent Dojo Developer Day we had some very productive discussions about the current state of UI Testing, where we see it going and what we can do to improve it in a cross-platform way. In this thread I hope to get support for making the practical steps necessary actually accomplish something useful quickly.

It is my intention to help 

standardize automated UI Testing 

across all JavaScript environments.

http://wiki.commonjs.org/wiki/UI_Testing

The plan, in a nutshell…

1) Standardize a low-level UI Testing API.

Similar to how the CommonJS Testing API is intended to be a standard low-level testing API that people can build high-level testing solutions on top of.

e.g. click(x,y)  tap(x,y)

2) Create wrappers to map existing UI Testing solutions to the new standard.
e.g. doh.robot, Selenium, Windmill, Syn.js

3) Collaborate with platform vendors to get them to implement the low-level API natively.
e.g. RIM, HP/Palm, Android, iOS, Fake.app, Selenium

4) Collaborate on an emulation UI Test Runner.
Might be more likely than getting native UI Testing support on mobile devices themselves.

5) Collaborate on a higher-level UI Testing solution that uses the low-level API.

e.g. click(myButton)  twoFingerSwipe(myWidget)
…

6) Profit! (j/k)

—

I want (#1) low-level UI Testing API to be designed and agreed upon by the community.

That's the only way that it'll ever get any traction.

I want everyone to be able to write simple tests that aren't entirely implementation-specific.

More than anything, I want all frameworks and all individual UI Developers to actually test their UIs.

I want testing to become an expected part of our jobs.

Eventually I want people to be hired based on their devotion and experience with UI Testing and have that knowledge translate from one environment to another and one job to another.

The low hanging fruit is with desktop browsers like IE & Firefox, but I want us to be forward thinking and make sure that the low-level UI Testing APIs are compatible with desktop, handheld, tablet, embedded (i.e. TVs), etc…

What is the best place to collaborate on these things?

Currently I have a wiki page up here: http://wiki.commonjs.org/wiki/UI_Testing

How are you currently doing UI Testing?

Thanks!

— Thomas Aylott – SubtleGradient – MooTools – Cloudera —

[Fábio M. Costa]

Hi Thomas,

I know some guys are are making an API around Python webdrivers, maybe this can be usefull:

http://github.com/cobrateam/splinter

They're main goal is to be able to use any webdriver they want, without changing any line of code from they're specs.

That's a great discussion.

--
Fábio Miranda Costa
twitter: @fabiomiranda
github: fabiomcosta

[Olivier El Mekki]

Hi,

As Fabio mentionned, there are already implementations in other
languages that can be used as a basis for discussions.

In the ruby world, there are the well-known webrat
( http://github.com/brynary/webrat )

but also good challengers like capybara :
( http://github.com/jnicklas/capybara )

Both can rely either on a virtual environment or on selenium. I think
checking the common methods for those, splinter and other languages
webdrivers api's would be a first step.

> How are you currently doing UI Testing?

Currently, I use cucumber/capybara/selenium. It's really slow , so it
is not intended to be run often. Also, I only test firefox this way.

As I said in a recent discussion in here, I have now to integrate js
tests in hudson. Node.js/jsdom/mootools-runner is where I look for now.

I've looked at JsTestDriver ( http://code.google.com/p/js-test-driver/ )
which can reportedly works with hudson (
http://cjohansen.no/en/javascript/javascript_continuous_integration_with_hudson_and_jstestdriver )
but it's way too much localhost related for me (I can't let some
browsers opened waiting for the remote server running hudson requesting
testing).

Still, its developpers would probably love to have a standard for
emulated UI manipulations.

--
Olivier El Mekki.

[Thomas Aylott]

---------- Forwarded message ----------
From: Dylan Schiemann <dy...@dojotoolkit.org>
Date: Mon, Nov 1, 2010 at 3:11 PM
Subject: Re: [Dojo-interest] Fwd: UI Testing Standardization
To: dojo-i...@mail.dojotoolkit.org


I think it's also worth taking a look at James Coglan's work:

http://github.com/jcoglan/presentations/tree/master/faye-london-ajax/ (
in particular, see
http://github.com/jcoglan/presentations/blob/master/faye-london-ajax/testing/slide.md
) with accompanying video at
http://skillsmatter.com/podcast/ajax-ria/faye-an-evented-server-side-app

Regards,
- -Dylan