Automated Web Site Testing

[Ed Brandmark]

I am interested in some suggestions for tools (or even if it is
possible) to test some web site code that I develop.

I write JavaScript code that is included in web pages and allows web
pages to use function calls to make various things display. This
could be a popup window, a popunder window, content into an IFRAME,
content directly into the page, etc.

The testing that needs to be done is that

a) this works on all browsers(Netscape 4 and above, IE 4 and above,
Opera)on various platforms (mac, windows 98, windows 2000, win XP)

we do use some imaging software - so that we can setup different
images on the same machine - ie different browsers (since only one IE
can be installed at a time - I think) and so on.

b) this displays correctly - I know this is somewhat arbitrary -
perhaps something that can capture pictures and write which test,
which browser etc.

c) needs to check correctly for various plugins - such as Flash - so
if possible be able to install and uninstall plugins.

NOTE: The test version of the JavaScript prints out whether it thinks
you have Flash installed - SO - again perhaps screen captures.

d) popups must be tested - and determined if for example a popup
appears or a pop-under appears and what is in focus and if the popup
closes by itself in a certain amount of time and/or if the popup
closes when you click on its contents.

e) when you click on some content - does it open a new window, load
over the current page, get stuck in the current IFRAME, etc.

But on the positive side:

a) No load testing necessary
b) No link testing necessary (ie valid links - we do need to test if
the target of the link works)

Any testing help - that can automate this would be appreciated.
Currently someone sits on various machines, installs and uninstalls
various browsers and plugins, and hits various web sites (that are
setup to test) and has to figure out if all displays well. Creating
the web sites is NOT a problem - and proabably needs to be done to
create the various combinations of tests. BUT currently I am limited
in how many test cases I can create NOT by creating them BUT by the
complaints of the testers.

I'm not sure that this kind of testing is feasible.

I've looked around and there seem to be a million tools - starting
with Mercury Interactive's Astra and Silk's Segue - but before I get
started evaluating these tools and spending a lot of time - I figured
I'd try to get some expert opinion from this group. Reading the
previous posts shows there is lots of expertise in this area - but
most of what I read is load testing, or function testing - like does a
link or pulldown work.

I can't imagine how this is done - but perhaps something that just
runs through all these tests, does screen shots, saves them and we can
print them - and flip through them - that would help. Anything that
relieves the level of testing, the tedium and boredom, and the ability
for us to create additional test pages - people get tired of running a
lot of tests - so if we had some automation we could run more tests.

Currently we create 'test' web pages, that call the various functions
and in some cases - write debug messages to the browser window (since
this is client side JavaScript - I can't write to a file) - to show
what is happening.

Thanks a lot.
Ed Brandmark

[evalid]

eValid, based on IE DLLs, is a full-featured browser that will do
all of the below except (a). It will handle popups, modal dialogs,
and do detailed timings. IFRAMEs are no problem. There is extensive
capability for validating content. And, while not free, is much more
cost effective than its counterparts. See the eValid Buyers' Guide
at <http://www.soft.com/eValid/guide.html>. Get an EVAL license
from <lice...@soft.com>. Complete details at <http://www.soft.com>.

[evalid]

You may wish to look at eValid, which will do almost all of what you
need. eValid, being built into a web browser, can do timing/tuning,
page mapping and metrics calculations, validation, and a lot of other
test functionality...all with 100% realism.

Check out the product at <http://www.e-valid.com> and get a license
from <lice...@soft.com>.

[lschwartz]

Elfriede Dustin is a fairly well-known author and lecturer on the subject
of software test automation. He has published a comparison of functional/
regresssion test automation tools and made it available to the public at
the following URL:
http://www.qualitywebsys.com/AppB.pdf (Adobe Acrobat document).

That comparison may save you a bit of work in your tool evaluation effort
(I use it as a teaching tool in my classes).

Thank you, Mr. Dustin, wherever you are.

===================================
Leonard Schwartz
Instructor
High Technology Program
Santa Clara Adult Education
www.scae.org
(SCAE is a member of Mercury Interactive's Academic Access Program and is
a training provider in the use of several test automation tools.)

[lschwartz]

Elfriede Dustin is a woman, therefore, in the 'parent' posting:

Wherever the masculine word "he," "him," or "Mr." occurs, substitute with the
appropriate feminine word.

Thanks to the folks out there who actually read the biographies of authors,
and informed me of her correct gender.

=====================
Leonard