Cross Browser Testing tips

[Kenny Chua]

Hi,

Does anyone have tips for writing and running a single suite of tests across multiple browsers (ie FF, Chrome and IE8+) and having the results consistent across said browsers?

From past experience, it is often easy getting tests to perform as expected in Chrome, then quite some time must be spent after in order to tweak the tests for FF and IE (eg try a different approach, explicit waits, etc)

Any advice on this front would be greatly appreciated!

Thanks

Kenny

[Simon Stewart]

Hi,

The first tip would be to run the tests on every browser from the very
start. Don't wait until you have 500 tests before starting the effort
as it'll only lead to a lot of pain.

Secondly, be aware that browsers work at different speeds. Your
IE-based tests are likely to be the slowest and to show the highest
flakiness, as they'll be the ones that highlight when an explicit wait
should be inserted (*sigh*) Also, be aware that different drivers
implement the native events thing differently. Opera and Chrome inject
straight into the event queue, whereas Firefox and IE fire OS-level
events. This means that actions such as "click" are "more async" on
some browsers than others: again, this typically shows up as flakiness
and can normally be addressed with an explicit wait.

Don't use xpath, but if you do, don't use complex xpath. Not all xpath
engines were created equal, and pushing the limits is the best way to
find out the quirks.

That should get you started, but good luck!

Simon

> --
> You received this message because you are subscribed to the Google Groups
> "webdriver" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/webdriver/-/bzwxfCTLUbAJ.
> To post to this group, send email to webd...@googlegroups.com.
> To unsubscribe from this group, send email to
> webdriver+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/webdriver?hl=en.

[darrell]

In  addition to what Simon has said, if it runs on one browser but not another or the results are not consistent, understand WHY it is failing and fix the problem, not the symptom.

Understand that APPEARS to work and ACTUALLY works are two different things. If there is a timing issue with a test case it will APPEAR to work on a fast browser but it will fail on a slow browser. If you get network delays or resource constraints, tests which APPEAR to work will ACTUALLY fail, even on the fast browsers like Chrome.

Darrell

[Andreas Tolf Tolfsen]

Adding to what Simon says…

Unless you have a big infrastructure available to run your tests on,
running them in parallell is a cheap option, and doing so locally on
your workstation helps to find many different problems.

I've found that the most expensive thing about interactive browser
testing is the startup cost of the web browser. So try to keep your
browser sessions for as long as possible. You typically don't want to
start another browser for every single test.

[MichaelF]

Run and run and rerun them, especially when adding in FF and Chrome, since these tend to update automatically a test that ran fine last week will be broken this week due to some browser code changes.  Also be aware that there are differences in how keys are used in the browsers, I often have to change tests as Click will work on one Browser but not another.  Remember that browser drivers with WebDriver tend to be another moving part.

[ravigupta]

Hey Kenny,

You can also follow the path of data driven approach where you keep the list of browsers in an excel and read it accordingly.

And can also follow the path of parallel execution using the TestNG framework as the Junit framework has some limitations.

I am currently using the IE and FF for my piece of testing and yes what is said before me by the people is right you can always keep on simultaneously testing rather executing on a whole.

I m yet test on chrome and safari(as my client have more apple users :) ) ..
 
Hope this would help.

ravi