Fitnesse and Selenium

[Giridharan]

Please let me know the difference between Fitnesse and selenium.

One of my project teams uses Fitnesse for web application testing.....But mine is Ajax based application. Can i use Fitnesse tool for this?

[fj]

Well, here's my 2-cents worth -

I evaluated FitNesse a while back and like the approach very much. However, my bottom-line answer is:

1) selenium is just a testing tool; FitNesse is a testing tool, integrated with a documentation system.

2) if you need to test web systems via the browser, Selenium probably has significantly superior test capabilities.


Both let you construct a tree-like hierarchy of tests whose success/fail outcomes are rolled upwards to the "root". Selenium, which I like very much, really has no facility for incorporating documentation. The concept for using FitNesse is that you'd document your system's requirements with the Wiki that's part of FitNesse. Heck - if you wanted to stop there, you could, and you'd just have your system documented. However, you can then return to the Wiki pages and insert tests of your system so you could, for instance, have a sentence defining a requirement followed immediately by a test table wherein that requirement is tested. It's a great concept and I like it very much.

My experimentation with FitNesse, however, was such that it does not have any tools for testing via the browser that are nearly as powerful or easy to use as Selenium. I won't spell out the details here, but there were one or two primary tools ("fixtures" in FitNesse/junit speak) repeatedly cited as being for testing a website via the browser, and upon examination, I found them to be cumbersome and limited as compared to Selenium's capabilities.

So, to summarize:

Selenium is your choice if you:
- do not need an integrated documentation system
- are testing via web/browser interface, especially if you have javascript that must execute, e.g. these "AJAX" sites that are the "hype-du-jour".

FitNesse is your choice if you:
- really want to document your requirements and optionally integrate them with your tests
- are not testing a website, or if you are, that the AUT has very elementary testing demands with respect to popups, javascript functions, etc.

Additional caveats/advice:
1. My basis is an evaluation of FitNesse that I did many months ago, so my info may be inaccurate now, esp regarding web testing fixtures. I first came across Selenium some months ago, shortly after having experimented with FitNesse, and at that time, the ease with which I could make Selenium (using selenese tests, not using the "driven" mode) drive a moderately heavy AJAX website made FitNesse's web testing fixtures and capabilties appear fairly rudimentary by comparison. I haven't checked FitNesse's web testing fixtures for a number of months; they may be much better now, you'll have to check for yourself.

2. There are more factors to consider, should you find yourself in a head-to-head comparison of the two tools. Here are a couple that come to mind:

2.a. think about exactly how you'd incorporate the FitNesse wiki with your existing version control and overall project setup.

I can't imagine anybody wanting to use the inherent versioning of the FitNesse wiki for any project. It's fine if you're running a wiki-based website where you're just presenting articles and content, but it is not adequate compared to using a "real" versioning/change management tool like starteam, vss, cvs, subversion, etc.

Fortunately, it's easy to turn this feature off in FitNesse. That still leaves you with figuring out how to incorporate FitNesse wiki's for multiple individual projects, using it thru branches & merges, and setting up to use the imported pages feature successfully on your projects.

Based on my old info about FitNesse, the tests, hence the wiki, must reside on the system where the AUT runs, much like Selenium core must in order to run selenese-based tests. One can surface remotely-hosted wiki content in FitNesse using some of its native capabilities, and http://fitnesse.org/ has (or had, at least) a recommended approach documented for how to setup a team of developers with their own local FitNesse instances running against their localhost sandboxes, and a central project wiki with the AUT server, etc.

Now, all this appears "do-able", but depending on your needs, may require some careful thought to setup something that works well for you. Selenium, both "core" and "rc" are much simpler to deploy into your dev process & project setup.

Once you have all that figured out for FitNesse, you have to get all your team trained on how it works -- remember, any non-trivial software project includes lots of folks besides just developers -- and how to use it without screwing it all up. Then, you have to get folks trained to enter content into the wiki properly. That's not hard, but it is one more thing folks would have to think about, e.g. "camel-casing" words, for example. So, there's some considerable planning, setup, and training overhead you should think about with really making a FitNesse solution realize its full potential for you.

2.b. SSL. Currently, there are some serious concerns regarding Selenium's ability to test SSL web pages. I never got into FitNesse enough to determine its capabilities in that area. So, I have no answers for you, only the tip that you should examine FitNesse's and Selenium's SSL capabilities compared to your needs.