I'm working on the business case for Watir right now. Why do you use it? Were you using something else beforehand? Did you compare Watir to other tools? Is Watir faster? more reliable? easier to use? Were you just following the crowd?
I'd love to hear your reasons. If you'd like to stay anonymous, send me your answers to me directly and I'll summarize.
We started out using it for regression testing, because the other tool we were using had some limitations with Javascript. We had thought about using it for awhile but we hired a Perl programmer and figured he could learn it easily, which he could. Of course, we also wanted to be cool and learn Ruby.
We found that it was so powerful, we could do a lot more with it. The most valuable thing we do with it is to set up data and scenarios that greatly speed up our manual regression testing and let us find interesting issues and try things that, if we had to do all the manual keystrokes, we'd never get to.
In some ways it's easy to use, in other ways hard for me, in that I am not an OO programmer and Ruby has been a challenge, and I have limited time to devote. We have an awesome set of scripts that cover most of our app, and we do add to it as the app grows, but we don't do as much as we could. I'm sure there are lots of new features it has that we haven't had time to even learn. My fellow tester is much better with it and got the scripts working with Vista and has done some good refactoring.
Some of the scripts my former fellow tester wrote were so complex, with amazingly intricate regular expressions and iterative logic, they are hard to maintain, even the 'real' programmers on our team can't figure them out. I recently just bagged one because it was just taking too long to try to keep it working, I wrote a simpler version of it in our other GUI regression tool.But that's the exception.
No issues with speed or reliability. Maintenance in general is easy because our tests are well designed with good use of modules and classes (at least in my inexpert opinion). Today I had tests failing because some HTML id names changed. I only had to change one module to fix them so that was really easy. I think you could make them hard to maintain if you didn't know what you were doing. -- Lisa
On Mon, Apr 14, 2008 at 7:21 PM, Bret Pettichord <b...@pettichord.com> wrote:
> I'm working on the business case for Watir right now. Why do you use it? > Were you using something else beforehand? Did you compare Watir to other > tools? Is Watir faster? more reliable? easier to use? Were you just > following the crowd?
> I'd love to hear your reasons. If you'd like to stay anonymous, send me > your answers to me directly and I'll summarize.
1. Open-source We can view and change the source code if need be. It is free
2. Uses a industrial strength non-proprietary language, i.e. Ruby (Unlike Rational Robot, QTP etc)
3. Ruby is dynamic and duck typed, terse and expressive etc Testers do not need to spend time and space assigning data-types and compiling. Auto testers need something up and running in a short space of time. Compare a Rational Functional Tester script with a Watir script which commits similar actions
4. Easy to use with a clean api I have had manual testers writing tests in minutes. You do not need to use a heavy IDE. Ruby comes with a text editor called Scite which is fast and integrates well with the Ruby interpreter.
*Watir has a cleaner API, less configuration issues etc than Selenium* Watir users can test through Firefox with FireWatir (with minor changes). We must note with Selenium that all browsers have different JavaScrpt engines. In my opinion Watir will soon become more popular then Selenium.
5. The Watir and Ruby community are an extemely humble and helpful lot (compare this with Java user-groups)
Watir was recommended a consulting firm we worked with, so it was one of the solutions I evaluated. My only serious choices were Watir, Selenium, or Twill. Since I'm not a programmer I ruled out all the unit test solutions. I needed something that mimicked the user experience.
I ruled out Selenium because from reading the documentation, I understood Selenium ran from a server, which just adds to the complexity of my test setup. I wanted something workstation based.
I couldn't get Twill to navigate our application's javascipt menus. Though this could have been a limitation of me, more than Twill. Regardless, Watir did navigates the menus with no problems.
I liked that Watir is Ruby based and at the time I really wanted a reason to learn Ruby, and this has proven to be a good choice. Not only can I pick up Ruby quickly, but I was even able to teach an intern how to use script tests with Watir.
The application I tested has approximately 250 pages and every time I found a repetitive test, I made a case for automating it. The time to write, debug, and test using Watir was usually faster than it would have been to test manually. However, running the automated tests are always faster than running the same tests manually.
I wanted something open source even though I could have gotten a budget for a closed source solution. I wanted to show my employer, at the time, that open source tools added value.
Bret Pettichord wrote: > I'm working on the business case for Watir right now. Why do you use it? > Were you using something else beforehand? Did you compare Watir to other > tools? Is Watir faster? more reliable? easier to use? Were you just > following the crowd?
> I'd love to hear your reasons. If you'd like to stay anonymous, send me > your answers to me directly and I'll summarize.
1. It is a great open source AND freely available alternative to HP (Mercury) Quick Test Professional, IBM Rational Robot, and other similar commercial solutions for automating Web Applications in Internet Explorer. 2. It performs faster and with lower overhead (Ruby Runtime Environment versus Quick Test Pro IDE). 3. Support is as easy to get as emailing a question to the Watir Google group and waiting a few minutes and it doesn't cost anything for experience-based support. Also the support experience is much better - we get more than we paid for (nothing/free for tons of real user experience as opposed to some person in India looking at answers in a FAQ on a computer screen who has never used the product before). 4. It is simple to develop our own framework around Watir. Also, when we want to run scheduled tests or create log-running tests like bots or spiders, Watir provides all of the functionality we need to get started with a browser-based version of these. 5. Bug fixes and patches are constant, always free, and easy to get a hold of. Being able to participate and provide solutions to defects is also nice. At very minimum we can at least understand why a problem is occurring by asking questions. 6. I know some people say, "you get what you pay for", implying that inexpensive or free products tend to be full of bugs and expensive, high-end, commercial products tend to be better. However our experience has been quite the opposite of this "ideal" - as soon as we started using Watir, drifting away from hours on the phone with Mercury or HP, to online conversations with real users, the cost-benefit ratio sky-rocketed. We have gotten and continue to get way more than we paid for. $0 for thousands of experienced users, FAQs, User Guides, extensions, and it's all way more worth our time than paying up to $10,000 (a year) just to sit and wait on the phone for hours, talking to someone who has no answers, tells you that their product "wasn't meant to be used that way", they can't support the methods your using, and so on.
You can quote me on any of these things.
Nathan
On Mon, Apr 14, 2008 at 7:21 PM, Bret Pettichord <b...@pettichord.com> wrote:
> I'm working on the business case for Watir right now. Why do you use it? > Were you using something else beforehand? Did you compare Watir to other > tools? Is Watir faster? more reliable? easier to use? Were you just > following the crowd?
> I'd love to hear your reasons. If you'd like to stay anonymous, send me > your answers to me directly and I'll summarize.
> 1. It is a great open source AND freely available alternative to HP > (Mercury) Quick Test Professional, IBM Rational Robot, and other similar > commercial solutions for automating Web Applications in Internet Explorer. > 2. It performs faster and with lower overhead (Ruby Runtime > Environment versus Quick Test Pro IDE). > 3. Support is as easy to get as emailing a question to the Watir > Google group and waiting a few minutes and it doesn't cost anything for > experience-based support. Also the support experience is much better - we > get more than we paid for (nothing/free for tons of real user experience as > opposed to some person in India looking at answers in a FAQ on a computer > screen who has never used the product before). > 4. It is simple to develop our own framework around Watir. Also, > when we want to run scheduled tests or create log-running tests like bots or > spiders, Watir provides all of the functionality we need to get started with > a browser-based version of these. > 5. Bug fixes and patches are constant, always free, and easy to get > a hold of. Being able to participate and provide solutions to defects is > also nice. At very minimum we can at least understand why a problem is > occurring by asking questions. > 6. I know some people say, "you get what you pay for", implying that > inexpensive or free products tend to be full of bugs and expensive, > high-end, commercial products tend to be better. However our experience has > been quite the opposite of this "ideal" - as soon as we started using Watir, > drifting away from hours on the phone with Mercury or HP, to online > conversations with real users, the cost-benefit ratio sky-rocketed. We have > gotten and continue to get way more than we paid for. $0 for thousands of > experienced users, FAQs, User Guides, extensions, and it's all way more > worth our time than paying up to $10,000 (a year) just to sit and wait on > the phone for hours, talking to someone who has no answers, tells you that > their product "wasn't meant to be used that way", they can't support the > methods your using, and so on.
> You can quote me on any of these things.
> Nathan
> On Mon, Apr 14, 2008 at 7:21 PM, Bret Pettichord <b...@pettichord.com> > wrote:
> > I'm working on the business case for Watir right now. Why do you use it? > > Were you using something else beforehand? Did you compare Watir to other > > tools? Is Watir faster? more reliable? easier to use? Were you just > > following the crowd?
> > I'd love to hear your reasons. If you'd like to stay anonymous, send me > > your answers to me directly and I'll summarize.
This is easy. I can quote the internal document I prepared when first
switching to Watir (from Rational Robot):
* tests are written in a real language, not a vendorscript.
* tests are easily added to version control.
* Watir and Ruby are widely used and support is easily found via
online forums.
* Eclipse (among other) provides a high-quality development
environment for Ruby.
* zero license cost.
Robot didn't come close to competing on any of these points.
What I've learned since then:
* programming ruby is fun. People are more likely to produce good
work doing something they enjoy.
* ruby programming is a skill you can apply elsewhere, unlike
skills in a vendorscript.
* the ability to modify the tool yourself is powerful.
* Compared to our previous setup, Watir tests can be developed
faster, run faster, and produce more usable output (now integrated to
a Bamboo CI server).
* upgrades are frequent and of high quality.
On Apr 15, 5:33 pm, Bret Pettichord <b...@pettichord.com> wrote:
1 Very low overhead deployment and light-weight development
environment.
2 Easy of use and an intuitive "model". Days, not weeks before you
got real tests running.
3 Free and Open!
4 Browser-based, close-as-you-can-get to real users. (and makes for a
great demo to non-engineers).
5 All the power of Ruby at your disposal, no "vendor-script"
constraints.
6 A great community behind it!!
I have used commercial tools. They work fine. But Watir has not
disappointed, and has all of the above going for it. It has been very
reliable for us.
-Mel
On Apr 14, 9:21 pm, Bret Pettichord <b...@pettichord.com> wrote:
> I'm working on the business case for Watir right now. Why do you use it?
> Were you using something else beforehand? Did you compare Watir to other
> tools? Is Watir faster? more reliable? easier to use? Were you just
> following the crowd?
> I'd love to hear your reasons. If you'd like to stay anonymous, send me
> your answers to me directly and I'll summarize.
On Mon, Apr 14, 2008 at 9:21 PM, Bret Pettichord <b...@pettichord.com> wrote:
> I'm working on the business case for Watir right now. Why do you use it? > Were you using something else beforehand? Did you compare Watir to other > tools? Is Watir faster? more reliable? easier to use? Were you just > following the crowd?
> I'd love to hear your reasons. If you'd like to stay anonymous, send me > your answers to me directly and I'll summarize.
On 14/04/2008, Lisa Crispin <lisa.cris...@gmail.com> wrote:
> In some ways it's easy to use, in other ways hard for me, in that I am not > an OO programmer and Ruby has been a challenge ...
+1. Not so much a shift but a journey where we may never arrive. Much of the problem lies with Basic. As testers we could have used QTP and Robot that script in Basic derived languages, and I would imagine that a person's first program is in VBA. We form bad habits. Basic is ruinous to the mind; we should seek its abolition.
It looks like most of my reasons are already covered here but here's
my ordered list of reasons anyway:
1. It's Free (as in beer)
Watir being free (as in beer) straight away makes it a very attractive
test tool. This has meant that automated testing has 'gotten in the
door' on projects where commercial automated testing tools would not
have been looked at in the first place.
Once Watir's benefits are evident, more and more team members want to
use it. It is then a simple, quick install on other's machines. The
process for purchasing and generating commercial test tool licenses is
often very lengthy and time consuming which means it ultimately won't
be running on as many machines as Watir will be.
2. It's Free (as in freedom)
Because Watir is open source and uses a modern object oriented
scripting language, it provides its users with the freedom to tailor
it to how they want to use it. Nothing is hidden and mysterious so
users can often solve their own problems without consulting others or
vendor support.
3. It's Flexible
Watir is a very flexible ruby library that supports many scripting
tasks. The main reason I have used it is to conduct automated
regression testing but I have also used it to create test data in
systems, schedule and run automated web site monitoring scripts
(complete with alerts), as well as one-off scripts that are quick to
write and get the job done.
4. It's Fast
Execution speed is as good as Quick Test Professional. The in built
browser synchronisation is better than Quick Test Professional.
The ramp up time for users to learn ruby and watir is very fast
compared to languages like TSL (WinRunner), SQA-Basic (RR) and Java
(RFT).
The installation and implementation of Watir is fast, easy and
lightweight. There are no server components to install like Selenium.
5. It's Fun
Ruby is a unique programming language in that it has been designed to
be fun and you can get better at using it every day. I love the 'Ah-
ha' moments when using Ruby where you realise that you can do
something just a little bit neater and more efficient.
> I'm working on the business case for Watir right now. Why do you use it?
> Were you using something else beforehand? Did you compare Watir to other
> tools? Is Watir faster? more reliable? easier to use? Were you just
> following the crowd?
> I'd love to hear your reasons. If you'd like to stay anonymous, send me
> your answers to me directly and I'll summarize.
My reason on recommending to use Watir in our work was heavily based on the capability of the test scripts to be maintainable and scalable. And Ruby won out over Perl because of the inherent OO in the design of the language.
It has to be maintainable because applications change over time. From a simple renaming of an HTML id to an outright change in button clicks, it becomes evident to compartmentalize things. It has to be scalable because the goal is to build on top of previous work and not scrap the underlying design (assuming it was well-designed in the first place). Adding new functionalities to the application, like additional steps in authentication for logging in, must not render previous work incapable. It's where most other scripting language comes up short.
