Watir tests - result reporting

[Babitha]

Hi,

I am new to Watir, and we are evaluating Watir as a tool to replace
Qtp.

One place I am stuck a little bit is regarding reporting the errors at
the end after all the test suites are done.

I have looked at various options those are discussed in the Watir
community, examples and blogs.

i started with Html reports from the examples, tweaked the code to
make the interface simpler - but found it to be still not simple
enough.

Then I looked at Test::Unit. The problem I find with Test::Unit is
that the output is in free text format, and consolidating the results
is a pain.

Please note we are not Agile practionitners. We are looking at Watir
as a testing tool for the testers - not for the developers. We do not
use BDD, so tools like Cucumber or RSpec probably will not suite us.
Also, I found even Rspec gives its output as free text format.

I am sure there must be a better simpler way. Probably I am just not
able to find any pointers towards it.

What I am looking at is a solution that will write the results to a
simple excel/csv file. I thought the Test::unit in itself could be
extended to do this, when any assertion passes or fails. I thought
this must be a simple thing to do, and probably somebody has already
done it. I also read in the Test::Unit documentation that Observers
can be added to Test::Unit TestResult. Is there any sample
implementations of this which will write it to an excel/csv as I
mentioned above?

Your help will be greatly appreciated.

Thanks,
Babitha

[Željko Filipin]

On Fri, Nov 12, 2010 at 7:03 AM, Babitha <babitha....@gmail.com> wrote:
> We do not
> use BDD, so tools like Cucumber or RSpec probably will not suite us.
> Also, I found even Rspec gives its output as free text format.

You can use RSpec just as a test runner for Watir tests, it does not have to be used in the "TDD/BDD way".

RSpec output is text by default, but you can specify that you want HTML report. It is as easy as (at command line):

spec my_specs.rb -f h:log\report.htm

More information on RSpec output:

http://rspec.info/documentation/tools/spec.html

Željko
--
watir.com - community manager
watirpodcast.com - host
testingpodcast.com - audio podcasts on software testing. all of them

[Alastair Montgomery]

Do you need to do any changes to your test scripts to run them with
RSpec?

On Nov 12, 9:33 am, Željko Filipin <zeljko.fili...@wa-research.ch>
wrote:

[Željko Filipin]

On Fri, Nov 12, 2010 at 12:44 PM, Alastair Montgomery <dood...@gmail.com> wrote:
> Do you need to do any changes to your test scripts to run them with
> RSpec?

It depends. :)

If you provide some sample code I could change it to use RSpec.

Željko

[Alastair Montgomery]

Hi,

Here is a small sample of the test scripts we are using at the moment,
can you tell if there is much work required to change it to RSpec?

require "rubygems"

require "logger"
require "test/unit"
require "watir"
require "library/CSVRead"
require "library/PARRead"
require "library/Login"
require "library/browseTree"

class WorkflowSelectorTest < Test::Unit::TestCase
def setup
#Setup Variables
myPAR = PARRead.new("001_parameters.txt")

#Create log file
$log = Logger.new(myPAR.logfile)
$log.debug "===Initializing Test Setup==="

testSite = myPAR.url
title = myPAR.title

#Choose Browser
Watir::Browser.default = myPAR.browser

#Start Test Run
$log.debug "===Start==="
$log.debug "Goto web page, "+ testSite
$browser = Watir::Browser.start(testSite)

#Login
tmp = Login.new($browser,myPAR.user,myPAR.password)

$log.debug "Test WebClient Title"
assert($browser.title.include? title)

#Wait for jQuery to render
Watir::Waiter.wait_until {$browser.label(:id,"acms-ws-select-
label").exists?}
$log.debug "Check workspace label"
assert($browser.label(:id, "acms-ws-select-label").exists?)
end

def teardown
$log.debug "===Test Teardown==="
#TODO Logout when implemented in client
if $DEBUG then
#Do not close the browser
puts "Finished"
else
$browser.close
end
$log.close
end

def testWorkflowSelector1
#Test choosing workspaces
myData = CSVRead.new("data/testWorkflowSelector1.csv")
workspaceName = myData.array

$log.debug "===testWorkflowSelector1==="
$log.debug "Test choosing workspaces"
$log.debug "Select workflow dropdown check"
assert($browser.select_list(:id, "ACMS-Workspace-Selector"))
$log.debug "Change workflow dropdown check"
dropDown = $browser.select_list(:id, "ACMS-Workspace-Selector")
i=0
while i < workspaceName.length
dropDown.select workspaceName[i][0]
dropDown.set workspaceName[i][0]
assert_equal(dropDown.value,workspaceName[i][0])
i+=1
end

$log.debug "===testWorkflowSelector1==="
end
end


Regards,
Alastair

On Nov 12, 1:40 pm, Željko Filipin <zeljko.fili...@wa-research.ch>
wrote:
> On Fri, Nov 12, 2010 at 12:44 PM, Alastair Montgomery <doodl...@gmail.com>

[Dave McNulla]

Alastair,

I did not try this, but I think it should look like this for the most
part. Don't forget to use the 'spec' command (not the 'ruby' command)
- I lost a day trying to figure out that mistake.

Here are some practices that I try to do with tests:
1. instead of making assumptions about data in the system, I try to
set that up in the before and after parts of the test. If your
workspace list is dynamic, you can inject the values for your test
into the database.
2. Each test tests one thing, finds one problem.
3. Be mindful of when to use :each and :all because that can affect
the efficiency that the tests run.
4. Less logging, let the system do that for me.

Good luck,

Dave
-------------------------

require "rubygems"
require "logger"
require "test/unit"
require "watir"
require "library/CSVRead"
require "library/PARRead"
require "library/Login"
require "library/browseTree"

describe "Workflow Selector" do

myData = CSVRead.new("data/testWorkflowSelector1.csv")

before(:each) do

#Setup Variables
myPAR = PARRead.new("001_parameters.txt")
#Create log file

testSite = myPAR.url
title = myPAR.title
#Choose Browser
Watir::Browser.default = myPAR.browser
#Start Test Run

$browser = Watir::Browser.start(testSite)
#Login
tmp = Login.new($browser,myPAR.user,myPAR.password)

Watir::Waiter.wait_until {$browser.label(:id,"acms-ws-select-
label").exists?}

end

it "should be invalid without a username" do
#Test choosing workspaces
workspaceName = myData.array
$browser.select_list(:id, "ACMS-Workspace-Selector")

dropDown = $browser.select_list(:id, "ACMS-Workspace-Selector")

workspaceName.length.times do |i|

dropDown.select workspaceName[i][0]
dropDown.set workspaceName[i][0]

dropDown.value.should equal workspaceName[i][0]
end
end

after(:each) do

#TODO Logout when implemented in client
if $DEBUG then
#Do not close the browser

else
$browser.close
end
end

end

[Željko Filipin]

On Sat, Nov 13, 2010 at 12:44 PM, Alastair Montgomery <dood...@gmail.com> wrote:
> Here is a small sample of the test scripts we are using at the moment,
> can you tell if there is much work required to change it to RSpec?

I think Dave ported it pretty god to RSpec. Let me know if you have further questions.

Željko

[Babitha]

Alastair,

Stmbled on this:

http://www.natontesting.com/2009/09/18/get-html-output-from-testunit-by-using-rspec/

Please see if it would work for you.

Thanks,
Babitha

> > Željko- Hide quoted text -
>
> - Show quoted text -

[Babitha]

Željko,

Thanks for your replies.

But, isn't there any solution that would give an excel report? HTML
report, even if color coded, I don't think is easy to run through.

Thanks,
Babitha

On Nov 15, 5:16 am, Željko Filipin <zeljko.fili...@wa-research.ch>
wrote:
> On Sat, Nov 13, 2010 at 12:44 PM, Alastair Montgomery <doodl...@gmail.com>

[Željko Filipin]

On Tue, Nov 16, 2010 at 10:12 AM, Babitha <babitha....@gmail.com> wrote:
> But, isn't there any solution that would give an excel report?

If you need simple table, you can export data as CSV file.

If you need more functionality, take a look at these:

http://rasta.rubyforge.org/
http://roo.rubyforge.org/

Željko