Selenium...

[aslak hellesoy]

...is driving me nuts. I don't use it, but people often come here and
on #cucumber asking questions about it.

Does anyone know how to get it going with Cucumber and Webrat? Any
volunteers to update the Wiki page?
http://wiki.github.com/aslakhellesoy/cucumber/setting-up-selenium

The state it's in right now provides no value, so I'd rather see the
page go than be up there and confuse people.
-Unless someone can help us fix it of course :-)

Aslak

[aidy lewis]

Most issues are not JS and OS, but functional issues. And thus Watir
is the better choice.


Aidy


2009/10/12 aslak hellesoy <aslak.h...@gmail.com>:

[Alf Mikula]

I think selenium more or less works if you install the selenium-rc gem and execute selenium-rc instead of using the "Selenium" gem, but I don't have enough info to update the doc at this time.

[Jim Meyer]

I'll try to look at this in the next day or two. I'm setting up a new laptop, so it'll be a good experiment.

--j

[Tero Tilus]

2009-10-12 23:58, aslak hellesoy:

> -Unless someone can help us fix it of course :-)

I definitely should at least document my setup. I have Cuke+Selenium
up and running fine.

--
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/

[Noel]

As a n00b that had a hard time getting things running I can agree that
the Selenium instructions leave something to be desired. At the same
time, the alternative solution's documentation were much better.
Maybe a good "how to test js with cuke" with best practices would be
enough to steer newcomers away.

I did try to get Culerity going but couldn't figure out what I needed
to do as far as JRuby, etc. In the end I stuck with Selenium since I
had gotten that working although I don't think I can reproduce.

I have not reviewed the wiki to see if there is anything there which
may not have been around when I went through this a few months ago.

I will volunteer to work with someone to develop this how-to.

-Noel

[Scott Johnson]

I got it running back in May by following the wiki directions, but
gave up due to what I feel are intractable problems with the way
webrat interacts with Selenium.

Specifically it tries to be "smart" about when to wait for the
browser, but Selenium is not designed to work that way. Selenium is
designed to do "Click X, then wait for page load" as one step. Webrat
would do "Click X", then on the next command (which could be
anything), try to figure out what to wait for first. I don't think
that's ever going to be reliable enough.

I brought this up on the Webrat mailing list at the time, but nobody
seemed to agree that it was a problem.

[wesgarrison]

> Maybe a good "how to test js with cuke" with best practices
> would be enough to steer newcomers away.

That's what I went looking for. I only knew about selenium, so that's
where I started, but now I'm not sure whether to use the jar or the
selenium-client gem or selenium with webrat or celerity or some other
solution.

So, I'm not sure what's next. I don't particularly care for Selenium,
I just want to do some js testing.

-- Wes Garrison

[Matt Wynne]

I had a dig around a few weeks ago, and my conclusion was that there
really isn't anything terribly satisfactory:

* Selenium
* pro + integrates with webrat
* pro + hits real browsers
* con - cumbersome and complex to set up
* con - slow-ass to run

* Celerity
* con - doesn't integrate with webrat (uses the watir API which IMO
is not nearly so nice)
* con - requires you to run your tests under jruby which means if
you want to use domain steps (e.g. Given steps that use models to set
up data) you need to spin up your whole stack under jruby
* pro + fast
* con - not (quite) a real browser

* Culerity
* pro + means you can run your test stack under ruby and still
leverage the speed of celerity
* con - still uses the watir API
* con - didn't seem very reliable

We still write all our JS for Songkick as progressive-enhancements
that we test manually. We occasionally unit-test more complex bits
using (the excellent) blue-ridge but mostly it's just code-n-fix for
the integration testing. Right now there doesn't seem to be a good
enough solution that it would give us sufficient value for the cost of
setting up and maintaining the tests. It's a shame, but that's where
we are right now.

cheers,
Matt

http://mattwynne.net
+447974 430184

[Bakken, Jari]

On 15.10.09 11.41, Matt Wynne <ma...@mattwynne.net> wrote:

On 15 Oct 2009, at 04:02, wesgarrison wrote:

* Celerity
* con - doesn't integrate with webrat (uses the watir API which IMO
is not nearly so nice)

Someone did a WebRat adapter for Celerity - I think maybe it's this: http://github.com/kamal/webrat. Haven't used that myself though.

Also the WebDriver project (http://code.google.com/p/webdriver/, will become Selenium 2.0) needs some Ruby love. I'm going to GTAC (http://www.gtac.biz) next week where I hope to get started on that (and likely wrap the result in the Watir API as well eventually, but that's primarily because I personally like it and my company is heavily invested in it). If anyone is interested in contributing to that, Simon Stewart and I have a wave going with some discussion (or we can take it off-Wave).

Jari

[Matt Wynne]

On 15 Oct 2009, at 11:04, Bakken, Jari wrote:

>
>
>
>
> On 15.10.09 11.41, Matt Wynne <ma...@mattwynne.net> wrote:
>
>
>
>
> On 15 Oct 2009, at 04:02, wesgarrison wrote:
>
>
> * Celerity
> * con - doesn't integrate with webrat (uses the watir API which IMO
> is not nearly so nice)
>
> Someone did a WebRat adapter for Celerity - I think maybe it's this: http://github.com/kamal/webrat
> . Haven't used that myself though.

I had a look at this and it seemed to be half-finished. That might be
unfair though - is anyone using it?

[Joseph Wilk]

Matt Wynne wrote:
> On 15 Oct 2009, at 04:02, wesgarrison wrote:
>
>
>>> Maybe a good "how to test js with cuke" with best practices
>>> would be enough to steer newcomers away.
>>>
>> That's what I went looking for. I only knew about selenium, so that's
>> where I started, but now I'm not sure whether to use the jar or the
>> selenium-client gem or selenium with webrat or celerity or some other
>> solution.
>>

I use the selenium-rc.jar.

>> So, I'm not sure what's next. I don't particularly care for Selenium,
>> I just want to do some js testing.
>>

Have you considered writing some js unit tests using something like
ScrewUnit (http://github.com/nkallen/screw-unit)?

>
> I had a dig around a few weeks ago, and my conclusion was that there
> really isn't anything terribly satisfactory:
>
> * Selenium
> * pro + integrates with webrat
> * pro + hits real browsers
> * con - cumbersome and complex to set up
>

I personally have not found Selenium complex to setup but then thats not
with it playing with Webrat. I've always found Webrat/Selenium setup
painful so far.

> * con - slow-ass to run
>

The real performance hit is in starting up the browser, once the browser
is running its fairly quick (but still not webrat/Celerity fast).

> * Celerity
> * con - doesn't integrate with webrat (uses the watir API which IMO
> is not nearly so nice)
>

I've spoked to quite a few people who are working on a Webrat/Celerity
adapter. Looking forward to seeing what people produce.

> * con - requires you to run your tests under jruby which means if
> you want to use domain steps (e.g. Given steps that use models to set
> up data) you need to spin up your whole stack under jruby
> * pro + fast
> * con - not (quite) a real browser
>
> * Culerity
> * pro + means you can run your test stack under ruby and still
> leverage the speed of celerity
> * con - still uses the watir API
> * con - didn't seem very reliable
>

There is also Watir (http://watir.com/). Which is similar to Selenium
but its written in Ruby. I generally find Watir nicer to work with but I
often find it slower than Selenium-rc.

> We still write all our JS for Songkick as progressive-enhancements
> that we test manually. We occasionally unit-test more complex bits
> using (the excellent) blue-ridge but mostly it's just code-n-fix for
> the integration testing. Right now there doesn't seem to be a good
> enough solution that it would give us sufficient value for the cost of
> setting up and maintaining the tests. It's a shame, but that's where
> we are right now.
>

A substantial part of the project I've been working on is Javascript. We
found the key in getting up to speed was having a good set of helper
methods wrapping the dirty details of the browser testing framework and
dealing with issues such as asynchronous tests.

--
Joseph Wilk
http://blog.josephwilk.net

[wesgarrison]

> Have you considered writing some js unit tests using something like
> ScrewUnit (http://github.com/nkallen/screw-unit)?

No, because I had no idea it existed. These types of projects could
be
put on the wiki, perhaps, on a "how to test js" page.

Also, I'd prefer staying in Ruby when possible instead of having tests
written in Ruby and JS. Just my preference, though, since I'm doing
the specs and don't really plan on implementing most of the
javascript.

That said, if I find screw-unit to be quickly picked up, I'd have no
issue using a JS based test methodology.

> There is also Watir (http://watir.com/). Which is similar to Selenium
> but its written in Ruby. I generally find Watir nicer to work with but I
> often find it slower than Selenium-rc.

Again, I'm doing more acceptance testing at the moment, so test speed
isn't as important to me. Getting it working in a timely manner is
more
important than it taking a few extra seconds. Naturally, I feel the
exact
opposite about unit tests since I'm the one writing the code for
those.
The JavaScript guy isn't in a hurry, right? :-)

Thanks for the pointers to other projects. I'll check them out and
see
what I can find!

-- Wes Garrison

[Ivo Dancet]

At first I was going to suggest to change the wiki but then I realised
that my approach may be too far from the intended approach and I don't
have enough experience with the intended usage to document it
properly. That's why I've written my environment for it down in a blog
post which you can find here:

http://injecting.by2.be/blog/2009/11/selenium_and_cucumber.html

Ivo Dancet
by2.be

Op 12-okt-09, om 23:58 heeft aslak hellesoy het volgende geschreven:

[Wincent Colaiuta]

On 12 oct, 22:58, aslak hellesoy <aslak.helle...@gmail.com> wrote:

> ...is driving me nuts. I don't use it, but people often come here and
> on #cucumber asking questions about it.

Just out of interest, what do you use to test JS and/or AJAX stuff
Aslak?

I've also had no end of trouble with Selenium. I've had it working on
and off for a while, but it seems very brittle and fragile; the latest
breakage I've had to sort out was in the move to Snow Leopard, where
the current version of Selenium RC (1.0.1) and recent versions of
Firefox (3.5.4 and 3.5.5, for example) both have breakage:

https://bugzilla.mozilla.org/show_bug.cgi?id=513747

http://jira.openqa.org/browse/SRC-743

There are supposedly workarounds, but they are either horrid (eg.
patching and compiling Selenium) or only work with Firefox 3.5.3. In
the end I had to switch to Safari instead and, fingers crossed, it
seems to be working for now with these settings:

Webrat.configure do |config|
config.mode = :selenium
config.application_port = 3000
config.selenium_browser_key = '*safari'
config.selenium_server_address = '127.0.0.1'
config.selenium_server_port = 4444
end

Before running the features I start the selenium-server.jar (part of
Selenium-RC 1.0.1) with:

java -jar /path/to/selenium-server.jar

I'm using version 1.2.17 of the selenium-client gem, webrat 0.5.3,
cucumber 0.4.3, and Rails 2.3.4.

Wish there was a better solution, but as far as AJAX testing goes I
haven't yet found a convincing alternative.

Cheers,
Wincent

[Caius Durling]

On 11 Nov 2009, at 20:10, Wincent Colaiuta wrote:

> Wish there was a better solution, but as far as AJAX testing goes I
> haven't yet found a convincing alternative.

We use Culerity, which runs our app under MRI ruby, but tests it from celerity under jRuby. Works quite well, and is the best method of testing we've found with JS on the page.

http://github.com/langalex/culerity
http://celerity.rubyforge.org/

C
---
Caius Durling
Brightbox
ca...@brightbox.co.uk
http://brightbox.co.uk/

[aslak hellesoy]

On Wed, Nov 11, 2009 at 9:10 PM, Wincent Colaiuta <w...@wincent.com> wrote:
> On 12 oct, 22:58, aslak hellesoy <aslak.helle...@gmail.com> wrote:
>
>> ...is driving me nuts. I don't use it, but people often come here and
>> on #cucumber asking questions about it.
>
> Just out of interest, what do you use to test JS and/or AJAX stuff
> Aslak?
>

I use Watir or one of its clones (including celerity).
Lately I have also been using WebDriver.

I have used Selenium in the past (heck - I wrote the initial Ruby
bindings years ago), and I know it works. It's just too much hassle
for me to set up.

> --
>
> You received this message because you are subscribed to the Google Groups "Cukes" group.
> To post to this group, send email to cu...@googlegroups.com.
> To unsubscribe from this group, send email to cukes+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cukes?hl=.
>
>
>