Running selenium from a VM?

[evan]

I posted a question about running fitnesse driven selenium tests
earlier today and I just found out that my company has a VM available
for this. Has anyone ran tests programmatically from a VM? If so,
any articles you can suggest or tips for this?

Thanks a ton!
Evan
evanm...@yahoo.com

[Ross Patterson]

Watch out for any timed waits. Everything runs slower in a virtual machine. If there are wait-for-n-seconds operations in the test client (e.g., Thread.Wait() in C#), you can expect tests to succeed sometimes and fail other times, unpredictably.

You'll probably also find that you have tests that currently succeed because the test script never out-paces the browser. You can expect those to fail unpredictably also when run in a slower machine. WaitForPageToLoad() will become your best friend :-)

But it all works. We run a grid of about 25 VMs under control of JetBrains' TeamCity build system, and we're quite happy. Except for the occasional morning when I come in and find out that the overnight tests failed en masse because some other VM on the same hypervisor got stuck in a loop and starved mine (and all the others) for a few hours. There's just no length of timeout that can get you through something like that :-)

Ross Patterson

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To post to this group, send email to seleniu...@googlegroups.com.
To unsubscribe from this group, send email to selenium-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/selenium-users?hl=en.

[Andrew]

Ross has some good points, I run 14 VMs under Virtualbox and have yet
to notice much slowness or delay (I'm not up to 25 yet), however the
physical machine is solely tasked to my VMs and nothing else. The
only issues I've noticed is that during a test, occasionally the RC
server fails to return results and tests fail either due to the server
not responding, or the server not returning a proper 'end of test'
message.

I do utilize waitForPageToLoad and sleep commands extensively, as much
to manage the test scenario as to ensure timings.

Some VM systems run slower than others. I set up Xen first and found
that 1 windows XP VM was quite pokey, yet running VirutalBox, a
Windows XP system was acceptably responsive. Not a speed demon, but
not horrible.

> evanmey...@yahoo.com

[darrell]

Evan,

I ALWAYS run my tests in a Vmware session. Having different images
with different configurations is much more economical than buying
dozens of computers. Additionally, the largest market is still
typically Internet Explorer and I'm on a Mac OS X machine. Ross has a
good point about using hard coded delays. This is true for all test
automation tools. You need to make your scripts event driven, i.e.
wait for an event to occur rather than waiting for a set period of
time.

If you are using Selenium RC and Java, have a read of
http://darrellgrainger.blogspot.com/2010/02/how-to-handling-timing-in-selenium.html.
It will detail the four types of events to wait for. The concepts
exist in all the languages you can use Selenium in but the examples
are all Java. This article is good even if you are not using VMs.
Timing can change due to multitasking, network delays, multiple people
using the same test machine, changes to the application you are
testing, etc. So you always want to use event driven timing.

Darrell

> evanmey...@yahoo.com