Launching an driving multiple instances of browsers in headless mode
Ben Lau
Does WebDriver allows multiple instances of browsers to be
instantiated and driven from separate independent threads, preferably
in headless mode?. This is to allow separate instances use different
input values.
For example, test framework like TestNG allow testing in separate
threads with different values.
A headless mode is likely to reduce memory consumption.
I know that HtmlUnit is already headless.
Some folks have use Xvfb for this purpose:
http://www.semicomplete.com/blog/geekery/xvfb-firefox.html
Thanks,
BK
Simon Stewart
Sorry for not getting back to you sooner! The short answer to your
question is that the HtmlUnitDriver definitely supports multiple
instances, the IE driver probably does (there's no reason for it not
to) and the Firefox driver doesn't yet[1] and the Safari driver can't.
As for headless mode, you can enable this by calling
"WebDriver.setVisible()". On those browsers that support headless mode
(IE and HtmlUnit) this will run them headless. For those browsers that
don't this is a no-op. Actually, the way that we've implemented
"sendKeys" on IE means that it should be visible --- I will need to
rectify this requirement.[2]
There's an open question about whether multiple instances of a driver
should share the same session; my gut instinct is that they shouldn't,
but there may be a technical reason why they may have to --- for
example, opening a new instance of Safari on OS X actually sets the
focus to an already open instance of the browser. It may have to be
something that's documented, and perhaps "queryable" from the driver.
Regards,
Simon
[1] See: http://code.google.com/p/webdriver/issues/detail?id=18
[2] http://code.google.com/p/webdriver/issues/detail?id=55
Mirko Nasato
I'm also interested in running multiple instances (headless mode is
optional). We'd like to set up an automated daily build (in Hudson)
that runs all functional tests for a project. There's a chance that
multiple projects gets built at the same time, starting multiple
WebDriver instances.
Initially we're going to use the HtmlUnit driver, that is pure Java
and has no problems with multiple instances, but we'd like to switch
to the FirefoxDriver at some point so we can properly test JavaScript
as well. Maybe Firefox could be started to listen on a dynamically
allocated port? I'll have a more in depth look at the FirefoxDriver as
I can.
On Feb 14, 10:22 am, "Simon Stewart" <simon.m.stew...@gmail.com>
wrote:
> There's an open question about whether multiple instances of a driver
> should share the same session; my gut instinct is that they shouldn't
Kind regards
Mirko
Simon Stewart
> Initially we're going to use the HtmlUnit driver, that is pure Java
> and has no problems with multiple instances, but we'd like to switch
> to the FirefoxDriver at some point so we can properly test JavaScript
> as well. Maybe Firefox could be started to listen on a dynamically
> allocated port? I'll have a more in depth look at the FirefoxDriver as
> I can.
There's a firefox user preference ("webdriver_firefox_port") that's
picked up in "webdriverserver.js" in the firefox extension. The
Firefox driver java code is currently hard-coded to use port 7055, but
it should be pretty easy to make that configurable. The only thing
that's really left to do is to change the "user.js" that's created to
pick a "good" port number.
> On Feb 14, 10:22 am, "Simon Stewart" <simon.m.stew...@gmail.com>
> wrote:
>
> >
> > There's an open question about whether multiple instances of a driver
> > should share the same session; my gut instinct is that they shouldn't
>
> In our case, it's clear that they should not.
It may be left as a documented implementation detail of the individual
drivers. For example, though it should be easy enough to make sessions
independent in Firefox, HtmlUnit and IE, because of the way that
Safari works, it may not be possible to have independent sessions
there.
Simon
Mirko Nasato
On 29/02/2008, Simon Stewart <simon.m...@gmail.com> wrote:
>
> There's a firefox user preference ("webdriver_firefox_port") that's
> picked up in "webdriverserver.js" in the firefox extension. The
> Firefox driver java code is currently hard-coded to use port 7055, but
> it should be pretty easy to make that configurable. The only thing
> that's really left to do is to change the "user.js" that's created to
> pick a "good" port number.
>
Thanks for the pointers. I've made the port configurable, both when
creating the profile and when starting Firefox. I've attached a patch
to issue #18 (although issue #4 would have been more appropriate).
Next I'd like to pick a random port and have the FirefoxDriver
automatically create a new profile on startup, and delete that profile
on quit.
BTW why do we need to start and stop Firefox after installing the
extension (in FirefoxLauncher#createBaseWebDriverProfile)? Everything
seems to work fine without that step, at least on Linux.
Shame that neither Java nor XPCOM seem to offer support for named
pipes rather than sockets, it would make this kind of inter-process
communication easier.
Kind regards
Mirko
Simon Stewart
> Thanks for the pointers. I've made the port configurable, both when
> creating the profile and when starting Firefox. I've attached a patch
> to issue #18 (although issue #4 would have been more appropriate).
Many thanks for taking the time to make a patch! It's really
appreciated. I'm planning on spending some time on Friday working on
WebDriver, so I'll take a proper look at it then.
> Next I'd like to pick a random port and have the FirefoxDriver
> automatically create a new profile on startup, and delete that profile
> on quit.
That should be pretty easy to do, assuming that you're starting and
stopping firefox through Java. Right now, we dump the profiles in the
system's temporary directory, but you're entirely correct that it
would be polite to tidy up after ourselves :)
> BTW why do we need to start and stop Firefox after installing the
> extension (in FirefoxLauncher#createBaseWebDriverProfile)? Everything
> seems to work fine without that step, at least on Linux.
That's the way that Firefox behaves when it comes across a profile
that's in an inconsistent state: it starts up, cleans up the profile
and then restarts so that it can be sure it has the correct
configuration. It'd be nice if there was a way to avoid this, and I'm
not happy with the heuristic approach we're taking here.
> Shame that neither Java nor XPCOM seem to offer support for named
> pipes rather than sockets, it would make this kind of inter-process
> communication easier.
Agreed. On the plus-side we do (accidentally) get the ability to run
firefox on one machine and the tests on another --- a remote webdriver
:) I'm slowly preparing to extract this out "officially" and provide
support for the other browsers to communicate with a remote instance
too.
*sigh* There's just not enough hours in the day :)
Regards,
Simon
Mirko Nasato
>
> Many thanks for taking the time to make a patch! It's really
> appreciated. I'm planning on spending some time on Friday working on
> WebDriver, so I'll take a proper look at it then.
>
to play with this stuff myself.
> > Next I'd like to pick a random port and have the FirefoxDriver
> > automatically create a new profile on startup, and delete that profile
> > on quit.
>
>
> That should be pretty easy to do, assuming that you're starting and
> stopping firefox through Java. Right now, we dump the profiles in the
> system's temporary directory, but you're entirely correct that it
> would be polite to tidy up after ourselves :)
>
Doh! I didn't notice the createCopyOfProfile() bit. Sometimes I just
make assumptions without checking them... :) I was assuming that since
it creates a new profile during installation, it would use *that*
profile when starting up. Instead it makes a copy.
Ideally, I'd also like to remove the need for the initial installation
step (i.e. rake install_firefox) altogether. Calling new
FirefoxDriver() should automatically create a new profile in a temp
dir, install the extension, and start Firefox.
The reason is that I'd like to package a webdriver-firefox.jar file
that also includes the fxdriver extension, upload that jar to our
maven repository, and then let the other developers start using
WebDriver just by adding it as a maven dependency like any other jar,
without any special installation step.
My first experiments show that it should be possible, but I need to
play with it a bit longer...
Kind regards
Mirko
Simon Stewart
> Doh! I didn't notice the createCopyOfProfile() bit. Sometimes I just
> make assumptions without checking them... :) I was assuming that since
> it creates a new profile during installation, it would use *that*
> profile when starting up. Instead it makes a copy.
The reason it does this is to pave the way for running multiple
instance of the FirefoxDriver. Firefox does at least two checks on
startup to see if it's already running. The first check is disabled by
the magic "MOZ_NO_REMOTE" environment variable, but the second check
looks for the lock file in the profile. If we didn't make a copy, this
second check would trip us up. It would also make assigning a unique
port for the firefox extension a tricky thing to do :)
> Ideally, I'd also like to remove the need for the initial installation
> step (i.e. rake install_firefox) altogether. Calling new
> FirefoxDriver() should automatically create a new profile in a temp
> dir, install the extension, and start Firefox.
That's a good solution, and one that I'd like to see.
> The reason is that I'd like to package a webdriver-firefox.jar file
> that also includes the fxdriver extension, upload that jar to our
> maven repository, and then let the other developers start using
> WebDriver just by adding it as a maven dependency like any other jar,
> without any special installation step.
>
> My first experiments show that it should be possible, but I need to
> play with it a bit longer...
It's entirely possible, and is what Selenium RC does right now. The
only problem with this approach is that it introduces additional
complexity when handling updates to the profile structure as Firefox
evolves. You can see this now when you start the new betas of Firefox
3 on a machine which was previously using Firefox 2: on start up, it
checks the extensions for compatibility and (I imagine) makes other
changes.
The ideal position to be in would be that the single firefox driver
works with both Firefox 2 and 3, so I'm not thrilled about the idea of
putting in a full profile as a ZIP into the firefox JAR (if you see
what I mean) but we can see how things go.
In other news, I've just checked in the fix that allows multiple
versions of firefox to be running simultaneously, so I'd appreciate
some feedback. "Works for me" would be great :) I'm not quite sure how
I'll implement access to an already running instance of Firefox, but
it's something I use on a semi-regular basis when developing the
driver, so it should appear relatively soon. Shout if you'd like it
back sooner.
Regards,
Simon
Mirko Nasato
On 07/03/2008, Simon Stewart <simon.m...@gmail.com> wrote:
>
>
> > Ideally, I'd also like to remove the need for the initial installation
> > step (i.e. rake install_firefox) altogether. Calling new
> > FirefoxDriver() should automatically create a new profile in a temp
> > dir, install the extension, and start Firefox.
> [...]
>
> It's entirely possible, and is what Selenium RC does right now. The
> only problem with this approach is that it introduces additional
> complexity when handling updates to the profile structure as Firefox
> evolves.
>
I haven't looked at what Selenium does, but it's possible to start
Firefox with a new anonymous profile just by pointing it to an empty
dir, e.g.
mkdir /tmp/test-profile; firefox -no-remote -profile /tmp/test-profile
Firefox will automatically populate the new profile on startup. Now
that seems to work fine even if the new dir already contains some
stuff, namely the fxdriver extension and some preferences.
So without any initial installation step, and without calling the
firefox binary multiple times (once with -CreateProfile, once for
actually starting the browser), one could start a WebDriver-enabled
Firefox with the following steps
1. create a new empty directory, e.g. /tmp/webdriver
2. copy the fxdriver extension to
/tmp/webdriver/extensions/fxdr...@googlecode.com
3. write a /tmp/webdriver/prefs.js with desired settings
4. start firefox with -profile /tmp/webdriver
As said, this *seems* to work fine (tested on Linux with Firefox 2 and
3, and on Windows with Firefox 2) but being an undocumented behaviour
I'd like to find out if it is reliable.
>
> The ideal position to be in would be that the single firefox driver
> works with both Firefox 2 and 3, so I'm not thrilled about the idea of
> putting in a full profile as a ZIP into the firefox JAR (if you see
> what I mean) but we can see how things go.
>
Good point. There's actually no need to package fxdriver.xpi into
webdriver-firefox.jar, it can be in a separate JAR, with its own
release cycle. So a Maven 2 project could declare its dependencies
with something like
<dependency>
<groupId>com.googlecode.webdriver</groupId>
<!-- brings in webdriver-common and json-simple as transitive dependencies -->
<artifactId>webdriver-firefox</artifactId>
<!-- no official releases; 280 is the svn revision -->
<version>0.0.280</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.googlecode.webdriver</groupId>
<artifactId>webdriver-firefox-extension</artifactId>
<!-- v0.4 works with Firefox 2; use v?.? for Firefox 3 -->
<version>0.4</version>
<scope>test</scope>
</dependency>
> In other news, I've just checked in the fix that allows multiple
> versions of firefox to be running simultaneously, so I'd appreciate
> some feedback.
Thanks for that! I'll give it a try tomorrow. That should allow us to
schedule automated builds that include Firefox tests.
Kind regards
Mirko
Mirko Nasato
> versions of firefox to be running simultaneously, so I'd appreciate
> some feedback. "Works for me" would be great :)
>
independent, i.e. they do not share the same session.
Mirko
Simon Stewart
There's currently a bit of a race condition if you try and start
multiple instances of firefox at the same time (because of how the
free port allocation works), but I'm working on a fix for that. Glad
to hear that it's working!
Simon
Simon Stewart
On Sun, Mar 9, 2008 at 2:40 PM, Mirko Nasato <mirko....@gmail.com> wrote:
> On 07/03/2008, Simon Stewart <simon.m...@gmail.com> wrote:
> I haven't looked at what Selenium does, but it's possible to start
> Firefox with a new anonymous profile just by pointing it to an empty
> dir,
The behaviour's a little inconsistent. On Linux, the browser process
starts once and then keeps on going. On OS X, the browser process
starts, populates the profile, quits and restarts. I've not tested on
Windows, but I suspect that it'll do the Mac thing rather than follow
the Linux way.
From a practical point of view, this means that on Linux, the process
starts once and remains running. On other OSs, the initial process
dies, and you need to figure out what's running now, which we can do
but is hardly elegant.
> one could start a WebDriver-enabled
> Firefox with the following steps
>
> 1. create a new empty directory, e.g. /tmp/webdriver
> 2. copy the fxdriver extension to
> /tmp/webdriver/extensions/fxdr...@googlecode.com
> 3. write a /tmp/webdriver/prefs.js with desired settings
> 4. start firefox with -profile /tmp/webdriver
>
> As said, this *seems* to work fine (tested on Linux with Firefox 2 and
> 3, and on Windows with Firefox 2) but being an undocumented behaviour
> I'd like to find out if it is reliable.
I think that'll work fine, except for the problems to do with watching
firefox try and repair what it thinks is a broken profile.
> > The ideal position to be in would be that the single firefox driver
> > works with both Firefox 2 and 3, so I'm not thrilled about the idea of
> > putting in a full profile as a ZIP into the firefox JAR (if you see
> > what I mean) but we can see how things go.
> >
> Good point. There's actually no need to package fxdriver.xpi into
> webdriver-firefox.jar, it can be in a separate JAR, with its own
> release cycle.
Sadly, the JAR and the XPI are more closely tied than might seem to be
the case. Most of the time, it's fine to version them independently,
but sometimes (just sometimes) we need to change the communication
protocol or modify the meaning of a command (for example, when
"setValue" was removed) I can imagine that it would cause misery for
someone trying to track this down if all they wanted to do was use the
code.
Regards,
Simon
Mirko Nasato
On 14/03/2008, Simon Stewart <simon.m...@gmail.com> wrote:
>
> On Sun, Mar 9, 2008 at 2:40 PM, Mirko Nasato <mirko....@gmail.com> wrote:
> > On 07/03/2008, Simon Stewart <simon.m...@gmail.com> wrote:
>
>
> > I haven't looked at what Selenium does, but it's possible to start
> > Firefox with a new anonymous profile just by pointing it to an empty
> > dir,
>
>
> The behaviour's a little inconsistent. On Linux, the browser process
> starts once and then keeps on going. On OS X, the browser process
> starts, populates the profile, quits and restarts. I've not tested on
> Windows, but I suspect that it'll do the Mac thing rather than follow
> the Linux way.
>
> From a practical point of view, this means that on Linux, the process
> starts once and remains running. On other OSs, the initial process
> dies, and you need to figure out what's running now, which we can do
> but is hardly elegant.
>
Damn, that's quite annoying. Why don't everybody just use Linux I wonder... ;-)
Thanks for looking into that. So it seems that the current approach,
i.e. installing a base profile first and then starting Firefox with a
temporary copy of that profile, is the best one after all.
I guess the quickest way to avoid the manual installation step is to
modify the FirefoxLauncher to automatically install the base profile
if it can't find a profile named "WebDriver". I'll give it a try in
the next few days.
Kind regards
Mirko