loadtesting with headless webkit

[daniel]

I am in charge of load testing a complex enterprise ajax application.
I currently use the grinder and script recording to accomplish this.
Because of the highly dynamic nature of the site large amounts of post
processing is necessary to get the playback scripts working. The
trouble is playback scripts are extremely delicate and have to be
maintained during our development cycle. It is essentially breaking
my goal of continuous testing. I don't want my test to break unless
an actual regression is detected. The only way I am going to keep up
with the development cycle is full automation and right now
maintaining worker scripts for driving the load is a huge bottleneck
that I simply need to eliminate.

I have been struggling with HTMLUnit. I have also been trying to use
the rhino javascript engine with envjs. With all of these tools I
keep hitting various limitations and I keep asking myself the same
question: Both firefox and webkit are open source, how come I can’t
create a real headless browser?

Now, the best solution is exactly what you guys are doing. I would
like to know how mature this project is? What kinds of problems need
to be solved? I would be willing to provide assistance in solving the
problems necessary to get this going as I am desperately in need of
this tool.

Thanks for taking the initiative and getting this started!

[daniel]

[Igor Viarheichyk]

Hi Daniel,

The project is relatively young, but it implements most of the WebDriver API except very recent one 

(asynchronous script execution). It is used in production for testing web applications, and looks pretty stable, 

thought some crashes were discovered recently. 

As for known limitations - it is based on one-year-old snapshot of WebKit code, so most recent browser features may be not supported. The only supported platform at the moment is Linux. It is completely headless meaning it does not render pages at all - and it is not possible for now to get page screenshots. However, page layout is calculated so tests can use visual positions of elements, taking into account that size of elements depending on system widgets or fonts differs from other browsers.

If these limitations are not critical, it makes sense to do a test run with webkitdriver, please let us know if you discover any issues with it.

Best regards,

Igor

--
You received this message because you are subscribed to the Google Groups "webkitdriver" group.
To post to this group, send email to webkit...@googlegroups.com.
To unsubscribe from this group, send email to webkitdriver...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/webkitdriver?hl=en.

[Baran ♪]

And feel free to send us patches if you want to contribute. 

- Baran

[Daniel Meyer]

Thank you so much for the reply.  This sounds great.

You mentioned that the asynchronous script execution is not supported (at least through the webdriver interface).  I have complex ajax interactions that need to be tested.  I suppose it would not be too hard to synchronize the AJAX requests like you can do with HTMLUnit.  However, because I am doing load testing it would be preferable to support asynchronous script execution.  How much work would be required to support this?

Also, in the wiki I read that "database, application cache and cookie storage is shared between all instances of WebKitDriver, living within a process."  I am going to need to run hundreds of instances on a single machine does this mean they all share the same cache?  Is there a way to remedy this and if so how hard would it be.

My last question is how hard would it be to get the webkit code up to date? 

In any case I will give this a shot tomorrow.

Thanks!

[Igor Viarheichyk]

On Thu, Jan 20, 2011 at 7:38 AM, Daniel Meyer <meye...@gmail.com> wrote:

Thank you so much for the reply.  This sounds great.

You mentioned that the asynchronous script execution is not supported (at least through the webdriver interface).  I have complex ajax interactions that need to be tested.  I suppose it would not be too hard to synchronize the AJAX requests like you can do with HTMLUnit.  However, because I am doing load testing it would be preferable to support asynchronous script execution.  How much work would be required to support this?

 Asynchronous javascript execution is supported, this is new WebDriver API for dealing with callback functions like those in setTimeout or XMLHttpRequest is not implemented yet, but planned to be implemented in nearest future.

Also, in the wiki I read that "database, application cache and cookie storage is shared between all instances of WebKitDriver, living within a process."  I am going to need to run hundreds of instances on a single machine does this mean they all share the same cache?  Is there a way to remedy this and if so how hard would it be.

This is correct, but WebKitDriver spawns separate process for each instance now, so each uses own database, application cache and cookie storage. 

My last question is how hard would it be to get the webkit code up to date? 

it is hard, there is lots of changes in code which should be merged, but it is planned to be done after synching with WebDriver api changes.