How to run FLB in parallel while functional testing?

[madhusudan g]

Hi,

 

Current issue:

I am running FLB to detect layout bugs when i am executing functional testing scenarios.

 

sequence of test steps in a test scenario:

1. go to the webpage

2. call flb to get layout bugs

3. As part of functional testing, enter <data in the page> like username, password, etc and submit button.

4. In the resulting pages of step3,

            do repeat steps 2 and 3

 

Now the Issue with above sequence is:

step 2 is taking long time and is delaying the step3 (functional test execution).

 

How can we prevent this issue?:

 

Since step 2 is only getting the details and not entering any form fields, i expect this time taking task can be run in parallel to step 3.

For this, one idea is, we should be able to create a virtual copy of webpage (but how to do? any idea?) and call flb asynchronously at step 2. flb can write its results for each page it got.

 

One way we can try is (has limitations), at step 2, we can send the current page url to flb thread so that it works in parallel. But this approach works only if URL is different for different pages.

 

I am looking for good suggestions in how we can avoid the extra time taken for layout checking.

 

Best regards

Madhusudan G

 

[Michael Tamm]

Here is my suggestion:

You should not mix functional and layout testing.

You should instead maintain a separate test suite for layout testing,

which can of course be run in parallel to any functional test suite.

Kind regards, Michael

P.S. Be aware, that when you analyse a page with FLB,
all JavaScript timers will be stopped.

2012/8/23 madhusudan g <madhusud...@gmail.com>

[madhusudan g]

True Michael, we can use separate test suite (r u talking about quick sanity tests? or using crawljax to click all the links of the applcation to traverse pages?) to traverse across pages for layout testing. But it might not cover all the pages of the application for layout testing.

 

Why I preferred functional test suite is the coverage, it traverse almost all pages in various supported flows of the application.

I do not want to mix functional testing and layout testing, i want layout testing should be done in background when functional testing is happening.

 

Please correct me if my approach is incorrect.

 

Thanks

Madhusudan G

[Michael Tamm]

True Michael, we can use separate test suite (r u talking about quick sanity tests? or using crawljax to click all the links of the applcation to traverse pages?) to traverse across pages for layout testing. But it might not cover all the pages of the application for layout testing.

I'm not sure what you mean with quick sanity test, but I definitely do not mean to crawl your web application. ;)

You know you web app best, so it should not be hard to programmatically visit most page types of your webapp.

I understand that your functional test suite as a side effect also visit all different pages of your webapp you you want to make use of this.

So here is another suggestion: You could spread several assertThatThereAreNoLayoutBugs() calls over your functional test suite,

but the implementation of this method should check a system property like "enable.flb" if it should actually perform a analysis

with FLB. This way you can run the functional test suite without the performance penalty of FLB if that system property is not set.

Furthermore you should add an additional build configuration to your CI server to run the functional test suite as a nightly build

with the system property "enable.flb" - this build will of course take longer, but at least you get the desired feedback once a day.

Kind regards, Michael

[madhusudan g]

Michael,

Regarding, "enable.flb" - this build will of course take longer in your response:

 

Actually i want to avoid this longer time and trying for ways. Because gui test suites normally takes more time than calling directly API/ WS calls. The more time is taken by gui tests for page loading, etc.

 

While test suite runs, iss there any way to create a copy of current window and pass it to layout test program, so that layout test will work on that copy in parallel and close that copy of the browser window after it finishes detection. This process will repeat till all the pages gets covered.

I am hoping in this process, the layout testing can be completed with test suite execution. ie. the flb execution current window can be completed by the time the test suite comes to the next window by entering details on the current page, click submit and comparing the actual and expected outputs.
 

I am not sure whether i was able to convey you the problem clearly. I want to talk you directly. Can I have your contact number?

 

Best regards

Madhusudan G

[Andrey Zamovskiy]

I did similar thing by extending get method of selenium TestCase base class.
e.g. each time page is loaded i pass instance of web driver to FLB.

I can share code, but i had more complicated requirements - our tests
written in python,
so i had to use py4j and selenium server to make it work.

--
Andrey

[Michael Tamm]

While test suite runs, iss there any way to create a copy of current window and pass it to layout test program, so that layout test will work on that copy in parallel and close that copy of the browser window after it finishes detection. This process will repeat till all the pages gets covered.

I am hoping in this process, the layout testing can be completed with test suite execution. ie. the flb execution current window can be completed by the time the test suite comes to the next window by entering details on the current page, click submit and comparing the actual and expected outputs.

Although I don't know if this is a good idea you could try the following: When you come to the point, where you want to create a copy of the current window, call WebDriver.getPageSource() to get the current DOM state as HTML. Then you need to add <base href="..."> into the <head> section and finally setup a little embedded HTTP server, which will serve the HTML. Than you can create a new FirefoxDriver, which requests the captured HTML and use it for layout bug detection.

Another idea might be, to start with two parallel FirefoxDriver instances which get the same commands until one is used for layout bug detection and the other for functional testing. You could encapsulate this in your own WebDriver implementation.

If have not implemented any of those ideas yet, so I can't give you any code. I just wanted to provide some ideas.

Kind regards, Michael

[madhusudan g]

Hi Andrey,

 

Thanks for your reply.

Please share the code that shares the webdriver object (duplicate instance of the current window) to FLB.

Currently, we are trying like:

1) while executing of test suite, in each page,

a) get the current url of the page

b) take the cookies of the current page

c) open a new tab with the current url (u get new webdriver object now pointing to the copy of the current page) and set the cookies.

d) pass this new webdriver object to flb server to execute upon.

e) while flb server is processing, test suite execution continues like providing input data and submit which lead to the stepa to step d again.

 

But this approach is working for our old applications but NOT working if the application is AJAX. Currently at step c, we are gettingblank page.

 

Best regards

Madhusudan G

[Andrey Zamovskiy]

Code attached.

Before you start, you need following dependencies:
py4j (http://py4j.sourceforge.net/)
selenium-server-standalone
(http://selenium.googlecode.com/files/selenium-server-standalone-2.25.0.jar)

Add py4j to local maven repository, something like this:
mvn install:install-file
-Dfile=/System/Library/Frameworks/Python.framework/Versions/2.7/share/py4j/py4j0.7.jar
-DgroupId=net.sf.py4j -DartifactId=py4j -Dversion='0.7'
-Dpackaging=jar

now you can build and run py4j gateway:
cd layoutbugs
mvn package
java -jar target/com.odesk.layoutbugs-1.0-jar-with-dependencies.jar

after that start selenium server:
java -jar selenium-server-standalone-2.25.0.jar

after that run example script:
python python.py

It will create selenium server session in python and pass it to py4j.

That's it. Feel free to ask any questions.

--
Andrey