does tellurium support parallel execution?

[Jason]

hi all,

I'm trying to parallelize our tellurium tests, and am running into
frequent cases where requests through the connector give me:

Exception : com.thoughtworks.selenium.SeleniumException:
The_server_localhost_failed_to_respond_with_a_valid_HTTP_response.:
java.lang.NullPointerException
at
org.codehaus.groovy.runtime.callsite.GetEffectivePojoFieldSite.acceptGetProperty(GetEffectivePojoFieldSite.java:
43)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:
237)
at
org.tellurium.connector.CustomSelenium.getActiveSeleniumSession(CustomSelenium.groovy:
76)

Is there anything that needs to be done for the selenium connector to
be thread-safe? Does it need synchronizaiton, or are lower levels in
the stack already synchronized?

My use-case is this:

I have multiple dslContexts, each of which represents a webapp (I have
2) that is being manipulated during a test. Setup is being done
through a TestNG listener, which establishes the DslContexts, does
connectUrl() on them, etc. Then my testcase begins.

As mentioned, I get frequent SeleniumExceptions when trying to use
parallel="methods" while executing.

From looking at the code, I see the the connector isn't synchronized,
nor is the CustomSelenium object. It also would seem to me that this:

// Get the active Selenium RC session
def CustomSelenium getActiveSeleniumSession(){
DefaultSelenium sel =
com.thoughtworks.selenium.grid.tools.ThreadSafeSeleniumSessionStorage.session()
CommandProcessor processor = sel.commandProcessor
CustomSelenium csel = new CustomSelenium(processor)
/*
if(this.userExtension != null &&
this.userExtension.trim().length() > 0){
File userExt = new File(this.userExtension);
// processor.setExtensionJs(userExt.getAbsolutePath())
processor.setExtensionJs(this.userExtension)
println "Add user-extensions.js found at given path: " +
userExt.getAbsolutePath() + " to Command Processor";
}
*/
csel.customClass = this.customClass
csel.passCommandProcessor(processor)

return csel
}

would have to be thread-safe or the csel object returned back may not
always be correct -- ie: CustomSelenium would have to be protected
like DefaultSelenium is, right?

thanks!
Jason

[Haroon Rasheed]

Hi Jason,

are you trying to run the tests against Selenium Grid?? what your selenium server host is in the TelluriumConfig file. I have not looked at the connector code for some time now. I know that com.thoughtworks.selenium.grid.tools.ThreadSafeSeleniumSessionStorage.session() is thread safe and if you are running the tests using Selenium Grid and the tests are going to get the selenium sessions from Selenium HUB then it would be easy to manage the parallel test execution. 

Cheers

Haroon

--
You received this message because you are subscribed to the Google Groups "tellurium-users" group.
To post to this group, send email to telluri...@googlegroups.com.
To unsubscribe from this group, send email to tellurium-use...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/tellurium-users?hl=en.

[Jason]

Hey Haroon,

Yeah, I'm running under the grid. The selenium server host is
localhost:4444 (but i'm not sure why you're asking...)

It seems that the parts of the code that CustomSelenium wraps around
the DefaultSelenium object would need to be ThreadLocal too, would
they not? Let me come up with a generic testcase (that only uses
tellurium stock framework code) for this that I can attach (and
presumably reproduce!)

Thanks,
Jason

On Mar 16, 4:17 pm, Haroon Rasheed <haroonz...@gmail.com> wrote:
> Hi Jason,
>
> are you trying to run the tests against Selenium Grid?? what your selenium
> server host is in the TelluriumConfig file. I have not looked at the
> connector code for some time now. I know
> that com.thoughtworks.selenium.grid.tools.ThreadSafeSeleniumSessionStorage.session()
> is thread safe and if you are running the tests using Selenium Grid and the
> tests are going to get the selenium sessions from Selenium HUB then it would
> be easy to manage the parallel test execution.
>
> Cheers
> Haroon
>

> > tellurium-use...@googlegroups.com<tellurium-users%2Bunsu...@googlegroups.com>

[Jason]

OK, I think this demonstrates the issue. There are 2 parts -- one is
the suite file, the other is the test.

Test first:

import org.tellurium.test.groovy.*;

import org.testng.annotations.*;
import static org.testng.AssertJUnit.*;
import org.tellurium.dsl.DslContext;

public class UltraSimpleParallelTest extends
TelluriumGroovyTestNGTestCase {

public void initUi() { }

@Test
public void testOne() {
1.upto(10) {
connectUrl("http://google.com");
assertEquals("location should be google", "http://
www.google.com/", connector.sel.getProperty("location"));
sleep(500)
}
}

@Test
public void testTwo() {
1.upto(10) {
connectUrl("http://amazon.com");
assertEquals("location should be amazon", "http://
www.amazon.com/", connector.sel.getProperty("location"));
sleep(500)
}
}
}

suite file:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite thread-count="2" skipfailedinvocationCounts="false" verbose="1"
name="parallel_test" junit="false" parallel="methods"
annotations="JDK" data-provider-thread-count="2">
<test name="parallel_test" junit="false" parallel="methods"
annotations="JDK">
<classes>
<class name="UltraSimpleParallelTest">
<methods>
<include name="testOne"/>
<include name="testTwo"/>
</methods>
</class>
</classes>
</test>
</suite>

[Jason]

Oops, I realize this test needs thread synchronization ( so that the
connectUrl() and asserts are done in a synchronized block), but for
now, just note that you'll get the selenium error:

com.thoughtworks.selenium.SeleniumException:
The_server_localhost_failed_to_respond_with_a_valid_HTTP_response

instead of just assertion failures...

On Mar 16, 6:41 pm, Jason <jleco...@gmail.com> wrote:
> OK, I think this demonstrates the issue.  There are 2 parts -- one is
> the suite file, the other is the test.
>
> Test first:
>
> import org.tellurium.test.groovy.*;
>
> import org.testng.annotations.*;
> import static org.testng.AssertJUnit.*;
> import org.tellurium.dsl.DslContext;
>
> public class UltraSimpleParallelTest extends
> TelluriumGroovyTestNGTestCase {
>
>     public void initUi() { }
>
>     @Test
>     public void testOne() {
>         1.upto(10) {
>             connectUrl("http://google.com");

>             assertEquals("location should be google", "http://www.google.com/", connector.sel.getProperty("location"));

>             sleep(500)
>         }
>     }
>
>     @Test
>     public void testTwo() {
>         1.upto(10) {
>             connectUrl("http://amazon.com");

>             assertEquals("location should be amazon", "http://www.amazon.com/", connector.sel.getProperty("location"));

[Jian Fang]

Jason,

Thanks for bringing this question up. You are right, we haven't thought of
the parallel execution in Tellurium Core yet. We will reproduce your problem
and see if we can have a quick fix.

More important concerns may include the singleton pattern we used in Tellurium
core. Perhaps we should consider both singleton and prototype objects like Spring
bean or Google Guice does. We need to think about this more deeply.

Thanks again,

Jian

--
You received this message because you are subscribed to the Google Groups "tellurium-users" group.
To post to this group, send email to telluri...@googlegroups.com.

To unsubscribe from this group, send email to tellurium-use...@googlegroups.com.

[Jian Fang]

Issue 408 has been created to track this problem.

http://code.google.com/p/aost/issues/detail?id=408

[Jian Fang]

I created a TestNG Java test case and use thread pool for parallel execution, and I got a different error.
Could you try my code to see what you get?

Thanks,

Jian

public class SampleTestNGParallelTest extends TelluriumTestNGTestCase {

    @BeforeClass
    public static void initUi() {
        connectSeleniumServer();
        useTrace(true);
    }

    @Test(threadPoolSize = 5, invocationCount = 10)
    public void testOne() {

      connectUrl("http://google.com");
      assertEquals("location should be google", "http://www.google.com/", connector.getSelenium().getProperty("location"));
        try {
            sleep(500);
        } catch (InterruptedException e) {
           
        }
    }

    @Test(threadPoolSize = 5, invocationCount = 10)
    public void testTwo() {
      connectUrl("http://amazon.com");
      assertEquals("location should be amazon", "http://www.amazon.com/", connector.getSelenium().getProperty("location"));
        try {
            sleep(500);
        } catch (InterruptedException e) {

        }
    }

    @AfterClass
    public static void tearDown() {
        showTrace();

[Jian Fang]

Has anyone tried Groovy Parallel Systems (GPars)?

http://gpars.codehaus.org/

Maybe we should consider adding parallel execution support in Core with GPars?