parallel test classes execution how to maintain correct objects for each of the threads ?

[80Vikram]

Hi All,

I need to run test classes in parallel on multiple android emulators

Scenario 2 is failing because correct object is not getting passed from one class to another

Please clarify how to maintain correct object between classes ?


In below execution

test 1 from 2nd class and test 2 from 1st class are getting passed.

Scenario 1: Passing

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="RunAll Test" parallel="tests" thread-count="2">

    <test name="Test-Nexus">
        <parameter name="appName_" value="dummy_Android_1" />
        <classes>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTest"/>        
        </classes>
    </test>
   
    <test name="Test-Samsung">
        <parameter name="appName_" value="dummy_Android_2" />
        <classes>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTest"/>          
        </classes>
    </test>


Scenario 2: Failing

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="RunAll Test" parallel="tests" thread-count="2">

    <test name="Test-Nexus">
        <parameter name="appName_" value="dummy_Android_1" />
        <classes>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTest"/>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTestClass2"/>
        </classes>
    </test>
   
    <test name="Test-Samsung">
        <parameter name="appName_" value="dummy_Android_2" />
        <classes>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTest"/>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTestClass2"/>
        </classes>
    </test>
   
</suite>

[Krishnan Mahadevan]

Not sure what do you mean by correct object is not getting passed. You would need to ensure that you build the synchronization between your classes.

 

TestNG provides you with the following methods to help you share data.

 

1. To share data between classes that belong to the same <test> use ITestContext.setAttribute() and iTestContext.getAttribute() accordingly. But you would need to ensure you build the co-ordination properly especially when within the <test> all your classes are running in parallel, because there can be definitely a data race.
2. To share data between multiple <tests> that belong to the same <suite> use ISuite.setAttribute() and iTestContext.getAttribute() accordingly.

 

From within any test method, you can get access to both these objects via

 

• ITestContext - Reporter.getCurrentTestResult().getTestContext()
• ISuite - Reporter.getCurrentTestResult().getTestContext().getSuite()

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[80Vikram]

Hi Krishnan,

Please find more details on this

1. With help of testng.xml setup
https://github.com/vikramvi/appium-parallel-execution-seleniumGrid/blob/master/testng.xml,

I could get test cases in single class to run in parallel mode on connected devices

https://github.com/vikramvi/appium-parallel-execution-seleniumGrid/blob/master/src/test/java/com/appium/seleniumgrid/parallel/poc/test_classes/AppiumParallelTest.java

2. Now I would like to replicate same setup on multiple test classes.
My query is how to pass along multiple drivers across test classes ?

Thanks,
Vikram

[⇜Krishnan Mahadevan⇝]

I still don't understand what do you mean by "My query is how to pass along multiple drivers across test classes ?"​

Why do test classes need to be provided with driver objects ?

Have you considered doing something like what I talk about in this below blog post of mine ?

https://rationaleemotions.wordpress.com/2013/07/31/parallel-webdriver-executions-using-testng/ 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

[80Vikram]

Hey Krishnan,

After going through your blog, few more queries popped up

1.  how is this approach different from adding thread-count = X to <suite name="Suite" parallel="methods">

meantime will modify the design to try out your solution.

Thanks,
Vikram


[⇜Krishnan Mahadevan⇝]

I was not saying it was the same either. The blog post that I shared talks about how to effectively manage your webdriver instantiation outside of a test class rather than combining it as part of the test class. These two are two different things.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

[80Vikram]

Hi Krishnan,

Thanks for answering queries patiently and with as much details.

I could able to refactor existing code to run multiple test cases ( from single class ) in parallel.

Need to still figure out running multiple classes though, hopefully I should find answer by tomorrow.


Also I made use of ISuiteListener to setup selenium grid sever and register device nodes before creating driver with IInvokedMethodListener. With your example code looks much cleaner.

Regards,
Vikram

[80Vikram]

Hey Krishnan,

I could got the parallel run ( same test cases from multiple classes ) on all the connected devices ( this achieves compatibility testing ) running with below setup

<suite name="RunAll Test" parallel="tests" thread-count="2">

<listeners>
<listener class-name="com.appium.testng.listeners.DriverListener"></listener>
</listeners>

    <test name="Test-Nexus">
        <parameter name="appName_" value="dummy_Android_1" />
        <classes>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTest"/>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTestClass2"/>
        </classes>
    </test>
   
    <test name="Test-Samsung">
        <parameter name="appName_" value="dummy_Android_2" />
        <classes>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTest"/>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTestClass2"/>
        </classes>
    </test>
   
</suite>

Query:

How can I achieve distributed parallel run ( to finish regression quickly, run test cases once on available devices connected to hub )
With Junit setup I've made use of maven failsafe plugin & thread count to distribute test cases to available/free nodes

I'm not understanding how to achieve same with TestNG

Thanks & Regards,
Vikram

[⇜Krishnan Mahadevan⇝]

Test distribution to nodes is an attribute of the Grid.

You can bump up your thread-count (the suite xml you have shared currently has 2) and that many number of threads would be spun off by TestNG and used to run tests.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

[80Vikram]

My requirement is to distribute test cases to available nodes, increasing thread count didn't solve this problem.

I agree that Selenium Grid takes care of distribution but how to control that from TestNG.

Below is the setting for maven failsafe plugin

                            <parallel>classes</parallel>
                            <threadCount>1</threadCount>
                            <reuseForks>false</reuseForks>

Thanks,
Vikram

[⇜Krishnan Mahadevan⇝]

You cannot control the TestSession distribution done by the Grid from outside of it.

The distribution logic it follows is pretty standard [ Works on a message queue concept ]

• Check if the incoming NewSession request provided desired capabilities matches with the capabilities available in the Node pool of the hub.
  
• If no match throw error and bail out.
• If match found check if the node in question has a free slot.
• If free slot available forward the NewSession request to the node.
• If no free slot available, dump the NewSession request back to the queue and continue to poll periodically for slot availability.

This cannot be controlled from outside of the Grid/Node infrastructure. This values are typically provided to the node at the time when it comes up.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

[80Vikram]

Thanks for detailed clarification.

Regards,
Vikram

[80Vikram]

Hey Krishnan,

For now to keep it simple, I'm using below approach

1 test runner for each of the test class.  It's kind of hard coding upfront node with test class.

<suite name="RunAll Test" parallel="tests" thread-count="2">
<listeners>
<listener class-name="com.appium.testng.listeners.DriverListener"></listener>
</listeners>
    <test name="Test-Nexus">
        <parameter name="appName_" value="dummy_Android_1" />
        <classes>
            <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTest"/>
          

        </classes>
    </test>
   
    <test name="Test-Samsung">
        <parameter name="appName_" value="dummy_Android_2" />
        <classes>

           <class name="com.appium.seleniumgrid.parallel.poc.test_classes.AppiumParallelTestClass2"/>
        </classes>
    </test>
   
</suite>

1. This can be optimized by finding all the connected nodes/devices upfront and assigning 1 test class to each one of them
2.  Problem with this approach, in case a node gets freed, it doesn't get assigned new job
3. I remember you were working on https://github.com/paypal/SeLion, does this framework takes care of distribution logic per your earlier answer ?

Thanks,
Vikram

[Krishnan Mahadevan]

>>>> 3. I remember you were working on https://github.com/paypal/SeLion, does this framework takes care of distribution logic per your earlier answer ?

 

No. SeLion wouldn’t do it either. Please understand that you are talking about two different JVMs trying to co-ordinate on the number of threads.

 

There’s a JVM #1 which is created when you launch your tests. Here you can manage your thread pool etc., using a TestRunner such as TestNG.

There’s a JVM #2 which is a standalone jar running as Hub. The distribution logic (which is again dependent on the number of slots available in JVM #3 (node1), JVM #4(node2) and so on) cannot be controlled from JVM #1

[80Vikram]

Thanks for detailed clarification again.

can you please comment on pt. 1 & 2 ?

Thanks & Regards,
Vikram