Dataproviders and Parallel execution

[vinay]

Hi All,


I am trying to run test method in parallel. Currently, I am reading
data for XML messages from excel sheet and creating them on the fly,
sending over JMS. The data is read using TestNG DataProvider
feature( as shown below ). But this reads only one message to go to
JMS queue at a given time.

I thought that I can achieve this using he DataProvider at the same so
that my messageLoaderTest can send them to JMS destination, like
below?

@DataProvider(name = "test1", parallel = true)
public Object[][] createData1() {
return new Object[][] {
{ "Cedric", new Integer(36) },
{ "Anne", new Integer(37) },
{ "A", new Integer(36) },
{ "B", new Integer(37) }
};
}

// This test method declares that its data should be supplied
by the Data
// Provider
// named "test1"
@Test(dataProvider = "test1", threadPoolSize = 5)
public void verifyData1(ITestContext testContext, String n1,
Integer n2) {
System.out.println("Context is " + testContext);
System.out.println(n1 + " " + n2);
}


Also, tried adding parallel= true to @dataProvider but got the runtime
error-

I had a look at the link http://groups.google.com/group/testng-users/browse_thread/thread/31d470e498e0de9f
,

but I do not get the error till the time I use parallel = true in the
Dataprovider.

Wrong number of arguments were passed by the Data Provider: found 1
but expected 2)
at
org.springframework.test.context.testng.AbstractTestNGSpringContextTests.run(AbstractTestNGSpringContextTests.java:
158)
at java.util.concurrent.FutureTask
$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Testing.xml config has

<suite name="testtool" junit="false" parallel="methods"
annotations="JDK" data-provider-thread-count="10">

Also, I download the 5.12.2beta.jar but does not seem to work. Is
there maven POM for this version?

Environment:
TestNG 5.12.1
Maven 2.3*
JDK 1.6


Please let me know if someone has an idea on the issue.
Thanks

[Cédric Beust ♔]

Hi Vinay,

The error you are getting:

Wrong number of arguments were passed by the Data Provider: found 1 but expected 2)

seems to be unrelated to the parallel aspect.  The code you are showing looks correct to me, are you sure it's that code that you are running and it's giving you the message above?  That message indicates that the test method expects two parameters but your data provider only supplied one.

If you can isolate this problem in a simple class and email it to me, it should be easy to figure this out.

And no, there is no Maven artifact for beta versions, only for releases.

--
Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
Cédric

[vinay sambyal]

Hi Cedric,

 

Please see below the sample code that I am trying and the error. I deployed the beta version to central repository but it did not help sadly. Please let me know if you are able to simultae the issue now.

 

Regards

 

public class SampleMessageLoaderTest2 {

 protected static Logger logger = Logger
   .getLogger(SampleMessageLoaderTest2.class);

 // This method will provide data to any test method that declares that its
 // Data Provider

 // is named "test1"

 @DataProvider(name = "test1", parallel = true)
 public Object[][] createData1() {

  return new Object[][] {
    { "Cedric", new Integer(36) },
    { "Anne", new Integer(37) },
    { "A", new Integer(36) },
    { "B", new Integer(37) }

  };

 }
 // This test method declares that its data should be supplied by the Data
 // Provider
 // named "test1"

 @Test(dataProvider = "test1", threadPoolSize = 5)
 public void verifyData1(ITestContext testContext, String n1, Integer n2) {
  System.out.println("Context is " + testContext);
  System.out.println(n1 + " " + n2);

 }

}

[TestRunner] Starting executor for test com. SampleMessageLoaderTest2 <<SampleMessageLoaderTest2.txt>> with time out:10000 milliseconds.
FAILED: verifyData1(verifyData1: Wrong number of arguments were passed by the Data Provider: found 2 but expected 3)
java.lang.IllegalArgumentException: wrong number of arguments
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

... Removed 9 stack frames
FAILED: verifyData1(verifyData1: Wrong number of arguments were passed by the Data Provider: found 2 but expected 3)
java.lang.IllegalArgumentException: wrong number of arguments
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

... Removed 9 stack frames
FAILED: verifyData1(verifyData1: Wrong number of arguments were passed by the Data Provider: found 2 but expected 3)
java.lang.IllegalArgumentException: wrong number of arguments
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

... Removed 9 stack frames
FAILED: verifyData1(verifyData1: Wrong number of arguments were passed by the Data Provider: found 2 but expected 3)
java.lang.IllegalArgumentException: wrong number of arguments
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

... Removed 9 stack frames

2010/7/7 Cédric Beust ♔ <ced...@beust.com>

[Cédric Beust ♔]

Hi Vinay,

I was wrong, adding parallel=true indeed triggers this bug.  I'll get back to you.

--
Cédric

[Cédric Beust ♔]

Hi Vinay,

I think I fixed the bug, can you try the beta?  http://testng.org/beta

Thanks.

--
Cédric

--
Cédric

[vinay sambyal]

+ testng...@googlegroups.com

On Thu, Jul 8, 2010 at 6:26 PM, <Vinay....@nomura.com> wrote:

Hi Cedric,

 

The beta works fine for me.

 

Here are some of things that I observed while testing my stuff.

 

1.    When I remove the parallel = true, attribute from the DataProvider, the time taken to run the test( Sample class which I sent yesterday ) is 0.407 sec

2.    And, if I run with  parallel = true, attribute from the DataProvider, the time taken to run the test( Sample class which I sent yesterday ) is 1.391 sec to 1.466 secs( when thread-count for providers is 10 ). I think this might be due to unnecessary context switching? As each time the verifyData1 is called the new threaded is printed in log, which I am printing by calling

System.out.println("ID is " + Thread.currentThread().getId()); inside veryFyData1 method.

I am hoping that this( parallelism) will be effective when the actual test will be like sending a request over network or some JMS kind of thing.

3.    The sure-fire report displays same threaded though even when I can see different thread Ids as mentioned in point 2.

 

 

Please let me know if my understanding is correct and see the settings.xml content as well( below ).

 

 

 

 

Settings.xml content—

 

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="testtool" junit="false" parallel="methods" annotations="JDK" data-provider-thread-count="2">

  <test verbose="2" name="com.SampleMessageLoaderTest2" junit="false" annotations="JDK" parallel="methods">

    <classes>

      <class name="com.SampleMessageLoaderTest2 "/>

    </classes>

  </test>

</suite>

 

 

Regards,

-Vinay

 

P.S: I have raised a request in office to unblock the post link so that we can post queries. But I am wondering if we can post the messages using official mail id( tried but thing it needs gmail account).

 

From: vinay sambyal [mailto:vinay....@gmail.com]
Sent: 08 July 2010 08:12
To: Cédric Beust ♔
Subject: Re: [testng-users] Dataproviders and Parallel execution

 

Hi Cedric,

 

I will try from office and let you know. Unfortuntaely, the google groups are blocked in office and neither can I send a mail as I do not see option join the group with non-gmail id.

 

Thanks

This e-mail (including any attachments) is confidential, may contain

proprietary or privileged information and is intended for the named

recipient(s) only. Unintended recipients are prohibited from taking action

on the basis of information in this e-mail and must delete all copies.

Nomura will not accept responsibility or liability for the accuracy or

completeness of, or the presence of any virus or disabling code in, this

e-mail. If verification is sought please request a hard copy. Any reference

to the terms of executed transactions should be treated as preliminary only

and subject to formal written confirmation by Nomura. Nomura reserves the

right to monitor e-mail communications through its networks (in accordance

with applicable laws). No confidentiality or privilege is waived or lost by

Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is

a reference to any entity in the Nomura Holdings, Inc. group. Please read

our Electronic Communications Legal Notice which forms part of this e-mail:

http://www.Nomura.com/email_disclaimer.htm

[vinay sambyal]

Hi Cedric/All,

I started running my tests but have the following problem when I run it on command line using maven. I tried running many times and each time it fails with the error below.

I have 25000 messages to sent( reading from excel and sending over JMS) and sometimes it is able to send 100 and sometimes 300 to 400.

[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: C:\Dev\PROJECT_NAME \target\surefire-reports
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
[TestRunner] Starting executor with time out:10000 milliseconds.
[TestRunner] Starting executor for test com.SampleMessageLoaderTest with time out:10000 milliseconds.
java.lang.InterruptedException: sleep interrupted
        at java.lang.Thread.sleep(Native Method)
        at org.testng.internal.PoolService.submitTasksAndWait(PoolService.java:106)
        at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1033)
        at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:137)
        at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:121)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580804,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=3, id=123, name=ASam, number=580802,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580807,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=3, id=123, name=peter, number=580800
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580801,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=3, id=123, name=ASam, number=580803,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580800,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580805,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580806,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580800,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580810,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=3, id=123, name=ASam, number=580812,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=3, id=123, name=ASam, number=580809,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580811,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=3, id=123, name=ASam, number=580808,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=3, id=123, name=ASam, number=580813,
PASSED: messageLoaderTest(org.testng.TestRunner@7cae6e, SampleMessage [type=1, id=123, name=ASam, number=580815,
Exception in thread "pool-2-thread-1" java.util.ConcurrentModificationException
        at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
        at java.util.AbstractList$Itr.next(AbstractList.java:343)
        at org.testng.reporters.TextReporter.logResults(TextReporter.java:81)
        at org.testng.reporters.TextReporter.onFinish(TextReporter.java:31)
        at org.testng.TestRunner.fireEvent(TestRunner.java:1259)
        at org.testng.TestRunner.afterRun(TestRunner.java:1036)
        at org.testng.TestRunner.run(TestRunner.java:557)
        at org.testng.SuiteRunner.runTest(SuiteRunner.java:311)
        at org.testng.SuiteRunner.access$000(SuiteRunner.java:33)
        at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:344)
        at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:151)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

[TestNG] Reporter org.testng.reporters.SuiteHTMLReporter@47858e failed
java.util.ConcurrentModificationException
        at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
        at java.util.AbstractList$Itr.next(AbstractList.java:343)
        at org.testng.reporters.SuiteHTMLReporter.generateMethodsChronologically(SuiteHTMLReporter.java:365)
        at org.testng.reporters.SuiteHTMLReporter.generateReport(SuiteHTMLReporter.java:68)
        at org.testng.TestNG.generateReports(TestNG.java:853)
        at org.testng.TestNG.run(TestNG.java:839)
        at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:74)
        at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:92)
        at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
        at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)
Tests run: 388, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.046 sec
Results :
Tests run: 388, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14 seconds
[INFO] Finished at: Thu Jul 15 20:30:40 BST 2010
[INFO] Final Memory: 16M/39M
[INFO] ------------------------------------------------------------------------

Just to add I have following in POM

Cedric- The version is the patch version which you gave to fix the parallel issue thing as per mail above.

 

The testng JAR at the compile time is testng-5.11-jdk15.jar, this is picked up automatic dependency due to Citrus integration tool that I am using.
Please let me know why its breaking.

<dependency>
                        <groupId>org.testng</groupId>
                        <artifactId>testng</artifactId>
                        <version>5.12.2</version>
                        <scope>test</scope>
                </dependency>
But, when I run from eclipse this error does not come and I am not able to use parallel=true. Not sure if that is due to some dependencies in eclipse or TestNG plugin of eclipse( I am using 3.5 ).

java.lang.IllegalArgumentException: wrong number of arguments

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:643)
        at org.testng.internal.MethodHelper$1.runTestMethod(MethodHelper.java:761)
        at org.springframework.test.context.testng.AbstractTestNGSpringContextTests.run(AbstractTestNGSpringContextTests.java:158)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.testng.internal.MethodHelper.invokeHookable(MethodHelper.java:769)
        at org.testng.internal.Invoker.invokeMethod(Invoker.java:552)
        at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:723)
        at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:73)
        at org.testng.internal.TestMethodWithDataProviderMethodWorker.call(TestMethodWithDataProviderMethodWorker.java:15)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

Any input would be great help as I was looking to produce some results for team.

Regards
-Vinay

2010/7/7 Cédric Beust ♔ <ced...@beust.com>

[Cédric Beust ♔]

Hi Vinay,

Odd, it looks like two reporters are generating their results in two different threads. I'm not sure how this could happen, but it looks like it is.

Until I find the time to investigate, here are a few suggestions that might unblock you:

• Disable the TextListener (`-useDefaultListeners false` and then re-add SuiteHTMLReporter).
• If this doesn't work, on line SuiteHTMLReporter:365, wrap the for loop in a synchronized(invokedMethods).

--
Cédric

--
Cédric