@Test(sequential=true)

[mikkin]

Hi,

We are facing this really strange error, when we use "sequential=true"
annotation to run the test classes in parallel. Basically, we have a
connection class in our framework that makes GET and POST calls to the
server. Now, in single threaded mode, things work just fine, but when
we try to run these tests in parallel, we are also forced to make the
GET and POST calls synchronized else the tests start skipping.

I was always of the opinion that when running tests in parallel, each
test will be run in one thread (with the usual thread re-use). And
this thread should have had its own instances of our classes to be
tested..isnt it?

Any help on this would be really appreciated.


Thanks
Mikkin

[Cédric Beust ♔]

Hi Mikkin,

A lot of factors can alter the way your tests are run in multithread mode, could you please post a short sample of your code, the testng.xml you are using, what you are seeing and what you expected instead?

--
Cedric

--

Cédric

[mikkin]

Here is the testNG.xml

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="XYZ Functional Tests by packages" parallel="true" thread-
count="10">
<test verbose="2" name="com.ABC.test.tests.sync" annotations="JDK">
<packages>
<package name="com.ABC.test.tests.sync"/>
</packages>
</test>
</suite>

So, when we run the test in single threaded mode (no parallel, no
thread-count) the tests run just fine without any error. But in multi
threaded mode, it fails saying " the connection is not open" and it
also starts skipping tests.

java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(Unknown Source)
at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown
Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(Unknown
Source)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.apache.commons.httpclient.HttpParser.readRawLine(Unknown
Source)
at org.apache.commons.httpclient.HttpParser.readLine(Unknown Source)
at org.apache.commons.httpclient.HttpConnection.readLine(Unknown
Source)
at org.apache.commons.httpclient.HttpMethodBase.readStatusLine
(Unknown Source)
at org.apache.commons.httpclient.HttpMethodBase.readResponse(Unknown
Source)
at org.apache.commons.httpclient.HttpMethodBase.execute(Unknown
Source)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry
(Unknown Source)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod
(Unknown Source)
at org.apache.commons.httpclient.HttpClient.executeMethod(Unknown
Source)
at org.apache.commons.httpclient.HttpClient.executeMethod(Unknown
Source)
at com.ABC.test.framework.connection.Connection.send(Connection.java:
177)
at com.ABC.test.framework.connection.Connection.doPost
(Connection.java:245)
at
com.ABC.test.tests.sync.CreateMonitorTest.checkRequiredParamsCreateMonitor
(CreateMonitorTest.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:
580)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:478)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:617)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:885)
at org.testng.internal.TestMethodWorker.invokeTestMethods
(TestMethodWorker.java:126)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:
110)
at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run
(ThreadUtil.java:132)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)




Thanks
Mikkin

> ***Cédric
> *

[Cédric Beust ♔]

Did you try parallel="tests"?

--
Cedric

--

Cédric

[mikkin]

I tried that now (since I didnt remember if I had tried it
before :) ), but it works more like single threaded. I can see only
one test method running at a time.


Thanks
Mikkin

> ***Cédric
> *

[Cédric Beust ♔]

On Thu, Mar 26, 2009 at 6:58 PM, mikkin <mikki...@gmail.com> wrote:

I tried that now (since I didnt remember if I had tried it
before :) ), but it works more like single threaded. I can see only
one test method running at a time.

Well, yes, that's the point.  I thought that you wanted to run all your tests in separate threads except for these two that you want sequential.

Please go back to my first message, I asked you a few very specific questions, can you please provide this information?

Thanks.

--
Cedric

[mikkin]

I am running the tests at package level, so my testng.xml (pasted in
previous thread) is calling a package which has about 75 test classes.

Regarding the sample code, I am not sure what part of code you would
like to see...

@Test(sequential=true)
public class CreateTest extends HomeTest {
private final static String CREATE_URL = "/create.abc";
// Test Variables
private Account account;
private Bucket bucket;

@BeforeMethod
public void setupTests() {
account = new Account();
Feedback fb = account.create();
Assert.assertTrue(fb instanceof Success, "Account Creation
failed" );
account.login();
bucket = new Bucket();
res = bucket.create();
}

@Test
public void onGet() {
Map<String, String> myMap = new HashMap<String, String>();
params.put("bucket_id", bucket.getPath());
Feedback fb = secureConn.doGet(CREATE_URL, myMap, null);
Assert.assertTrue(fb instanceof Success,
"Create call should work over GET" );
}
}

All I want testNG to do is run the test classes inside that package
(specified in testng.xml) in parallel. So basically I would see
simultaneous running of methods (so if my thread-count=10, then at
most 10 methods from 10 different classes will start their run and the
threads would continue running different methods of that same class
till its over, and then switch to next class present in the package)


On Mar 26, 7:02 pm, Cédric Beust ♔ <cbe...@google.com> wrote:

[Cédric Beust ♔]

On Thu, Mar 26, 2009 at 7:30 PM, mikkin <mikki...@gmail.com> wrote:

All I want testNG to do is run the test classes inside that package
(specified in testng.xml) in parallel. So basically I would see
simultaneous running of methods (so if my thread-count=10, then at
most 10 methods from 10 different classes will start their run and the
threads would continue running different methods of that same class
till its over, and then switch to next class present in the package)

Isn't that exactly what you are seeing and that's what you *didn't* want to happen, based on your first message?

If you want to run most of your methods in parallel except a few that need to run in sequence, the easiest way is to use dependencies.

--

Cédric