invocationCount + invocationTimeOut

[Tam]

Hi Folks,

We want to be able to run a specific Test for a specified TimeOut..  This will give me a ability to see how a test case is doing over a period and keep it running to track down issues with the environment. 

Below is my test, which is getting invoked 10 Times but not coming out in 10 Milliseconds.  More over, if invocationTimeOut is not specified @Before and @After are getting executed before each test which is as expected but when specified @Before and @After is getting executed just once.   

Any pointers/help would really appreciate.

package com.tools.test.practice.mytests;

import org.testng.annotations.AfterMethod;

import org.testng.annotations.BeforeMethod;

import org.testng.annotations.Test;

public class Test1 {

@Test(invocationCount=10,invocationTimeOut=10)

public void test1() {

System.out.println("TestMethod");

}

@BeforeMethod

public void beforeMethodTest1() {

System.out.println("BeforeMethod");

}

@AfterMethod

public void afterMethodTest1() {

System.out.println("AfterMethod");

}

}

Thanks Much

[Krishnan Mahadevan]

Tameem,

If you look at the javadocs for the invocationTimeout [ http://testng.org/javadocs/org/testng/annotations/Test.html#invocationTimeOut() ]

 you will notice the following:

The maximum number of milliseconds that the total number of invocations on this test method should take. This annotation will be ignored if the attribute invocationCount is not specified on this method. If it hasn't returned after this time, it will be marked as a FAIL.

So ideally speaking your invocationTimeout value should be calculated as :

Time taken per invocation * Number of invocations.

Here's a sample that shows how this works :

public class TimeOutExample {
    //Here the expectation is that to execute testMethod 10 times it should not take more than 900 ms.
    //But every iteration is going to sleep for 100 ms. So running it for 10 times is going to take 1 second.
    @Test (invocationTimeOut = 900, invocationCount = 10)
    public void testMethod() throws InterruptedException {
        Thread.sleep(100); //Sleep for 0.1 seconds
    }
}

When this method is executed you will see the below exception

org.testng.internal.thread.ThreadTimeoutException: Method org.testng.internal.TestNGMethod.testMethod() didn't finish within the time-out 900

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[Pratik Jain]

Sorry for Posting this in old thread.

Hi Krishnan,

I am seeing weird test output print when I am using InvocationTimeOut.

for example: -  If my test is 

@Test(invocationCount=5)

public void testMethod() throws InterruptedException {

Thread.sleep(100);

System.out.println("Print....");

}

then output is 

===============================================

    Default test

    Tests run: 5, Failures: 0, Skips: 0

===============================================

===============================================

Default suite

Total tests run: 5, Passes: 5, Failures: 0, Skips: 0

===============================================

But when I use InvocationTimeOut like below example

@Test(invocationCount=5,invocationTimeOut=1000)

public void testMethod() throws InterruptedException {

Thread.sleep(100);

System.out.println("Print....");

}

then output is 

[RemoteTestNG] detected TestNG version 7.0.0

Print....

Print....

Print....

Print....

Print....

PASSED: testMethod

===============================================

    Default test

    Tests run: 1, Failures: 0, Skips: 0

===============================================

===============================================

Default suite

Total tests run: 1, Passes: 1, Failures: 0, Skips: 0

===============================================

In this case it is considering total run is 1.

Also, in case of failure of InvocationTimeOut it print as 

@Test(invocationCount=5,invocationTimeOut=1)

public void testMethod() throws InterruptedException {

Thread.sleep(100);

System.out.println("Print....");

}

===============================================

    Default test

    Tests run: 2, Failures: 1, Skips: 0

===============================================

===============================================

Default suite

Total tests run: 1, Passes: 0, Failures: 1, Skips: 0

===============================================

 It seems bug to me. Could you please check Thanks

Pratik

[Krishnan Mahadevan]

Pratik,

Yes, what you see as a behavior by TestNG is perhaps a bit counter intuitive and maybe even consuming. But TestNG here is working as designed.

 

Let me explain my understanding of the current behavior that TestNG is exhibiting.

 

@Test(invocationCount=5)
public void testMethod() throws InterruptedException {
  Thread.sleep(100);
  System.out.println("Print....");
}

 

In the above sample, we are telling TestNG that we want it to invoke testMethod “5” times. That explains the below result

 

Print....

Print....

Print....

Print....

Print....

 

===============================================

Default Suite

Total tests run: 5, Passes: 5, Failures: 0, Skips: 0

===============================================

 

But when you have a sample that looks like below:

 

@Test(invocationCount = 5, invocationTimeOut = 1000)

public void testMethod() throws InterruptedException {
  Thread.sleep(100);
  System.out.println("Print....");
}

 

Here we are telling TestNG that we want it to run “testMethod” to run 5 times but the overall timeout across all the invocations would be 1000 milli seconds. That is why the 5 iterations gets wrapped into just 1 invocation and reported as below:

 

Print....

Print....

Print....

Print....

Print....

 

===============================================

Default Suite

Total tests run: 1, Passes: 1, Failures: 0, Skips: 0

===============================================

 

Quoting the javadocs as well,

 

The maximum number of milliseconds that the total number of invocations on this test method should take. This annotation will be ignored if the attribute invocationCount is not specified on this method. If it hasn't returned after this time, it will be marked as a FAIL.

 

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.com/

--