Accessing actual test instance in ITestListener

[Vladimir]

Hi,

How would I access actual test class instance after all test methods
in that class have been run? I want to write a listener that would,
after all test methods have been run, traverse test instance fields
all the way to Object and nullify all instance fields.

Regards,
Vladimir

[Cédric Beust ♔]

Why not do this in an @After method?

--

Cédric

[Vladimir]

On Oct 9, 12:15 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
>
>
> Why not do this in an @After method?

That is another option, create a new super class for all existing test
classes and implement @AfterClass method. But then we have to ensure
all existing test subclasses which declare @AfterClass method invoke
superclass @AfterClass method. My understanding is TestNG does not
traverse class hierarchy to invoke all @After methods in a hierarchy,
right?

Fast forward to now and that is how I thought doing this through a
listener would be a great idea - until I got stuck looking through API
unable to figure out how to proceed.

BTW we want to null all instance fields since TestNG keeps a reference
to a test instance until entire test suite has been finished. The we
discovered we were leaking memory :)

[Cédric Beust ♔]

Hi Vladimir,

On Fri, Oct 9, 2009 at 9:30 AM, Vladimir <dov...@gmail.com> wrote:

On Oct 9, 12:15 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
>
>
> Why not do this in an @After method?

That is another option, create a new super class for all existing test
classes and implement @AfterClass method. But then we have to ensure
all existing test subclasses which declare @AfterClass method invoke
superclass @AfterClass method. My understanding is TestNG does not
traverse class hierarchy to invoke all @After methods in a hierarchy,
right?

No, TestNG should definitely traverse the entire hierarchy.  Please let me know if this is not the case.

 

Fast forward to now and that is how I thought doing this through a
listener would be a great idea - until I got stuck looking through API
unable to figure out how to proceed.

BTW we want to null all instance fields since TestNG keeps a reference
to a test instance until entire test suite has been finished. The we
discovered we were leaking memory :)

Mmmh, I'm not quite sure this is going to help.  If A has a reference to B which has a reference to C, setting C to null is not going to drastically change your memory usage, unless I'm missing something specific to your application...

--

Cédric

[Vladimir]

On Oct 9, 12:42 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
> Hi Vladimir,
>
> On Fri, Oct 9, 2009 at 9:30 AM, Vladimir <dov...@gmail.com> wrote:
>
> > On Oct 9, 12:15 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
>
> > > Why not do this in an @After method?
>
> > That is another option, create a new super class for all existing test
> > classes and implement @AfterClass method. But then we have to ensure
> > all existing test subclasses which declare @AfterClass method invoke
> > superclass @AfterClass method. My understanding is TestNG does not
> > traverse class hierarchy to invoke all @After methods in a hierarchy,
> > right?
>
> No, TestNG should definitely traverse the entire hierarchy.  Please let me
> know if this is not the case.

Not the case, at least when I run a test class from Eclipse (5.9.0.5).

> Mmmh, I'm not quite sure this is going to help.  If A has a reference to B
> which has a reference to C, setting C to null is not going to drastically
> change your memory usage, unless I'm missing something specific to your
> application...
>

We have hundreds of test classes and very often we set rather
heavyweight objects to instance fields of those test classes. Over
time these accumulate and are unnecessarily around. We started to see
lots of OOMs. Booted up a profiler and discovered where the culprit
was. After nulling instance fields test suite started to fly with no
OOMs....

[Cédric Beust ♔]

On Fri, Oct 9, 2009 at 9:59 AM, Vladimir <dov...@gmail.com> wrote:

On Oct 9, 12:42 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
> Hi Vladimir,
>

> On Fri, Oct 9, 2009 at 9:30 AM, Vladimir <dov...@gmail.com> wrote:
>
> > On Oct 9, 12:15 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
>
> > > Why not do this in an @After method?
>
> > That is another option, create a new super class for all existing test
> > classes and implement @AfterClass method. But then we have to ensure
> > all existing test subclasses which declare @AfterClass method invoke
> > superclass @AfterClass method. My understanding is TestNG does not
> > traverse class hierarchy to invoke all @After methods in a hierarchy,
> > right?
>
> No, TestNG should definitely traverse the entire hierarchy.  Please let me
> know if this is not the case.

Not the case, at least when I run a test class from Eclipse (5.9.0.5).

Can you post your code showing that parent's @After method are not being called?

 

> Mmmh, I'm not quite sure this is going to help.  If A has a reference to B
> which has a reference to C, setting C to null is not going to drastically
> change your memory usage, unless I'm missing something specific to your
> application...
>

We have hundreds of test classes and very often we set rather
heavyweight objects to instance fields of those test classes. Over
time these accumulate and are unnecessarily around. We started to see
lots of OOMs. Booted up a profiler and discovered where the culprit
was. After nulling instance fields test suite started to fly with no
OOMs....

Ah, I see, it's the fields that you are setting to null that are big objects.

For what it's worth, you should be able to create a generic "nullifyInstanceFields()" that does this through reflection and that will work for any class.

--

Cédric

[Vladimir]

On Oct 9, 1:08 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
>
> Can you post your code showing that parent's @After method are not being
> called?

import org.testng.annotations.AfterClass;
import org.testng.annotations.Test;

@Test
public class A {

public void test() {}

@AfterClass
public void destroy() {
System.out.println("Invoked A-->destroy");
}
}


import org.testng.annotations.AfterClass;
import org.testng.annotations.Test;

@Test
public class B extends A{

public void test2() {}

@AfterClass
public void destroy() {
System.out.println("Invoked B-->destroy");
}
}

[Parser] Running:
/Users/vladimir/workspace/infinispan/core/temp-testng-
customsuite.xml

Invoked B-->destroy
PASSED: test
PASSED: test2

===============================================
org.infinispan.api.B
Tests run: 2, Failures: 0, Skips: 0
===============================================


===============================================
infinispan-core
Total tests run: 2, Failures: 0, Skips: 0
===============================================

> Ah, I see, it's the fields that you are setting to null that are big
> objects.

Yes :)

>
> For what it's worth, you should be able to create a generic
> "nullifyInstanceFields()" that does this through reflection and that will
> work for any class.

Yeah that is the idea! Thanks Cedric

[Vladimir]

Oh, I can not override a method :)

[Cédric Beust ♔]

On Fri, Oct 9, 2009 at 10:26 AM, Vladimir <dov...@gmail.com> wrote:

Oh, I can not override a method :)

Yes, I was going to point that out...

Use a different name and you should be okay.

--

Cédric

[ajay mehra]

Hi All,
can u one point out how can we run a testng test parallely in remote machines with out giving parameters in xml because xml use is good for small number of testcases but i have a test case which i have to run on 20 machines.and i am using testng+selenium+eclipse environment and not able to run my test parallely.itz running sequentially and atacking only one host machine.
i created 20 instances using @factory annotation to run on 20 machines but itz creating 20 instances in my machine thats bz itz not able to catch other host machine ip's.may be i am missing something.would really appreciate if u can provide sm solution to this.

 

here is my test:
import com.thoughtworks.selenium.*;
import org.testng.annotations.*;

 

public class Xampperf02
{
 private Selenium selenium;

 

 @BeforeClass
 @Parameters({"selenium.host","selenium.port","selenium.browser","selenium.url",})
 public void startSelenium(String host, String port, String browser, String url) {
 
 this.selenium = new DefaultSelenium(host, Integer.parseInt(port), browser, url);
 this.selenium.start();
 this.selenium.open(url);
 
 
 }
 
 
 @AfterClass(alwaysRun=true)
 public void stopSelenium() {
 this.selenium.stop();
 }
 @Test(invocationCount=2)
 public void testXampperf02() throws Exception {
  selenium.open("http://202.53.87.166/xampperf/index.html");
  selenium.click("link=Login");
  selenium.waitForPageToLoad("30000");
  String loginID = selenium.getExpression("login");
  selenium.type("login", selenium.getEval("'" + loginID + "'+Math.round((3790-3741) * Math.random() + 3741)"));
  selenium.type("password", "password");
  selenium.click("Submit");
  selenium..waitForPageToLoad("30000");
  selenium.click("link=Total Members list");
  selenium.waitForPageToLoad("30000");
  selenium.click("link=Logout");
  selenium.waitForPageToLoad("30000");
  selenium.click("link=Login");
  selenium.waitForPageToLoad("30000");
 }
}

 

My factory classs is as follows:(here i am creating 20 instances of my test class Xampperf02()

 

import org.testng.annotations.Factory;
import org.testng..annotations.Test;

 

@Test
public class Factory1
{
  @Factory
  public Object[] createInstances()
  {
    return new Object[] {new Xampperf02(),
                        new Xampperf02(),
                        new Xampperf02(),
                                 .  
                                 .
                                 .
                         new Xampperf02(),
                          };

 

 

 

 

 

  }
}

 

and here goes my xml file:

 

<suite name="GoogleSuit"  verbose="3">

 

<test name="Ajay">
<parameter name="selenium.host" value="172.16.2.78"></parameter>
<parameter name="selenium.host" value="172.16.2.123">
<parameter name="selenium.host" value="172.16.2.76">
<parameter name="selenium.host" value="172.16.2.91">
                      ............(20 ips of differnt host)
<parameter name="selenium.port" value="5533"></parameter>
<parameter name="selenium.url" value="http://202.53.87.166/xampperf/index.html"></parameter>
<parameter name="selenium.browser" value="*firefox"></parameter>

 

 

 

<classes>
<class name="Factory1">

 

</class>
</classes>

 

</test>

 

</suite>.

 

In the xml file,i am passing 20 ip values but my test is detecting only one ip and creating all 20 instances in that single host machine leaving remaining 19 ip's in idle.

 

 

2009/10/9 Cédric Beust ♔ <cbe...@google.com>

[Carsten]

Hi all,

I really like the idea of accessing actual test instances using
listeners (testng keeps the references anyway).

I know that @AfterXXX should be used for cleanup, but the problem is
that many developers regularly forget to release their test-
resources.

And if there are memory issues there are only a few options to fix
this (please correct me if i forgot other solutions):

- cleanup the actual test-classes in @AfterXXX: this would mean to
profile the unittests or to manually go over the test-classes and
search for "bad" tests
- cleanup the test-classes in @AfterXXX from a base-class: forces all
test classes to extend from a common base class
- split up the test-suites: should be done because it logically makes
sense, not because of memory issues
- increase the available memory

I think it would help if a listener can register to the @AfterXXX-
methods and that the actual test-instance would be somehow accessible.
This would allow a cleanup of at least some of the allocated resources
without modifying the actual test classes.

Carsten

> ***Cédric
> *

[Cédric Beust ♔]

On Tue, Oct 13, 2009 at 9:41 AM, Carsten <carsten.g...@googlemail.com> wrote:

I think it would help if a listener can register to the @AfterXXX-
methods and that the actual test-instance would be somehow accessible.
This would allow a cleanup of at least some of the allocated resources
without modifying the actual test classes.

Do you mean listeners that would be invoked after @After methods and that would have access to the test instances?

--

Cédric

[Carsten]

Yes, this would offer a better tracking of memory issues.

A listener can check for unreleased resources in e.g. @AfterClass:
check that all members of this particular class are set to null

Also interesting for @AfterSuite: Iterate over all test instances and
report members that are not released (which is most probably always a
test error).

Carsten

On Oct 13, 6:45 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
> On Tue, Oct 13, 2009 at 9:41 AM, Carsten

> <carsten.guberna...@googlemail.com>wrote:

>
> > I think it would help if a listener can register to the @AfterXXX-
> > methods and that the actual test-instance would be somehow accessible.
> > This would allow a cleanup of at least some of the allocated resources
> > without modifying the actual test classes.
>
> Do you mean listeners that would be invoked after @After methods and that
> would have access to the test instances?
>
> --

> ***Cédric
> *