How to skip test methods inherited from extended abstract test classes.

[Krishantha Samaraweera]

Hi all,

I'm trying to skip tests based on custom annotation  implemented using IAnnotationTransformer. Skipping tests were successful when set custom annotation in class level, however test methods inherited from extended base class didn't get ignored. If i set the same custom annotation in abstract class, then everything works as expected, but I need to use my abstract class in other places and cannot tag it with custom annotation.

What is the best way to ignore test methods from abstract class. Any help would be greatly appreciated.

public abstract class AbstractTestCase {

    @Test()
    public void testMethod1(){
        System.out.println("selenium abstarct1");
    }

    @Test()
    public void testMethod2(){
        System.out.println("selenium abstarct2");
    }

}


@SetEnvironment(executionEnvironments = {ExecutionEnvironment.STANDALONE})
public class TestSelenium extends AbstractTestCase{

    @Test()
    public void runSelenium1() {
        System.out.println("runSelenium()");
    }

    @Test()
    public void runSelenium2() {
        System.out.println("runSelenium()");
    }

}

Thanks,
Krishantha.

[Krishnan Mahadevan]

Can you also show us how does your Annotation Transformer look like ?

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at http://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[Felik Junvianto]

Hi all,

Sorry for bringing up the old thread. I also recently encountered the same problem, but on different situation. So my peer put all of the test method (annotated with @Test annotation) on the abstract class, and chose to extend the abstract class with concrete method for preparing some stuffs (e.g.: using mock or connection to test db, etc). Some illustrations:

public abstract class BaseTest {

    protected abstract Data getData();

    @Test

    public void test1() {

        // test here

    }

    @Test

    public void test2() {

       // test here

    }

}

@Test

public class ConcreteTest1 extends BaseTest {

    @Override

    protected Data getData() {

    } 

}

[⇜Krishnan Mahadevan⇝]

Felik,

Can you please help mention what is the issue you are facing ? I didn't understand that part from your post.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.

Visit this group at https://groups.google.com/group/testng-users.

[Felik Junvianto]

Hi Krishnan,

Sorry for the inconvenience, I think my post got cut by connection problem. Below is the complete illustration:

public abstract class BaseTest {
   protected abstract Data getData();

   @Test
   public void test1() {

      /* test here using getData() */
    }


  @Test
   public void test2() {
      /* test here using getData() */

    }
}

@Test
public class ConcreteTest1 extends BaseTest {
    @Override
    protected Data getData() {

      /* implementation of getData() */
    }
}

@Test
public class ConcreteTest2 extends BaseTest {
   @Override
   protected Data getData() {
      /* implementation of getData() */    
   }
}

By running using gradle + testng configuration, the test will be run like this (some rough illustration again, assume the result of each respective test is represented by the entailing comment):

- ConcreteTest1::test1() // success

- ConcreteTest1::test2() // success

- ConcreteTest2::test1() // failed

- ConcreteTest2::test2() // failed

Now, for reducing the not-helping-failure-count on monitoring dashboard (another "signal vs. noise" philosophy), I need to temporarily skip test from ConcreteTest2 (short term countermeasure until the owner have a time to fix the test).  What is the best and concise way to achieve this? I have tried registered excluded group and tag ConcreteTest2 under that group but to no avail.

@Test(groups = "ignored") // not working
public class ConcreteTest2 extends BaseTest {
   @Override
   protected Data getData() {
      /* implementation of getData() */    
   }
}

Below are the snippet of the gradle configuration for testng:

...

test {
  ...
  useTestNG() {
    suites "../../config/testNG.xml"
    excludeGroups 'broken', 'manual', 'slow', 'ignored'
  }
  ...
}

...

and here are the content of testNG.xml:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="TestNG Suite" time-out="2000">
  <test name="All Tests">
    <packages>
      <package name=".*">
        ... // unrelated XMl configuration
      </package>
    </packages>
  </test>
</suite>

Many thanks for suggestion.

Regards,

Felik Junvianto

[⇜Krishnan Mahadevan⇝]

Felik,

I was able to do this using a beanshell expression as a method selector in my suite xml file.

The below resources should add more context to beanshells :

• TestNG documentation : http://testng.org/doc/documentation-main.html#beanshell
• My blog post : https://rationaleemotions.wordpress.com/2013/08/21/beanshell-and-testng/

Here's how my classes look like :

package organized.chaos.forums;

import org.testng.annotations.Test;

public abstract class SimpleBaseTest {
    protected abstract String getData();

    @Test
    public void test1() {
        System.err.println(getClass().getName() + ":: + test1() with getData() ---> " + getData());
    }


    @Test
    public void test2() {
        System.err.println(getClass().getName() + ":: + test2() with getData() ---> " + getData());
    }

    @Test
    public static class ConcreteTest1 extends SimpleBaseTest {
        @Override
        protected String getData() {
            return "Hello World";
        }
    }


    @Test
    public static class ConcreteTest2 extends SimpleBaseTest {
        @Override
        protected String getData() {
            return "Goodbye World";
        }
    }
}

Here's how my suite xml looks like :

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">

<suite name="Class1Suite" parallel="false" verbose="2">
    <test name="Class1Test">
        <method-selectors>
            <method-selector>
                <script language="beanshell">
                    <![CDATA[whatClass = System.getProperty("class", "all");
                    return testngMethod.getInstance().getClass().getName().endsWith(whatClass) == false;
                ]]>
                </script>
            </method-selector>
        </method-selectors>
        <classes>
            <class name="organized.chaos.forums.SimpleBaseTest$ConcreteTest1"/>
            <class name="organized.chaos.forums.SimpleBaseTest$ConcreteTest2"/>
        </classes>
    </test> 
</suite> 

Here's my execution log from the terminal  (Here I am trying to instruct TestNG to skip tests from ConcreteTest2 class):

➭ mvn clean test -Dclass=ConcreteTest2

[INFO] Scanning for projects...

[INFO]

[INFO] ------------------------------------------------------------------------

[INFO] Building testbed 1.0-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

 T E S T S

-------------------------------------------------------

Running TestSuite

[XmlSuite] [WARN] 'parallel' value 'false' is deprecated, default value will be used instead: 'none'.

...

... TestNG 6.9.11 by Cédric Beust (ced...@beust.com)

...

organized.chaos.forums.SimpleBaseTest$ConcreteTest1:: + test1() with getData() ---> Hello World

organized.chaos.forums.SimpleBaseTest$ConcreteTest1:: + test2() with getData() ---> Hello World

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.378 sec - in TestSuite

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 5.141 s

[INFO] Finished at: 2016-07-19T09:52:26+05:30

[INFO] Final Memory: 24M/289M

[INFO] ------------------------------------------------------------------------

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

[Felik Junvianto]

Hi Krishnan,

Many thanks for the solution suggestion. I will try to implement this workaround.

Regards,

Felik Junvianto.