How to add tests programmatically to current suite xml

[Tatery]

Hello,

I'd like to execute tests using testng xml and maven but in addition to that some tests have to be run programmatically. It means that some tests from xml suite have to run other tests programmatically and afterwards I'd like to have only one test report. 

Do you know if something like this is possible?

[Tatery]

@Cedric is there a way to modify testNg context at runtime? This would be really nice feature :)

[Cédric Beust ♔]

It might be but it's not supported, so the behavior is undefined.

Adding tests dynamically is typically performed by factories or data providers, wouldn't that work for you?

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

[Tatery]

Many thanks for the prompt reply.

I very often use factories and data providers but this time both of them do not cover all my requirements. I'm looking for a way to execute existing tests programmatically as part of bigger test suite. Since context modifying at runtime is not possible what do you thing about other approach: read xml suite in @BeforeSuite method then add new tests programmatically and execute tests based on this new in memory suite:

@BeforeSuite

public void initTestProg(ITestContext context) throws InterruptedException {

XmlSuite xmlSuite = context.getCurrentXmlTest().getSuite();

// Create an instance of XmlTest and assign a name for it.

XmlTest xmlTest = new XmlTest(xmlSuite);

xmlTest.setName("Sample Test");

// Create a list which can contain the classes that you want to run.

List<XmlClass> myClasses = new ArrayList<XmlClass>();

myClasses.add(new XmlClass("testing.testngtst.Basic1"));

myClasses.add(new XmlClass("testing.testngtst.Basic2"));

// Assign that to the XmlTest Object created earlier.

xmlTest.setXmlClasses(myClasses);

//....(more code here)

testNG.setXmlSuites(mySuites);

testNG.run();

}

But how to break execution of the original suite at the end of @BeforeSuite method?

Do you have any thoughts how to implement context modifying at runtime? I don't know testNg implementation in details but if you give me some hints I could try to implement this feature.

W dniu poniedziałek, 8 maja 2017 23:15:35 UTC+2 użytkownik Cédric Beust ♔ napisał:

It might be but it's not supported, so the behavior is undefined.

Adding tests dynamically is typically performed by factories or data providers, wouldn't that work for you?

-- 

Cédric

On Mon, May 8, 2017 at 2:12 PM, Tatery <tatery...@gmail.com> wrote:

@Cedric is there a way to modify testNg context at runtime? This would be really nice feature :)

W dniu poniedziałek, 8 maja 2017 10:37:14 UTC+2 użytkownik Tatery napisał:

Hello,

I'd like to execute tests using testng xml and maven but in addition to that some tests have to be run programmatically. It means that some tests from xml suite have to run other tests programmatically and afterwards I'd like to have only one test report. 

Do you know if something like this is possible?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.

[Cédric Beust ♔]

With this code, you would run a TestNG inside a TestNG. It would work but the reports at the end would probably be confusing.

Maybe you could take over the whole process and in your own main(), create an XmlSuite that is the sum of testng.xml and your own programmatically created tests, and then invoke testng.run().

​

-- 

Cédric

To unsubscribe from this group and stop receiving emails from it, send an email to testng-users+unsubscribe@googlegroups.com.

[⇜Krishnan Mahadevan⇝]

@Tatery

I think you can try upgrading to the latest version of TestNG and then try leveraging the listener IAlterSuiteListener. This interface let's you alter the suite contents programmatically via listeners.

Please see this thread for more info.

https://groups.google.com/forum/m/#!topic/testng-users/u4HwjBEUzhY

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

[Tatery]

@Krishnan from my point of view there is no significant difference in running tests programmatically with "old way" and with IAlterSuiteListener. I use ReportNg and after tests execution methods from classes handled with IAlterSuiteListener are not available on report.

[⇜Krishnan Mahadevan⇝]

If ReportNG is generating reports via an implementation of IReporter interface implementation then this should not happen. If it's doing via some other listener implementation mechanism then yes it could very well be possible.

[Tatery]

For sure, ReportNG use IReporter. 

In this example: https://github.com/cbeust/testng/blob/master/src/test/java/test/listeners/AlterSuiteListenerTest.java, class AlterSuiteListener1SampleTest is implemented inside "main" class AlterSuiteListenerTest1, in this case test methods from AlterSuiteListener1SampleTest are available in report. However, when you move AlterSuiteListener1SampleTest to separate file then methods are not attached to report. Any idea what is wrong?

[Ashish khanna]

Hello,

I also want to execute my TestNg Testsuite  programmatically but It shows configuration failure when I try to run it. The reason which I got is that it is not able to run Before and After Annotation.

Can anyone help me with it.

List<XmlSuite> suites = new ArrayList<XmlSuite>();

suites.add(suite);

TestNG tng = new TestNG();

tng.setXmlSuites(suites);

tng.run();

[travis.n...@gmail.com]

I have similar object when running test with maven and surefire plugin. My intention is to get suite definition from another source rather than the suite file that I define on surefire configration:

Here is the procedure I use but fail to do so:

Configuration and procedure:

1. Maven build goals: clean test

2. Surefire configuration

 <configuration>
 <suiteXmlFiles>
 <suiteXMLFile>testng.xml</suiteXMLFile>
 </suiteXmlFiles>

3. On testng.xml I only defines one test, in that test I have one dummy test class which will be override later with setTest() of ISuite.

<suite name="HourlyTest">
<test name="OnDemandTest">

<classes>

  <class name="com.auto.DummyTest" />

</classes>
</test>
</suite>

4. I over onStart() method of ISuiteListener and do following actions:

• Get current suite
• Instant a new XmlTest
• Instant all XmlClass which I want to run from class names
• setClasses() for list XmlClass to the XmlTest
• setTest() for the test we just instantiated above

At the end of this procedures, even the test has new configuration with new list of test classes but I found that only dummy test class is executed.

It look like that even I change the test instance of current suite, testng still ignore my change at runtime. Can testng support this way of suite configuration at runtime? Or I have to move away from surefire and do this with a void main on src/main?

Thanks and Regards,

Truong Nguyen

[Krishnan Mahadevan]

Please upgrade to the latest released version of TestNG viz., 6.14.3 and use org.testng.IAlterSuiteListener for your usecase.

 

Thanks & Regards

Krishnan Mahadevan

 

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

 

--