Wrong number of test executions when a TestNG suite xml calls other suite xmls

[ngogovski]

Hello TestNG guys,

We have hit a problem with latest TestNG Eclipse plugin when calling a
testNG suite xml from another suite xml.

Here is how to reproduce the problem:
1. Create a simple java project with 1 class <PROJECT ROOT>/src/com/
Test1Class.java which has just 1 test method.
2. Create 2 suite xml files placed in <PROJECT ROOT>/suites:
2.1. the first xml is called suite1.xml and has the following content:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite1" verbose="1">
<test name="Suite1 tests">
<classes>
<class name="com.Test1Class" />
</classes>
</test>
</suite>

2.2. the second xml is called complex_suite.xml and has the following
content:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Complex suite" verbose="1">
<suite-files>
<suite-file path="suites/suite1.xml" />
</suite-files>
</suite>

Now run the complex_suite.xml and you will get the following output in
the console:

[TestNG] Running:
D:\Eclipse_workspace\TestTestNG\suites\suite1.xml

running testMethod1()

===============================================
Suite1
Total tests run: 1, Failures: 0, Skips: 0
===============================================

[TestNG] Running:
D:\Eclipse_workspace\TestTestNG\suites\suite1.xml

running testMethod1()

===============================================
Suite1
Total tests run: 2, Failures: 0, Skips: 0
===============================================

[TestNG] Running:
D:\Eclipse_workspace\TestTestNG\suites\complex_suite.xml


===============================================
Complex suite
Total tests run: 4, Failures: 0, Skips: 0
===============================================

As you can see the test method is called twice, while expected only
one execution.


It is interestig to note that changing the complex_suite.xml to:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Complex suite" verbose="1">
<suite-files>
<suite-file path="suite1.xml" />
</suite-files>
</suite>

produces the following output in the console:

java.io.FileNotFoundException: D:\Eclipse_workspace\TestTestNG
\suite1.xml (The system cannot find the file specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at java.io.FileInputStream.<init>(FileInputStream.java:66)
at org.testng.xml.Parser.parse(Parser.java:165)
at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:
271)
at org.testng.TestNG.run(TestNG.java:980)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:
202)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173)
[TestNG] Running:
D:\Eclipse_workspace\TestTestNG\suites\suite1.xml

running testMethod1()

===============================================
Suite1
Total tests run: 1, Failures: 0, Skips: 0
===============================================

[TestNG] Running:
D:\Eclipse_workspace\TestTestNG\suites\complex_suite.xml


===============================================
Complex suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

So it complains it cannot find the internal suite1.xml, but indeed
runs the test method just once as expected.

I've tried calling many internal suites from the complex one and the
issue is the same: all tests are run once and then are run once again
in the same order.

thank you for your help :)
Nikolay

[ssimmons]

I've seen this issue too, and it's not limited to the TestNG Eclipse
plugin. I'm using the maven surefire plugin to fire of my parent
suite, and when I build the project I get the same issue. I'm guessing
the problem is in TestNG itself, not any related plugin.

[Cédric Beust ♔]

Yes, it looks like a TestNG issue, I'll look into it.

-- 

Cédric

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Jostar]

Hi,

Any progress about that?
It's really disappointing not beeing able to run nested suites properly any
more (i don't remember having that problem in the past).
Using testng-6.5.1

greets,
Arno

--
View this message in context: http://old.nabble.com/Wrong-number-of-test-executions-when-a-TestNG-suite-xml-calls-other-suite-xmls-tp33407862p33982619.html
Sent from the testng-users mailing list archive at Nabble.com.

[Cédric Beust ♔]

Sorry, no, I haven't had the time to look into it.

-- 

Cédric

[justme]

Is there a previous version that can be reverted to where this is not an issue?

-- 

Cédric

>> testng-users+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/testng-users?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "testng-users" group.
> To post to this group, send email to testng...@googlegroups.com.
> To unsubscribe from this group, send email to

> testng-users+unsubscribe@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/testng-users?hl=en.
>
>
>

--
View this message in context: http://old.nabble.com/Wrong-number-of-test-executions-when-a-TestNG-suite-xml-calls-other-suite-xmls-tp33407862p33982619.html
Sent from the testng-users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.

To unsubscribe from this group, send email to testng-users+unsubscribe@googlegroups.com.

[Cédric Beust ♔]

All the previous versions have their own update site at http://beust.com/eclipse-old

-- 

Cédric

To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/t-c1beQ87XsJ.

To post to this group, send email to testng...@googlegroups.com.

To unsubscribe from this group, send email to testng-users...@googlegroups.com.

[Jostar]

Can you maybe describe in a few lines how the "collecting" of classes/methods
to execute is working, so i'll have a look at it myself. May i could help to
"faire avancer le schmilblick" ;-)
That issue is really bothering me!!!

Greets & Thx in advance,
Arno

Sorry, no, I haven't had the time to look into it.

--
Cédric




On Fri, Jun 8, 2012 at 9:23 AM, Jostar <arno...@hispeed.ch> wrote:

>
> Hi,
>
> Any progress about that?
> It's really disappointing not beeing able to run nested suites properly
> any
> more (i don't remember having that problem in the past).
> Using testng-6.5.1
>
> greets,
> Arno
>
>
>

--
View this message in context: http://old.nabble.com/Wrong-number-of-test-executions-when-a-TestNG-suite-xml-calls-other-suite-xmls-tp33407862p34030730.html

[Cédric Beust ♔]

Hi,

I actually fixed the duplicate bug in the Eclipse plug-in over the weekend, can you please update your plug-in and confirm it's working for you?

Thanks.

-- 

Cédric

[Jostar]

I did and yes it does.
Thx a lot!

View this message in context: http://old.nabble.com/Wrong-number-of-test-executions-when-a-TestNG-suite-xml-calls-other-suite-xmls-tp33407862p34066279.html