TestNG fails when generating reports

[Tai Thach]

Hi,

I'm using Ant to invoke TestNG on several different test suites (each with it's own testng.xml file). The problem I'm running into is intermittent. It doesn't occur all the time, but it occurs frequently enough to be a real pain. Basically all the test suites are run and all the tests are executed, but it looks like when TestNG is generating the reports for one of the suites, some exception occurs and then it just prints that the tests have failed: 

   [testng] ===============================================
   [testng] Suite testng_parallel
   [testng] Total tests run: 1112, Failures: 1, Skips: 0
   [testng] ===============================================
   [testng] 
   [testng] 2016-10-13 14:00:39,530 [main] INFO cv.test.testng.BasicExecutionListener - END EXECUTING TESTNG
   [testng] [Utils] Attempting to create C:\cv\1010\logs\testng-failed.xml
   [testng] [Utils]   Directory C:\cv\1010\logs exists: true
   [testng] Creating C:\cv\1010\logs\testng-failed.xml
   [testng] [Utils] Attempting to create C:\cv\1010\logs\Suite testng_parallel\testng-failed.xml
   [testng] [Utils]   Directory C:\cv\1010\logs\Suite testng_parallel exists: true
   [testng] Creating C:\cv\1010\logs\Suite testng_parallel\testng-failed.xml
   [testng] [TestNG] Time taken by [FailedReporter passed=0 failed=0 skipped=0]: 102 ms
   [testng] [TestNG] Time taken by org.testng.reporters.EmailableReporter2@20deea7f: 147 ms
   [testng] The tests failed.

When this happens, the reports don't get generated properly so the # of tests executed is not properly recorded. I've spent a few days trying to debug this, but I have not made much progress. I know that when a suite is successful, multiple reporters are run, but for one of the suites, it fails while running the reporters (in red):

$ grep -n -E "Suite testng_\w*$|org.testng.reporters" /cygdrive/c/cv/1010/build/20161013_2.out
27277:   [testng] Suite testng_first
86881:   [testng] Suite testng_parallel
86896:   [testng] [TestNG] Time taken by org.testng.reporters.EmailableReporter2@20deea7f: 147 ms
129505:   [testng] Suite testng_1
129510:   [testng] [TestNG] Time taken by org.testng.reporters.jq.Main@68999068: 1246 ms
129533:   [testng] [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@2c767a52: 523 ms
129534:   [testng] [TestNG] Time taken by org.testng.reporters.XMLReporter@1ea9f6af: 203 ms
129617:   [testng] [TestNG] Time taken by org.testng.reporters.JUnitReportReporter@393671df: 618 ms
129618:   [testng] [TestNG] Time taken by org.testng.reporters.EmailableReporter2@1dde4cb2: 42 ms
211188:   [testng] Suite testng_2
211524:   [testng] [TestNG] Time taken by org.testng.reporters.jq.Main@68999068: 3159 ms
211549:   [testng] [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@2c767a52: 379 ms
211616:   [testng] [TestNG] Time taken by org.testng.reporters.XMLReporter@1ea9f6af: 225 ms
211995:   [testng] [TestNG] Time taken by org.testng.reporters.JUnitReportReporter@393671df: 1008 ms
212062:   [testng] [TestNG] Time taken by org.testng.reporters.EmailableReporter2@1dde4cb2: 209 ms
321193:   [testng] Suite testng_3
321198:   [testng] [TestNG] Time taken by org.testng.reporters.jq.Main@68999068: 3330 ms
321223:   [testng] [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@2c767a52: 474 ms
321224:   [testng] [TestNG] Time taken by org.testng.reporters.XMLReporter@1ea9f6af: 355 ms
321675:   [testng] [TestNG] Time taken by org.testng.reporters.JUnitReportReporter@393671df: 723 ms
321676:   [testng] [TestNG] Time taken by org.testng.reporters.EmailableReporter2@1dde4cb2: 102 ms

I've tried increasing the "verbose" attribute to 10 for the suite that fails, but it doesn't seem to print any additional errors. Initially I thought it was  a memory issue, but increasing the memory for the TestNG task did not fix the issue. 

I'm running ant 1.71. and testng-6.9.12.jar.

Any suggestions / ideas on how to fix this?

Thanks,

Tai

[Mrunal Gosar]

Are you running your test suites in parallel. Can you try to run them in single thread and see if it still fails, and also specify different output directory for each of the test suites.

[Tai Thach]

Hi Mrunal,

The problem I'm running into is not that the tests are failing, it's that TestNG fails when trying to run the reporters after all the tests are executed. When the reporters fail, the number of unit tests it reports is not accurate (ie instead of reporter 5000 unit tests, it reports only 4000).

Tai

[Charu]

Hi,

I am facing this same issue with TestNG 6.9.8

Please suggest any possible resolution. 

Thanks and Regards

Charu

[⇜Krishnan Mahadevan⇝]

You are working with a very old version of TestNG. Please try again using the latest released version of TestNG viz., 6.14.3 and post back the results with a sample (if the problem exists) so that we can use the sample to reproduce the issue. 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To post to this group, send email to testng...@googlegroups.com.
Visit this group at https://groups.google.com/group/testng-users.
For more options, visit https://groups.google.com/d/optout.

--

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/