TEST-AllTests.xml and junitreport

[sjlib]

I can sure use some help regarding generating the html junitreport
from the output of the fluint ant task. The issue seems to be that the
fluint ant task generates a file, TEST-AllTests.xml. From what I can
tell, this would be the exact same file that junitreport would produce
after doing its merge given a fileset.

However, I wind up with an empty TESTS-TestSuites.xml and hence an
empty html report.

generate-fluint-report:
[junitreport] the file C:\projects\test-output\fluint\TEST-
AllTests.xml is not a valid testsuite XML document
[junitreport] Processing C:\projects\test-output\fluint\report\TESTS-
TestSuites.xml to C:\projects\test-output\fluint\report\junit-
noframes.html

So the question is , how are people doing this ?

The release blog for 1.1.0 says :
XML compliant output with most CI servers as well as the JUnitReport
task in Ant and the Surefire Reporting plugin in Maven.

I just cant figure out how to provide the output to junitreport. The
only mechanism I see is junitreports fileset and what that gives me is
the WARNING mentioned above and an empty html report.

Surely, someone is doing this already successfully ?

Thanks, Steve

[Brian]

@sjlib - Looks like from the Ant documentation, that TEST-
TestSuites.xml is the default filename that is used to merge all of
the tests results. That being said, if you're getting an error from
the junitreport task then that means something is going on with the
output produced by the AIR Test Runner. I have a feeling it may not
like the top level tags we generate even though Maven's Surefire
plugin may be ok with them. If there is a way to just run an xsl on
the XML generated by fluint that may be a stopgap solution, but I'm
unsure if this is possible.

If you could file an issue as a defect on our google code site with
sample output, it would help out a lot. 1.1.1 is sealed but we can
definitely get this fix into 1.2. Once you file the bug I'll try to
dig in and figure out what's up. Thanks.

-Brian

[sjlib]

Well, the issue is and isn't with what the Air Test Runner outputs,
meaning,
the file itself is what junitreport expects, however, the junitreport
ant task, by design does a merge on xml files, whose root node is
testsuite, not testsuites. That is, it expects individual files and
produces the same exact file as Air Test Runner outputs.
The issue is there's no way to feed the Air Test Runner output
directly to junitreport (as far as I can see ...)

Now, there are a couple of things. Maybe no one actually is doing what
I tried to do or I haven't figured out the way to do it. Either is
possible. I'm guessing the problem may be circumvented if using CI
tools like Cruise Control. I'm not sure.

However, I was able to achieve what I was after by altering the source
of TestRunnerWindow.as in airtestrunner. Instead of writing the single
file, I loop over the testsuite XML elements and write those to
individual files.

//TestRunnerUtils.writeToFile(results, dir, this.REPORT_FILE_NAME);
for each(var testsuite:XML in results.testsuite)
{
TestRunnerUtils.writeToFile(testsuite, dir, testsuite.@name + ".xml");
}

With this solution, junitreport gets its input exactly as it expects
and builds it's TESTS-TestSuites.xml properly and .. voila, I get my
junit HTML report of Fluint tests.

I'd be happy to provide any further information if this becomes useful
for incorporating into airtestrunner. Or, if indeed, you'd still like
me to file a defect.

Thanks, Steve

> > Thanks, Steve- Hide quoted text -
>
> - Show quoted text -

[Brian]

@steve - Sounds like you've done some great research into the issue.
If you don't mind still filing an issue on the Google Code site would
help us greatly in getting this fixed. If you could mention your
approach to the solution as well, it is appreciated. We'll have to
regression test your solution with a few of the CI servers as well as
the Maven surefire plugin to make sure all meshes well, but this is a
great start. If you'd like to attach a patch file to the issue as
well, even better.

Thanks for the help in getting this started.

-Brian

[Arnoud]

Hi, had the same issues. dpuint gave better output ;-)

for now a solution just fix the output with ant and use the xslt task:

cheers, Arnoud

<!--
replace nl.bla.Class to nl.bla::Class in the report
to make it compatible with junit-frames.xsl
-->
<replaceregexp
file="${report.dir}/TEST-AllTests.xml"
match='\.([A-Z][A-Za-z0-9_$]*")'
replace="::\1"
byline="true"/>

<!-- convert
testsuite
name="net.digitalprimates.fluint.unitTests.frameworkSuite.testCases::TestASComponentUse"
to
name="AddressDtoAssemblerImplTest"
package="com.albumprinter.chameleon.data.business.assembler.impl"
to make it compatible with junit-frames.xsl
-->
<replaceregexp
file="${report.dir}/TEST-AllTests.xml"
match='testsuite name="([^:]+)::(.*)"'
replace='testsuite name="\2" package="\1"'
byline="true"/>

<xslt basedir="${report.dir}" destdir="${report.dir}/html"
style="${build.home}/xsl/junit-frames.xsl">
<mapper type="glob" from="*.xml" to="*.html" />
<param name="output.dir" expression="${report.dir}/html" />
</xslt>