[testng-users] Combine/collate multiple testng-results.xml files into one

[Dirk Hain]

Hi,

I have searched on this topic for a while and could not find an 'easy'
way to combine the testng-results.xml files from multiple suites into
one file.
I have multiple (maven) projects which contain tests and all produce
their own testng-results.xml on project level. I also have a parent
project that I execute to invoke all the sub-projects. Ideally I would
like to configure testng in the parent project to take all sub-project
testng-results.xml files and combine them into one which will be
stored on the parent project level.

If anybody has had a similar case I would love to hear how you solved
this problem.

Thanks!
-Dirk

[Tomás Pollak]

Hudson does this. It presents the test results for a multi-module maven project in one consolidated view (however it doesn't provide you with a merged testng-results.xml file, is that what you need or would the Hudson page be enough?)

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Dirk Hain]

Hi Tomas,

Thanks for your response. I need a combined testng-results.xml because I am integrating into another reporting framework. I guess if Hudson does this there must be a way. Hopefully I don't have to find and merge the files myself :-)

Thanks,

-Dirk

[Nalin Makar]

Hudson doesn't merge the XML files. It only parses multiple testng xml files into hudson specific test objects to report the results as one.

It should be pretty straight forward though using some xpath.

-nalin

[Michael]

I recently had the same need. While I do not know whether it is
effective in all situations, it works for me. Essentially, I have an
ant task that applies an XSLT stylesheet to merge each testng-
results.xml into a single file.

The details of my solution can be found at
http://blog.meesqa.com/2010/09/11/combine-multiple-testng-resultsxml-files-into-a-single-xml-file/

Hope this helps.

Michael

[Dirk Hain]

Michael,

Thanks a lot for your solution. An ant task will work just fine for my
requirements! Great post!!!

Cheers,
-Dirk

[Cédric Beust ♔]

Great post, Michael. I added a link to your article in the main documentation:

http://testng.org/doc/misc.html

-- 

Cédric

On Sat, Sep 11, 2010 at 9:35 PM, Michael <mens...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
Cédric

[elli_ileev]

Hello everybody!
Do you know how to generate the testng-xslt report from multiple Jenkins jobs? I think, your solution will help me solve my problem too, I have tried to merge the testng-merge.xml, unfortunately it does not merge it, instead it saves the last executed job results only.
-Currently I have the multiple .jenkins/jobs/* , and I would like to generate the common testng-xslt report from these jobs. For now I am able to generate the xslt report only from one job at a time.
If you have idea what I am trying to do, please give me a hand.
Thanks a lot!

[Yulia Kantarovich]

Hi!

How can I merge those results in one consolidated view?

[Trinadh Kumar]

Hi Mike, 

I had same issue but i am not able found your blog anymore , can you please share the details in email of another url ?

Thanks

Trinadh

[Michael]

Blog is temporarily offline.  It should be back up in a week or so.  Until then, here are the contents (hopefully the formatting will stay in place):

Combine multiple testng-results.xml files into a single XML file

A couple of weeks ago, I realized I needed to combine multiple testng-results.xml files into to do some consolidated reporting. I did some research and came up with a solution that works for me. Recently, the testng-users Google group had a thread that asked the same thing. This post explains how I did it - not necessarily the best way but works for me now.

Basically, I use an XSLT stylesheet to merge the various XML files into a single file. As you will see below, the solution I used only merges 2 files at a time so I create a loop that merges each file into a master file. Let's start with the master ant task:




...
<taskdef resource="net/sf/antcontrib/antlib.xml"/>
...
<target name="merge-results">
    <delete dir="${report.dir}/testng_summary_report"/>
    <delete file="${report.dir}/start-merge.xml" />
    <delete file="${report.dir}/testng-merge.xml" />

    <mkdir dir="${report.dir}/testng_summary_report"/>
    <copy 
        file="empty-testng-results.xml" 
        tofile="${report.dir}/testng-merge.xml" />

    <for param="xmlFile">
        <path>
            <fileset dir="${report.dir}" >
                <include name="**/testng-results.xml" />
            </fileset>
        </path>

        <sequential>

            <copy 
                file="${report.dir}/testng-merge.xml" 
                toFile="${report.dir}/start-merge.xml" />
                <xslt style="testng-merge.xsl"
                    destdir="${report.dir}"
                    in="${report.dir}/start-merge.xml"
                    out="${report.dir}/testng-merge.xml">

                    <param name="with" expression="@{xmlFile}" />
                </xslt>
            <delete file="${report.dir}/start-merge.xml" />
        </sequential>
    </for>
</target>

After some initial cleanup, the ant task copies the "starting" XML file (empty-testng-results.xml). It looks like this:




<?xml version="1.0" encoding="UTF-8"?>
<testng-results>
    <reporter-output>
    </reporter-output>
</testng-results>

The task then uses the antlib for command to loop through all of the testng-results.xml files and applies the XSLT transform to each in turn. I did not write the XSLT stylesheet. I found it at http://www2.informatik.hu-berlin.de/~obecker/XSLT/merge/merge.xslt.html. For convenience, I include it here:

<!--
   Merging two XML files
   Version 1.6
   LGPL (c) Oliver Becker, 2002-07-05
   obe...@informatik.hu-berlin.de
-->
<xslt:transform 
    xmlns:xslt="http://www.w3.org/1999/XSL/Transform" 
    xmlns:m="http://informatik.hu-berlin.de/merge" 
    version="1.0" 
    exclude-result-prefixes="m">

<!-- Normalize the contents of text, comment, and processing-instruction
     nodes before comparing?
     Default: yes -->
<xslt:param name="normalize" select="'yes'" />

<!-- Don't merge elements with this (qualified) name -->
<xslt:param name="dontmerge" />

<!-- If set to true, text nodes in file1 will be replaced -->
<xslt:param name="replace" select="false()" />

<!-- Variant 1: Source document looks like
     <?xml version="1.0"?>
     <merge xmlns="http://informatik.hu-berlin.de/merge">
        <file1>file1.xml</file1>
        <file2>file2.xml</file2>
     </merge>         
     The transformation sheet merges file1.xml and file2.xml.
-->
<xslt:template match="m:merge">
   <xslt:variable name="file1" select="string(m:file1)" />
   <xslt:variable name="file2" select="string(m:file2)" />
   <xslt:message>
      <xslt:text />Merging '<xslt:value-of select="$file1" />
      <xslt:text />' and '<xslt:value-of select="$file2" />'<xslt:text />
   </xslt:message>
   <xslt:if test="$file1='' or $file2=''">
      <xslt:message terminate="yes">
         <xslt:text>No files to merge specified</xslt:text>
      </xslt:message>
   </xslt:if>
   <xslt:call-template name="m:merge">
      <xslt:with-param name="nodes1" select="document($file1,/*)/node()" />
      <xslt:with-param name="nodes2" select="document($file2,/*)/node()" />
   </xslt:call-template>
</xslt:template>

<!-- Variant 2:
     The transformation sheet merges the source document with the
     document provided by the parameter "with".
-->
<xslt:param name="with" />

<xslt:template match="*">
   <xslt:message>
      <xslt:text />Merging input with '<xslt:value-of select="$with" />
      <xslt:text>'</xslt:text>
   </xslt:message>
   <xslt:if test="string($with)=''">
      <xslt:message terminate="yes">
         <xslt:text>No input file specified (parameter 'with')</xslt:text>
      </xslt:message>
   </xslt:if>

   <xslt:call-template name="m:merge">
      <xslt:with-param name="nodes1" select="/node()" />
      <xslt:with-param name="nodes2" select="document($with,/*)/node()" />
   </xslt:call-template>
</xslt:template>

<!-- ============================================================== -->

<!-- The "merge" template -->
<xslt:template name="m:merge">
   <xslt:param name="nodes1" />
   <xslt:param name="nodes2" />

   <xslt:choose>
      <!-- Is $nodes1 resp. $nodes2 empty? -->
      <xslt:when test="count($nodes1)=0">
         <xslt:copy-of select="$nodes2" />
      </xslt:when>
      <xslt:when test="count($nodes2)=0">
         <xslt:copy-of select="$nodes1" />
      </xslt:when>

      <xslt:otherwise>
         <!-- Split $nodes1 and $nodes2 -->
         <xslt:variable name="first1" select="$nodes1[1]" />
         <xslt:variable name="rest1" select="$nodes1[position()!=1]" />
         <xslt:variable name="first2" select="$nodes2[1]" />
         <xslt:variable name="rest2" select="$nodes2[position()!=1]" />
         <!-- Determine type of node $first1 -->
         <xslt:variable name="type1">
            <xslt:apply-templates mode="m:detect-type" select="$first1" />
         </xslt:variable>

         <!-- Compare $first1 and $first2 -->
         <xslt:variable name="diff-first">
            <xslt:call-template name="m:compare-nodes">
               <xslt:with-param name="node1" select="$first1" />
               <xslt:with-param name="node2" select="$first2" />
            </xslt:call-template>
         </xslt:variable>

         <xslt:choose>
            <!-- $first1 != $first2 -->
            <xslt:when test="$diff-first='!'">
               <!-- Compare $first1 and $rest2 -->
               <xslt:variable name="diff-rest">
                  <xslt:for-each select="$rest2">
                     <xslt:call-template name="m:compare-nodes">
                        <xslt:with-param name="node1" select="$first1" />
                        <xslt:with-param name="node2" select="." />
                     </xslt:call-template>
                  </xslt:for-each>
               </xslt:variable>
      
               <xslt:choose>
                  <!-- $first1 is in $rest2 and 
                       $first1 is *not* an empty text node  -->
                  <xslt:when test="contains($diff-rest,'=') and not($type1='text' and normalize-space($first1)='')">
                     <!-- determine position of $first1 in $nodes2
                          and copy all preceding nodes of $nodes2 -->
                     <xslt:variable name="pos" select="string-length(substring-before( $diff-rest,'=')) + 2" />
                     <xslt:copy-of select="$nodes2[position() &lt; $pos]" />
                     <!-- merge $first1 with its equivalent node -->
                     <xslt:choose>
                        <!-- Elements: merge -->
                        <xslt:when test="$type1='element'">
                           <xslt:element name="{name($first1)}" namespace="{namespace-uri($first1)}">
                              <xslt:copy-of select="$first1/namespace::*" />
                              <xslt:copy-of select="$first2/namespace::*" />
                              <xslt:copy-of select="$first1/@*" />
                              <xslt:call-template name="m:merge">
                                 <xslt:with-param name="nodes1" select="$first1/node()" />
                                 <xslt:with-param name="nodes2" select="$nodes2[position()=$pos]/node()" />
                              </xslt:call-template>
                           </xslt:element>
                        </xslt:when>
                        <!-- Other: copy -->
                        <xslt:otherwise>
                           <xslt:copy-of select="$first1" />
                        </xslt:otherwise>
                     </xslt:choose>
      
                     <!-- Merge $rest1 and rest of $nodes2 -->
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2[position() &gt; $pos]" />
                     </xslt:call-template>
                  </xslt:when>

                  <!-- $first1 is a text node and replace mode was
                       activated -->
                  <xslt:when test="$type1='text' and $replace">
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2" />
                     </xslt:call-template>
                  </xslt:when>

                  <!-- else: $first1 is not in $rest2 or
                       $first1 is an empty text node -->
                  <xslt:otherwise>
                     <xslt:copy-of select="$first1" />
                     <xslt:call-template name="m:merge">
                        <xslt:with-param name="nodes1" select="$rest1" />
                        <xslt:with-param name="nodes2" select="$nodes2" />
                     </xslt:call-template>
                  </xslt:otherwise>
               </xslt:choose>
            </xslt:when>

            <!-- else: $first1 = $first2 -->
            <xslt:otherwise>
               <xslt:choose>
                  <!-- Elements: merge -->
                  <xslt:when test="$type1='element'">
                     <xslt:element name="{name($first1)}" namespace="{namespace-uri($first1)}">
                        <xslt:copy-of select="$first1/namespace::*" />
                        <xslt:copy-of select="$first2/namespace::*" />
                        <xslt:copy-of select="$first1/@*" />
                        <xslt:call-template name="m:merge">
                           <xslt:with-param name="nodes1" select="$first1/node()" />
                           <xslt:with-param name="nodes2" select="$first2/node()" />
                        </xslt:call-template>
                     </xslt:element>
                  </xslt:when>
                  <!-- Other: copy -->
                  <xslt:otherwise>
                     <xslt:copy-of select="$first1" />
                  </xslt:otherwise>
               </xslt:choose>

               <!-- Merge $rest1 and $rest2 -->
               <xslt:call-template name="m:merge">
                  <xslt:with-param name="nodes1" select="$rest1" />
                  <xslt:with-param name="nodes2" select="$rest2" />
               </xslt:call-template>
            </xslt:otherwise>
         </xslt:choose>
      </xslt:otherwise>
   </xslt:choose>
</xslt:template>

<!-- Comparing single nodes: 
     if $node1 and $node2 are equivalent then the template creates a 
     text node "=" otherwise a text node "!" -->
<xslt:template name="m:compare-nodes">
   <xslt:param name="node1" />
   <xslt:param name="node2" />
   <xslt:variable name="type1">
      <xslt:apply-templates mode="m:detect-type" select="$node1" />
   </xslt:variable>
   <xslt:variable name="type2">
      <xslt:apply-templates mode="m:detect-type" select="$node2" />
   </xslt:variable>

   <xslt:choose>
      <!-- Are $node1 and $node2 element nodes with the same name? -->
      <xslt:when test="$type1='element' and $type2='element' and local-name($node1)=local-name($node2) and namespace-uri($node1)=namespace-uri($node2) and name($node1)!=$dontmerge and name($node2)!=$dontmerge">
         <!-- Comparing the attributes -->
         <xslt:variable name="diff-att">
            <!-- same number ... -->
            <xslt:if test="count($node1/@*)!=count($node2/@*)">.</xslt:if>
            <!-- ... and same name/content -->
            <xslt:for-each select="$node1/@*">
               <xslt:if test="not($node2/@* [local-name()=local-name(current()) and namespace-uri()=namespace-uri(current()) and .=current()])">.</xslt:if>
            </xslt:for-each>
         </xslt:variable>
         <xslt:choose>
            <xslt:when test="string-length($diff-att)!=0">!</xslt:when>
            <xslt:otherwise>=</xslt:otherwise>
         </xslt:choose>
      </xslt:when>

      <!-- Other nodes: test for the same type and content -->
      <xslt:when test="$type1!='element' and $type1=$type2 and name($node1)=name($node2) and ($node1=$node2 or ($normalize='yes' and normalize-space($node1)= normalize-space($node2)))">=</xslt:when>

      <!-- Otherwise: different node types or different name/content -->
      <xslt:otherwise>!</xslt:otherwise>
   </xslt:choose>
</xslt:template>

<!-- Type detection, thanks to M. H. Kay -->
<xslt:template match="*" mode="m:detect-type">element</xslt:template>
<xslt:template match="text()" mode="m:detect-type">text</xslt:template>
<xslt:template match="comment()" mode="m:detect-type">comment</xslt:template>
<xslt:template match="processing-instruction()" mode="m:detect-type">pi</xslt:template>

</xslt:transform>


At the end of this process, the merged results will be found in a file named testng-merge.xml.

[Trinadh Kumar]

Thank you very much for reply .

Small help needed, i am very new xsld and i am confused with below setup. Can you please guide me more details on below.

1. <taskdef resource="net/sf/antcontrib/antlib.xml"/> ,  dont have resource in my machine , what should i do to create that ?

2. Reg:- <xslt:transform  file , where should i give my xml files names which need to be merged, and what is the file name i have to give for this template, and where should i save in my project ?

3. is it possible to get details steps on this please.

Thanks

Trinadh

[Michael]

It has been some 7 years since I wrote that post so I am no longer familiar with the details...  I will try to give some general guidance.

1) antcontrib is an old ant extension library.  It can be found at http://ant-contrib.sourceforge.net/ 

2) The original files are stored in some directory specified by the variable ${report.dir}

Essentially, it is looking at all subdirectories within the the report.dir directory for any files with the name testng-results.xml

           <fileset dir="${report.dir}" >
                <include name="**/testng-results.xml" />
            </fileset>

3) Sorry I cannot provide more detail steps - I no longer have the environment when I first created the post.