[JIRA] (JENKINS-58369) xUnit Plugin problem with Boost test XML output that contains messages

9 views
Skip to first unread message

fdfrancisduffy@gmail.com (JIRA)

unread,
Jul 6, 2019, 1:46:03 PM7/6/19
to jenkinsc...@googlegroups.com
Francis Duffy created an issue
 
Jenkins / Bug JENKINS-58369
xUnit Plugin problem with Boost test XML output that contains messages
Issue Type: Bug Bug
Assignee: Nikolas Falco
Attachments: junit_test_bad.xml, junit_test_good.xml, xunit_test_bad.xml, xunit_test_good.xml
Components: xunit-plugin
Created: 2019-07-06 17:45
Environment: Jenkins version 2.179, xunit plugin version 2.3.1
Labels: xunit junit test-results parse-error
Priority: Minor Minor
Reporter: Francis Duffy

I compile the following minimal C++ Boost test program

 

#define BOOST_TEST_MODULE ExampleTestSuite
#include <boost/test/included/unit_test.hpp>
#include <boost/test/unit_test.hpp>

BOOST_AUTO_TEST_SUITE(MyTestSuite_1)

BOOST_AUTO_TEST_CASE(MyTest_1_1) {
    // BOOST_TEST_MESSAGE("Running MyTest_1_1");
    int i = 1;
    BOOST_CHECK_EQUAL(1, i);
}

BOOST_AUTO_TEST_SUITE_END()

BOOST_AUTO_TEST_SUITE(MyTestSuite_2)

BOOST_AUTO_TEST_CASE(MyTest_2_1) {
    int i = 1;
    BOOST_CHECK_EQUAL(1, i);
}

BOOST_AUTO_TEST_SUITE_END()

and run the test using the parameters

 

 

--log_format=XML --log_level=test_suite --report_level=no --result_code=no --log_sink=xunit_test.xml

The XML output is in the attached file xunit_test_good.xml and the xUnit plugin generated JUnit output is in the attached file junit_good.xml. Everything is as I would expect here and the table that is rendered on the Jenkins GUI has ExampleTestSuite at the top level with MyTestSuite_1 and MyTestSuite_2 nested below it.

However, if I comment in the BOOST_TEST_MESSAGE line in the code, I get the message in the XML output as in the attached file xunit_test_bad.xml. This gets converted to the JUnit output in junit_test_bad.xml by the xUnit plugin. The nesting is now messed up in that ExampleTestSuite appears with a nested MyTestSuite_2 but ExampleTestSuite.MyTestSuite_1 appears alongside ExampleTestSuite.

This is coming from the classname attribute in the generated JUnit output. Would it be possible to fix this so that test message output does not interfere with the generation of the classname attribute and hence the nesting of the test suites in the report? Alternatively, is there a workaround that I can apply to keep the log message but still have the expected nesting of the test suites?

 

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

fdfrancisduffy@gmail.com (JIRA)

unread,
Jul 6, 2019, 1:47:02 PM7/6/19
to jenkinsc...@googlegroups.com
Francis Duffy updated an issue
Change By: Francis Duffy
I compile the following minimal C++ Boost test program

 
{code:java}

#define BOOST_TEST_MODULE ExampleTestSuite
#include <boost/test/included/unit_test.hpp>
#include <boost/test/unit_test.hpp>

BOOST_AUTO_TEST_SUITE(MyTestSuite_1)

BOOST_AUTO_TEST_CASE(MyTest_1_1) {
    // BOOST_TEST_MESSAGE("Running MyTest_1_1");
    int i = 1;
    BOOST_CHECK_EQUAL(1, i);
}

BOOST_AUTO_TEST_SUITE_END()

BOOST_AUTO_TEST_SUITE(MyTestSuite_2)

BOOST_AUTO_TEST_CASE(MyTest_2_1) {
    int i = 1;
    BOOST_CHECK_EQUAL(1, i);
}

BOOST_AUTO_TEST_SUITE_END()

{code}

and run the test using the parameters

 

 
{code:java}
--log_format=XML --log_level=test_suite --report_level=no --result_code=no --log_sink=xunit_test.xml{code}
The XML output is in the attached file {{xunit_test_good.xml}} and the xUnit plugin generated JUnit output is in the attached file {{junit_good.xml}}. Everything is as I would expect here and the table that is rendered on the Jenkins GUI has {{ExampleTestSuite}} at the top level with {{MyTestSuite_1}} and {{MyTestSuite_2}} nested below it.

However, if I comment in the {{BOOST_TEST_MESSAGE}} line in the code, I get the message in the XML output as in the attached file {{xunit_test_bad.xml}}. This gets converted to the JUnit output in {{junit_test_bad.xml}} by the xUnit plugin. The nesting is now messed up in that {{ExampleTestSuite}} appears with a nested {{MyTestSuite_2}} but {{ExampleTestSuite.MyTestSuite_1}} appears alongside {{ExampleTestSuite}}.


This is coming from the {{classname}} attribute in the generated JUnit output. Would it be possible to fix this so that test message output does not interfere with the generation of the {{classname}} attribute and hence the nesting of the test suites in the report? Alternatively, is there a workaround that I can apply to keep the log message but still have the expected nesting of the test suites?

 

kim.randell@vicon.com (JIRA)

unread,
Oct 14, 2019, 11:32:03 AM10/14/19
to jenkinsc...@googlegroups.com
Kim Randell commented on Bug JENKINS-58369
 
Re: xUnit Plugin problem with Boost test XML output that contains messages

This comes from this line:

https://github.com/jenkinsci/xunit-plugin/blob/5f017f9da41e987f3644c65f10658660519abfe5/src/main/resources/org/jenkinsci/plugins/xunit/types/boosttest-1.2-to-junit-4.xsl#L176

Note that $elt/@file is only populated if the TestCase element has a first child with a file attribute (line 172). Therefore, if there is a message or assertion failure in the test, classname is the suite name concatenated with the filename (minus extension), otherwise it is just the suite name. It's unclear to me why the filename should be used in the classname in any case.

 

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages