[JIRA] (JENKINS-59659) XUnit plugin result displays confused by aggregation of parallel builds

15 views
Skip to first unread message

ddumont@gmail.com (JIRA)

unread,
Oct 4, 2019, 9:54:02 AM10/4/19
to jenkinsc...@googlegroups.com
Dan Dumont created an issue
 
Jenkins / Bug JENKINS-59659
XUnit plugin result displays confused by aggregation of parallel builds
Issue Type: Bug Bug
Assignee: Nikolas Falco
Components: xunit-plugin
Created: 2019-10-04 13:53
Priority: Blocker Blocker
Reporter: Dan Dumont

So we use pipeline to build several platforms of our codebase, and prod/nonprod etc.  They all produce a unit test report, and the paths to the result files all contain the specific parallel branch they were built in.

I was hoping this would be enough for it to properly identify the tests, but it seems that unless the tests have the build specific info in the test name, the graphing and result display of the tests gets very confused and will combine some tests based on race conditions which leads to a very spiky looking graph.

I confirmed this by using the jest config in a node project we build the same way.  Jest lets you set the pattern used to build the suite name and test name. Once I set this to include the build info, the graph flattened out and remained consistent.

In other projects we use other frameworks that you support googletest, etc.  I don't know if each of them support the option to configure the test suite/name patterns...     Could you please look at how the plugin could properly handle parallel builds for test display and graphing?

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

nfalco79@hotmail.com (JIRA)

unread,
Oct 7, 2019, 6:22:02 AM10/7/19
to jenkinsc...@googlegroups.com
Nikolas Falco commented on Bug JENKINS-59659
 
Re: XUnit plugin result displays confused by aggregation of parallel builds

I have some questions:

  • Do you have a jenkins job per branch (multibranch pipeline) or one job that build a specific branch?
  • Is the workspace folder shred among all build? Is the workspace wipe out?
  • What do you mean with parallel? Do you have a single job that where you have parallelized the test stage (which one that produce reports) or you have one job but parallel executor of the same job in the same workspace?

nfalco79@hotmail.com (JIRA)

unread,
Nov 1, 2019, 1:36:02 PM11/1/19
to jenkinsc...@googlegroups.com

nfalco79@hotmail.com (JIRA)

unread,
Nov 1, 2019, 1:36:03 PM11/1/19
to jenkinsc...@googlegroups.com

nfalco79@hotmail.com (JIRA)

unread,
Nov 1, 2019, 1:36:03 PM11/1/19
to jenkinsc...@googlegroups.com

nfalco79@hotmail.com (JIRA)

unread,
Nov 1, 2019, 1:45:02 PM11/1/19
to jenkinsc...@googlegroups.com
 
Re: XUnit plugin result displays confused by aggregation of parallel builds

Any news? If not I will close this issue.
In pipeline there is the association with the stage. This means you can run parallel step and process reports. Each report is associated with a specific stage. In non pipeline environment we can not support stage association. This means that tests name have to be unique across parallel execution.

nfalco79@hotmail.com (JIRA)

unread,
Nov 1, 2019, 1:47:02 PM11/1/19
to jenkinsc...@googlegroups.com
Nikolas Falco edited a comment on Bug JENKINS-59659
Any news? If not I will close this issue.
In pipeline there is the association with the stage. This means you can run parallel step and process reports. Each report is associated with a specific stage. Path of the reports are not taken into account.

In non pipeline environment we can not support stage association. This means that tests name have to be unique across parallel execution.

nfalco79@hotmail.com (JIRA)

unread,
Nov 1, 2019, 2:11:02 PM11/1/19
to jenkinsc...@googlegroups.com
Nikolas Falco closed an issue as Not A Defect
 

Path of report file could not be used to distinguish test. Perfom a xunit step in a different atep to differentiate them as explained in the improvement of linked issue.

Change By: Nikolas Falco
Status: Open Closed
Resolution: Not A Defect
Reply all
Reply to author
Forward
0 new messages