[workflow-plugin] Parallel JUnitResultArchiver steps result in proper junitResult.xml but wrong build.xml

[Logan Glickfield]

I'm having trouble with a pipeline I'm trying to set up. The build is simultaneously succeeding but having failing tests.

I have 2 nodes configured in parallel that run some python tests and dump the results to xml. I then call the JUnitResultArchiver step in each node.

The first node to finish (about 20 minutes) has all passing tests. The second node finishes later (about 45 minutes) with a couple failing tests. In the build folder, if I look at junitResult.xml, it has all the tests from both nodes including the failing tests. But if I look at build.xml, there is one TestResultAction and it only contains information about the tests from the first node and thus indicates success.

Here's how this looks in the UI:

I'm on Jenkins 1.652 and JunitPlugin 1.11. Any ideas what may be wrong or how I can further debug?

Thanks for the help!

-Logan

P.S. Stripped down representation of my pipeline's flow below:

stage 'Setup'
node {
    git...
    stash 'project'
}

stage 'Tests'
tests = [:]
tests['Basic Tests'] = node {
    unstash 'project'
    sh 'py.test -v --junitxml=reports/junit.xml || true'
    step([$class: 'JUnitResultArchiver', testResults: 'reports/junit.xml'])
}
tests['System Tests'] = parallelNode {
    unstash 'project'
    sh 'py.test -v --no-small --no-medium --large --junitxml=reports/junit.xml || true'
    step([$class: 'JUnitResultArchiver', testResults: 'reports/junit.xml'])
}
parallel tests

[Logan Glickfield]

Oops. s/parallelNode/node (forgot to clean up 2nd instance in sample code).

[Logan Glickfield]

I've tried a couple different things:

- Using unique names for the different output .xml files (instead of both `junit.xml`)

- Running tests serially

- Getting the tests to complete in the opposite order

However, no matter what I've tried, only the first node to call `JUnitResultArchiver` has any bearing on the final build status or in the UI.

Should I file this as a bug, or do others have suggestions?

[Logan Glickfield]

Current hunch is junit-plugin is throwing out the additional pytest output because of https://github.com/jenkinsci/junit-plugin/blob/master/src/main/java/hudson/tasks/junit/TestResult.java#L251-L258. The pytest xml files have a hardcoded testsuite 'name' and no 'id' or 'timestamp' attribute.

Since 'id' and 'timestamp' are optional, is there any expectation in the schema 'name' should be unique? If not, this seems like a junit-plugin bug. If there is a unique expectation for 'name', it seems like a pytest bug.