Distinguish multiple test results

[Fabian Cenedese]

Hello

We have a declarative pipeline where each step executes several
tests in sequence, each one producing a cppunit report. This works
out fine except that failed tests can't be distinguished.

Jenkins (pseudo code):

stage('Parallel Execution') {
parallel {
stage('stage1') {
steps {
script {
DoTest(...) // e.g. Debug, Release, other compilers etc
DoTest(...)
DoTest(...)
}
}
}
stage('stage2') {
steps {
script {
DoTest(...)
DoTest(...)
DoTest(...)
}
}
}

If now a test fails in the DoTest routine the report looks like this:

Test Result (3 failures / -1)
Parallel Execution / stage1 / FailedTest
Parallel Execution / stage1 / FailedTest
Parallel Execution / stage1 / FailedTest

This makes it hard to find out which call really failed, especially if the
test only failed once and not each time. I've seen that xUnit has
'checksName', but that is only used for SCM, apparently not internally.
Is there a way to give xUnit a custom name, similar to checksName,
to be used for the results instead of the default stage name? My goal
would be something like (or any other way):

Parallel Execution / stage1-call1 / FailedTest
Parallel Execution / stage1-call2 / FailedTest
Parallel Execution / stage1-call3 / FailedTest

Thanks

bye Fabi

[christop...@googlemail.com]

Hi,

I got a similar issue with calling tests in debug and release variant. For me results don't even show up twice, instead it's a matter of coincidence, which results show up at all (individually for each single test case). In my case it's about nunit and gtests which write their result in xml files.

From this list I learnt, the best solution is to modify the xml files. So I added a line of sed to create individual test names for the debug and release tests.

That's not what I would call a great solution but it works.

BR,

Christoph