This is spin-off issue from https://issues.jenkins-ci.org/browse/JENKINS-54255
i think that it is better not to abuse old closed issue with this one.
here is the summary of discussion i had with Nicholas.:
======================================
i see that xunit thresholds produce non deterministic results in parallel pipe runs
This is how i define xunit thresholds:
{code:java}
xunit thresholds: [failed(unstableThreshold: '0',failureNewThreshold: '0')],
tools: [Custom(customXSL: "${JENKINS_URL}/userContent/pytest-to-junit.xsl",
deleteOutputFiles: true,
failIfNotNew: true,
pattern: "${env.DOWNLOADED_ARTIFACTS_PATH}/${env.TEST_RESULTS_PATH}/*.xml",
skipNoTestFiles: true,
stopProcessingIfError: true)]
{code}
I run few parallel branches of jenkins pipe, each in its own workspace and each pipe branch uploads it test results independently.
I have some build new failed tests:
|!https://swdvp-jnk01.anpa.corp.amazon.com:9080/static/da57df94/images/48x48/clipboard.png!|Test Result (40 failures / +17)|
i expect to get failed status, but xunit still sets the build to be unstable:
{code:java}
01:57:59.736 INFO: Starting to record.
01:57:59.736 INFO: Processing Custom Tool
01:58:00.146 INFO: Stopping recording.
01:58:00.147 INFO: Check 'Failed Tests' threshold.
01:58:00.147 INFO: The new number of tests for the threshold 'Failed Tests' exceeds the specified "new failure threshold" value.
{code}
I use:
* [Jenkins ver. 2.160|https://jenkins.io/]
* xUnit plugin 2.3.2
On this Nicolas kindly replied:
Anyway if this is your configuration (it's not clear from your post, seems commented)
{noformat}
xunit thresholds: [failed(unstableThreshold: '0',failureNewThreshold: '0')],
{noformat}
the unstable build status is correct and is in accordance with what the tooltips document.
|
|
