[JIRA] (JENKINS-61405) Stage post conditions doomed by build result becoming unstable

6 views
Skip to first unread message

jdoyle@iso-ne.com (JIRA)

unread,
Mar 9, 2020, 4:03:03 PM3/9/20
to jenkinsc...@googlegroups.com
Jim D created an issue
 
Jenkins / Bug JENKINS-61405
Stage post conditions doomed by build result becoming unstable
Issue Type: Bug Bug
Assignee: Andrew Bayer
Components: pipeline-model-definition-plugin
Created: 2020-03-09 20:02
Environment: Jenkins ver. 2.176.3
Pipeline: Declarative 1.3.9
Priority: Major Major
Reporter: Jim D

In a pipeline with multiple sequential stages that typically have either an unstable or a success result, stage post conditions can't be used to take any meaningful action, because one unstable stage dooms the subsequent stages to fire the unsuccessful post condition instead of the success post condition.

Pipeline:

 

pipeline {
    agent any
    stages {
        stage('First') {
            steps {
                echo 'Before step with error'
                catchError(message: 'Step had error.', stageResult: 'UNSTABLE') {
                    sh 'nonexistentcommand'
                }
                echo 'After step with error'
            }
            post {
                success {
                    echo 'First stage post success'
                }
                unsuccessful {
                    echo 'First stage post unsuccessful'
                }
            }
        }
        stage('Second') {
            steps {
                echo 'Stage after unstable stage'
            }
            post {
                success {
                    echo 'Second stage post success'
                }
                unsuccessful {
                    echo 'Second stage post unsuccessful'
                }
            }
        }
    }
}

Console Output:

 

Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in ...
[Pipeline] {
[Pipeline] stage
[Pipeline] { (First)
[Pipeline] echo
Before step with error
[Pipeline] catchError
[Pipeline] {
[Pipeline] sh
+ nonexistentcommand
.../script.sh: line 1: nonexistentcommand: command not found
[Pipeline] }
ERROR: Step had error.
ERROR: script returned exit code 127
[Pipeline] // catchError
[Pipeline] echo
After step with error
Post stage
[Pipeline] echo
First stage post unsuccessful
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Second)
[Pipeline] echo
Stage after unstable stage
Post stage
[Pipeline] echo
Second stage post unsuccessful
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: FAILURE

Resolved issue https://issues.jenkins-ci.org/browse/JENKINS-52114 references making the post conditions "more stage specific", but from this test, the conditions don't seem stage-specific at all.  It's really unclear what the current behavior for the stage post conditions is supposed to be. 

Unresolved issue https://issues.jenkins-ci.org/browse/JENKINS-59469 is a little bit similar, in that it asks for a variable with the stage result that can be used in the post section.  That would address our overall use case here too (which is sending per-stage notifications).  But just having post conditions that referred to the stage would be helpful.

If there are any workarounds for the current behavior, I'd love to give them a try.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

jdoyle@iso-ne.com (JIRA)

unread,
Mar 9, 2020, 5:02:03 PM3/9/20
to jenkinsc...@googlegroups.com
Jim D updated an issue
Change By: Jim D
In a pipeline with multiple sequential stages that typically have either an unstable or a success result, stage post conditions can't be used to take any meaningful action, because one unstable stage dooms the subsequent stages to fire the unsuccessful post condition instead of the success post condition.

Pipeline:

 
{noformat}

pipeline {
    agent any
    stages {
        stage('First') {
            steps {
                echo 'Before step with error'
                catchError(message: 'Step had error.', stageResult: 'UNSTABLE') {
                    sh 'nonexistentcommand'
                }
                echo 'After step with error'
            }
            post {
                success {
                    echo 'First stage post success'
                }
                unsuccessful {
                    echo 'First stage post unsuccessful'
                }
            }
        }
        stage('Second') {
            steps {
                echo 'Stage after unstable stage'
            }
            post {
                success {
                    echo 'Second stage post success'
                }
                unsuccessful {
                    echo 'Second stage post unsuccessful'
                }
            }
        }
    }
}{noformat}
Console Output:

 
{noformat}
{noformat}

Resolved issue https://issues.jenkins-ci.org/browse/JENKINS-52114 references making the post conditions "more stage specific", but from this test, the conditions don't seem stage-specific at all.  It's really unclear what the current behavior for the stage post conditions is supposed to be. 

Unresolved issue https://issues.jenkins-ci.org/browse/JENKINS-59469 is a little bit similar, in that it asks for a variable with the stage result that can be used in the post section.  That would address our overall use case here too (which is sending per-stage notifications).  But just having post conditions that referred to the stage would be helpful.

If there are any workarounds for the current behavior, I'd love to give them a try.

jdoyle@iso-ne.com (JIRA)

unread,
Mar 11, 2020, 4:17:02 PM3/11/20
to jenkinsc...@googlegroups.com
Jim D commented on Bug JENKINS-61405
 
Re: Stage post conditions doomed by build result becoming unstable

Please note, right now I'm testing a workaround that's based on the code in pipeline-model-extensions/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/model/BuildCondition.java.

That's the code that seems to return a counter-intuitive result that mixes some stage-specific things with non-stage-specific things.

I can create a global pipeline library method that basically pulls out the 3 components that BuildCondition uses, errorResult, execResult, and prevResult, and inspect and use them directly:

@NonCPS
Map<String,String> getStageConditionResults () {
    Map<String,String> results = [:]
    String stageName = env.STAGE_NAME
    WorkflowRun run = currentBuild.getRawBuild()
    Result errorResult = Result.SUCCESS;
    List<FlowNode> startAndEnd = CommonUtils.findPossiblyUnfinishedEndNodeForCurrentStage(stageName, run.getExecution());
    if (startAndEnd.size() == 2 && startAndEnd.get(0) != null && startAndEnd.get(1) != null) {
        DepthFirstScanner scanner = new DepthFirstScanner();
        if (scanner.setup(startAndEnd.get(1), Collections.singletonList(startAndEnd.get(0)))) {
            WarningAction warningAction = StreamSupport.stream(scanner.spliterator(), false)
                .map({node -> node.getPersistentAction(WarningAction.class)})
                .filter(Objects.&nonNull)
                .max(Comparator.comparing({warning -> warning.getResult().ordinal}))
                .orElse(null);
            if (warningAction != null) {
                errorResult = warningAction.getResult();
            }
        }
    }
    Result execResult = run.getExecution().getResult()
    Result prevResult = run.getResult();
    if (prevResult == null) {
        prevResult = Result.SUCCESS;
    }
    if (execResult == null) {
        execResult = Result.SUCCESS;
    }
    results['errorResult'] = errorResult.toString()
    results['execResult'] = execResult.toString()
    results['prevResult'] = prevResult.toString()
    return results
}

What I notice is that unstable stages have UNSTABLE errorResult and successful stages have SUCCESS errorResult. But once one stage has UNSTABLE prevResult, all subsequent stages have UNSTABLE prevResult. It looks like in the BuildCondition code, that UNSTABLE prevResult will trump the SUCCESS errorResult and return UNSTABLE as the overall Result for a successful stage, firing stage post conditions like "unsuccessful". Really what would be good is a pair of stage post conditions like success/unsuccessful that ignores the UNSTABLE prevResult.

In any case, I'm going to try using this pipeline library method in our stage post sections to detect when a stage is unstable and requires notifications.

Is there any chance similar logic could be implemented by the existing post conditions, or added as new post conditions?

Reply all
Reply to author
Forward
0 new messages