Stopping a build in pipeline vs freestyle mode: Different "Results"

[Philipp Mahlberg]

Hello everybody,

I initially asked this question a week ago, but somehow it didn't go through. Since I am quite certain that this is rather a dev than a user question, I will try to bring it up here again...

I am developing a Jenkins plugin that can be used in a traditional Freestyle project as well as in Pipeline mode. In certain circumstances, I want the plugin to abort the entire Jenkins job as a continued build process of the project is not of any use anymore.

Thus, I am throwing an AbortException within the run method of the StepExecution class:

run.setResult(jenkinsResult);
if (jenkinsResult == Result.ABORTED) {
    throw new AbortException("Aborted due to failure during QF-Test build step");
}

In pipeline mode, this works as expected. The first line is not even needed, even without it, the build stops and will be marked as ABORTED.

However, in freestyle mode, the exception gets caught here: https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/Run.java#L1886

The build seems to terminate (as requested), but the result will be forced to FAILURE (instead of ABORTED, which is what I would expect). I know that I can achieve the ABORT state by "interrupting" the build. But interrupting seems to encode the user actively stopping the build e.g. by clicking the red cross. Whereas the AbortException message gets mentioned on the console log, the interrupt simply stops the build without further notice. In this light, I wouldn't say that my use case (the plugin stopping the build autonomously) should be handled by the interrupt mechanism.

Is this inconsistency between result states somehow intended? (Recall, the pipeline mode does call this an aborted build right away) Or am I misinterpreting the build result concept in Jenkins?

Thanks, Philipp

[Jesse Glick]

On Wed, Jul 29, 2020 at 8:44 AM Philipp Mahlberg
<philipp....@qf-software.com> wrote:
> run.setResult(jenkinsResult);

Do not call `setResult`. Use exceptions only.

> In pipeline mode, this works as expected. The first line is not even needed, even without it, the build stops and will be marked as ABORTED.

Must be a mistake. `AbortException`, despite the name, should produce
`Result.FAILURE`. It is the stock marker for a build which failed not
due to any bug in Jenkins (an expected failure), so no stack trace is
printed.

> I know that I can achieve the ABORT state by "interrupting" the build. But interrupting seems to encode the user actively stopping the build e.g. by clicking the red cross.

For Pipeline builds, `FlowInterruptedException` encodes a cause and a
result, defaulting to `ABORTED`. There is no exact equivalent for
traditional job types but (IIRC) any `InterruptedException` will count
as `ABORTED`.

[Philipp Mahlberg]

Hi Jesse,
thanks a lot for your comments; I hope the penny has dropped now (see below...).

You were right -- there was still a `setResult(ABORTED)` call preceding the exception in the plugin. This survives in pipeline mode buts gets overwritten in a Freestyle project which caused the observed inconsistency.

From what I understand, the build result mechanism works as follows:
* Within a  `run`-method of a step I can call run.setResult() with an argument of type SUCCESS, UNSTABLE or FAILURE -- depending on the outcome I want to indicate.
* In case of an error that is still "normal" in terms that I might happen during a build process but is such severe that I want to abort the build I throw an Exception e.g. of type AbortException. Jenkins, in turn, marks such builds as FAILED.
* The other two possible outcomes, namely ABORTED or NOT_BUILD are beyond the scope of "normal" build execution and as such there should be in general no need to set them from a run method of a step. This is probably also the reason why one can specify an (abnormal) cause when triggering an build that should be treated as ABORTED.

Is this correct now? This would still leave an ambiguity between "failed, but completed" and "normally aborted" (which gets also marked as failed) builds but apart from this it looks rather consistent...

[Jesse Glick]

On Thu, Jul 30, 2020 at 12:07 PM Philipp Mahlberg
<philipp....@qf-software.com> wrote:
> Within a `run`-method of a step I can call run.setResult()

You _can_ but you _should not_. `setResult` should only ever be called
by infrastructure code (in Jenkins core or Pipeline foundation plugins
like `workflow-job`). (There is currently an exception carved out for
`UNSTABLE` though it is better to use `WarningAction`.)

> In case of an error that is still "normal" in terms that I might happen during a build process but is such severe that I want to abort the build

Let us say _fail_ the build.

> I throw an Exception e.g. of type AbortException. Jenkins, in turn, marks such builds as FAILED.

Correct.

> The other two possible outcomes, namely ABORTED or NOT_BUILD are beyond the scope of "normal" build execution

Well, `ABORTED` is pretty common; and `NOT_BUILT` is used occasionally
for example from the `milestone` step to indicate that the build was
not “aborted” per se, it just did not run some or all of its normal
stages or whatever.

> This would still leave an ambiguity between "failed, but completed" and "normally aborted" (which gets also marked as failed)

Perhaps you are misinterpreting. For Pipeline at least (traditional
build types are mostly analogous):

`FAILURE` means the build did not complete all of its expected
operations: a “fatal error” in the Javadoc. For example, some `sh`
step had a nonzero exit status and there was nothing catching that, so
everything after that point was skipped. `AbortException` is an
exception like any other, unwinding the call stack, with the sole
difference that it suppresses printing a stack trace at the end
because the failure is thought to be a user-level problem rather than
an internal bug.

If the Groovy program defined by a Pipeline script returned normally
(execution reaches the end of the implicit `Script.run` method), then
the build is by default `SUCCESS`, though it could be `UNSTABLE` if
some nonfatal problems were recorded, such as test failures.

`ABORTED` means a `FlowInterruptedException` terminated the script,
either because the build as a whole was actually interrupted (e.g.,
user clicks red *X*) or because some step threw it and nobody caught
it (e.g., `timeout` exceeded). A `FlowInterruptedException` may also
set another result if desired, and it can include causes that get
printed to the log and shows in the build summary.

[Jesse Glick]

Anyway, back to your original post: I have no idea what this QF-Test
thing is, but the vast majority of the time, for either Pipeline or
traditional builds, what you want to do is throw `AbortException` and
that is all.