[Plugin Development] API to listen to the end of the execution of a step and to collect its result?

[Cyrille Le Clerc]

Dear community,

Context: 

I'm trying to implement an OpenTelemetry instrumentation of Jenkins, starting injecting distributed traces in job executions. See https://github.com/cyrille-leclerc/opentelemetry-plugin

Question: 

What is the recommended way to listen to the end of the execution of a step and to collect its result?

I would like to listen the end of execution of `stage`and `git` steps to end a trace span reporting the result of the step.

I found the `GraphListener#onNewHead(FlowNode)` API and I identify the end of stage steps with `StepEndNode` containing a   whose descriptor is a `StageStep.DescriptorImpl` but

• I am not clear on if I should track the `StepEndNode` containing a `LabelAction` or an `ArgumentsAction`
• On none of the StepEndNode give me access to the error thrown during the execution to indicate a failure,  `StepEndNode#getError()` and `StepEndNode.getExecution.getCauseOfFailure()` return null during  `GraphListener#onNewHead(FlowNode)`

I'm wondering if I have to add a `BodyExecutionCallback` to the `CpsBodyInvoker` but I didn't find an API to do this implementing a listener of the pipeline executions.

I have a similar request for `git` steps that don't even have a  `StepEndNode`, how can I intercept the end of execution and the result/error of the execution of a `git` step.

I hope my question is detailed enough.

Cyrille

[Mark Waite]

I don't know the answer to your listener question.  This is probably not what you want to hear, but I think that you should focus on the checkout step rather than the git step.  Per the git step documentation:

More advanced checkout operations require the checkout step rather than the git step.  

The git step is a simplified shorthand for a subset of the more powerful checkout step:

checkout([$class: 'GitSCM', branches: [[name: '*/master']],
    userRemoteConfigs: [[url: 'http://git-server/user/repository.git']]])

 

I hope my question is detailed enough.

Cyrille

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/db5376b1-3888-46f0-9429-9b34e3940ed9n%40googlegroups.com.

[Andrey Babushkin]

You can take a look at Job and Stage monitoring plugin (https://github.com/jenkinsci/github-autostatus-plugin). I think it does the thing you're looking for.

[Cyrille Le Clerc]

Many thanks Andrey. It seems to be spot on, I'll look at this as soon as my day job ends.

@Mark you are right. I simplified a bit just mentioning the `git`step but I'm interested in instrumenting as many SCM integration as possible.

Cyrille

[Jeff]

I'm the author of the "Job and Stage monitoring plugin"

Have you considered extending that plugin? I don't entirely understand what you're trying to accomplish, but to the extent it overlaps with your proposed plugin, it's probably better for the community not to have two similar plugins. Most users aren't going to understand the differences between two similar plugins, which is confusing and frustrating.

The "Job and Stage monitoring plugin" watches jobs, and allows code to take arbitrary actions at the end of each step in a job, and also at the end of the job. Said code can either be hosted in the JSM plugin, or an entirely separate plugin.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/a1012b64-1979-47a5-9646-832e21376146n%40googlegroups.com.

[Jesse Glick]

On Mon, Jan 11, 2021 at 8:24 PM 'Cyrille Le Clerc' via Jenkins Developers <jenkin...@googlegroups.com> wrote:

• I am not clear on if I should track the `StepEndNode` containing a `LabelAction` or an `ArgumentsAction`

Depends on whether you are interested in the close of the body (which a block-scoped step may run zero, one, or more times), or of the step itself.

• On none of the StepEndNode give me access to the error thrown during the execution to indicate a failure,  `StepEndNode#getError()` and `StepEndNode.getExecution.getCauseOfFailure()` return null during  `GraphListener#onNewHead(FlowNode)`

`onNewHead` is going to be called as soon as the new node is created, before some of its metadata is actually initialized. Probably there should be a better API here, but you can just use `GraphListener` in such a way that when a new head is added, you process metadata from its parent node(s), which by that time should be finalized.

I have a similar request for `git` steps that don't even have a  `StepEndNode`, how can I intercept the end of execution and the result/error of the execution of a `git` step.

Same. When the next node appears, the atom node for the step should have any `ErrorAction` attached.

[Cyrille Le Clerc]

Hello Jeff,

It's a great idea to include the tracing and the OpenTelemetry capabilities in the "Job and Stage monitoring plugin".

Would you have a moment to have a video conversation to align on the vision and to discuss the way to move forward?

Cyrille

[Jeff]

Happy to have a chat. What time zone are you in? I'm on the US West Coast (PST). I'm not sure whether anyone else in the community is interested, but replying to the group just in case.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8be42453-51dd-40ae-bce4-fe543a6271a6n%40googlegroups.com.

[Cyrille Le Clerc]

Thank you Jesse for the guidance.

I have introduced in my code higher level listener interface to simplify the usage of the GraphListener API. See skeleton of the listener API below.

@Jeff thanks, I am sending you a direct message to organise a quick call.

Cyrile

----

public interface PipelineListener {

@Nonnull
static List<PipelineListener> all() {
return ExtensionList.lookup(PipelineListener.class);
}

/**
* Just before the pipeline starts
*/
void onStartPipeline(@Nonnull FlowNode node, @Nonnull WorkflowRun run);

/**
* Just before the `stage`step starts
*/
void onStartStageStep(@Nonnull StepStartNode stepStartNode, @Nonnull String stageName, @Nonnull WorkflowRun run);

/**
* Just after the `stage` step ends
*/
void onAfterEndStageStep(@Nonnull StepEndNode stageStepEndNode, @Nonnull String stageName, @Nonnull WorkflowRun run);

/**
* Just before the `parallel` branch starts
*/
void onStartParallelStepBranch(@Nonnull StepStartNode stepStartNode, @Nonnull String branchName, @Nonnull WorkflowRun run);

/**
* Just before the `parallel` branch ends
*/
void onEndParallelStepBranch(@Nonnull StepEndNode stepStepNode, @Nonnull String branchName, @Nonnull WorkflowRun run);

/**
* Just before the atomic step starts
*/
void onAtomicStep(@Nonnull StepAtomNode node, @Nonnull WorkflowRun run);

/**
* Just after the atomic step
*/
void onAfterAtomicStep(@Nonnull StepAtomNode stepAtomNode, @Nonnull WorkflowRun run);

/**
* Just after the pipeline ends
*/
void onEndPipeline(@Nonnull FlowNode node, @Nonnull WorkflowRun run);
}
----