Starting an asynchronous step

[Jeff Wilson]

I'm trying to write a new pipeline plugin for a build step that is long-running and prone to network delays and timeouts, so it would appear that an asynchronous step is the best way to implement this. However, I have questions about how to implement the StepExecution.start() method to start this asynchronous step. I can't find a (canonical) example -- does anyone have a pointer to such? Alternatively, a good development guide that shows how to implement this type of step?

[Robert Sandell]

Perhaps you've already read this guide? https://github.com/jenkinsci/workflow-step-api-plugin/blob/master/README.md#creating-an-asynchronous-step

/B

2018-04-06 16:30 GMT+02:00 Jeff Wilson <geek.m...@gmail.com>:

I'm trying to write a new pipeline plugin for a build step that is long-running and prone to network delays and timeouts, so it would appear that an asynchronous step is the best way to implement this. However, I have questions about how to implement the StepExecution.start() method to start this asynchronous step. I can't find a (canonical) example -- does anyone have a pointer to such? Alternatively, a good development guide that shows how to implement this type of step?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1652f66e-46bc-4d94-8e30-dac8993e817c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Robert Sandell

Software Engineer

CloudBees, Inc.

E: rsan...@cloudbees.com

Twitter: robert_sandell

[Jeff Wilson]

Indeed. It simply says to implement the start() method as needed. There are no examples of what an implementation might look like, nor is there a description/documentation of what to do.

On Friday, April 6, 2018 at 12:12:47 PM UTC-4, Robert Sandell wrote:

Perhaps you've already read this guide? https://github.com/jenkinsci/workflow-step-api-plugin/blob/master/README.md#creating-an-asynchronous-step

/B

2018-04-06 16:30 GMT+02:00 Jeff Wilson <geek.m...@gmail.com>:

I'm trying to write a new pipeline plugin for a build step that is long-running and prone to network delays and timeouts, so it would appear that an asynchronous step is the best way to implement this. However, I have questions about how to implement the StepExecution.start() method to start this asynchronous step. I can't find a (canonical) example -- does anyone have a pointer to such? Alternatively, a good development guide that shows how to implement this type of step?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1652f66e-46bc-4d94-8e30-dac8993e817c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jesse Glick]

On Fri, Apr 6, 2018 at 10:30 AM, Jeff Wilson <geek.m...@gmail.com> wrote:
> I can't find a (canonical) example -- does anyone
> have a pointer to such?

The `sleep` step is probably the simplest to understand. The most relevant bits:

https://github.com/jenkinsci/workflow-basic-steps-plugin/blob/7d42af656e09eda765befc8d2267064e1a46a824/src/main/java/org/jenkinsci/plugins/workflow/steps/SleepStep.java#L74-L128

[Jeff Wilson]

In this example, the important parts are in the setUpTimer() method, as are as follows:

task = Timer.get().schedule(new Runnable() {

   @Override public void run() {

      getContext().onSuccess(null);

   }

}, end - now, TimeUnit.MILLISECONDS);

So, we basically handle our own multi-threading in the start() command, as in this example?

Does that change if, within the StepExecution, I want to trigger a BuildStep asynchronously? 

[Jesse Glick]

On Fri, Apr 6, 2018 at 4:48 PM, Jeff Wilson <geek.m...@gmail.com> wrote:
> So, we basically handle our own multi-threading in the start() command, as
> in this example?

Yes, it is an asynchronous API—the Pipeline runtime tells your step
when to start, and you tell it when it is finished. What you do in the
meantime is your business. Perhaps nothing is happening in Jenkins—you
may simply be passively awaiting a webhook or something.

> Does that change if, within the StepExecution, I want to trigger a BuildStep
> asynchronously?

I do not think you want to do that. `BuildStep.perform` is synchronous
(blocking).

If the only thing you are doing is calling some synchronous Jenkins
APIs, and you cannot get around that easily, then you probably just
want to extend `SynchronousNonBlockingStepExecution`. The step will
then not survive a Jenkins restart—since the underlying API it is
calling could not either.

[Jeff Wilson]

Probably I'm using the wrong terminology. I have a long running build action that involves sending commands to a REST API, getting the results, parsing them and writing out artifacts that could be used by a later build step. This step can also break the build based on the output of the REST calls.  Obviously the Jenkinsfile would have to include things like the URI to the REST API and timeouts, etc. I'm simply looking for an example in case I'm missing out on built in functionality of the pipeline APIs, rather than doing the whole thing myself including the multithreading. The Jenkins pipeline APIs are fairly dense and under-documented (IMO), so I'm having a bit of difficulty getting started.

[Jeff Wilson]

And it would have to survive a Jenkins restart, I would think.

[Jesse Glick]

On Sun, Apr 8, 2018 at 12:06 PM, Jeff Wilson <geek.m...@gmail.com> wrote:
> I have a long running build action
> that involves sending commands to a REST API, getting the results, parsing
> them and writing out artifacts that could be used by a later build step.

Most similar functions use `SynchronousNonBlockingStepExecution` and
abort on Jenkins restart under the guess that the time window here is
pretty small and a restart in the middle is unlikely. Perhaps you can
do better if you want to put in the effort, but I am not sure how—if
you are making a REST call from the Jenkins JVM, then a restart will
of course break that TCP connection.

Simpler options:

· Use `SynchronousNonBlockingStepExecution` but have users wrap the
step in `retry` if they care enough about a single build passing. (I
have not personally tested the combination.)
· Do not bother with a step at all, or a Jenkins plugin for that
matter. Write a little Python program or whatever that does the same
thing and have people call it via `sh`/`bat`. Kills a whole flock of
birds with one little stone.

[Scott Hebert]

Hi,

I wrote a plugin (not battle-tested but well test-covered) that may be helpful for Jeff's requirements.

You can see the code here: https://github.com/jenkinsci/fedora-module-build-system-plugin

This step: https://github.com/jenkinsci/fedora-module-build-system-plugin/blob/master/src/main/java/com/redhat/fedora/buildsystem/mbs/pipeline/SubmitModuleBuildRequestStep.java, is the one that makes a REST api call 

I then have an example here of how to call it: https://github.com/jenkinsci/fedora-module-build-system-plugin/blob/master/src/test/resources/com/redhat/fedora/buildsystem/mbs/simpleSubmit.groovy

and a more complete example that includes a timeout and waitUntil here: https://github.com/jenkinsci/fedora-module-build-system-plugin/blob/master/src/test/resources/com/redhat/fedora/buildsystem/mbs/complete.groovy

Hope that helps.

Scott

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2L82Yi4SZ2zQ_mE%2BJ05ZMocgX7gERA9MEYRyQL1_b6Qg%40mail.gmail.com.