Setting envrioment variables from plugin running in pipeline script

[andrew...@xtra.co.nz]

I have a plugin I've been updating to support pipeline script (with some help from this forum - thanks guys!).  During testing I discovered that the plugin is not setting environment variables when run from a pipeline script, but works fine in a freestyle job.

Can anyone suggest anything I might have overlooked?

Thanks in advance

  Andrew

Pipeline Script

This outputs an empty value for %BUILDMASTER_APPLICATION_ID%.

node {
    buildMasterSelectApplication applicationId: '1', deployableId: '1'
    bat 'echo BUILDMASTER_APPLICATION_ID = %BUILDMASTER_APPLICATION_ID%'
}

Code

This is how I'm setting the environment variable

public class SelectApplicationBuilder extends Builder implements SimpleBuildStep {

    @Override
    public void perform(Run<?, ?> run, FilePath workspace, Launcher launcher, TaskListener listener) throws InterruptedException, IOException {

        run.addAction(new VariableInjectionAction("BUILDMASTER_APPLICATION_ID", applicationId));
    } 

}

[Jesse Glick]

On Sun, Mar 5, 2017 at 3:35 AM, <andrew...@xtra.co.nz> wrote:
> run.addAction(new
> VariableInjectionAction("BUILDMASTER_APPLICATION_ID", applicationId));

Well what is this `VariableInjectionAction`?

See JENKINS-29144 / JENKINS-29537 / JENKINS-42499.

Currently `SimpleBuildStep` is not well suited to tasks which _return_
information to the build. You are probably better off creating a
Pipeline-specific `SynchronousNonBlockingStepExecution` or similar, so
that the script can assign the return value to a local variable,
rather than going through magical environment variable names—which
anyway will be confusing if called multiple times, and totally unsafe
if those calls are from `parallel` branches.

In other words there is a good chance that the design of the plugin as
conceived for freestyle projects just does not make sense for Pipeline
in general.

[andrew...@xtra.co.nz]

The SynchronousNonBlockingStepExecution has done the trick, however I now have two classes I need to support, one for the freestyle plugin and one for the pipeline plugin.  Is there any way to get these to reuse anything?  

There is alot of dupication from the jelly files through to databound setters, etc and that's before we even get to what the plugin does.

public class SelectApplicationBuilder extends Builder implements SimpleBuildStep, ResourceActivity {

    ...

}

 

public class SelectApplicationPipeline extends Step {

    ...

}

[Jesse Glick]

On Wed, Jun 20, 2018 at 1:26 AM <andrew...@xtra.co.nz> wrote:
> I now have two classes I need to support, one for the freestyle plugin and one for the pipeline plugin. Is there any way to get these to reuse anything?

Sure, most implementation can be reused.

> There is alot of dupication from the jelly files through to databound setters

If you have a complex configuration form, you can `st:include` a
common page. You can even factor the Java parts of the configuration
(incl. `FormValidation doCheckXXX` and the like) into a common
`Describable` which both the `Builder` and the `Step` use as a field,
via `f:property`, though to make the Pipeline syntax reasonable you
will need to override some methods in `StepDescriptor`, and then
JEP-201 and `job-dsl` will be unhappy…too much stuff relying on
reflection. Arguably clearer and less error-prone to just copy the
Java method signatures, using a common implementation for any method
bodies over a line long.

Since over a year has now passed, I will add that I do not know
offhand whether you can use a `Step` with a `String` return value as
the right-hand side of an expression in the `environment` section of
Declarative, or otherwise bind variables procedurally. You might
instead consider switching to a `SimpleBuildWrapper`, which _can_
define environment variables (consistently for freestyle or Pipeline),
and which additionally can be used from Declarative Pipeline. This
would show up in the Build Environment section for a freestyle config
form; for Pipeline, it might look something like (Scripted):

node {
withBuildMasterApplication(applicationId: '1', deployableId: '1') {

bat 'echo BUILDMASTER_APPLICATION_ID = %BUILDMASTER_APPLICATION_ID%'
}
}

or (Declarative):

pipeline {
agent {…whatever…}
options {
withBuildMasterApplication applicationId: '1', deployableId: '1'
}
stages {
stage('Main') {
steps {

[andrew...@xtra.co.nz]

I'd been considering the options idea but didn't know how to go about it, so keen to go down that route.

Do you have any idea why...

This pipleline:

pipeline {
  agent any
  options {
    withBuildMasterApplication applicationId: '1'

  }
  stages {
    stage('Main') {
      steps {
        bat 'echo BUILDMASTER_APPLICATION_ID = %BUILDMASTER_APPLICATION_ID%'
      }
    }
  }
}

gives this error:

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 4: Invalid option type "withBuildMasterApplication". Valid option types: [buildDiscarder, catchError, checkoutToSubdirectory, disableConcurrentBuilds, overrideIndexTriggers, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, waitUntil, withContext, withCredentials, withEnv, ws] @ line 4, column 5.
       withBuildMasterApplication applicationId: '1'

Using this code:

public class SelectApplicationBuildWrapper extends SimpleBuildWrapper implements ResourceActivity, BuildMasterSelectApplication {
    private String applicationId;

    @DataBoundConstructor
    public SelectApplicationBuildWrapper(String applicationId) {
        this.applicationId = applicationId;
    }

    public String getApplicationId() {
        return applicationId;
    }

    @Override
    public void setUp(Context context, Run<?, ?> run, FilePath workspace, Launcher launcher, TaskListener listener, EnvVars initialEnvironment) throws IOException, InterruptedException {
        context.env("BUILDMASTER_APPLICATION_ID", "1");
    }

    @Symbol("withBuildMasterApplication")
    @Extension
    public static final class DescriptorImpl extends BuildWrapperDescriptor {
        @Override
        public boolean isApplicable(AbstractProject<?, ?> item) {
            return true;
        }

        @Override
        public String getDisplayName() {
            return "Select BuildMaster Application";
        }
    }

    // ResourceActivity
    @Override
    public ResourceList getResourceList() {
        ResourceList list = new ResourceList();
        Resource r = new Resource("BuildMaster Application " + this.applicationId);
        list.w(r);

        return list;
    }

    // ResourceActivity
    @Override
    public String getDisplayName() {
        return "BuildMaster Application Resource";
    }
}

[Jesse Glick]

On Thu, Jun 21, 2018 at 6:09 AM <andrew...@xtra.co.nz> wrote:
> Do you have any idea why...
>

> options {
> withBuildMasterApplication applicationId: '1'
> }
>
> gives this error:

>
> WorkflowScript: 4: Invalid option type "withBuildMasterApplication". Valid option types: [buildDiscarder, catchError, checkoutToSubdirectory, disableConcurrentBuilds, overrideIndexTriggers, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, waitUntil, withContext, withCredentials, withEnv, ws] @ line 4, column 5.
> withBuildMasterApplication applicationId: '1'
>
> Using this code:
>
> public class SelectApplicationBuildWrapper extends SimpleBuildWrapper implements ResourceActivity, BuildMasterSelectApplication {

My guess is that Declarative is failing to properly list wrapper
metasteps (i.e., `SimpleBuildWrapper` via `CoreWrapperStep`). Clearly
it is detecting block-scoped non-meta `Step`s, since for example
`waitUntil` makes absolutely no sense in `options` so it must have
been listed automatically. Check an existing released wrapper metastep
like `xvnc`¹ and if that does not work either, file a bug report in
`pipeline-model-definition-plugin`. @abayer would likely know.

¹ https://github.com/jenkinsci/xvnc-plugin/blob/3bab24d0b92e3df33dc5e5fe8d94713e0095ebd5/src/main/java/hudson/plugins/xvnc/Xvnc.java#L45-L294

[Robert Sandell]

@Override
        public boolean isApplicable(AbstractProject<?, ?> item) {
            return true;
        }

A pipeline is not an AbstractProject. So I doubt it even works in scripted, it needs some other descriptor type I guess. Does it show up in the snippetizer?

/B

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3MxSvFAv_%3DdGn2F43PHj0an0jBeurXjtvVbAgC7id%3DFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Robert Sandell

Software Engineer

CloudBees, Inc.

E: rsan...@cloudbees.com

Twitter: robert_sandell

[Jesse Glick]

On Thu, Jun 21, 2018 at 10:41 AM Robert Sandell <rsan...@cloudbees.com> wrote:
> @Override
> public boolean isApplicable(AbstractProject<?, ?> item) {
> return true;
> }
>
> A pipeline is not an AbstractProject. So I doubt it even works in scripted, it needs some other descriptor type I guess.

Nope, this is just part of `BuildWrapperDescriptor`. There is no
`isApplicable(Job)` method. Pipeline ignores this method.

[andrew...@xtra.co.nz]

This is showing up in a freestyle job under the "Build Environment" section, but it is still not treated as a valid option in the declarative pipeline.

I've had a look at CheckoutToSubdirectory.java (first plugin that mentioned in error message that I could find the code for) and that is extending DeclarativeOption. Should I be extending from that class and will that show up in a freestyle job?

[andrew...@xtra.co.nz]

In addition to working in the freestyle build options section, it also works under steps as below.  The only hint I've seen in the code I've looked at so far is one class extending DeclarativeOption, but that would take me down maintaining two seperate plugins again.

pipeline {

  agent any

// Fails :-(  

//options {

  //  withBuildMasterApplication

  //}

  stages {

    stage('Main') {

      steps {

        withBuildMasterApplication {

           echo "id = $BUILDMASTER_APPLICATION_ID"

        }

      }

    }

  }

}

output "id = 11"

On Friday, June 22, 2018 at 3:17:50 PM UTC+12, andrew...@xtra.co.nz wrote:

[Jesse Glick]

As I initially suspected, this is a limitation of Declarative Pipeline.

https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/e1820b04c95eed794d4b407d3379beb8a858b85d/pipeline-model-definition/src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/generator/OptionsDirective.java#L117-L121

As you can see, it is only considered true block-scoped `Step`s, not
metasteps via `SimpleBuildWrapper` + `CoreWrapperStep`. Then again, it
also rejects those steps requiring a `Launcher` or `FilePath` (i.e.,
running inside a workspace), so even fixing this in Declarative would
require a core API change:

https://issues.jenkins-ci.org/browse/JENKINS-46175

In the meantime, it looks like you could define a block-scoped `Step`
and have it work in `options`, if that is important enough to outweigh
the bother of having a separate Pipeline-specific extension and a bit
of duplication of configuration machinery.

[andrew...@xtra.co.nz]

One last attempt...

My step class will now appear as an option, however I don't see any way of setting environment variables with it.  Is this possible?

I did try "def app = buildMasterSelectApplication2 applicationId: '1'" in the options but got an error message that this is not a valid option.

I have an alternative implementation using SimpleBuildWrapper that does set the environment variables. I'm happy with that as my fallback as I've spent enough time on this issue (and thanks for your advice), however the "global" options syntax does appeal.

 

pipeline {

  agent any

  options {

    buildMasterSelectApplication2 applicationId: '1'

  }

  stages {

    stage('main') {

      steps {

           echo "id = $BUILDMASTER_APPLICATION_ID"

      }

    }

  }

}

public class SelectApplicationPipeline extends Step {

    private String applicationId;

    @DataBoundConstructor

    public SelectApplicationPipeline(String deployableId) {

        this.applicationId = applicationId;

    }

    public String getApplicationId() {

        return applicationId;

    }

    @Override

    public StepExecution start(StepContext context) throws Exception {

        return new SelectApplicationPipeline.Execution(context, applicationId);

    }

    @Extension

    public static class DescriptorImpl extends StepDescriptor {

        @Override

        public Set<? extends Class<?>> getRequiredContext() {

            return new HashSet<>(Arrays.asList(Run.class)); 

// , Launcher.class, TaskListener.class));

        }

        @Override

        public String getFunctionName() {

            return "buildMasterSelectApplication2";

        }

        @Override

        public String getDisplayName() {

            return "BuildMaster: Select Application 2";

        }

        @Override

        public boolean takesImplicitBlockArgument() {

            return true;

        }

    }

    public static class Execution extends SynchronousNonBlockingStepExecution<BuildMasterApplication> implements BuildMasterSelectApplication {

        private final String applicationId;

        public Execution(StepContext context, String applicationId) {

            super(context);

            this.applicationId = applicationId;        }

        @Override

        protected BuildMasterApplication run() throws Exception {

            BuildMasterApplication a = new BuildMasterApplication();

            a.applicationId = Integer.valueOf(this.getApplicationId());

            a.releaseNumber = "1.2.0";

            // This doesn't appear to do anything

            this.getContext().get(Run.class).addAction(new VariableInjectionAction("BUILDMASTER_APPLICATION_ID", String.valueOf(a.applicationId)));

            return a;

        }

    }

}

public class VariableInjectionAction implements EnvironmentContributingAction {

    private String key;

    private String value;

    public VariableInjectionAction(String key, String value) {

        this.key = key;

        this.value = value;

    }

    @Override

    public void buildEnvVars(AbstractBuild<?, ?> build, EnvVars envVars) {

        if (envVars != null && key != null && value != null) {

            envVars.put(key, value);

        }

    }

    public String getDisplayName() {

        return "VariableInjectionAction";

    }

    public String getIconFileName() {

        return null;

    }

    public String getUrlName() {

        return null;

[Jesse Glick]

On Sat, Jun 23, 2018 at 12:15 AM <andrew...@xtra.co.nz> wrote:
> public static class Execution extends SynchronousNonBlockingStepExecution<BuildMasterApplication> implements BuildMasterSelectApplication {

This is wrong. For a block-scoped step, you would need to extend
`StepExecution` directly.

https://github.com/jenkinsci/workflow-step-api-plugin/#creating-an-asynchronous-step
https://github.com/jenkinsci/workflow-basic-steps-plugin/blob/7955d583910839d8eee469e92558567d6a46121d/src/main/java/org/jenkinsci/plugins/workflow/steps/EnvStep.java#L85-L89