Making a plugin compatible with pipeline: Exprting env vars

[Martin Weber]

Hi,

for AbstractBuildS, my plugin [1] parses a generated file, grabs a specific
value (usually /usr/bin/make on linux) from the file and exposes that value as
an environment variable named "CMAKE_BUILD_TOOL". This allows users to
reference the actual build tool in subsequent build steps that execute shell
scripts.

Is there a way with pipeline to accomplish something similar?

BTW, I use
((AbstractBuild<?, ?>) run).getEnvironments()
.add(Environment.create(exportedEnvVars))
to export the variable, is there a better way to do that?

Martin

[1] https://wiki.jenkins-ci.org/display/JENKINS/CMake+Plugin

--
Cd wrttn wtht vwls s mch trsr.

[Jesse Glick]

On Sat, May 21, 2016 at 2:49 PM, Martin Weber <fifteen...@gmail.com> wrote:
> for AbstractBuildS, my plugin [1] parses a generated file, grabs a specific
> value (usually /usr/bin/make on linux) from the file and exposes that value as
> an environment variable named "CMAKE_BUILD_TOOL". This allows users to
> reference the actual build tool in subsequent build steps that execute shell
> scripts.
>
> Is there a way with pipeline to accomplish something similar?

There are two options.

You could declare a dependency on `workflow-step-api` and define a
custom `Step` for this purpose, returning a `String` value.

Or, with only a core dependency, you could have a `SimpleBuildWrapper`
which would function as a build wrapper in either a freestyle project
or a Pipeline build, which could do some setup and/or teardown and
also set environment variables within the block.

Currently there is not a way for a `SimpleBuildStep` (non-block-scoped
step) to add global environment variable definitions to Pipeline
builds. I am hesitant to add one, since adjusting the build’s overall
environment is hazardous when you are dealing with multiple nodes
within one build: these variables are frequently specific to a single
machine (and it sounds like `CMAKE_BUILD_TOOL` would fall into that
category). Block-scoped steps more safely support complex builds such
as

def doTheBuild(platform) {
node(platform) {
wrap([$class: CmakeWrapper]) {
sh '$CMAKE_BUILD_TOOL clean world'
}
}
}
parallel linux32: {
doTheBuild('linux32')
}, linux64: {
doTheBuild('linux64')
}, win: {
doTheBuild('win')
}

where the value of the environment variable is distinct in each branch.

[Martin Weber]

Am Montag, 23. Mai 2016, 11:28:57 schrieb Jesse Glick:
> On Sat, May 21, 2016 at 2:49 PM, Martin Weber <fifteen...@gmail.com>
wrote:
> > for AbstractBuildS, my plugin [1] parses a generated file, grabs a
> > specific
> > value (usually /usr/bin/make on linux) from the file and exposes that
> > value as an environment variable named "CMAKE_BUILD_TOOL". This allows
> > users to reference the actual build tool in subsequent build steps that
> > execute shell scripts.
> >
> > Is there a way with pipeline to accomplish something similar?

Thank you for your explanation.

[...]

> Currently there is not a way for a `SimpleBuildStep` (non-block-scoped
> step) to add global environment variable definitions to Pipeline
> builds. I am hesitant to add one, since adjusting the build’s overall
> environment is hazardous when you are dealing with multiple nodes
> within one build: these variables are frequently specific to a single
> machine (and it sounds like `CMAKE_BUILD_TOOL` would fall into that
> category). Block-scoped steps more safely support complex builds such
> as

I wasn't aware the of the 'multiple nodes within one build' feature yet. Now I
totally agree, it would be hazardous to add `CMAKE_BUILD_TOOL` to global
environment. But adding it to a per-job/per-node environment would be fine,
but is there such an environment in jenkins?


Exporting `CMAKE_BUILD_TOOL` to global environment in a *freestyle* project is
just there to help solve some corner cases and allows users to call
"$CMAKE_BUILD_TOOL clean world" from a shell script.
But these cases are rare, since CmakePlugin itself can invoke
$CMAKE_BUILD_TOOL -- as a sub-step, without any shell.
See this the pipeline equivalent of './configure', 'make clean world',
'DESTDIR=${WORKSPACE}/artifacts/dir make install'

step([$class: 'CmakeBuilder',
buildDir: 'build',
buildType: 'Debug',
cleanBuild: true,
installationName: '3.5.2',
sourceDir: 'testprojects/C-subsrc/src',
steps: [
[args: 'clean world'],
[args: 'install', envVars: 'DESTDIR=${WORKSPACE}/artifacts/dir']
]
])


So I think the plugin`s help text should just mention that $CMAKE_BUILD_TOOL
is *not* available in *pipeline* jobs.

Martin

[Jesse Glick]

On Mon, May 23, 2016 at 2:45 PM, Martin Weber <fifteen...@gmail.com> wrote:
> I wasn't aware the of the 'multiple nodes within one build' feature yet. Now I
> totally agree, it would be hazardous to add `CMAKE_BUILD_TOOL` to global
> environment. But adding it to a per-job/per-node environment would be fine,
> but is there such an environment in jenkins?

Not really. There *is* a per-node environment, but it is not specific
to a build or even a job. Block scope from a wrapper is what we have.

> this the pipeline equivalent of './configure', 'make clean world',
> 'DESTDIR=${WORKSPACE}/artifacts/dir make install'
>
> step([$class: 'CmakeBuilder',
> buildDir: 'build',
> buildType: 'Debug',
> cleanBuild: true,
> installationName: '3.5.2',
> sourceDir: 'testprojects/C-subsrc/src',
> steps: [
> [args: 'clean world'],
> [args: 'install', envVars: 'DESTDIR=${WORKSPACE}/artifacts/dir']
> ]
> ])

You should not do this. `SimpleBuildStep` does not support the key
“durability” feature of Pipeline builds, so it is intended only for
very quick steps. Pending JENKINS-26055, it would be better for users
to run `sh` or `bat` steps (after using the `tool` step to establish
an executable path) and not use the Cmake plugin at all.

[Martin Weber]

Am Montag, 23. Mai 2016, 15:11:40 schrieb Jesse Glick:
> On Mon, May 23, 2016 at 2:45 PM, Martin Weber <fifteen...@gmail.com>
wrote:

[...]

> > this the pipeline equivalent of './configure', 'make clean world',
> > 'DESTDIR=${WORKSPACE}/artifacts/dir make install'
> >
> > step([$class: 'CmakeBuilder',
> >
> > buildDir: 'build',
> > buildType: 'Debug',
> > cleanBuild: true,
> > installationName: '3.5.2',
> > sourceDir: 'testprojects/C-subsrc/src',
> > steps: [
> >
> > [args: 'clean world'],
> > [args: 'install', envVars: 'DESTDIR=${WORKSPACE}/artifacts/dir']
> >
> > ]
> >
> > ])
>
> You should not do this. `SimpleBuildStep` does not support the key
> “durability” feature of Pipeline builds, so it is intended only for

Does that mean I should not have invented these 'sub-buildsteps'?
I did it since cmake is a *cross platform* buildscript generator and jenkins
users should not have to bother
-1) whether to invoke 'sh' or 'cmd.exe' to do the actual build (depending on
node OS).
-2) whether the actual build tool is just 'make' or one of '/usr/bin/gmake',
'/usr/bin/ninja', 'C:\MinGW\bin\mingw-32-make', 'C:\watcom\make', ... (My
fault, You missed that point, my groovy example above does not show the fact
that cmake has an option 'Generator' to control which *kind* of buildscript to
generate [1]. The $CMAKE_BUILD_TOOL variable in the plugin corresponds to the
given 'Generator' option.)

I see, the “durability” feature is of concern. Would it make sense to
implement this as a 'AbstractStepExecutionImpl'?



BTW: I was always wondering why jenkins has no build step to run a program
directly. Jenkins has 'Invoke shell' (on *nix) or 'Invoke Batch' (on windows),
thus forcing users to add OS-detection logic to their jobs. It could be so
simple to provide an 'Invoke Command' build-step: java.lang.ProcessBuilder
will do it right, if you give it "mvn' it will invoke 'mvn.bat' found on $PATH
on windows; if you give it "C:/whatever/make" it will invoke
"C:/whatever/make.exe".

Martin

[1] https://cmake.org/cmake/help/v3.0/manual/cmake-generators.7.html#id4

[Patrick Wolf]

That is captured here, Martin and on the list of things to do:

https://issues.jenkins-ci.org/browse/JENKINS-26055

 

Martin

[1] https://cmake.org/cmake/help/v3.0/manual/cmake-generators.7.html#id4
--
Cd wrttn wtht vwls s mch trsr.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/2671743.EiD8DKVuB7%40linux.

For more options, visit https://groups.google.com/d/optout.

--

[Jesse Glick]

On Tue, May 24, 2016 at 3:54 PM, Martin Weber <fifteen...@gmail.com> wrote:
> I see, the “durability” feature is of concern. Would it make sense to
> implement this as a 'AbstractStepExecutionImpl'?

Pending JENKINS-26055 it would be too hard to do that.

> I was always wondering why jenkins has no build step to run a program
> directly. Jenkins has 'Invoke shell' (on *nix) or 'Invoke Batch' (on windows),
> thus forcing users to add OS-detection logic to their jobs. It could be so
> simple to provide an 'Invoke Command' build-step

XShell plugin IIRC