general Pipeline help

[Brent Kilgore]

I am trying to set up a pipeline job that imitates what we have now in jenkins as a huge complicated mess of batch script steps.  I am having issues getting my foot in the door so to speak. I am running newest 2.0 build in windows

The first issue is i cant seem to find any good references for the pipeline groovy interface anywhere.  Specifically, the ONLY information I've seen for the git/scm/etc method are random snippets of code floating about.  the pipeline documentation on the jenkins site seems to just regurgitate the same hello world and/or some VERY specific scenarios that do not apply at all.  I need a comprehensive API documentation (parameters, return types, etc.)

Here is the workflow what I'm trying to implement.

1. git fetch
2. <git checkout localbranch... since im reusing a folder dedicated for this, this should no-op>
3. git merge upstream-branch
   
4. if merge conflicts, pause for manual resolution.  <--- this is the "killer feature" i was wanting from pipelines
5. git push origin localbranch
6. <perform incremental build by executing perl script>
7. <perform unit tests>
8. <email results>

The rest of my issues can probably be resolved with some documentation, but ill list them here anyway:

• I cannot get the git step to use credentials (ssh specifically). My git server DOES have a self-signed certificate, but i have verification turned off globally. If i omit the credentials and jenkins server as my windows user (which is setup to use ssh) the checkout/clone succeeds.
  
• I cant figure out how to make the git step do anything but checkout.  For step 4, I have had to resort to using bat steps to perform other git tasks.
  
• I cant figure out how to prevent the job from failing if the bat step fails.
• once resolved: how do i retrieve the return code from the bat step?
• I assume executing a perl script is also done with a bat step?

[Brent Kilgore]

sorry, forgot to paste my script in it's current state:

node {

  

  stage "Setup"

   git branch: DEVELOPMENT_BRANCH,

      url: GIT_REPO

   stage 'Refresh Shared Branch'

   echo "Refresh ${DEVELOPMENT_BRANCH} from ${UPSTREAM_BRANCH}"

   

   def ret = bat "git merge ${UPSTREAM_BRANCH}"

   echo ret

   if( ret )

   {

       input message: 'Merge Conflict detected', ok: 'Merge Resolved'

   }

   stage 'Build'

   echo "Incrementally building ${DEVELOPMENT_BRANCH} in ${GIT_WORKSPACE}"

   

   stage 'unittest'

   echo "unittesting"

[Craig Rodrigues]

Hi,

Thanks for describing your workflow and posting your script.
You want to do some non-trivial stuff in your script, but I think it is doable.

You mentioned that you are doing this on Windows, so you are correctly

invoking 'bat' to run commands.  You won't be able to get away from doing that.

You've probably already done this, but I strongly advise you to bookmark this link:

https://github.com/jenkinsci/pipeline-plugin/#getting-started

That is the best starting point for all the docs and examples

related to pipeline.  Pipeline is a work in progress, so things are not perfect,

but they are definitely improving every day.

The next thing I would advise you to do, is to take a look at this pipeline which I

wrote:

https://github.com/freebsd/freebsd-ci/blob/master/scripts/build/build-test.groovy

That is a non-trivial build -> test -> report results workflow.  It is Unix-based,

so it isn't Windows, but you'll get some ideas (hopefully!). 

Also, bookmark this link which I wrote which talks about global variables in pipelines:

https://groups.google.com/d/msg/jenkinsci-users/P7VMQQuMdsY/bHfBDSn9GgAJ

I'll take a whack at trying to answer some of your questions.

The 'git' pipeline step is basically only used to checkout stuff.

If you want to see all the parameters you can specify with it,

then go to the screen for configuring your job, and do:

Pipeline

      -> Snippet Generator

            -> Sample Step

                  -> git

                       -> Generate Groovy

The snippet generator will show you what parameters you can add to the git command.  Using the snippet generator will help you a lot in terms

of figuring out how to use pipeline steps.  It is not a comprehensive API reference, but it is better than nothing.

Now, in terms of running bat commands, I think you are on the right track.

If you want to take a look at how the bat command is implemented, look

at these two files:

https://github.com/jenkinsci/workflow-durable-task-step-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/BatchScriptStep.java
https://github.com/jenkinsci/durable-task-plugin/blob/master/src/main/java/org/jenkinsci/plugins/durabletask/WindowsBatchScript.java

It looks like the bat step runs the command you specify under "cmd /c",

and then echoes ERRORLEVEL after executing the command.

It is not clear to me if ERRORLEVEL is propagated back up through groovy in a way in which you can read it in your script.  I am assuming no.

If you go back to the build-test.groovy which I wrote, you will see that

I catch exceptions, since the Bourne shell step doesn't have a return value which is useful.  I think the WindowsBatchScript step also doesn't have a useful return value.  I think what you can try doing, is see if the

bat step throws an exception if the underlying Windows command fails.

If that is the case, you can try catching exceptions, like how I did in build-test.groovy.

Good luck!

--

Craig