Best practices for develop/release branch model with pipeline

[Sathyakumar Seshachalam]

New to Jenkins pipeline.

My process is that developers work off of develop branch (Feature branches and merges of-course).

At any point in time, a release branch is branched off of develop  and then deployed to a stage environment, Once Accepted/approved, the same release branch is deployed into prod. (All immutable deployments).

So am looking at atleast two stages that are only conditionally and manually entered  - stages being deploy to stg, deploy to prod and condition being the branch prefix. (Each stage will have build steps like deploy binaries, launch, run functional tests etc.,) and an automatic stage that is triggered only once per day (nightly) with build steps like deploy binaries, lunch, run and tear down).

Is this kind of a workflow feasible with pipelines. If yes, Are there any recommendations/suggestions/pointers. 

Thanks,

Sathya

[Michael Lasevich]

I am not sure the stages you are talking about are same as what Jenkins Pipelines calls stages.

Jenkins, at its core, is a job server. In Pipelines, a stage is a segment of a job. Stages of a build job would be something like "Build Binaries" or "Upload Build Artifacts" - something that is part of one logical job. What you are talking is a deployment process which is really a separate job from a build job, and not really a "stage" of build. 

So, my approach would be (and is, in some cases):

* Set up a Pipeline build for the develop branch 

* Make sure the build job archives either deployment artifact(s) or pointer to them - something that can be used for deployment.

* Set up a separate deployment job (can also be Pipeline) that takes in parameters for a build run and target environment (stage, QA, UA, PreProd, Production, whatever), and grabs artifacts/pointers from the selected run and performs a deployment

Now, if you want to get fancy, you make that first "build" job a MultiBranch job that builds both develop and some versions of the feature branches (I've used /feature/build/* pattern) and then modify the selection of the job run to select from multiple branches (need to write a Groovy based Parameter selector for that) - and now you can deploy builds from feature branches for testing BEFORE they are merged into develop

HTH,

-M

[Sathyakumar Seshachalam]

Thanks,

> And then modify the selection of the job run to select from multiple branches (need to write a Groovy based Parameter selector for that) - and now you can deploy builds from feature branches for testing BEFORE they are merged into develop

If there are any examples / code snippets on how to do this, will greatly help me. 

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/bLO1Y2ylLq8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/8cfea0f2-bd05-4aac-ab89-21ce5cf21cda%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Lasevich]

Quick and dirty version of a job lister that takes two parameters "job" (multibranch job name) and "old_job" (non-mb job name) and produces output that can be used by Active Choices Parameter to present a list of builds to select. Value is a | delimited set of values, so you may want to parse it something like this:

deploy_info = BuildToDeploy.split('\\|');
deploy = [
    version: deploy_info[0],
    id: deploy_info[1],
    branch: java.net.URLDecoder.decode(deploy_info[2], "UTF-8"),
    time: new Date(Long.valueOf(deploy_info[3])),
    url: deploy_info[4],
    project: deploy_info[5]
]

Here is the script:

import hudson.model.*

import hudson.node_monitors.*

import hudson.slaves.*

import java.util.concurrent.*

import groovy.time.*

now=new Date()

jenkins = Hudson.instance

search_mb=[job]

search_non_mb=[old_job]

full=[:];

builds=[:];

for (item in jenkins.items){

  if (item.name in search_mb){

    for (job in item.getAllJobs()){

      for (build in job.getBuilds()){

          full[build.displayName] = [ 

            name: build.displayName, 

            id: build.id,

            branch: job.name, 

            job_name: item.name, 

            time: build.getTime(), 

            ts: build.getTime().getTime(),

            url: build.getUrl(),

            building: build.building,

            failed: (build.result == hudson.model.Result.FAILURE),

            project: "${item.name}/${job.name}"

            

          ];

        }

  }

    

  }else if (item.name in search_non_mb){

    job = item

    

    for (build in job.getBuilds()){

      def parameters = build?.actions.find{ it instanceof ParametersAction }?.parameters

      branch=job.name

      parameters.each{

        if (it.name == "branch"){

          branch = it.value

        }

      }

      

      full[build.displayName] = [ 

        name: build.displayName, 

        id: build.id,

        branch: branch, 

        job_name: item.name, 

        time: build.getTime(), 

        ts: build.getTime().getTime(),

        url: build.getUrl(),

        building: build.building,

        failed: (build.result == hudson.model.Result.FAILURE),

        project: "${item.name}"

        

      ];

    }

  }

}

full = full.sort { -it.value.ts }

for (build in full){

  item = build.value

  branchDecoded=java.net.URLDecoder.decode(item.branch, "UTF-8");

  TimeDuration duration = TimeCategory.minus(now, item.time)

  if (duration.days >= 7) {

    duration=duration.minus(new TimeDuration(duration.hours, duration.minutes, duration.seconds, duration.millis))

  } else if (duration.days >= 1) {

    duration=duration.minus(new TimeDuration(0,duration.minutes,duration.seconds, duration.millis))

  }else if (duration.hours >= 1) {

    duration=duration.minus(new TimeDuration(0,0,duration.seconds, duration.millis))

  } else {

    duration=duration.minus(new TimeDuration(0,0,0, duration.millis))

  }

  timestamp = "${duration} ago"

  value="${item.name}|${item.id}|${item.branch}|${item.ts}|${item.url}|${item.project}"

  display="${item.name} (${branchDecoded}, ${timestamp})"

  if (item.building){ 

    display += " **BUILDING**" 

  } else if (item.failed){

    display += " **FAILED**" 

  }

  builds[value] = display

}

return builds

To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

[Graham Hay]

This is interesting, it's something I've been struggling with while converting our current system over to the new world (I really like the stage view!).

We currently have a "pipeline" of freestyle jobs, that pass artifacts down the line. Build (and test) -> Deploy to stage -> Deploy to prod. The last build uses pinned artifacts ("keep forever"), which allows us to rollback by unpinning the latest build. This is something I'd like to keep.

The one thing I don't understand is where I would define the deploy jobs? Isn't the whole point that everything is now in the Jenkinsfile, and under version control? Also, I don't seem to be able to point my existing freestyle job at the new pipeline build job, to retrieve artifacts. So what am I missing?

Thanks,

Graham

[Michael Lasevich]

There are lots of answers to this and I am not going to pretend I know the "right" answer for you, but here are a few things you may want to consider when figuring things out:

* In this "pipeline" of jobs you describe, only the initial build has access to the source repository - rest use artifacts from other jobs to perform the next phase of the process

* Deployment process is typically consistent of two parts - a part that is specific to the your environment/organization but not specific to release you are deploying and part that is specific to the particular version you are deploying. While the line is often blurry, you are much better off not mixing them - and keeping the non-deployment specific bits in a separate repo. That is your Deployment Job, which is independent of the particular version you are deploying. So, changes in org/env specific process are pretty much independent of the changes in your project or changes in how to deploy your artifacts  - so it makes no sense to version them together and store in one repo

* The Global Libraries are a pretty handy place to store common workflows - like your global environment/org deployment process. Alternatively you can just run an independent repo for that part as it is an independent project/codebase

* Deployment artifact is a good place to include release specific deployment instructions - there is nothing stopping you from creating an additional deployment artifact that contains not only your code build, but also instructions on specific steps to deploy this particular version. You can even package those instructions as a pipeline script which gets executed from your other job. You can even have multiple jobs for different phases of the deployment process.

That should hopefully put you on the right path for your needs.

-M