Suggestions for advanced git/jenkins build integration

[Owen B. Mehegan]

My team has 30+ git repos, which all use the same post-receive hook to trigger Jenkins build jobs when people push a commit to the 'master' branch (the hook maintains a mapping of repo -> Jenkins job). Work done in any other branch does not trigger a build automatically, but developers can build their branch using the Jenkins UI - 'branch' is a string parameter for all jobs.

All of our build jobs also have a list parameter of "deploy environment" which, if selected, causes the job to build a package and push it to an environment for test. But our workflow doesn't lend itself to continuous deploy (yet), so most people commit, then go to the Jenkins UI and do another build with the deploy option set if they want a deploy. This is obviously wasteful since it performs the build steps twice, unnecessarily, and requires a change of context when someone wants their commit to go to a test env.

People are asking me for an easier way to specify that a commit should be deployed. The ideal solution seems to involve mind-reading, which is not yet supported by any Jenkins plugin I've seen. I'm curious if anyone has come up with a good workflow for this. We have a couple of possible solutions, but haven't tried any yet:

* Some 'magic phrase' in the commit message, which is parsed by the post-commit hook. This pattern is sometimes used by code review or bug tracking systems, but I don't love the idea of adding this type of metadata to the commit text.
* Every repo has a branch named for each test environment (we have several), and if people want to deploy to that env, they merge to that branch and push.
* Every repo has a git tag named for each test environment, and people update the tag with 'git tag -f' when they want to trigger a deploy.
* Every repo contains a file which defines some parameters that are interpreted by the post-receive hook. Branches can then do different things. But at best this gives us branch-level resolution, and people are asking for commit-level. Also I dislike the idea of magic files in a repo driving build/deploy behavior. Seems like it's overloading things.
* People just stop complaining and use the damn Jenkins UI.

I would welcome any other suggestions, or thoughts about the ones we have come up with! Thanks in advance :)

[Stuart Montgomery]

Comments inline…

On May 21, 2013, at 3:15 PM, "Owen B. Mehegan" <omeh...@gmail.com> wrote:

My team has 30+ git repos, which all use the same post-receive hook to trigger Jenkins build jobs when people push a commit to the 'master' branch (the hook maintains a mapping of repo -> Jenkins job). Work done in any other branch does not trigger a build automatically, but developers can build their branch using the Jenkins UI - 'branch' is a string parameter for all jobs.

All of our build jobs also have a list parameter of "deploy environment" which, if selected, causes the job to build a package and push it to an environment for test. But our workflow doesn't lend itself to continuous deploy (yet), so most people commit, then go to the Jenkins UI and do another build with the deploy option set if they want a deploy. This is obviously wasteful since it performs the build steps twice, unnecessarily, and requires a change of context when someone wants their commit to go to a test env.

People are asking me for an easier way to specify that a commit should be deployed. The ideal solution seems to involve mind-reading, which is not yet supported by any Jenkins plugin I've seen. I'm curious if anyone has come up with a good workflow for this. We have a couple of possible solutions, but haven't tried any yet:

* Some 'magic phrase' in the commit message, which is parsed by the post-commit hook. This pattern is sometimes used by code review or bug tracking systems, but I don't love the idea of adding this type of metadata to the commit text.
* Every repo has a branch named for each test environment (we have several), and if people want to deploy to that env, they merge to that branch and push.

I prefer this approach, especially compared to the next idea of using tags. Branches conceptually lend themselves to being a "moving target", whereas tags (using `git tag -f`) seems less fitting since most of the time, a the ref a tag points to should not change.

And if you used branches and only want a subset of contributors to have access to deploy to certain environments, you could enforce branch-specific security requirements using a Git hook script.

* Every repo has a git tag named for each test environment, and people update the tag with 'git tag -f' when they want to trigger a deploy.
* Every repo contains a file which defines some parameters that are interpreted by the post-receive hook. Branches can then do different things. But at best this gives us branch-level resolution, and people are asking for commit-level. Also I dislike the idea of magic files in a repo driving build/deploy behavior. Seems like it's overloading things.
* People just stop complaining and use the damn Jenkins UI.

You could also use the Jenkins CLI or REST API to write a script to trigger a "deployment" job, if people wanted to avoid using the Jenkins UI.

I would welcome any other suggestions, or thoughts about the ones we have come up with! Thanks in advance :)

-- 
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Stuart

[Kevin Fleming (BLOOMBERG/ 731 LEXIN)]

You *do* want them to use the Jenkins UI, but what you could add to the mix is the Promoted Builds plugin. This allows someone (with appropriate permissions) to select an *existing* successful build and promote it into another chain of job(s), so that it won't have to be run again. Your deployment job could then be the target of the promotion operation, and extract the build artifacts from the promoted build so that it can deploy them.

----- Original Message -----
From: jenkins...@googlegroups.com
To: jenkins...@googlegroups.com
At: May 21 2013 18:15:42

My team has 30+ git repos, which all use the same post-receive hook to trigger Jenkins build jobs when people push a commit to the 'master' branch (the hook maintains a mapping of repo -> Jenkins job). Work done in any other branch does not trigger a build automatically, but developers can build their branch using the Jenkins UI - 'branch' is a string parameter for all jobs.

All of our build jobs also have a list parameter of "deploy environment" which, if selected, causes the job to build a package and push it to an environment for test. But our workflow doesn't lend itself to continuous deploy (yet), so most people commit, then go to the Jenkins UI and do another build with the deploy option set if they want a deploy. This is obviously wasteful since it performs the build steps twice, unnecessarily, and requires a change of context when someone wants their commit to go to a test env.

People are asking me for an easier way to specify that a commit should be deployed. The ideal solution seems to involve mind-reading, which is not yet supported by any Jenkins plugin I've seen. I'm curious if anyone has come up with a good workflow for this. We have a couple of possible solutions, but haven't tried any yet:

* Some 'magic phrase' in the commit message, which is parsed by the post-commit hook. This pattern is sometimes used by code review or bug tracking systems, but I don't love the idea of adding this type of metadata to the commit text.
* Every repo has a branch named for each test environment (we have several), and if people want to deploy to that env, they merge to that branch and push.

* Every repo has a git tag named for each test environment, and people update the tag with 'git tag -f' when they want to trigger a deploy.
* Every repo contains a file which defines some parameters that are interpreted by the post-receive hook. Branches can then do different things. But at best this gives us branch-level resolution, and people are asking for commit-level. Also I dislike the idea of magic files in a repo driving build/deploy behavior. Seems like it's overloading things.
* People just stop complaining and use the damn Jenkins UI.