Best practices: many distinct jobs vs parameterized jobs

[Rich Schumacher]

Hey folks,

I'm using the Job DSL plugin to create CI/CD pipelines for a bunch of projects. The projects themselves are very similar and therefore the jobs for build and release will be nearly identical, differing only in stuff like Git repo to clone, email address for notifications, etc. I've been going back and forth about using distinct jobs for each project or to use a parameterized job for each step of the pipeline. The former leads to a proliferation of jobs that are nearly identical. The latter leads to jobs that might have 3-6 parameters and means it's difficult to use directly since you'll need to define all the parameters. The nice thing is that the Job DSL plugin makes either case pretty easy to support.

So my question is really about best practices and preferences: many distinct jobs vs a handful of parameterized jobs? Are there any non-obvious pro/cons that I'm missing?

Thanks in advance!

[wo...@anomalizer.net]

I follow the principle of "one job per codebase (git repo)". The problem with reusing the same job across multiple codebases is that it makes any and every trend data invalid for the job in question. By trend data, I mean things such as

• Build success rates
• Average build times
• Test success rates
• Test coverage data
• Static analysis trends
• Artifacts sizes
• ... etc. etc.

The way to think about what should run under a single job should be influenced by what kind of grouping would ensure that the trend data is meaningful.

The one place where this gets ambiguous is dealing with branches and forks for a give code base. There are situations wherein it makes sense to consolidate build trends across branches and orks into a single job and there are situations wherein it is best to track them separately. The only way to track it separately in jenkins 1.x is to have one job for each. Jenkins 2.x has a mechanism wherein you have have a single job (pipeline) across the branches and yet, track their statistics separately.

[Daniel Beck]

> On 13.08.2016, at 16:23, wo...@anomalizer.net wrote:
>
> The only way to track it separately in jenkins 1.x is to have one job for each. Jenkins 2.x has a mechanism wherein you have have a single job (pipeline) across the branches and yet, track their statistics separately.

The Pipeline suite of plugins is compatible with 1.x, but not available out of the box.

Also, there are one or two plugins that implement a similar "multi branch" behavior with freestyle based configuration.

[Ginga, Dick]

FWIW, I have implemented a 2-tier build approach where the first job prepares the source code workspace, pulling from whatever SCM. The second tier is the builder. It takes, as one parameter, that is the $WORKSPACE of the tier 1 job, performs all the building, testing, scanning in THAT job’s workspace. At completion, the tier 1 job can now report all is own successes, build times, test results, etc

 

This model enables several things:

 

1.       One core builder common to lots of branches

2.       building from separate branches

3.       re-building past builds (the tier 1 job populates a workspace from a particular commit)

4.       tier 1 jobs that produces Release Candidates can run extra step like labeling the source, creating checksums, virus scanning the workspace, etc

5.       even build developer workspaces as long as the location is network accessible

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1fd10095-47a2-4d3f-82fe-1b3f10cadd92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rich Schumacher]

Thanks for the great feedback, everyone! These are all good arguments and I've been convinced of the benefits of using distinct jobs for each project. Thankfully, the Job DSL plugin makes this trivial.

Cheers!