Can I saperate the template and job definition in different files?

[link89]

Hi, I found that in jenkins-job-builder the template definition and the job defintion have to be in the same file, while in my case I have a monorepo that I hope the each projects can define their jobs in their project folders. Does jjb supports to define job this way? 

[Vsevolod Fedorov]

Hi,

No, they do not have to be in the same file. Not even in the same directory. We, for example, always have them in different places.

It seems like you have some other problem it your definitions do not work.

Vsevolod.

On 9/21/22 11:20, link89 wrote:

Hi, I found that in jenkins-job-builder the template definition and the job defintion have to be in the same file, while in my case I have a monorepo that I hope the each projects can define their jobs in their project folders. Does jjb supports to define job this way?  --
You received this message because you are subscribed to the Google Groups "jenkins-job-builder" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkins-job-bui...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkins-job-builder/23c7a334-c131-47dd-8186-7a3352d2bbbbn%40googlegroups.com.

[link89]

Hi, thank you for the quick reply. I don't know how to make it actually. I don't start to use template yet because I don't know how to put them in different file.

What I want to do is to put  the common part in template.yml, and then use it in job1.yml, job2.yml, etc.

[Andrew Grimberg]

As Vsevolod mentioned, templates and job definitions definitely don't
need to live in the same file.

If they did the global-jjb [0] library that Linux Foundation Release
Engineering maintains would not work at all!

If you would like some examples of how to do all of this see [1], [2],
[3], and [4] (I can provide more ;)

1 - 3 Are some of our more complex projects. 4 is less complex in that
it has very few "custom" templates outside of what global-jjb provides.
1 - 3 have a lot of custom templates. So you can see a mixture of
localized and library templates in both cases.

-Andy-

[0]
https://docs.releng.linuxfoundation.org/projects/global-jjb/en/latest/index.html
[1] https://github.com/opendaylight/releng-builder
[2] https://github.com/onap/ci-management
[3] https://github.com/fdio/ci-management
[4] https://github.com/o-ran-sc/ci-management

--
Andrew J Grimberg
Manager Release Engineering
The Linux Foundation

NOTICE: The Linux Foundation supports their employees with flexible work
hours. If you recieve mail from me outside of standard business hours
please be aware that I do not expect a response until the next standard
business day.

[link89]

Thank you for the sharing, those real world examples do help me a lot.

Now I totally understand how JJB works. I think it just treats configurations as a database. The command line tool merges items in different files into a list as a database, and uses name/id as foreign key to reference to other items. The command line tool will process those references and turn them into the final job definitions to create jobs in Jenkins.

It's really smart to use the idea of a database to make it possible to reuse configurations, as yaml doesn't provide such features. However, if you change the id/name of an item, or use a duplicate name/id accidentally,  or refactor names of variable placeholders, it will be horrible to ensure all of the existing configuration files are still valid. IMO lack of support in code refactoring and strict type checking would be a nightmare for large projects.

I think nowadays we may use cue-lang to achieve the same result more elegantly. It can use spec to define the schema of complicated items like #Job, #SCM, and reuse configuration by importing packages and data merge instead of id reference. I don’t know if there are already attempts to use such ideas. Personally I believe cue-lang + jjb would be a perfect combination for CI management. 

[Andrew Grimberg]

On 9/22/22 01:36, link89 wrote:
> Thank you for the sharing, those real world examples do help me a lot.

You're welcome. I hand those examples out to a lot of folks ;)

> It's really smart to use the idea of a database to make it possible to
> reuse configurations, as yaml doesn't provide such features. However, if
> you change the id/name of an item, or use a duplicate name/id
> accidentally,  or refactor names of variable placeholders, it will be
> horrible to ensure all of the existing configuration files are still
> valid. IMO lack of support in code refactoring and strict type checking
> would be a nightmare for large projects.

JJB has a verify command that can be used (techincally the test command
which does actually fully try to generate) if conflicting IDs are
present it will error out. Unfortunately it can be hard to figure out
_where_ the error is coming from sometimes. We strongly recommend the
use of SCM around it because of this! ;)

As for large projects. We support > 12,000 jobs across multiple Jenkins
servers using JJB and our global-jjb library. So, yes it can be hard to
deal with sometimes, it's actually really workable. We've been using it
like this for well over 5+ years now.

-Andy-

[Weihong Xu (link89)]

As for large projects. We support > 12,000 jobs across multiple Jenkins
servers using JJB and our global-jjb library. So, yes it can be hard to
deal with sometimes, it's actually really workable. We've been using it
like this for well over 5+ years now

Multiple Jenkins servers support is one of the reason I prefer jenkins-job-builder to Job DSL. But on the other hand groovy is much more powerful than yaml to express complicate configuration.

I have decided to use cue-lang to generate configuration and then use jenkins-job-builder to create/update jobs. Maybe  I can tell if it is a good choice or not in the future :)