Draft JEP: Configuration as Code

[nicolas de loof]

Hi,

Some of you who joined Contributor Summit at Jenkins World'17 already know about the resurrecting effort to provide configuration-as-code for Jenkins management (JENKINS-31094)

Both Praqma and CloudBees have been investigating on this topic last months, and have wrote prototypes to cover this need. As we learned about each other effort we decided to join forces, and make this public as a JEP proposal.

Our joined prototype is here : https://github.com/jenkinsci/configuration-as-code-plugin

We also wrote a document to explain our goals and design decisions. We used JEP template so we can now propose this for JEP acceptance (I guess this is the right place to do so)

We'll be happy to hear from you about this.

[Oleg Nenashev]

Hello,

Would it be possible to stage the plugin in the experimental update center?
I would like to evaluate it for my use-cases, but I need it in the experimental update center to bundle it into my Docker images.

Thanks in advance,
Oleg

вторник, 7 ноября 2017 г., 10:49:59 UTC+1 пользователь nicolas de loof написал:

[Jesse Glick]

On Mon, Nov 20, 2017 at 2:52 AM, Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> I would like to evaluate it for my use-cases, but I need it in the
> experimental update center to bundle it into my Docker images.

Maybe you should just change your demo build style to not depend on
the update center. See `workflow-aggregator-plugin/demo/` for an
example.

[eric.s...@docker.com]

Saw all of the talk about this from the post-FOSDEM hackday work and I'm very interested in presenting about it at the DFW JAM this month.   The repo links to a slide deck presentation, are those slides up to date and something I might use for a JAM presentation?   If so, is there a recording of any talks they were used at that I could watch?

Thanks,

Eric S.

[nicolas de loof]

This slides deck is indeed expected to be used for JAM presentations

unfortunately it hasn't been used for a recorded session yet

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5335be02-ae53-4e01-bb87-0088d3351b42%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Eric Smalling]

Sounds like I may need to be the first to try then! :) 

On Wed, Feb 14, 2018 at 2:44 PM, nicolas de loof <nicolas...@gmail.com> wrote:

This slides deck is indeed expected to be used for JAM presentations

unfortunately it hasn't been used for a recorded session yet

2018-02-14 21:30 GMT+01:00 eric.smalling via Jenkins Developers <jenkinsci-dev@googlegroups.com>:

Saw all of the talk about this from the post-FOSDEM hackday work and I'm very interested in presenting about it at the DFW JAM this month.   The repo links to a slide deck presentation, are those slides up to date and something I might use for a JAM presentation?   If so, is there a recording of any talks they were used at that I could watch?

Thanks,

Eric S.

On Tuesday, November 7, 2017 at 3:49:59 AM UTC-6, nicolas de loof wrote:

Hi,

Some of you who joined Contributor Summit at Jenkins World'17 already know about the resurrecting effort to provide configuration-as-code for Jenkins management (JENKINS-31094)

Both Praqma and CloudBees have been investigating on this topic last months, and have wrote prototypes to cover this need. As we learned about each other effort we decided to join forces, and make this public as a JEP proposal.

Our joined prototype is here : https://github.com/jenkinsci/configuration-as-code-plugin

We also wrote a document to explain our goals and design decisions. We used JEP template so we can now propose this for JEP acceptance (I guess this is the right place to do so)

We'll be happy to hear from you about this.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5335be02-ae53-4e01-bb87-0088d3351b42%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/6TjlxEqHUEs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznzO9maEP0_e1%3D%2BxxkaYF%2Bskte%3DxXZAUdq5ybNpONGgng%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

--

Eric Smalling

Solution Architect | Docker Inc.

eric.s...@docker.com

682.233.3742

   @ericsmalling

[Ewelina Wilkosz]

Nice! good luck Eric, and if you need some clarification I can try to explain via mail or short online meeting

When is the meetup?

On Wednesday, February 14, 2018 at 9:54:04 PM UTC+1, Eric Smalling wrote:

Sounds like I may need to be the first to try then! :) 

On Wed, Feb 14, 2018 at 2:44 PM, nicolas de loof <nicolas...@gmail.com> wrote:

This slides deck is indeed expected to be used for JAM presentations

unfortunately it hasn't been used for a recorded session yet

2018-02-14 21:30 GMT+01:00 eric.smalling via Jenkins Developers <jenkin...@googlegroups.com>:

Saw all of the talk about this from the post-FOSDEM hackday work and I'm very interested in presenting about it at the DFW JAM this month.   The repo links to a slide deck presentation, are those slides up to date and something I might use for a JAM presentation?   If so, is there a recording of any talks they were used at that I could watch?

Thanks,

Eric S.

On Tuesday, November 7, 2017 at 3:49:59 AM UTC-6, nicolas de loof wrote:

Hi,

Some of you who joined Contributor Summit at Jenkins World'17 already know about the resurrecting effort to provide configuration-as-code for Jenkins management (JENKINS-31094)

Both Praqma and CloudBees have been investigating on this topic last months, and have wrote prototypes to cover this need. As we learned about each other effort we decided to join forces, and make this public as a JEP proposal.

Our joined prototype is here : https://github.com/jenkinsci/configuration-as-code-plugin

We also wrote a document to explain our goals and design decisions. We used JEP template so we can now propose this for JEP acceptance (I guess this is the right place to do so)

We'll be happy to hear from you about this.

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5335be02-ae53-4e01-bb87-0088d3351b42%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/6TjlxEqHUEs/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznzO9maEP0_e1%3D%2BxxkaYF%2Bskte%3DxXZAUdq5ybNpONGgng%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Eric Smalling]

The next meetup is a week from today, not sure I can pull of researching enough on this to present on it that fast though.  Next month is already booked with A.Bayer coming in town so the next opportunity would be some time in mid-late April.

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/6TjlxEqHUEs/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznzO9maEP0_e1%3D%2BxxkaYF%2Bskte%3DxXZAUdq5ybNpONGgng%40mail.gmail.com.

[Oleg Nenashev]

Hi Eric,

You can also find my old slides about Jenkins Config Management with Groovy Scripts and Configuration-as-Code Plugin here: https://speakerdeck.com/onenashev/moscow-jenkins-meetup-number-3-groovy-hooks-and-configuration-as-code-plugin.

The slides have been created in November when the plugin was in very early stage, but you may get some info there.

Can send you PowerPoint version if needed.

Hopefully it helps,

Oleg

2018-02-15 15:17 GMT+01:00 'Eric Smalling' via Jenkins Developers <jenkin...@googlegroups.com>:

The next meetup is a week from today, not sure I can pull of researching enough on this to present on it that fast though.  Next month is already booked with A.Bayer coming in town so the next opportunity would be some time in mid-late April.

On Feb 14, 2018, at 2:44 PM, nicolas de loof <nicolas...@gmail.com> wrote:

This slides deck is indeed expected to be used for JAM presentations

unfortunately it hasn't been used for a recorded session yet

2018-02-14 21:30 GMT+01:00 eric.smalling via Jenkins Developers <jenkinsci-dev@googlegroups.com>:

Saw all of the talk about this from the post-FOSDEM hackday work and I'm very interested in presenting about it at the DFW JAM this month.   The repo links to a slide deck presentation, are those slides up to date and something I might use for a JAM presentation?   If so, is there a recording of any talks they were used at that I could watch?

Thanks,

Eric S.

On Tuesday, November 7, 2017 at 3:49:59 AM UTC-6, nicolas de loof wrote:

Hi,

Some of you who joined Contributor Summit at Jenkins World'17 already know about the resurrecting effort to provide configuration-as-code for Jenkins management (JENKINS-31094)

Both Praqma and CloudBees have been investigating on this topic last months, and have wrote prototypes to cover this need. As we learned about each other effort we decided to join forces, and make this public as a JEP proposal.

Our joined prototype is here : https://github.com/jenkinsci/configuration-as-code-plugin

We also wrote a document to explain our goals and design decisions. We used JEP template so we can now propose this for JEP acceptance (I guess this is the right place to do so)

We'll be happy to hear from you about this.

-- 
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5335be02-ae53-4e01-bb87-0088d3351b42%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/6TjlxEqHUEs/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANMVJznzO9maEP0_e1%3D%2BxxkaYF%2Bskte%3DxXZAUdq5ybNpONGgng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/6TjlxEqHUEs/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5E6C9341-41A0-40D5-B479-EF231AE900C0%40docker.com.

[Camila Naranjo]

Hi Nicolas,

In this moment I'm trying to use the plugin configuration as code in an Jenkins instance previously configurated. I want to know if is possible that Jenkins recognize the jenkins.yaml when Jenkins first has some configurations, and if it is possible how I can do it, because I was trying but jenkins doesn't recognize the file.

Excuse me if this is not the right space for the question.

Thanks

[nicolas de loof]

Sure it can (and yes this is the right place to ask)

Which version of the plugin are you running ? Where is your jenkins.yaml file located on disk ?

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8f98982d-845c-45ab-bef0-c55f181aa56d%40googlegroups.com.

[Camila Naranjo]

Thanks for answer me :)

The version's plugin is 9.0-snapshot and the jenkins.yaml is on disk

[nicolas de loof]

On disk for sure, but where compared to your Jenkins home ? You might have to pass environment variable to select a non default location

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/45642d2b-dc2d-453e-be99-07e865f9e9b1%40googlegroups.com.

[Camila Naranjo]

Yes, on jenkins and in the machine I configured CASC_JENKINS_CONFIG  to the jenkins file.

[nicolas de loof]

on JENKINS_URL/configuration-as-code does plugin report this file being identified as configuration source ?

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/195903ad-2ad1-4d9b-bd12-b8acb95649d9%40googlegroups.com.

[Camila Naranjo]

No :(

[nicolas de loof]

then double check environment variable is well set for jenkins master (can use groovy script console to run :

System.getenv()

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/bae76a19-f9cf-497b-9bcb-9d74cd8051bc%40googlegroups.com.

[Camila Naranjo]

I get this:

 {PATH=/sbin:/usr/sbin:/bin:/usr/bin, GEM_PATH=file:/var/opt/jenkins/jenkins-labo/jenkins/plugins/ruby-runtime/WEB-INF/lib/stapler-jruby-1.209.jar!/gem, SHELL=/bin/bash, NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat, LOGNAME=jenkins, XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt, PWD=/, USER=jenkins, LANG=en_US.UTF-8, SHLVL=2, HOME=/home/jenkins, _=/usr/bin/java}

[nicolas de loof]

ok, so you're wrong when you say " I configured CASC_JENKINS_CONFIG  to the jenkins file"

you probably configured some env variable, but not set for user running jenkins process.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/160748ea-39d2-4540-9e99-00cff305d375%40googlegroups.com.

[Camila Naranjo]

Sorry for the question but I am newbie about configuration, can you explain me how can I set the enviroment for jenkins process.

[nicolas de loof]

depends how you have installed jenkins, which OS you run, etc

maybe simpler for you to have jenkins.yaml in your JENKINS_HOME as default location

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/f866070b-2250-4242-8b0b-6d62783925d4%40googlegroups.com.

[Sverre Moe]

Would Configuration as Code allow us to have job configuration in VCS?

Is it possible with JCasC to get job configuration as well, or is the JobDSL plugin the way to go for that?

Our Pipeline jobs are pretty much identical, the only thing that differs are the SCM URL, and Browser URL.

I have written a number of Groovy scripts to bulk update Jenkins jobs. Lately this has given me problems with Multibranch pipeline jobs.

Updating the configuration through groovy scripts caused the configuration to be "invalid", and the next Branch Indexing did a Takeover and all branches where triggered.

Having job configuration as code in files would help. Not just with changing the job configuration, but also when we need to add new projects. Simply adding a line in the job configuration, checking it in to Git, and having Jenkins add the Multibranch Pipeline job.

[nicolas de loof]

No, casc do only support initial job setup. If you adopted Jenkinsfile you mostly don't need to setup anything in job anyway, just a top level git organization.

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/7d9d4135-9684-4cc5-95f9-d9145c7bd03a%40googlegroups.com.

[Sverre Moe]

Jenkinsfile alone does not help.

We do not have a top level git organization, not using either GitHub, GitLab or Bitbucket. Just plain simple Gitolite.

1) Adding a new Multibranch Pipeline project in Jenkins.

It happens occasionally we need to add a new git repository to build, or maybe we want to remove one project or a branch or two.

Would have been nice to define all jobs in a configuration file. Simply adding a new entry and Jenkins would create a new Multibranch Pipeline.

I could inspect that configuration file to see if we where missing some projects, instead of searching in the Jenkins UI.

2) Updating the configuration for 200 Multibranch Pipeline projects.

Initially I created all projects with a Groovy script when we started to use Pipeline. A lot has happened since then.

* We needed to add SCM Browser configuration to all jobs as we didn't do that initially.

* We needed to add Advanded Clone Behaviour, to fetch tags.

In each of these cases I created a Groovy script to do this and update all our Multibranch Pipeline projects and had to study the API Javadocs or the source code to find how to write those Groovy scripts.

The worst problem I encountered last week was when I change the configuration through Groovy script it caused the configuration to become "invalid", and a Takeover occurred at next Branch Indexing causing all branches to start building.

What about next time we need to change the configuration again, for say adding Discover Tags to the SCM.

Having the list of jobs and their configuration in a file in VCS would give us much greater control.

If a job configuration didn't work, I could just revert back to the previous configuration.

I do not want that configuration in a Jenkinsfile, duplicated within 200+ git repositories. Change one tiny thing would be a nightmare with massive merge requests for them all. It is the reason why our Jenkinsfile are as spares as possible and all logic is in the Shared Library scripted pipeline.

I could perhaps write an configuration change within the Shared Pipeline, but it would not take affect for all projects at once, only when a Branch Indexing started a new build.

[Jesse Glick]

On Sat, Jun 23, 2018 at 11:23 AM Sverre Moe <sverr...@gmail.com> wrote:
> Adding a new Multibranch Pipeline project in Jenkins.
> It happens occasionally we need to add a new git repository to build, or maybe we want to remove one project

You would need to use Job DSL or the equivalent to manage the list of
repository folders.

> or a branch or two.

Use Job DSL to update the branch include/exclude settings.

> I could perhaps write an configuration change within the Shared Pipeline, but it would not take affect for all projects at once, only when a Branch Indexing started a new build.

If you are referring to things defined in the `properties` step, this
is a known limitation of the multibranch system. There is an RFE filed
(in `job-dsl-plugin` with `pipeline` label, which I cannot look up for
you right now because JIRA is dead) which outlines some thoughts for a
future system which would blend the flexibility of Job DSL for
large/complex systems with the ease of use of multibranch for the most
common setups.

For purposes of JEP-201, I think the takeaway is that while static
YAML configuration of top-level items is a desirable option, there
will always be situations where that is inadequate and we will need
something closer in spirit to Job DSL (though preferably redesigned to
run as an external “function” of some kind).