Mass Jenkins job updates

[Bob Bick]

Hi,

 

We have many Jenkins jobs defined that were copied from a “template” Jenkins job. Now, I discovered that I’d like to add a new build step to most (not all) of the jobs.

 

There seems to be several ways to modify the jobs:

1)      Brute force – Edit each job by clicking the Jenkins “Job Configure” button for each job. (Yuck)

2)      Groovy Script Console - This has the downside that I could screw up the configurations if I have a bug in the groovy script.

3)      Job DSL Plugin

4)      Jenkins Enterprise Templates – Not really sure whether this can even be done.

5)      Manually edit config.xml files

6)      Other???

 

Before investing time in coming up with a solution, I would like to know how other people have addressed mass updates to Jenkins jobs; and what worked well (or not so well).

 

Any feedback/guidance would be much appreciated.

 

Thanks,

 

Bob

*** *** ***
This message contains information which may be confidential and privileged. Unless you are the addressee (or authorized to receive for the addressee), you may not use, copy or disclose to anyone the message or any information contained in the message. If you have received the message in error,  please advise the sender by reply e-mail and delete the message.

[teilo]

I use option 4 - Jenkins enterprise

This won't be a drop in solution for you as your jobs need to be templatized before you can update them by just tweaking the template. You could do this with some groovy but that has drawbacks if you mess up your groovy.

But Jenkins enterprise has other benefits (at least to me) that the initial pain of moving jobs too templates was worth it.

Configuration slicing didn't do it to me, groovy can be an issue if you only have a subset of your jobs share the same config - as you have to maintain a whitelist/blacklist.

The other option is wget and some sed/perl/... scripting

[Matthew...@diamond.ac.uk]

I often find the low-tech approach works. For simple bulk changes I edit the config.xml files using the bash command line (this is Linux, obviously). Something like this:

 

cd /path/to/Jenkins-home/jobs/

find -mindepth 1 -maxdepth 1 -type d -name "jobname.pattern.of.interest* " | sort | xargs -i --verbose sed --in-place 's#delete_materialize_on_completion=true</propertiesContent>#delete_materialize_on_completion=false\nbuckminster_result_delete=true</propertiesContent>#g' {}/config.xml

 

You need to check the result carefully, of course. More complex changes could probably be done using a simple python script.

 

Hope that helps

Matthew

 

From: jenkins...@googlegroups.com [mailto:jenkins...@googlegroups.com] On Behalf Of Bob Bick
Sent: 20 May 2013 22:45
To: jenkins...@googlegroups.com
Subject: Mass Jenkins job updates

 

Hi,

 

We have many Jenkins jobs defined that were copied from a “template” Jenkins job. Now, I discovered that I’d like to add a new build step to most (not all) of the jobs.

 

There seems to be several ways to modify the jobs:

1)      Brute force – Edit each job by clicking the Jenkins “Job Configure” button for each job. (Yuck)

2)      Groovy Script Console - This has the downside that I could screw up the configurations if I have a bug in the groovy script.

3)      Job DSL Plugin

4)      Jenkins Enterprise Templates – Not really sure whether this can even be done.

5)      Manually edit config.xml files

6)      Other???

 

Before investing time in coming up with a solution, I would like to know how other people have addressed mass updates to Jenkins jobs; and what worked well (or not so well).

 

Any feedback/guidance would be much appreciated.

 

Thanks,

 

Bob

 

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 

[CHAVANNE Robert]

Hi Bob,

For this I have used essentially configuration slice plugin and jenkin cli.
For the last one, using batch command in a shell you can get all config.xml
(get-job), modify them and then update the job using update-job command. Or
you can use the template in the same way to generate different configuration
and create/update your jobs.

Regards,

Robert

[Bob Bick]

Thank you for the response.

Just so I understand what you are saying... If you create a job based on a template, and then you make a change to the template, all jobs are automatically updated based on the template? I have not tried this; however, I would assume that you'd need to modify the template and then re-run the transformation with the exact same parameters to re-create each job. If so, that seems like a hassle because you'd need to re-run the transformation for many jobs which would be inefficient.

Do I understand that correctly?

Thanks,

Bob

-----Original Message-----
From: jenkins...@googlegroups.com [mailto:jenkins...@googlegroups.com] On Behalf Of teilo
Sent: Tuesday, May 21, 2013 4:03 AM
To: jenkins...@googlegroups.com
Subject: Mass Jenkins job updates

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[teilo]

Hi Bob,

   "Just so I understand what you are saying... If you create a job based on a template, and then you make a change to the template, all jobs are automatically updated based on the template?"

Yes - if your jobs are based on that template then they are all automatically updated.

    "I have not tried this; however, I would assume that you'd need to modify the template and then re-run the transformation with the exact same parameters to re-create each job. "

That's not how the plugin works - when you create a job you no longer create a "FreeStyle" job or a "Maven2" job, you create a "Mytemplate" job.  The only thing the job will ask you for is the parameters defeined by the template, and these are stored in the job.  The template then converts these parameters using the definition stored in the template.

  "If so, that seems like a hassle because you'd need to re-run the transformation for many jobs which would be inefficient. "

That would be ineficient - which is why it doesn;t work like that.  

Basically, if your job has been created from a template then once you update the template then the job is automatically updated.

However your jobs are not created from a tempalte, so you would first have to create a template and then convert (recreate?) your jobs to be based on the template.

/James

[Bob Bick]

Thanks James, that clears it up!

 

I guess a template could not add a new parameter. Is that correct? Or, maybe there is a way to define a default value that all jobs would get?

 

Bob

[teilo]

You can add new parameters, so long as you handle the fact that the parameter may be null in the transformer used by the template.

There's a bit about this in my presentation from the Jenkins User event in London, and the online docs from cloudbees should give you more details of this.

Others have pointed out there may be other ways to solve the same issue with various advantages/disadvantages.

[Bob Bick]

Thanks James.

 

I am trying out the CloudBees template approach using Groovy template transformations.

 

Overall, it seems nice; however, it is not clear how to access attributes in Groovy code. The CloudBees doc shows this trivial example of adding Groovy code:

 

<foo>

  <%

    for (int i=0; i<100; i++) {

      if ((i%2)==0) {

  %>

  <data>${i}</data>

  <%

      }

    }

  %>

</foo>

 

I’d like to use an attribute whose value is provided by the user when the job is instantiated. Here is a simplified example of what I am trying to do:

<foo>

  <%

      def x = “a” + $someAttribute

  %>

  <data>${x}</data>

</foo>

 

Unfortunately, the above example does not work. Does anyone know if it is possible to access attribute values in the Groovy template code?

 

Thanks in advance.

[teilo]

its just someAttribute not $someAttribute (although my groovy isn't the best and $someAttribute could also be correct syntax :/ )

You need to make sure that the template creates valid XML that represents a job that is known to Jenkins.

Might be best to start by loading a an existing job and then adding a single parameter and then changing the groovy transform.  

If not the error you get back although cryptic and in martian does give you somewhat of a clue as where to look.

I wanted to open source our templates, but I have more urgent things blocked in the approval queue so I have not pursued it.

Maybe someone with karma could allow read access to the Jenkins templates at https://jenkins.ci.cloudbees.com/template/ unless they contain secret info?

/James

[Bob Bick]

I tried someAttribute and it did not work either.

 

The Groovy code is executed prior to the user entering the template attributes. This makes using Groovy code not very useful since it can’t be dynamic.

[teilo]

That somehow may depend on what you are evaluating – but it is dynamic on job save, so if you update the parameters the job is updated.

 

e.g. my template has

“checkstyle” in the template is a checkbox.

 

--snip --

  <reporters>

    <hudson.plugins.checkstyle.CheckStyleReporter>

<% if (checkstyle) { %>

      <thresholds>

        <unstableTotalAll>0</unstableTotalAll>

      </thresholds>

<% } // end if (checkstyle) %>

    </hudson.plugins.checkstyle.CheckStyleReporter>

-- snip –

 

Or – where I have an option for maven parallel building (-Txxx)

  <goals>-e ${parallelBuilds} -Dmaven.test.failure.ignore=false test checkstyle:checkstyle pmd:pmd findbugs:findbugs</goals>

And as it was added at the start of the template I have a default value

<%

/*

  * New parameters may be null so set a suitable default.

  */

 if (parallelBuilds == null) {

   parallelBuilds = "";

 }

%>

 

Other places – where I need it just cope with the fact the value hasn’t been set (but this is partly due to the fact that I create template jobs inside a template folder J - and then force the user update the config by disabling the jobs until they have entered the required info)

  <mavenName>${maven_version == null ? "" : maven_version.getName()}</mavenName>

 

I suggest you raise a support call with cloudbees or provide the template and full transform and error, this is no longer a Jenkins issue and doesn’t really belong on this list.

 

/James

[JonathanRRogers]

On Monday, May 20, 2013 5:44:48 PM UTC-4, Bob Bick wrote:

We have many Jenkins jobs defined that were copied from a “template” Jenkins job. Now, I discovered that I’d like to add a new build step to most (not all) of the jobs.

 I used the Job DSL plugin for a while but that turned out to be less flexible than I needed. Now, I use Python scripts which use the jenkinsapi library to grab a template job's config, replace the necessary parts, and create or update target jobs with the modified config. The excellent lxml library makes it easy to modify the XML document of the config.

[Matt Legrand]

This is also my approach. For simple changes, I edit the config.xml files directly. It is usually easy to figure out what needs to change from the context. That it's flat text makes it quick to edit and review. This does require a "Reload Configuration from Disk" after the edits are made.