Anyway to programatically update a Jenkinsfile?

[Jeff Sault]

Does anyone know of any libraries/tools that will allow me to add/remove/update sections of a Jenkinsfile programatically? I have a load of different projects which I need to update to include some new mandatory parameters. I'd like to go down the 'shared library' route but in this instance its not really possible. Parsing the jenkinsfile is non-trivial but I assume theres something in groovy/jenkins land which can already do the job.

Thanks

Jeff

[jeremy mordkoff]

I use a templating tool (m4) so I can build up a groovy script using macros. In the first stage of my Jenklinsfiie, I call a bash script that generates a groovy script, which I then load and execute. The loaded script has N stages. Note that if the ordering of your stages changes, jenkins will stop showing history, so I try to always add stages at the end. 

[Jeff Sault]

Thanks but i dont think it suits in my case, i need to update a bunch of jenkinsfiles owned by other teams. Surely theres a way to parse a Jenkinsfile into an object, make changes and then write the updated Jenkinsfile back out to the filesystem?

[Slide]

Pipeline scripts are a subset of groovy, so you may want to look in the direction of the available groovy AST modification methods. I am not sure how well it will work on declarative (though it should), but it's the best bet I believe.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/db51294d-c934-47bf-87f7-a3f113cf618d%40googlegroups.com.

--

Website: http://earl-of-code.com

[Tim Black]

I have to agree with Jeremy on this. Using a templating engine like m4 or jinja2 is superior to programmatic generation of groovy, if you need to do it on the fly. Why? Simplicity and Readability.

I inherited a build system that is based on using a multibranch pipeline with a Jenkinsfile that loads and executes a groovy script that's generated on the fly based on input from structured data from json files found alongside the Jenkinsfile in the repo. So, yeah, it's basically a hand-rolled solution to parametrizing a job. It sounds like that's basically what you're trying to accomplish as well, Jeff.

The part I don't like is the way we're generating the groovy code. We've a library of python code that uses classes and a streaming approach to outputting all the syntax required to implement the pipeline. First of all, there's a high learning curve to using this approach - when trying to determine what the groovy output will look like you have to run the code and look at the output, which is a whole software development process in and of itself. When making changes, instead of using a template with variable substitution and loops perhaps, with the programatic approach, you have to write (and test and debug) code to make the groovy code you want. A template already resembles what the final output (groovy) will look like, so it's quite intuitive to work on. Also, every time you need to use a new groovy construct, with the programatic approach, you have to add support for it in your code gen library. With the template you just write it, based on the pipeline syntax documents. This brings up another reason why templated approach is superior: Programatic approach obfuscates the details of the language  from the code gen developer. Looking back and forth between the jenkins pipeline syntax docs and a template is straightforward, not so with a library of classes defined to generate groovy.

So, that's my $0.02. I plan to refactor our code gen library to use jinja2 templating to make our pipelines more intuitive and easier to work with.

Another thing related to keep in mind is that, as far as I can tell, the "load" command, which is the mechanism you would use to actually run your generated pipeline script, is designed to work with Scripted Pipeline, not Declarative. This distinction is very poorly documented by Cloudbees, IMO. Please chime in if you have more info on dynamic generation and running of pipeline code.

[Martin d'Anjou]

Use a template engine to generate Jenkins files, I suggest the StringTemplate template engine.

It's a purely declarative template.

There are examples in the intro doc that you can try in Groovy.

Martin

[Torsten Reinhard]

we don´t use Jenkinsfile(s) at all - instead we generate all the jobs using the Job-DSL.

The jobs than either have the full (generated) code in it - or even better they just call some functions of a shared library.

Any modifications can be applied by a new version of the shared library - or by a commit that triggers and updates the JobGen job which will update all jobs.

It´s often a philosophical discussion about pros and cons of Jenkinsfile(s) - for real life projects it has a lot of disadvantages from my point of view.

It doesn´t solve your problem - but maybe there´s a way to move towards this approach of managing it?

Good luck, Torsten