GoCD Templates vs templated pipelines (e.g. in Python to YAML plugin and config repos)

[chris...@gmail.com]

Hi all,

I'm currently using a python script to generate a .gocd.yaml file, containing a collection of pipelines.

I'm now looking at ways to handle multiple branches for a specific repository, with a pipeline per branch.

I figured I could use GoCD Templates (as in, the <templates> element) to take the stages and then use the parameters to control various minor differences, with materials specified in each pipeline from the template (unfortunately, I don't think I can put the materials in the template - right?)

I bumped into an issue when I discovered that I can't define a template using the YAML config plugin. This lead me to a series of issues/discussions/PRs on this and related topics:
https://github.com/tomzo/gocd-yaml-config-plugin/issues/211
https://github.com/gocd/gocd/issues/6123#issuecomment-547636050 (seems this would work, but requires Groovy DSL plugin?)
https://groups.google.com/g/go-cd/c/JalrejeFlOc/m/xpLsZGZLAgAJ

I can fairly easily expand my Python code to simply generate more pipelines (and then control whatever I want to share/not share, because it's expanded as a bunch of static strings before GoCD ever sees it), but then the addition/removal of any pipelines requires commits to the config-repo, and then updating on the server.
I was initially motivated to use templates on the basis that I thought I might be able to use the web UI to create new copies of a template (even one defined in a YAML file, back when I thought that was possible).

I note also that I can't clone yaml-defined pipelines to make a server-defined pipeline with editable fields.

How are people doing this kind of thing?
Should I just update the config-repo each time I need to add/remove a pipeline for a new/deleted branch?
Does anyone have opinions about the Groovy DSL plugin instead (and its applicability to this use case?)

Thank you if you read this far!

I suspect I'm doing something silly, or have a bad mental model, but with that in mind, I'm probably looking in the wrong places for solutions...

Best regards,
Christian

[Chad Wilson]

The groovy plugin can definitely do this (with or without a linked template) based on webhook triggers for branch/PR creation in your source control, but it does currently have some security compromises if your config repo is open to arbitrary users to make commits. The scripting engine is not (currently) sandboxed, basically. If it's a single user or limited user environment it may be fine, though - if you're OK with Groovy. It'd probably only be worthwhile if you were able to replace your python with groovy.

Or you can create your own plugin with jython 😅

Other than this, there is https://github.com/getsentry/gocd-jsonnet-config-plugin although I haven't used it, and it doesn't seem super well maintained as to dependencies. Not sure if it can dynamically detect branches etc.

Overall, this wasn't designed to be super-easy in GoCD as GoCD is relatively opinionated about discouraging branch-based development, and templates are an older feature that was largely intended to be replaced with whatever templating approach on top of config repos you find easiest.

-Chad

--
You received this message because you are subscribed to the Google Groups "GoCD Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/go-cd/94f0136a-983a-46bd-9247-4f238d849101n%40googlegroups.com.