Migrating (or simply reimplimenting) from Go Version: 18.9.0 to lateast and greatest

[Fenn]

We have a large legacy installation of GoCD with many pipelines.
We are running Go-server 18.9.0. When we export the XML config it is
175247 lines long and 7.3M. We have 891 pipelines.
It will sometimes freeze up and we need to restart the server.
We are interested in migrating to a newer version - interested in
hearing any recommendations, war stories or advice.
Should we Migrate (or even try) or just start up a new instance and
reimpliment pipelines one at a time? Is Gomatic (
https://github.com/SpringerSBM/gomatic ) worth learning for this?
Very interested to hear from anyone with experience or even just ideas!
Brian Fennell

[Fenn]

I see the latest version is 20.10.0 - so we will have to skip 2 major versions.

[Aravind SV]

Hello Brian,

This thread from the mailing list is relevant:
https://groups.google.com/g/go-cd/c/6Rufx-ewWpU/m/egjJwQx8AgAJ

About gomatic: Last I heard, it wasn’t compatible with the latest versions of GoCD. I’d look into one of the pipelines as code plugins supported by GoCD instead:

https://github.com/tomzo/gocd-yaml-config-plugin
https://github.com/gocd-contrib/gocd-groovy-dsl-config-plugin
https://github.com/tomzo/gocd-json-config-plugin

Once you upgrade, you will even be able to programmatically export pipelines to one of those formats:
https://api.gocd.org/current/#export-pipeline-config-to-config-repo-format

Cheers,
Aravind

PS: A full backup before you continue would be my recommendation.

[Fenn]

Thanks, Aravid!

Those links help.

Any idea on how to pick a new strategy to keep the XML config from
getting so huge? It takes 40 minutes to start up the go-server and we
think it is related to the giant XML config (or the in-memory data
structure that is its analog). I find myself wondering if we should
have more than one server - or a cluster of servers - but the server
seems to be designed to be "one and only one". Perhaps we could split
up development and production across two go-servers, but that only
gives us approximately a factor of 2.

Currently the only way we have to semi-automate refactoring is to cut
and paste the whole XML from the web UI into a text file, have a
script modify the XML and then paste the entire XML back into the web
UI (then check for errors after trying to save it). Sometimes we step
on each others toes and end up rolling back someone's changes by
mistake using this method.

I think the new API offers better approaches than this (like the yaml
features you mentioned) but haven't had a chance to test them out
because we are running an old version.

Reading the docs and googling gets a hodgepodge of things from
different versions and it is hard to work out what might really work
for us. I am hoping to learn from someone else's best practices by
reaching out to this list.

Brian Fennell