I'm trying to work out how to best set up GoCD to build a series of libraries.
At present, I use Jenkins to do this, and a single pipeline receives a triggering git webhook (from any one of ~50 repositories) and then uses a single repository containing builder-specific code to essentially allow what GoCD uses Configuration Repositories for (as I understand from this group and the docs) to then clone and build the triggering repo's code.
The code is built for 6 different target/release/debug combinations (3 targets, all with debug+release). On Jenkins, my current system carries out these builds in series, making the time 6x longer than it might otherwise be (excluding consideration of e.g. setup and publication after build).
The bulk of my question relates to Stage 2 in my explanation below.
My current thought for GoCD looks something like:
- Stage 1 (setup, static testing)
- Job 1 (only job)
- Checkout the git material that triggers the build
- Parse a file giving dependencies
- Install any necessary dependencies using one of the 6 targets (probably 32-bit Windows, debug)
- Run static analysis (this can take some non-negligible time, due to the tools :/ )
- Run tests on the source code
- Stage 2 (Build 6 target combinations in parallel)
- Jobs 1-6 (using https://docs.gocd.org/current/advanced_usage/admin_spawn_multiple_jobs.html ? ) Here I want to have each job use a different Docker container, from one of 3 images (per target, rel+debug can use the same image). Questions below the rest of job config explanation
- Checkout git code (or copy from host?)
- Install target-specific dependencies
- Compile library
- Run tests on the compiled code
- Build a package for installation
- Publish back to host via artifact?
- Stage 3 (post-build)
- Publish to Github releases
- Publish to the feed from which packages are installed in 1.1.3 and 2.{1-6}.2
To run the jobs in parallel, it seems I need multiple agents - but that I can install multiple agents on a single host (
https://docs.gocd.org/current/advanced_usage/admin_install_multiple_agents.html). Alternatively, I looked at elastic agents, but this seems like it might be more complex than required? If I have an image that I pass various configuration variables into, and it produces the compiled library after testing, what's the best way to integrate this into GoCD?
There will only be the 6 agents (maybe 7 in future?) required for build, and I don't want to scale beyond a single host computer. I think running them all together is fine (they won't use multiple cores effectively, and I can devote a computer specifically to this build process).
Do I need a separate agent for the stage 1/3 parts? On Jenkins I'm running all of the pipeline on the master node, which isn't the 'best-practice' method but was easier to setup.
Should I instead use only a single agent and manually write script code to start 6 containers in parallel, and then gather them? (I'd prefer not to, this seems like it will avoid some of the nice visualization and scheduling being done by GoCD.)
Any guidance on what I should do/read/attempt would be appreciated.
Thanks!
Christian