How to nest execution of dependent Pipelines, for execution on a single machine ?

[boito...@gmail.com]

Hi,

  (For full disclosure, the following question was first asked on SO a few weeks ago.)

  We are a company of few developers, which will try to implement Continuous Integration for our codebase. It is constituted of half a dozen repositories, with dependencies between them. Ideally, each repository will be built when commits are pushed on origin (for predefined branches).

 As far as we understand, this means that we create a separate Pipeline for each of our own repo. But because of dependencies, some of the Pipelines downstream would somehow need to first build the upstream Pipelines they depend upon.

---

  We could take an example: let's assume libA is a dependency of libB which is a dependency of clientC.

libA pipeline

libA has external dependencies, so we can write the pipeline build-A-pipe to build it : one of the stages being responsible for gathering such external dependencies, and a subsequent stage actually invokes the build command.

This pipeline is directly invoked when a new commit to libA is pushed on origin.

libB pipeline

libB would ideally be built within a separate pipeline, called build-B-pipe. During the stage to gather libB dependencies, we have to build libA. It seems to us that the recommended way to achieve such behaviour is to call

build job: 'build-A-pipe'

within the pipeline that builds libB. (This way we respect the DRY principle: reuse the build-A-pipe, which already describes all steps and details required to successfully build libA.)

clientC pipeline

Now, if we wanted to build clientC, we would follow a similar procedure. There would be a call

build job: 'build-B-pipe'

in the dependencies gathering stage of the pipeline building clientC. The issue is that it results in nested calls to the build command, which deadlocks the single machine :

* at the top level calling build job: 'build-B-pipe' schedules build-B-pipe, and starts it on the master machine (our only "execution node").
* build-B-pipe then calls build job: 'build-A-pipe', which is then scheduled but cannot start, as the only "execution node" is already taken.

---

What would be the preferred way to approach this problem, to make this inherently sequential build work within Jenkins ?

Thank you for reading,