[blueocean] multi-platform builds with multi-stage completely parallel tracks

[Leandro Lucarella]

Hi, I'm trying to build a pipeline that in principle it sounds like it
should be very simple. I basically have a project that I want to build
and test for Ubuntu trusty and xenial. So basically I want to run them
in separate nodes (which will use Docker).

I want to visualize this in blueocean like this:

Checkout Build Test Deploy

O----------O--------O--------O
trusty trusty trusty trusty

O----------O--------O--------O
xenial xenial xenial xenial

But I can't nail it. TL;DR, is there any way to achieve this? If yes,
how? If not, is it planned? If yes, any ETAs?

Things that I tried:

I can run 2 "jobs" in different nodes completely in parallel, but then
stages inside the nodes are not visualized:

parallel(
'xenial': {
node {
stage("Setup") {
sh 'echo xenial setup'
}
stage("Build") {
sh 'echo xenial build'
}
}
},
'trusty': {
node {
stage("Setup") {
sh 'echo trusty setup'
}
stage("Build") {
sh 'echo trusty build'
}
}
}
)

This shows:

Parallel

O
xenial

O
trusty

If I add top-level stages outside of the node{}, then I get a closer
visualization, but then both track don't run completely in parallel:

stage ("Setup") {
parallel (
'xenial': {
node {
stage("Run") {
sh 'echo xenial setup'
}
}
},
'trusty': {
node {
stage("Run") {
sh 'echo trusty setup'
}
}
}
)
}
stage ("Build") {
parallel (
'xenial': {
node {
stage("Run") {
sh 'echo xenial build'
}
}
},
'trusty': {
node {
stage("Setup") {
sh 'echo trusty build'
}
}
}
)
}

And on top of not being really parallel, the synchronization points are
shown in the visualization (which makes sense if there are
synchronization points:

Checkout Build Test Deploy
O-----.--.------O-----.-.------O-----.-.------O
trusty | | trusty | | trusty | | trusty
| | | | | |
O----´ `-----O-----´ `-----O-----´ `------O
xenial xenial xenial xenial


So, questions again for the people that read this far :)
Is there any way to achieve this? If yes, how? If not, is it planned?
If yes, any ETAs?


Thanks a lot!

--
Leandro Lucarella
Technical Development Lead
Sociomantic Labs GmbH <http://www.sociomantic.com>

[Michael Pailloncy]

As far as I know, stage blocks inside parallel tasks are deprecated => see https://github.com/jenkinsci/pipeline-examples/blob/master/docs/BEST_PRACTICES.md#parallelism

Have you tried something like : 

stage ("Setup") {
    parallel (
        'xenial': {

            node("xenial") {

               sh 'echo xenial setup'
            }
        },
        'trusty': {

            node("trusty") {

               sh 'echo trusty setup'
            }
        }
    )
}
stage ("Build") {
    parallel (
        'xenial': {

            node("xenial") {

               sh 'echo xenial build'
            }
        },
        'trusty': {

            node("trusty") {

               sh 'echo trusty build'
            }
        }
    )
}

IIUC, this pipeline should do what you want. 

If not, have you tried to create your pipeline with BlueOcean Pipeline Editor Plugin ? 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20170703125023.20deb51c%40labs-064.localdomain.
For more options, visit https://groups.google.com/d/optout.

[James Dumay]

Michael's solution is probably the closest to that for the time being. We are looking at adding a "matrix" style pipeline capability to Declarative and Blue Ocean in the future. Watch and vote for JENKINS-40986 

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

[leandro....@sociomantic.com]

On Tuesday, 4 July 2017 06:43:58 UTC+2, mpapo - Michael Pailloncy wrote:

As far as I know, stage blocks inside parallel tasks are deprecated => see https://github.com/jenkinsci/pipeline-examples/blob/master/docs/BEST_PRACTICES.md#parallelism

Have you tried something like : 

stage ("Setup") {

[snip]

Yes, sort of, it's my second example but without the `stage` inside the `parallel` call. I guess I still get the (undesired) synchronization points with that scheme.
 

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

[leandro....@sociomantic.com]

On Tuesday, 4 July 2017 08:57:23 UTC+2, James Dumay wrote:

Michael's solution is probably the closest to that for the time being. We are looking at adding a "matrix" style pipeline capability to Declarative and Blue Ocean in the future. Watch and vote for JENKINS-40986 

OK, thanks, I will watch that issue. I guess there is no way to know an ETA for it, right? I don't see any in the issue.

[James Dumay]

Not at the moment, no. Sorry

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/hkFSNrURL6A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/358b94bc-3c5c-462c-9733-c786205c1f16%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

James Dumay,
Director of Product Management.
CloudBees, Inc.

Meet with me: https://calendly.com/jdumay
Skype: jdumaycb Twitter: i386

[leandro....@sociomantic.com]

On Tuesday, 4 July 2017 10:50:09 UTC+2, leandro....@sociomantic.com wrote:

On Tuesday, 4 July 2017 06:43:58 UTC+2, mpapo - Michael Pailloncy wrote:

As far as I know, stage blocks inside parallel tasks are deprecated => see https://github.com/jenkinsci/pipeline-examples/blob/master/docs/BEST_PRACTICES.md#parallelism

Have you tried something like : 

stage ("Setup") {

[snip]

BTW, this jenkins plugin testing library should probably be updated if stages inside parallel should not be used, my example was based on this:

 https://github.com/jenkins-infra/pipeline-library/blob/master/vars/buildPlugin.groovy

This is a problem I find very often, pipelines seem to change too fast and I always find contradicting examples, and is very hard to find a true source of up-to-date documents. I hope this improves soon :)