Multi-Platform matrix build with jenkins2 Pipeline

[Matt Hauck]

Hello! My team is looking to move to jenkins2 and struggling a bit with how to do a multip-platform build. I'm really excited about the direction it is moving in with the pipline builds -- I implemented something like this myself (github.com/tanium/pyjenkins), but glad to see something better being built directly into the platform.

One difficulty I've run into so far is the less-than-ideal for multi-platform builds with pipelines. We have a build that needs to run in windows, linux, mac, etc. Right now we have this setup with a matrix job. The only way I can see to do this with the pipeline plugin is using `parallel` (i.e. https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md#creating-multiple-threads)

However, there is one rather large setback of doing it this way: artifacts and output get intermingled.

It is pretty crucial to us to be able to quickly and clearly tell which platform the build failed on, as well as to distinguish the artifacts from platform from the artifacts from another platform. I have the build currently running with `parallel` steps, but I find it really really difficult to find out what the build is doing either while running or after it finishes. 

Also, there does not appear to be an option to setup an older-style matrix build that uses the Jenkinsfile for its execution (for understandable reasons), so I don't even see a way to move forward incrementally until multi-platform builds have better support. 

I should also say that the matrix job is a really awesome feature of jenkins that, from a survey of other build systems I did once, is kinda unique to jenkins. Support for having the exact same build definition across that executes across multiple platforms does not really exist outside the matrix job definition in Jenkins, as far as I could tell. It would be sad to see this great feature become a second-class citizen that can't use all the cool shiny new toys. 

Are there plans to add better support for multi-platform builds / matrix jobs? Is there something I'm missing? 

[Matt Hauck]

Clarification: "artifacts and output get intermingled" is confusing. I meant to say that standard output for all the various parallel nodes gets intermingled. Something like this:

[ windowsNode ] Doing some stuff
[ linuxNode ] Doing different stuff
[ macNode ] yet more stuff

It also seems that artifacts (whether files or test results) also get intermingled and centralized rather than kept separate in connection to the node they were built on. This point I am less clear on though.

[Matt Hauck]

bump.

[martin...@gmail.com]

I'm also in the same boat as you. Have you found a solution to this yet?

The Blue Ocean UI does mitigate the problem slightly, but I still find it difficult to figure out what is going on. We use a large set of plugins that does not show up in Blue Ocean (test and coverage graphs, console parsing, etc.), so I would prefer to stick with the old UI for now. However, I don't see any clear separation of the platforms unless I go through the pipeline steps themselves. Another major challenge I'm having is how to generate email reports. For matrix projects, we generate a report summarizing errors, warnings, test results, etc. per platform. As far as I see it, even the test results gets aggregated from all the platforms, making it impossible to figure out at what node the test failed. This is the pipeline I am currently working on: https://github.com/martinmine/rootspi/blob/pipelines/jenkins/pipeline/continuous.groovy 

I'm also having the same feeling as you that there is something I'm missing. I see many benefits of using pipelines (version control of the build logic, generating artifacts, the dynamic build environment introduced by scripting the pipeline, etc), but this is a major issue we need to get across to properly utilize pipelines in our build infrastructure.

[Mark Waite]

I'm not sure how you would present a separation of logs without using stages to represent the agents in your multi-configuration project.

I thought that the way to see the separation of those logs was to have each parallel get its own stage declaration and then I can view the results for a specific stage from the "logs" pop-up that appears with each of the stages.  For example, refer to https://ci.jenkins.io/job/Plugins/job/git-client-plugin/job/master/ to see how it presents the separation of Windows and Linux stages, and shows the console output for those stages.

Test results are aggregated at the top level (like https://ci.jenkins.io/job/Plugins/job/git-client-plugin/job/master/73/testReport/) and do not seem to have a way to showing the results by stage.  It would be a nice enhancement to have test results from a stage also visible in that stage in the old UI, but I'm not sure that would fit with the UI that is included in the stage "logs".

Isn't the better investment to put effort into making those popular plugins fit well within the Blue Ocean UI?

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/06da6375-1697-48eb-a813-4bee345916a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[martin...@gmail.com]

I'm aware you can separate them by using stages, and I wasn't arguing against using them. However, we rely on parser plugins such as Log Parser Plugin to parse the logs. This plugin does only see the log as one aggregated chunk, and as with the tests there are no way of separating them from each other. 

[Matt Hauck]

Nope, no solution to it yet. Glad to hear others are struggling with this too. Our solution so far as to not use Pipeline job configurations, sadly, but stick to good old Matrix job configurations together with tanium/pyjenkins for "configuration as code".

If you had separate stages for each platform, then you would lose concurrency of your multip-platform build, right? That would be a non-starter, imo. 

[Matt Hauck]

By the way, looks like the new "Blue Ocean" UI pretty much solves the issue I had with this. Looks like there is everything we need to move forward with multi-platform pipline builds. yay!