[JIRA] [remoting] (JENKINS-22494) Multiconfiguration project does not respect label restrictions

7 views
Skip to first unread message

sade@avr-vr.com (JIRA)

unread,
Jun 8, 2015, 1:34:03 PM6/8/15
to jenkinsc...@googlegroups.com
Samuel Delisle commented on New Feature JENKINS-22494
 
Re: Multiconfiguration project does not respect label restrictions

Is there any design issue that prevents a plugin from achieving the proposed behavior? If plugins can control whether or not a node is willing to execute a particular job, it would seem pretty straight-forward to have one that allow you to specify label restrictions (or inherit those of the parent job).
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

sade@avr-vr.com (JIRA)

unread,
Jun 8, 2015, 1:34:03 PM6/8/15
to jenkinsc...@googlegroups.com
Samuel Delisle edited a comment on New Feature JENKINS-22494
Is there any design issue /constraint  that prevents a plugin from achieving the proposed behavior? If plugins can control whether or not a node is willing to execute a particular job, it would seem pretty straight-forward to have one that allow you to specify label restrictions (or inherit those of the parent job).

mark.earl.waite@gmail.com (JIRA)

unread,
Jun 8, 2015, 2:09:02 PM6/8/15
to jenkinsc...@googlegroups.com

Martin Skinner and Samuel Delisle I don't see how this could be implemented as an enhancement request without breaking the existing behavior. The existing behavior allows the flyweight task (initial creation of a workspace) to be limited to certain machines, then a completely different (and much richer) set of specifications (extensible by plugins) can be used to control which labels are used to execute the job configurations.

Today, the first limitation (initial workspace creation) does not apply to the second specification. Considering the number of Jenkins installations, I'm reasonably confident that means there are installations and jobs in those installations which depend on the current behavior.

sade@avr-vr.com (JIRA)

unread,
Jun 8, 2015, 5:31:01 PM6/8/15
to jenkinsc...@googlegroups.com

I was actually wondering why there did not seem to be any plugin that did this. There are lots of plugin that affect what gets executed where in some way, but I couldn't find any that did this with labels. I finally found this plugin that pretty much does what is said here. You put "return Label1&&Label2" or whatever condition you want in the script box, and it will affect both the parent job and its children.

mark.earl.waite@gmail.com (JIRA)

unread,
Jun 8, 2015, 7:46:02 PM6/8/15
to jenkinsc...@googlegroups.com

I use the elastic axis plugin with labels all the time. Coupling that with the implied labels plugin has allowed to to state that I want a job to run on all Linux and all Windows machines with the expression "linux || windows"

mark.earl.waite@gmail.com (JIRA)

unread,
Jun 8, 2015, 11:33:03 PM6/8/15
to jenkinsc...@googlegroups.com
Mark Waite edited a comment on New Feature JENKINS-22494
I use the [elastic axis plugin|https://wiki.jenkins-ci.org/display/JENKINS/Elastic+Axis] with labels all the time.  Coupling that with the [ platform labeler plugin|https://wiki.jenkins-ci.org/display/JENKINS/PlatformLabeler+Plugin] and the [ implied labels plugin|https://wiki.jenkins-ci.org/display/JENKINS/Implied+Labels+Plugin] has allowed to to state that I want a job to run on all Linux and all Windows machines with the expression "linux || windows"

dbeck@cloudbees.com (JIRA)

unread,
Jun 10, 2015, 8:23:02 AM6/10/15
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
 
Jenkins / New Feature JENKINS-22494
Change By: Daniel Beck
Component/s: matrix-project-plugin
Component/s: remoting

brian.murrell@intel.com (JIRA)

unread,
Feb 2, 2016, 8:31:02 AM2/2/16
to jenkinsc...@googlegroups.com
Brian J Murrell commented on New Feature JENKINS-22494
 
Re: Multiconfiguration project does not respect label restrictions

This is basically the same thing I asked for in JENKINS-9531 5 years ago.

myatlyuk@gmail.com (JIRA)

unread,
Aug 16, 2016, 7:14:07 PM8/16/16
to jenkinsc...@googlegroups.com

Another workaround for the issue is to leverage NodeLabel parameter plugin and parameterize the build with node label parameter. That seems to work for us.
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

yuhsiang.cheng@gmail.com (JIRA)

unread,
Dec 7, 2016, 7:08:04 AM12/7/16
to jenkinsc...@googlegroups.com
Hermes Cheng updated an issue
 
Change By: Hermes Cheng
Priority: Major Critical

yuhsiang.cheng@gmail.com (JIRA)

unread,
Dec 7, 2016, 7:08:04 AM12/7/16
to jenkinsc...@googlegroups.com
Hermes Cheng commented on New Feature JENKINS-22494
 
Re: Multiconfiguration project does not respect label restrictions

Multiconfiguration project is a default feature of Jenkins v2.0, and I believe this feature is very important for a smooth CI process.

mark.earl.waite@gmail.com (JIRA)

unread,
Dec 7, 2016, 8:45:02 AM12/7/16
to jenkinsc...@googlegroups.com

Hermes Cheng can you please explain why you can't use the elastic axis plugin (as described in my earlier reply) or the NodeLabel parameter plugin to achieve the results you want?

mark.earl.waite@gmail.com (JIRA)

unread,
Dec 7, 2016, 8:47:01 AM12/7/16
to jenkinsc...@googlegroups.com
Mark Waite edited a comment on New Feature JENKINS-22494
[~hermescheng] can you please explain why you can't use the elastic axis plugin (as described in my [earlier reply|https://issues.jenkins-ci.org/browse/JENKINS-22494?focusedCommentId=229776&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-229776]) or the NodeLabel parameter plugin to achieve the results you want?

Also, you may want to refer to my [much earlier comment|https://issues.jenkins-ci.org/browse/JENKINS-22494?focusedCommentId=199241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-199241] explaining why I believe the multiconfiguration projects behave this way.

yuhsiang.cheng@gmail.com (JIRA)

unread,
Dec 7, 2016, 9:44:02 PM12/7/16
to jenkinsc...@googlegroups.com
Hermes Cheng updated an issue
Change By: Hermes Cheng
Priority: Critical Major

yuhsiang.cheng@gmail.com (JIRA)

unread,
Dec 7, 2016, 11:11:02 PM12/7/16
to jenkinsc...@googlegroups.com
Hermes Cheng commented on New Feature JENKINS-22494
 
Re: Multiconfiguration project does not respect label restrictions

Hi Mark Waite,
Yes, elastic axis plugin works well. I just thought as a default feature, it should support the original 'label expression' restricting functionality instead depending on another plugin. Lower the priority since it has a workaround.

gert.massa@siemens.com (JIRA)

unread,
Feb 8, 2017, 11:20:03 AM2/8/17
to jenkinsc...@googlegroups.com
Gert Massa updated an issue
 
Change By: Gert Massa
Priority: Major Critical

aprice@tibco.com (JIRA)

unread,
Mar 30, 2017, 4:56:05 AM3/30/17
to jenkinsc...@googlegroups.com
Adrian Price commented on New Feature JENKINS-22494
 
Re: Multiconfiguration project does not respect label restrictions

I'm curious as to why this is still open some three years after it was first logged. Isn't it an absolutely fundamental requirement that a build honour its job configuration? As things currently stand the matrix cell jobs get run anywhere, which is surely not the intend behind restricting where the top-level matrix itself can run.
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 30, 2017, 8:40:02 AM3/30/17
to jenkinsc...@googlegroups.com

Adrian Price, the multi-configuration project honors the axes assigned to it. If an agent label is used as the condition, that agent label is honored. I don't know why it was reopened, since I believe the code is working as designed, and the design is such that it meets the needs.

Saying that the "top level matrix runs" the job is a misnomer as far as I can tell, since it is only used for the initial flyweight task, and is not used to perform any of the build actions defined in the job.

Martin Skinner suggested reopening this as a feature request with the idea that "Restrict where this project can run" should be automatically included in the conditions which define the axes. I disagree with that, since it will be a major compatibility change for existing users, but I suspect that is the reason it has been reopened.

I consider this to be a "Won't Fix" request, since it is behaving as designed, and the proposed feature request of automatically including the "restrict where this job runs" in the axis definition would break compatibility for some portion of the 80 000+ installations of Jenkins.

aprice@tibco.com (JIRA)

unread,
Mar 30, 2017, 10:39:02 AM3/30/17
to jenkinsc...@googlegroups.com
Adrian Price edited a comment on New Feature JENKINS-22494
I'm curious as to why this is still open some three years after it was first logged. Isn't it an absolutely fundamental requirement that a build honour its job configuration? As things currently stand the matrix cell jobs get run anywhere, which is surely not the intend intent behind restricting where the top-level matrix itself can run . ?

aprice@tibco.com (JIRA)

unread,
Mar 30, 2017, 11:38:02 AM3/30/17
to jenkinsc...@googlegroups.com

Mark Waite, thanks for the quick response. I suppose that in many case one would indeed want the matrix cell jobs to run on different nodes appropriate to particular combinations of matrix axis variables.

Is there any documentation for your suggestion of using an agent label as the condition? It's not clear to me how to proceed - all I want to do is restrict where the matrix cell jobs run.

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 30, 2017, 11:44:06 AM3/30/17
to jenkinsc...@googlegroups.com

Adrian Price, if you want to use all agents, or all agents having a particular label, then you can install the elastic axis plugin and configure an axis on the multi-configuration job using elastic axis. Refer to the earlier comment for more on that technique.

If you want to use specific agents, then define an axis based on the label, and list the labels you want included.

As an example, for the git plugin, I have a job which defines an "operating-system" axis with values CentOS-6, CenOS-7, Debian-7, Debian-8, Ubuntu-14, Ubuntu-16, windows. Those labels are generated automatically for me by the platformlabeler plugin, so I get good platform coverage without managing label assignments to agents.

mark.earl.waite@gmail.com (JIRA)

unread,
Mar 30, 2017, 11:45:04 AM3/30/17
to jenkinsc...@googlegroups.com
Mark Waite edited a comment on New Feature JENKINS-22494
I use the [elastic axis plugin|https://wiki.jenkins-ci.org/display/JENKINS/Elastic+Axis] with labels all the time.  Coupling that with the [platform labeler plugin|https://wiki.jenkins-ci.org/display/JENKINS/PlatformLabeler+Plugin] and the [implied labels plugin|https://wiki.jenkins-ci.org/display/JENKINS/Implied+Labels+Plugin] has allowed me to to state that I want a job to run on all Linux and all Windows machines with the expression "linux || windows"

arnt.work@gmail.com (JIRA)

unread,
Sep 27, 2018, 9:56:04 AM9/27/18
to jenkinsc...@googlegroups.com

I would like to add my reasons on why a way to specify where to run the eventual builds (and where not) is also needed, independent from the axis'.

I have a master with no compiler, and several slaves with compilers. I want my build to be build in (let's see the example simple) 2 flavors, so I define 1 axis with flavor1 and flavor2 as the value. Every slave can build each of the flavors, they are just different build settings (think building with different defines in C/C++). However, for speed, I would like my 2 builds to run on 2 separate slaves if they are free. And they cannot be run on the master.

As I see it, there is no solution for this currently: Adding a 'slave' axis makes no sense, I don't want each combination of the other axis to be build on each slave, I only want to build each combination of my axis to be build once. And I don't want to lock certain combinations to certain build machines, these machines might be busy while others are idle, and of course I'd want to use the idle machines then.
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

arnt.work@gmail.com (JIRA)

unread,
Sep 27, 2018, 9:57:03 AM9/27/18
to jenkinsc...@googlegroups.com
Arnt Witteveen edited a comment on New Feature JENKINS-22494
I would like to add my reasons on why a way to specify where to run the eventual builds (and where not) is also needed, independent from the axis'.

I have a master with no compiler, and several slaves with compilers. I want my build to be build in (let's see the example simple) 2 flavors, so I define 1 axis with flavor1 and flavor2 as the value. Every slave can build each of the flavors, they are just different build settings (think building with different defines in C/C++). However, for speed, I would like my 2 builds to run on 2 separate slaves if they are free. And they cannot be run on the master.

As I see it, there is no solution for this currently: Adding a 'slave' axis makes no sense, I don't want each combination of the other axis to be build on each slave, I only want to build each combination of my axis to be build once. And I don't want to lock certain combinations to certain build machines, these machines might be busy while others are idle, and of course I'd want to use the idle machines then.


 

I'll try the NodeLabel Parameter Plugin suggested in JENKINS-23459 next.

arnt.work@gmail.com (JIRA)

unread,
Sep 27, 2018, 10:08:02 AM9/27/18
to jenkinsc...@googlegroups.com
Arnt Witteveen edited a comment on New Feature JENKINS-22494
I would like to add my reasons on why a way to specify where to run the eventual builds (and where not) is also needed, independent from the axis'.

I have a master with no compiler, and several slaves with compilers. I want my build to be build in (let's see the example simple) 2 flavors, so I define 1 axis with flavor1 and flavor2 as the value. Every slave can build each of the flavors, they are just different build settings (think building with different defines in C/C++). However, for speed, I would like my 2 builds to run on 2 separate slaves if they are free. And they cannot be run on the master.

As I see it, there is no solution for this currently: Adding a 'slave' axis makes no sense, I don't want each combination of the other axis to be build on each slave, I only want to build each combination of my axis to be build once. And I don't want to lock certain combinations to certain build machines, these machines might be busy while others are idle, and of course I'd want to use the idle machines then.

 
My trials may help others:
* I' ll try ve tried the NodeLabel Parameter Plugin suggested in JENKINS-23459 next . This make the build a paramterized build, which is something I don't want.

arnt.work@gmail.com (JIRA)

unread,
Sep 27, 2018, 11:03:13 AM9/27/18
to jenkinsc...@googlegroups.com
Arnt Witteveen edited a comment on New Feature JENKINS-22494
I would like to add my reasons on why a way to specify where to run the eventual builds (and where not) is also needed, independent from the axis'.

I have a master with no compiler, and several slaves with compilers. I want my build to be build in (let's see the example simple) 2 flavors, so I define 1 axis with flavor1 and flavor2 as the value. Every slave can build each of the flavors, they are just different build settings (think building with different defines in C/C++). However, for speed, I would like my 2 builds to run on 2 separate slaves if they are free. And they cannot be run on the master.

As I see it, there is no solution for this currently: Adding a 'slave' axis makes no sense, I don't want each combination of the other axis to be build on each slave, I only want to build each combination of my axis to be build once. And I don't want to lock certain combinations to certain build machines, these machines might be busy while others are idle, and of course I'd want to use the idle machines then.

 

My trials may help others:
* I've tried the ' NodeLabel Parameter Plugin ' suggested in JENKINS-23459 . This make the build a paramterized build, which is something I don't want.
* I had success using the 'Groovy Label Assignment plugin' suggested above, using this script to keep the 'flyweight job' (that starts the axis combination jobs) on the master, and run the actual builds on any node with a label 'CanRunMainBuild' (which could be refined further to run only certain axis on certain node labels, if you need it):

if (currentJob.getClass().toString() == "class hudson.matrix.MatrixProject") {
// for the matrix "parent/flyweight" job
return "master";
} else {
// for the child jobs
return "CanRunMainBuild";
}

arnt.work@gmail.com (JIRA)

unread,
Sep 27, 2018, 11:25:04 AM9/27/18
to jenkinsc...@googlegroups.com
Arnt Witteveen edited a comment on New Feature JENKINS-22494
I would like to add my reasons on why a way to specify where to run the eventual builds (and where not) is also needed, independent from the axis'.

I have a master with no compiler, and several slaves with compilers. I want my build to be build in (let's see the example simple) 2 flavors, so I define 1 axis with flavor1 and flavor2 as the value. Every slave can build each of the flavors, they are just different build settings (think building with different defines in C/C++). However, for speed, I would like my 2 builds to run on 2 separate slaves if they are free. And they cannot be run on the master.

As I see it, there is no solution for this currently: Adding a 'slave' axis makes no sense, I don't want each combination of the other axis to be build on each slave, I only want to build each combination of my axis to be build once. And I don't want to lock certain combinations to certain build machines, these machines might be busy while others are idle, and of course I'd want to use the idle machines then.

 

My trials may help others:
* I've tried the 'NodeLabel Parameter Plugin' suggested in JENKINS-23459 . This make the build a paramterized build, which is something I don't want.
* I had limited success using the 'Groovy Label Assignment plugin' suggested above, using this script to keep the 'flyweight job' (that starts the axis combination jobs) on the master, and run the actual builds on any node with a label 'CanRunMainBuild' (which could be refined further to run only certain axis on certain node labels, if you need it):


if (currentJob.getClass().toString() == "class hudson.matrix.MatrixProject") {

 // for the matrix "parent/flyweight" job

return "master";

} else {
// for the child jobs
return "CanRunMainBuild";
}

However, now slaves run all the subjobs simultaneously, and the Throttle concurrent builds plugin doesn't block that no matter what I try :(

arnt.work@gmail.com (JIRA)

unread,
Sep 27, 2018, 11:34:05 AM9/27/18
to jenkinsc...@googlegroups.com
success using the 'Groovy Label Assignment plugin' suggested above, using this script to keep the 'flyweight job' (that starts the axis combination jobs) on the master, and run the actual builds on any node with a label 'CanRunMainBuild' (which could be refined further to run only certain axis on certain node labels, if you need it):
However, now slaves run all the subjobs simultaneously, and the Throttle concurrent builds plugin doesn't block that when using the 'Throttle this project alone' functionality. I think this is JENKINS-33940, and the suggested workaround there (use a category) does work.

arnt.work@gmail.com (JIRA)

unread,
Sep 27, 2018, 11:34:06 AM9/27/18
to jenkinsc...@googlegroups.com
Arnt Witteveen edited a comment on New Feature JENKINS-22494
I would like to add my reasons on why a way to specify where to run the eventual builds (and where not) is also needed, independent from the axis'.

I have a master with no compiler, and several slaves with compilers. I want my build to be build in (let's see the example simple) 2 flavors, so I define 1 axis with flavor1 and flavor2 as the value. Every slave can build each of the flavors, they are just different build settings (think building with different defines in C/C++). However, for speed, I would like my 2 builds to run on 2 separate slaves if they are free. And they cannot be run on the master.

As I see it, there is no solution for this currently: Adding a 'slave' axis makes no sense, I don't want each combination of the other axis to be build on each slave, I only want to build each combination of my axis to be build once. And I don't want to lock certain combinations to certain build machines, these machines might be busy while others are idle, and of course I'd want to use the idle machines then.

 

My trials may help others:
* I've tried the 'NodeLabel Parameter Plugin' suggested in JENKINS-23459 . This make the build a paramterized build, which is something I don't want.
* I had limited success using the 'Groovy Label Assignment plugin' suggested above, using this script to keep the 'flyweight job' (that starts the axis combination jobs) on the master, and run the actual builds on any node with a label 'CanRunMainBuild' (which could be refined further to run only certain axis on certain node labels, if you need it):

if (currentJob.getClass().toString() == "class hudson.matrix.MatrixProject") {

{ // for the matrix "parent/flyweight" job
return "master";
}

else {

{ // for the child jobs
return "CanRunMainBuild";
}
However, now slaves run all the subjobs simultaneously, and the Throttle concurrent builds plugin doesn't block that no matter what when using the 'Throttle this project alone' functionality. I try : think this is JENKINS-33940, and the suggested workaround there ( use a category) does work.

shaishap@gmail.com (JIRA)

unread,
Dec 4, 2019, 2:25:07 AM12/4/19
to jenkinsc...@googlegroups.com

Any plans to fix this?

If binding using Node axis, is there any workaround that does not require running all configurations on all nodes with label?
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

mark.earl.waite@gmail.com (JIRA)

unread,
Dec 4, 2019, 5:21:05 AM12/4/19
to jenkinsc...@googlegroups.com

Read the preceding comments for suggestions that allow filtering of matrix configurations

shaishap@gmail.com (JIRA)

unread,
Dec 15, 2019, 5:08:03 AM12/15/19
to jenkinsc...@googlegroups.com

Thanks. I read it, but it does not allow the run of each configuration on one host only.

i.e. If I have 2 config values, and 4 potential build agents, creating a 2x4 would run 8 jobs, instead of 2.

Is there a way to solve this?

mark.earl.waite@gmail.com (JIRA)

unread,
Dec 16, 2019, 4:17:04 AM12/16/19
to jenkinsc...@googlegroups.com

Shai Shapira all the details of using exclusions are included in the comments.

If that's not enough, you might consider switching to the Jenkins pipeline. Blog posts for Declarative pipeline and scripted pipeline describe the techniques to define matrix configurations and to exclude portions of the matrix.
Reply all
Reply to author
Forward
0 new messages