[JIRA] (JENKINS-16521) Multi-Project Throttle Categories don't work with Matrix jobs

196 views
Skip to first unread message

chris@simplistix.co.uk (JIRA)

unread,
Jan 29, 2013, 8:22:52 AM1/29/13
to jenkinsc...@googlegroups.com
Issue Type: Bug Bug
Assignee: abayer
Components: matrix, throttle-concurrents
Created: 29/Jan/13 1:21 PM
Description:

I've set up two Multi-Project Throttle Categories, filesystem and postgres, both with maximum total concurrent builds of 0 and maximum concurrent builds per node of 1.

This is because only one job that uses these categories can run on a host at once, if two try, they'll stomp all over each other.

So, I've configured two jobs, both matrix jobs, and both configured with "Throttle Concurrent Builds" checked, "throttle this projects as part of one or more catogories" checked and both the categories defined above checked.

However, when a node has more than one executor, I'll frequently see multiple configurations from the matrix being run on the same node at the same time. Sure enough, they stomp all over each other.

Environment: master and 9 slaves all running SUSE Linux.
Jenkins 1.480.2
throttle-concurrents: 1.7.2
Project: Jenkins
Priority: Major Major
Reporter: Chris Withers
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

chris@simplistix.co.uk (JIRA)

unread,
Jan 29, 2013, 8:26:54 AM1/29/13
to jenkinsc...@googlegroups.com

This issue would appear to be related:

https://issues.jenkins-ci.org/browse/JENKINS-12240

...but suggests a desire for the opposite behaviour: as many configurations of a particular job as possible should run on a node.

However, I'm sure there are just as many people who want the behaviour I'm describing, here's an example:

https://issues.jenkins-ci.org/browse/JENKINS-13619

chris@simplistix.co.uk (JIRA)

unread,
Jan 29, 2013, 9:09:54 AM1/29/13
to jenkinsc...@googlegroups.com

I've just verified that this is a regression in 1.7.2.

1.7.1 behaves as expected, so I'll downgrade to that as a workaround...

chris@simplistix.co.uk (JIRA)

unread,
Feb 1, 2013, 5:45:53 AM2/1/13
to jenkinsc...@googlegroups.com

w@wael.org (JIRA)

unread,
Mar 13, 2013, 10:36:53 AM3/13/13
to jenkinsc...@googlegroups.com

I tried 1.7.1 but had the same unwanted behaviour as 1.7.2!

chris@simplistix.co.uk (JIRA)

unread,
Apr 25, 2013, 12:11:33 PM4/25/13
to jenkinsc...@googlegroups.com

I'm also now seeing this on 1.7.1 with 1.480.3 :-/

judoole@gmail.com (JIRA)

unread,
May 15, 2013, 8:34:58 AM5/15/13
to jenkinsc...@googlegroups.com

Same here. Version 1.72 and 1.513 of Jenkins. Is this, https://issues.jenkins-ci.org/browse/JENKINS-12240, supposed to fix it? It is included in version 1.72 according to commits.

judoole@gmail.com (JIRA)

unread,
May 15, 2013, 9:44:58 AM5/15/13
to jenkinsc...@googlegroups.com

Update: Cloned tag 1.7.1 and installed and it seems to work using Category.
Created a "Matrix-job" category using "Maximum Total Concurrent Build = 4" and "Maximum Concurrent Builds Per Node = 4". The job itself is configured with "Maximum Concurrent Builds Per Node = 1" and "Maximum Total Concurrent Build = 4". I am not using any slaves though.

chris@simplistix.co.uk (JIRA)

unread,
May 17, 2013, 6:40:58 AM5/17/13
to jenkinsc...@googlegroups.com

Yeah, I thought it was working, and then found overlapping matrix child jobs on some slave nodes.
So, have to default to one executor per slave, which means those slaves can't be used for any other jobs (that don't use the throttled resources), which is a waste

hitswar@gmail.com (JIRA)

unread,
May 24, 2013, 12:20:58 AM5/24/13
to jenkinsc...@googlegroups.com

Hi Andrew(Abayer),
Can you please fix this and release 1.7.3 fast.
or if any one has already done fix..please do the pull request to get it merged.

samu.wikstedt@iki.fi (JIRA)

unread,
Feb 12, 2014, 11:34:38 AM2/12/14
to jenkinsc...@googlegroups.com

Hi,
is this behaviour somehow related to issue we've see lately with matrix jobs and throttle-plugin: throttle plugin do not recognize slave nodes that has been added after the matrix jobs has been gone to queue. Resulting those jobs going to slave nodes that has been there before job launch and new nodes doing nothing.

chris@simplistix.co.uk (JIRA)

unread,
Feb 16, 2014, 6:54:38 AM2/16/14
to jenkinsc...@googlegroups.com

No, that sounds entirely unrelated. Please file a separate issue for that.

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 8, 2014, 1:48:17 PM3/8/14
to jenkinsc...@googlegroups.com
Oleg Nenashev started work on Bug JENKINS-16521
Change By: Oleg Nenashev (08/Mar/14 6:47 PM)
Status: Open In Progress

andrea.curtoni@nice-software.com (JIRA)

unread,
May 15, 2014, 4:35:04 AM5/15/14
to jenkinsc...@googlegroups.com

Hi,
we have another issue with matrix and throttle-concurrents, I cannot say if it is related to this.

We have 4 executors on a node and 2 matrix projects using them. Each matrix project spawns 4 jobs. Jobs from the same matrix project can run in parallel on the same node but they conflict with jobs from other projects. So we configured the two jobs with the same category and specified to run 1 job per node. Unfortunately jobs from different projects in the same category are spawned to the same node.
We would expect to have 4 jobs from the same project... or at least 1 job from a single project and 3 free executors, not mixed project jobs.

We did the following:

Global configuration:
Category Name: mock

  • Maximum Total Concurrent Builds: 0
  • Maximum Concurrent Builds Per Node: 1

Job configurations:
Throttle Concurrent Builds => checked
Throttle this project as part of one or more categories => selected

  • Maximum Total Concurrent Builds: 0
  • Maximum Concurrent Builds Per Node: 1
  • Multi-Project Throttle Category: mock

We also tried to specify the per-node global conf without any help. We also tried to set the limit for the master node (which is used as the main executor for the matrix jobs). No luck.

Note the same configuration with other non-matrix projects works fine.

Jenkins version: 1.559
Throttle Concurrent Builds Plug-in version: 1.8.2

Should I open another issue for this problem?

o.v.nenashev@gmail.com (JIRA)

unread,
Jun 1, 2014, 9:16:23 AM6/1/14
to jenkinsc...@googlegroups.com

@Andrea
This is a similar issue.
I'll provide a Dev Build of the plugin in JENKINS-13619

o.v.nenashev@gmail.com (JIRA)

unread,
Jun 1, 2014, 9:16:23 AM6/1/14
to jenkinsc...@googlegroups.com
Oleg Nenashev resolved Bug JENKINS-16521 as Duplicate

Marking as duplicate.
Let's track the further activity in JENKINS-13619

Change By: Oleg Nenashev (01/Jun/14 1:15 PM)
Status: In Progress Resolved
Resolution: Duplicate
Reply all
Reply to author
Forward
0 new messages