Question about Maven "Build modules in parallel" setting

[David Alayachew]

Hello Jenkins Team,

I have a GitHub repo with a Maven project with over 50 sub-modules, all in the same repo. All of those sub-modules are under a single aggregate pom file.

I created a single Maven job pointed to the aggregate pom, and on the first run, all sub-modules were discovered as expected.

Now, I want these jobs to build in parallel. So, I checked the checkbox that says "Build modules in parallel".

But when I looked at the nodes used, all of the builds occurred on a single node, even though there are many other nodes available. And to be clear, it was using all executors of that node, but still limiting itself to a single node.

So, that meant that, even though the queue had 40+ jobs queued up, once all the executors on that single node were occupied, it didn't matter if any of the other nodes were free, no work would be picked up by them.

On a second attempt, a different node was used, but the same behaviour -- all sub-modules only executed on this second node.

Is this behaviour overridable? Or am I potentially doing things wrong? Again, I am only using the Jenkins Maven Job, not a Jenkinsfile or anything like that.

And as a workaround, I could just make a new Maven Job for each Maven sub-module. Doing it this way, now I do actually run on many free nodes.

But that has the downside of increasing the maintenance burden immensely. Remember, I have over 50 sub-modules. If I need to add new functionality, that's 50 Jenkins jobs that I need to update. Not ideal at all.

Thank you for your time and help.

[muni bhasker]

Hi 

You have slack group so I cloud join.

Thank you 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/jenkinsci-users/CAA9v-_M1Ex2E%2Bn2MGwH6RfT3ek0T23QX6VyQKcmRfbUhJ6N79w%40mail.gmail.com.

[David Alayachew]

Hello,

I do not understand your question. What do you mean by Slack group? I do not know of any Slack group, so if there is one, I cannot grant you access to it.

If you are asking about having the ability to join this google groups discussion, it appears that you have already successfully done so.

To view this discussion visit https://groups.google.com/d/msgid/jenkinsci-users/CAO54Onviqjzw__p6MnVw7yG6LF0khG7QONxbYRFRCYdYvh_Hfw%40mail.gmail.com.

[David Alayachew]

Hello Jenkins Team,

I still have not heard back regarding my query last week about the Maven Jenkins build options. Could someone take a look and respond?

Thank you for your time and consideration.

David Alayachew

[Mark Waite]

On Tue, 23 Sep 2025 at 7:05 PM, David Alayachew wrote:

Hello Jenkins Team,

I have a GitHub repo with a Maven project with over 50 sub-modules, all in the same repo. All of those sub-modules are under a single aggregate pom file.

I created a single Maven job pointed to the aggregate pom, and on the first run, all sub-modules were discovered as expected.

 

Now, I want these jobs to build in parallel. So, I checked the checkbox that says "Build modules in parallel".

But when I looked at the nodes used, all of the builds occurred on a single node, even though there are many other nodes available. And to be clear, it was using all executors of that node, but still limiting itself to a single node.

So, that meant that, even though the queue had 40+ jobs queued up, once all the executors on that single node were occupied, it didn't matter if any of the other nodes were free, no work would be picked up by them.

On a second attempt, a different node was used, but the same behaviour -- all sub-modules only executed on this second node.

Is this behaviour overridable?

No, that is not overridable as far as I know.

 

Or am I potentially doing things wrong? Again, I am only using the Jenkins Maven Job, not a Jenkinsfile or anything like that.

I think it is a mistake to use the Maven job type in Jenkins.  Use a Pipeline.  It gives you more precise control and allows you to choose techniques that better suit your specific needs.

Refer to the "Risks" section of the Maven job type documentation for a detailed description why you should use a Pipeline job instead of the Maven job type.

 

And as a workaround, I could just make a new Maven Job for each Maven sub-module. Doing it this way, now I do actually run on many free nodes.

But that has the downside of increasing the maintenance burden immensely. Remember, I have over 50 sub-modules. If I need to add new functionality, that's 50 Jenkins jobs that I need to update. Not ideal at all.

 

Yes, if you need fine-grained control of job parallelism and then you'll need to manage that fine-grained control.

Mark Waite 

[David Alayachew]

Thanks for the response Mark.

Yes, me and my whole team had a nice sitdown going through the exact Risks section you are talking about, making sure that this is what we want to do. After a fairly long back and forth, we felt that the risks were worth putting up with in order to get the benefits.

As for the Jenkinsfile, that is the last resort. I have to clarify, the example I gave, of 50 sub-modules, was a severe understatement. We actually have nearly 200 modules total, each with their own specific build logic. A big part of the reason why we still wanted the Maven Build Job even in spite of the scary failures is because the Maven Build Job is documented to handle each of our use cases out-of-the-box. This one about the nodes being used is literally the only thing that is not running as expected. A Jenkinsfile, in comparison, will take significantly more effort. And while we're willing to if needed, this first pathway was enticing enough that it's worth considering for us.

But based on what you are saying, it kind of looks like we are going to be making a Jenkinsfile. As a last ditch effort, I see links to the Jira at the top of the link you shared. Assuming that I don't find any linked issues that are already describe our feature request, do you think they would be fine if I submitted one?

Thanks again for your time, help, and speedy response.

David Alayachew

--

You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/jenkinsci-users/b59c26a3-c445-4759-8016-4515c13b3803n%40googlegroups.com.

[David Alayachew]

And to be clear, I am more than happy to make the change myself and offer a complete pull request.

I am moreso making sure that a request and offer like this would even be welcome to the team. It seems yes, but while I have you here, I figured I'd ask.