Increase concurrent nodes on a single execution (local executor)

44 views
Skip to first unread message

Ben Kaplan

unread,
Apr 27, 2023, 5:58:31 PM4/27/23
to rundeck-discuss
Hey,
We noticed when stress testing our ENV that when an execution is started over more than 10 (local executor) nodes only 10 "nodes" are actively running at a time, where the rest are pending in a queue.

In this specific use-case all nodes are of type "local executor" so I can't set/edit the thread pool count that you can do with remote-execution.

Any ideas on which setting can control the max concurrent-nodes for Rundeck local executor?

rac...@rundeck.com

unread,
Apr 28, 2023, 9:21:23 AM4/28/23
to rundeck-discuss
Hi Ben,

Take a look at this. Try to add "quartz.threadPool.threadCount = 20"  parameter on your "rundeck-config.properties" config file and then restart your Rundeck instance.

Regards.

Ben Kaplan

unread,
Apr 28, 2023, 9:38:47 AM4/28/23
to rundeck...@googlegroups.com
Thanks, ya that's the first option I tried. No matter it's value no more than 10 nodes are running in parallel. I can confirm that rundeck recognizable this value change in the system report page but I guess there is another limit somewhere?

--
You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rundeck-discuss/daf7aae0-1d75-4153-8fdc-d6e2d1a8b2c3n%40googlegroups.com.


Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

rac...@rundeck.com

unread,
Apr 28, 2023, 10:51:08 AM4/28/23
to rundeck-discuss
Hi Ben, Which Rundeck version are you using?

Greetings.

Ben Kaplan

unread,
Apr 28, 2023, 12:11:46 PM4/28/23
to rundeck...@googlegroups.com
We're running 4.10.1 version

On Fri, Apr 28, 2023, 17:51 rac...@rundeck.com <rac...@rundeck.com> wrote:
Hi Ben, Which Rundeck version are you using?

Greetings.

On Friday, April 28, 2023 at 9:38:47 AM UTC-4 ben.k...@redis.com wrote:
Thanks, ya that's the first option I tried. No matter it's value no more than 10 nodes are running in parallel. I can confirm that rundeck recognizable this value change in the system report page but I guess there is another limit somewhere?

On Fri, Apr 28, 2023, 16:21 http://rac...@rundeck.com <http://rac...@rundeck.com> wrote:
Hi Ben,

Take a look at this. Try to add "quartz.threadPool.threadCount = 20"  parameter on your "rundeck-config.properties" config file and then restart your Rundeck instance.

Regards.

On Thursday, April 27, 2023 at 5:58:31 PM UTC-4 ben.k...@redis.com wrote:
Hey,
We noticed when stress testing our ENV that when an execution is started over more than 10 (local executor) nodes only 10 "nodes" are actively running at a time, where the rest are pending in a queue.

In this specific use-case all nodes are of type "local executor" so I can't set/edit the thread pool count that you can do with remote-execution.

Any ideas on which setting can control the max concurrent-nodes for Rundeck local executor?

--
You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rundeck-discuss/daf7aae0-1d75-4153-8fdc-d6e2d1a8b2c3n%40googlegroups.com.


Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

--
You received this message because you are subscribed to a topic in the Google Groups "rundeck-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rundeck-discuss/YIAqR7SK_fM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rundeck-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rundeck-discuss/226a24dd-09fe-40d1-bfe3-c6b2cce5d2e6n%40googlegroups.com.

rac...@rundeck.com

unread,
Apr 28, 2023, 1:38:22 PM4/28/23
to rundeck-discuss
Hi Ben,

I tested the "quartz.threadPool.threadCount" successfully on 4.12.

1. Set "quartz.threadPool.threadCoun"t with a custom value, e.g.: 20. And restart your instance.
2. Then, on your job, set the thread count with the number of nodes that you have in your model source (Edit your job, go to "nodes tab" and then set the Thread Count textbox).
3. Save and run your job.

If I understood the problem correctly, this approach should achieve your goal

Regards.

Ben Kaplan

unread,
Apr 28, 2023, 2:02:36 PM4/28/23
to rundeck...@googlegroups.com
Thanks but when using local executor the nodes tab does not have any options available to modify in the UI (thread count can only be adjusted when not using "localhost").

We're trying to increase the localhost (local executor type) concurrency in an execution, it feels like there is some limit configured somewhere that applies to localhost we just don't know where it is.

On Fri, Apr 28, 2023, 20:38 rac...@rundeck.com <rac...@rundeck.com> wrote:
Hi Ben,

I tested the "quartz.threadPool.threadCount" successfully on 4.12.

1. Set "quartz.threadPool.threadCoun"t with a custom value, e.g.: 20. And restart your instance.
2. Then, on your job, set the thread count with the number of nodes that you have in your model source (Edit your job, go to "nodes tab" and then set the Thread Count textbox).
3. Save and run your job.

If I understood the problem correctly, this approach should achieve your goal

Regards.

On Friday, April 28, 2023 at 12:11:46 PM UTC-4 ben.k...@redis.com wrote:
We're running 4.10.1 version

Ben Kaplan

unread,
Apr 30, 2023, 9:17:14 AM4/30/23
to rundeck-discuss
ok found the issue... we use a parent job to call a child job and I was looking at the wrong child job :(
increasing the thread count of the correct job fixes this (as expected).
sorry for the confusion and thanks for the help.

Ben

Reply all
Reply to author
Forward
0 new messages