Unable to run more than 8 jobs in parallel in rundeck.

[sarath botlagunta]

I set the quartz Thread pool size to 20.

             quartz.threadPool.threadCount = 20

I was hoping, I can run atleast 16 to 18 jobs in parallel even If I discount few threads for rundeck background tasks.

But I can only run maximum of 8 jobs in parallel.

See the attached Image of System Info which shows only 40% ratio for Scheduler and I could not figure out how to make my rundeck run more than 8 jobs.

Any help or guidance will be appreciated.

[rac...@rundeck.com]

Hi Saranth,

Could you elaborate? Which Rundeck version are you using? how did you test the parallel jobs? Consider the instance’s memory/workload, take a look at this.

e.g: Rundeck 5.2.0, launched with the following params:

java -jar -server -Xmx4096m -Xms1024m rundeck-5.2.0-20240410.war

And the following quartz config:

quartz.threadPool.threadCount=50

I’ve tested with 10 jobs simultaneously, and the scheduler worked as expected.

Hope it helps!

​

[sarath botlagunta]

I submitted 13 jobs in parallel and only 8 running at given time and for the remaining 5 they are staying in some queue and starting running as previous jobs are finishing.

I submitted 20 commands with each command is sleep 200 and they are running fine with 20 in parallel.

Only difference I can think about jobs is, each step in a job is referencing a job in some other project.

I have a Project SystemJobs -> Job1
                                               -> Job 2
                                               -> Job 3

I have a Project UserJobs -> LoadTest1 Job -> /Systemjobs/Job1  <args>

                                                LoadTest2 Job -> /Systemjobs/Job1 <args>

                                                LoadTest3 Job -> /Systemjobs/Job1 <args>

                                                                           

I have created around 13 such LoadTest jobs with each job just referencing the jobs from SystemJobs project.

Is there any limitation with this setup that prevents me from reaching my 20 thread limit when running these load test jobs created like this but not seeing that limitation with commands.

Also why 8? It is always exactly 8.

Regards,

Sarath

                  

--
You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rundeck-discuss/a8e2eb21-d9e3-441d-baca-0f22a1257bb1n%40googlegroups.com.