Help on hosting Jenkins controllers

[Raviraj Karasulli]

We have a large Jenkins installation, we have scaled the controller vertically.

Currently our memory allocated is ~40 GB for the controller. We want to check if it is possible to scale the controller with the horizontal scaling. If we scale it with the kubernetes cluster or a managed instance group, how does the controllers work? Any scheduled jobs will be triggered by all the controllers?

[James Nord]

Hi,

I am assuming you are already using multiple build agents.

Jenkins OSS does not support horizontal scaling of controllers.

CloudBees CI is the only option I know of that supports horizontally scaling controllers.

https://www.cloudbees.com/blog/revolutionize-your-jenkins-environment-with-ha-hs

Other than that so you know where your memory is going?

Pipeline groovy will consume memory on the controller, so performing things like json /yaml processing in groovy

Rather than as a shell step can be a factor limiting scalability.

Regards

James

[Raviraj]

hi James,

We have considered the CloudBees CI and are deterred by the costs of licensing.

Yes your guess is right, we have 100s of pipeline jobs running, nothing runs on the controller. 

We use declarative pipeline jobs along with shared libraries. Also, there are many shell interpolations present in the job.

Is it possible to make all the pipeline jobs consume resources from a dedicated agent rather than consuming it from the controller?

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/qPCZO5tMEXQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/jenkinsci-users/bad54753-80fd-4adb-8d40-bb753e895294n%40googlegroups.com.

--

Regards,

Raviraj