Jenkins master in Docker optimal setup

[Eddard]

Hello,

the software engineering team i work in is putting up a new ci/cd platform based on jenkins which will orchestrate the continuous delivery
pipeline for more than 50 micro-services in a near future. We already have 10 of them being built and deployed through that platform.

we have come to the realization that out setup light not be scalable and are looking for an alternative architecture/setup for the platform.

• Right now we have a bare meta machine with a docker-engine on it which allow ton a docker container with the master jenkins in it.
• when we run a pipeline build, we launch a  standard ubuntu container as a slave agent which  runs on the same docker host as the jenkins master, this slave container
• the ubuntu container slave has a docker and docker-compose installed which in turn are used to launch container on the same docker-host of the master to run all the tasks of the pipeline.

This is not very optimal and we are looking for alternatives, if you have a ci-platform setup based on jenkins master running in a container and has scaled to run many many builds would you please share your experiences? and what would

you consider an optimal and scalable jenkins in docker setup/architecture?

Thanks in advance

[Baptiste Mathus]

Hey, 

You should probably start with the general things like https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds https://wiki.jenkins-ci.org/display/JENKINS/Consideration+for+Large+Scale+Jenkins+Deployment or https://jenkins.io/doc/book/architecting-for-scale/ and Googling about "Scaling jenkins" to get a grasp about the domain.

About your question, one of the first things to know/do when scaling Jenkins, is to not run builds on the Master. Basically, you generally set the executor to 0, (or a very very low number and drastically select the ones you allow running there).

About using Docker, using a Docker Swarm cluster as a build cluster, used by the master like it has virtual infinite capacity works well (using the docker-plugin), though probably not the one with the most Docker spirit way (see docker-slaves-plugin, though not the same goal). 

<shameless-plug>If you speak french, I demoed that using basically https://github.com/batmat/jez at the ToulouseJUG/JAM here.</shameless-plug>

HTH

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b12830f9-c5cf-411f-a489-5cf2ada3aafd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Omeka]

hello thank you for you response

Eddard Omka

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.