setup of Jenkins infrastructure

[LnT]

Hi ,

I'm preparing topology of new setup of Jenkins infrastructure. We will be given  60 Virtual VMs (Consists of 45 Windows + 15 Linux VMs)

Target :
Need to submit the high level topology for the jenkins infrastructure for Test Automation and Generation of reports.

My Plan is
1. Create Jenkins master in in windows VM (Provided 16GB RAM)
2. Make rest all VMs as jenkins slaves
   2.1 - 3 VMs will be dedicated to Utility jobs like report generations
   2.2 - Win7,Win8 and Win10 VMs will be dedicated to automation test suits
   2.3 - Linux VMs will be dedicated to automation test suits

Pros : Maintenance is easy w.r.t all 20 users in my team.
Cons : I suspect , 3000+ jobs will be created and heavy burden over Jenkins master

I would need help - do we have any best practices documentation for setup.

Please advice.

Regards,
LnT

[Dirk Heinrichs]

Hi,

> My Plan is
> 1. Create Jenkins master in in windows VM (Provided 16GB RAM)

We were also running our Jenkins master on Windows for the past couple of years, but we finally migrated it to Linux last year. Windows really (still) doesn't make a good server OS (even though it has "Server" in the name). So if there's no real reason other than "manager wants it to be this way", I'd recommend against running any server on Windows.

> Cons : I suspect , 3000+ jobs will be created and heavy burden over Jenkins master

Please define "heavy burden". We usually don't run that much on the master, all the heavy lifting is done on our agent nodes.

HTH...

Dirk

[LnT]

Thanks so much Dirk.

I will convince for Linux Jenkins master.

Burden meaning - if 3000+ Jenkins jobs are created then

even though jobs run @ slaves

but respective build information will be stored @ Jenkins master. Hence it is seeing as burden.

What would you suggest - if Jenkins page load is damn slow ?

Regards,

LnT

[Dirk Heinrichs]

Hi,

> Burden meaning - if 3000+ Jenkins jobs are created then
> even though jobs run @ slaves
> but respective build information will be stored @ Jenkins master. Hence it is seeing as burden.

We've got approx. 1000 jobs, and don't experience any problems on the master (VM with 4 vCPU and 16G RAM). We also use the "Archive Artifact" plugin a lot, which stores to-be-kept artifacts on the master for later retrieval. Some of those artifacts are sized ~3G.

> What would you suggest - if Jenkins page load is damn slow ?

A general mistake we made was to put all the jobs into the main page. It was just lately that we discovered the wonders of the Folder plugin and started to use it to organize our jobs. Views also help here, but I - personally - like the folders more.

Bye...

Dirk

[LnT]

Very much grateful to your feedback Dirk.

Thanks a lot.

[Artur Szostak]

We have a setup with ~1000 matrix jobs building on about 50 platform / version combinations (i.e. one subjob is one platform combination). Total volume of the build logs approaches 1.5TB on a btrfs volume with compression enabled (I believe raw is about 2.5TB). If you were talking about freestyle jobs then our setup is way bigger than what you will have, otherwise its a similar size. Our experience has been that the Jenkins master becomes IOPS bound. You will really want a data storage system that can give you a high enough IOPS rate for the master. Otherwise the start up times are very long and the Jenkins webpage slows down badly. SSD is ideal if you can afford it, but only necessary for big setups with high amounts of jobs and build logs.

Cheers

Artur

________________________________________
From: jenkins...@googlegroups.com <jenkins...@googlegroups.com> on behalf of LnT <tlnar...@gmail.com>
Sent: 01 March 2018 09:35
To: Jenkins Users
Subject: Re: setup of Jenkins infrastructure

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com<mailto:jenkinsci-use...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6b3570ad-779f-4874-960f-a7ec7b3e1c8d%40googlegroups.com<https://groups.google.com/d/msgid/jenkinsci-users/6b3570ad-779f-4874-960f-a7ec7b3e1c8d%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.