Affordable solution for Jenkins CI and build slaves in the cloud?

[Eddie Sutton]

I no longer want to deal with backup issues of our CI system.

I have an ESXi server running on a Mac Mini hosting a Jenkins server and multiple build machines on a Synology NAS datastore targeting Android, iOS, Linux, macOS, and Linux. 

Any success stories or recommendations?

Thanks in advance,

-Ed

[Victor Martinez]

If you mean to backup your Jenkins server, then read below:

   In my case I've been using tools such as:

• Puppet: https://github.com/jenkinsci/puppet-jenkins
  
• Chef: https://github.com/chef-cookbooks/jenkins
  
• Ansible: https://github.com/geerlingguy/ansible-role-jenkins
  

        NOTE: You can use docker (https://hub.docker.com/r/jenkins/jenkins/) although the complexity will be with a live and continuous up to date instance, in my opinion. 

   The second part is about using a subset of the Configuration as Code tools such as:

• Pipelines: https://jenkins.io/doc/book/pipeline/
  
• JobDSL: https://github.com/jenkinsci/job-dsl-plugin/wiki
  
• Jenkins Job Builder: https://docs.openstack.org/infra/jenkins-job-builder/
  

    Finally, you can automate the last bits and pieces using Groovy and the https://wiki.jenkins.io/display/JENKINS/Post-initialization+script

The above suggestion is based on the Configuration as Code paradigm and being able to specify the current jenkins instance state either programmatically or with some configuration in code. 

My two cents

[Artur Szostak]

> If you mean to backup your Jenkins server, then read below:
>
> In my case I've been using tools such as:
>

> * Puppet: https://github.com/jenkinsci/puppet-jenkins
> * Chef: https://github.com/chef-cookbooks/jenkins
> * Ansible: https://github.com/geerlingguy/ansible-role-jenkins
>

How is this related to backup? This just deals with installation and configuration management.

[Victor Martinez]

My answer was suggesting how to move from traditional backup processes since there are already tools/frameworks, as you just mentioned, which help to configured everything fully as code. I guess, that's why it's related to backup :)

[Artur Szostak]

OK, but that does not help with the build logs etc. i.e. If I loose my server, I not only want to recreate the instance, but I want to restore the build logs. This requirement may not be for everyone. But for people who consider the build history valuable it would be important to back that up also.

Unfortunately I do not think that Jenkins has an enterprise grade backup solution available out of the box. You have to put it together by yourself. And then there is no way to do it 100% correctly without a full shutdown of the Jenkins master at some point. Which is unfortunate, since that makes it impossible to have a high availability Jenkins that is also being backed up regularly and automatically at the same time. One can get close with clever use of a copy-on-write snapshotting file systems like Btrfs, but you will not have 100% guarantee that the snapshot is consistent since Jenkins has no quiescing facility.
________________________________________
From: Victor Martinez <victormar...@gmail.com>
Sent: 17 August 2017 13:52:08
To: Jenkins Users
Cc: Artur Szostak
Subject: Re: Affordable solution for Jenkins CI and build slaves in the cloud?

[corneil....@gmail.com]

How about using JFrog Artifactory in the cloud and configure Jenkins to push build logs to Artifactory https://www.jfrog.com/confluence/display/RTF/Build+Integration#BuildIntegration-SupportedPlugins

Corneil du Plessis about.me/corneil

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2022a80edcf74ceaacc3c68a3e4eaf61%40partner.eso.org.

For more options, visit https://groups.google.com/d/optout.

[Victor Martinez]

Yep, I agree you might need to either use the current backup plugins:

- https://wiki.jenkins.io/display/JENKINS/thinBackup

- https://wiki.jenkins.io/display/JENKINS/PeriodicBackup+Plugin

Or create your own backup process.

Although, I just found that Cloudbees provides an enterprise plugin which already sync with AS3 among others:

- https://go.cloudbees.com/docs/cloudbees-documentation/cje-user-guide/index.html#backup

Cheers