How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate
Palanilkunnathil Melemuriyil, Vinod P
Hi All,
We have an old version (1.651.3) of Jenkins running on a server (RHEL) with many jobs configured to automate our day to day activities.
I would like to migrate to the latest stable release of Jenkins.
For that, I would want to have a second instance of Jenkins running (with the latest stable release of Jenkins) on the same server and slowly migrate all the jobs and users to the new jenkins instance after thorough testing.
Can anyone guide me on how all the jobs and all the configurations which I have already in place in the old instance can be migrated in one go (maybe by copying some directories or files from the older version) to the new instance and I start testing the new instance to see if all jobs are running without any issues.
What would be the best approach to migrate to the latest version of Jenkins and at the same time have the old version running till we decommission it in due course.
Also right now, we don’t have a backup strategy in place for our Jenkins instance. I am wondering if by chance the Jenkins instance crashes how do I recover it?
What are the directories/configuration files I need to backup (on a daily basis) so that in case Jenkins crashes, I can bring it up by having the backup files restored.
Appreciate your help on this.
Thank You
Vinod
Palanilkunnathil Melemuriyil, Vinod P
Could anyone guide me on the below query?
Thank You
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-users/0EBBBB91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13.
For more options, visit https://groups.google.com/d/optout.
Artur Szostak
With regards to migration, for small Jenkins instances you should be OK with:
1) Stopping the new Jenkins instance.
2) Copying the XML configuration files and job directory from $JENKINS_HOME from the old instance to the new instance.
3) Start the new instance up.
For larger and more complex setups the probability of this working tends towards zero. In that case the better strategy is copying the configuration and jobs over one at a time while going through a stop, copy, start and verify cycle each time. This is extremely painful though. If you can it is better to script migration of the jobs through the REST APIs, while both instances are up and running. You will still need to consider how to verify your migration.
If you do have a large setup and have not done so already you might want to take this opportunity to put in the effort and build your new instance so that the whole configuration is created automatically (ideally in a virtualised server). You will again have to use the REST APIs and tools like jenkins-job-builder. For larger setups I think this becomes a must, since the default Jenkins GUI is not scalable for administrative tasks. Try maintain >60 build nodes and >1000 jobs with the existing Jenkins GUI and you will know what I mean.
With regards to backup, if you want a true and full backup strategy then I am not aware of any built in facility or plugin that can help. There are a few that will skim the XML configs of the jobs and a few other bits and pieces, but nothing that will restore a server to its original state if the whole thing burns down.
When I first setup our Jenkins server a few years ago the best option I could come up with was using https://wiki.jenkins.io/display/JENKINS/Exclusive+Execution+Plugin and rsync within an exclusive job to snapshot $JENKINS_HOME (in our case excluding any job workspaces, just keeping their configs and build logs). The snapshot itself can then be backed up properly in the background while Jenkins returns to work.
This worked OK, but it is not very scalable. At some point the snapshot starts taking way too long. An improved strategy I plan to implement on our next upgrade is to use Btrfs (ZFS can work also) as the underlying data store for #JENKINS_HOME. This will allow to perform copy on write snapshots which are very quick.
I will also switch to using https://wiki.jenkins.io/display/JENKINS/Build+Blocker+Plugin rather than https://wiki.jenkins.io/display/JENKINS/Exclusive+Execution+Plugin.
Note that I am relying on the fact that Jenkins does not do any serious I/Os while the snapshotting is taking place. Otherwise the backup will never be consistent. This is a major pitfall and until Jenkins provides a real quiescing infrastructure ,100% safe online backups will never be possible. However, I believe that with Btrfs one can mitigate this by performing 2 COW snapshots with a small interval of time between them and see if there is any difference between those snapshots. If not then there is a high probability that you have a consistent backup.
If you do not care for online backups then you can of course avoid the consistency problem by shutting down Jenkins, doing the backup and then starting it up again. But our Jenkins instance takes > 40min to start, so I stick to online backups, since I want to do them regularly.
Hope you find some of this useful.
Cheers
Artur
________________________________________
From: jenkins...@googlegroups.com <jenkins...@googlegroups.com> on behalf of Palanilkunnathil Melemuriyil, Vinod P <v.palanilkunna...@cgi.com>
Sent: 17 July 2017 17:40:20
Subject: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0EBBBB91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13<https://groups.google.com/d/msgid/jenkinsci-users/0EBBBB91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13?utm_medium=email&utm_source=footer>.
Palanilkunnathil Melemuriyil, Vinod P
Thank You
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4b7919d4cb71482398ec9a444a6b5de1%40partner.eso.org.