Backup/Restore large repository in Nexus 3.x

[Anatoly Laskin]

We plan to switch from Nexus version 2.12 to version 3.2.

What can you recommend for a large volume backup (1-3 TB) of BLOB storage in Nexus 3.2?

The documentation (http://books.sonatype.com/nexus-book/3.2/reference/backup.html) describes:

1. Configure the Export configuration & metadata for backup task to export databases.

2. Run the scheduled task to export the databases to the configured folder.

3. Back up custom configurations in your installation and data directories around the same time you run the export task.

4. Back up all blob stores.

I think that the implementation of 1 to 3 steps of the common scenario will not take long. Suspends access to the database until the backup is complete.

But the execution of 4 step can last for more than an hour, even if we talk about incremental copying. During this time, access to the database will be restored.

What will happen if, during the backup of the BLOB storage, the artifact will be deleted from the repository. The database will delete the entry, the group of blob files will be deleted in the BLOB store, but the backup will continue to refer to the file from the BLOB of the repository, since the backup was created relatively long ago.

When restoring a backup, how does this collision manifest itself?

Or is this situation not critical for such large volumes?

Can I first make a backup copy of the BLOB repository, and then run the scheduled task to export the databases to the configured folder?

Regards,

Anatoly Laskin

[Bruno Miguel Custódio]

Bumping. Can anyone please shed some light on this?

[Wes Wannemacher]

What I would imagine is that even when you have a really large blobstore, you should still be using a backup technology that enforces some consistency at the point that the backup is initiated. A good example is the shadow copy stuff that windows does (https://en.wikipedia.org/wiki/Shadow_Copy). What I would envision is that the database exports are dropped into a known location that will be backed up as well, but not likely a part of the blobstore snapshotting. The backup initiates on the blob store as near to the same time as the database export as can be achieved to minimize collisions. All of the database backups are initiated as close to simultaneously as the host will allow because each database export happens in a separate thread. During the database export, the databases are marked read-only. This will stop someone from deleting components in Nexus. Therefore, if the blobstore is backed up using shadow copy or lvm snapshot (http://www.tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html) while the database export is in progress, you should end up with a consistent backup. 

The downside to this approach is that nexus will be read-only while the database exports are taking place. In my testing, this is usually a very short time period, but your mileage can vary based on your hardware. 

-Wes

On Wed, Jun 7, 2017 at 6:45 AM, Bruno Miguel Custódio <brunomc...@gmail.com> wrote:

Bumping. Can anyone please shed some light on this?

--
You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.
To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/42cba303-110b-4c60-aa3e-1ed4620658d7%40glists.sonatype.com.

For more options, visit https://groups.google.com/a/glists.sonatype.com/d/optout.

[Bruno Miguel Custódio]

In my case that's not an option. Isn't there really anything built into Nexus 3 that allows for consistent backups of the blob stores? That would be odd, to say the least.

[Brian Fox]

The idea would be to do a database export to disk, then backup the disk such that you have consistency between the data and the blobs. The blobs themselves are otherwise files that you would backup normally.

Unless you can take an outage and or stop writes to the system during the backup, there will be some slight mutations.

On Wed, Jun 7, 2017 at 3:59 PM, Bruno Miguel Custódio <brunomc...@gmail.com> wrote:

In my case that's not an option. Isn't there really anything built into Nexus 3 that allows for consistent backups of the blob stores? That would be odd, to say the least.

--

You received this message because you are subscribed to the Google Groups "Nexus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nexus-users+unsubscribe@glists.sonatype.com.
To post to this group, send email to nexus...@glists.sonatype.com.

To view this discussion on the web visit https://groups.google.com/a/glists.sonatype.com/d/msgid/nexus-users/d2fb0393-b0ac-497f-a808-068b3bc492f1%40glists.sonatype.com.