Noob question for best practices (volume sizing)

[Brad Fairfield]

I'm sorry if this is not the appropriate place to ask.  If not, could someone let me know where to go.

I have been doing backups for various clients and employers for around twenty five years, but I am new to Bareos.  We will be backing up approx. 12 servers (a mix of Linux and Windows), and we are wanting to provide 6 months of backups.  We estimate it will be approx. 30 TB of full backups and 10-12 TB of incremental backups in that six month time frame.  We will be doing file backups, and our Bareos server will have over 105 TB of internal storage.

We plan on doing full backups every 6 months with regular incrementals.  With a retention time of just under 365 days.  That way it is recycling the full backup volumes from a year ago, rather than starting a third full backup at the one year point.

The big question is on size of the backup volumes.  I have never had the option to change that.  What are the things to consider when you are determining the backup volume (FULL and Incremental) sizes.  Is it better to keep them lower, thinking the server can more efficiently handle the small files when it has to open, write and save the files.  But then you could potentially have thousands of backup volumes that you have to manage.  Or can you set it much larger without any concerns to reduce the number of files in the backup volume directory.

Also, the thought is how much data could be lost if one of the backup volumes becomes damaged.

Then, that lead to what happens in Bareos if you are trying to do a larger restore job, and it calls for a backup volume that is damaged.  Does it fail the entire restore job, or does it just report it in a message and move on the the next volume it can access?

Brad

[Christoph Haas]

(Totally the right place to ask.)

 A couple of thoughts…

The number of volumes should not matter that match. My volumes are 10 GB for differential and incremental backups. And 50 GB for full backups. I would not mind having many volume files. Consider putting different pools into different directories on your 105 TB-Storage, so that not all files are in one directory and making operations slow.

I prefer to have smaller volumes and rather increase the number of parallel backups. I am backing up 150 VMware servers and run backups of 10 of them in parallel. I also set this in its pool:

Maximum Volume Jobs = 1

That way the storage device (using disk volumes) can write on 10 different volumes at the same time thus speeding up the backups. It could keep your backup time windows lower if that's helping you.

(Try not to be too intimidated by the sheer number of configuration files and options in Bareos. You can spend a month tuning all those little knobs but you will probably not even need it.)

Do you run full backups every 6 months and then only incrementals in between? That scares me. Only one incremental backup/volume would have to be corrupt to invalidate the entire chain since the last full backup. What about something like…

• Full every 6 months
• Differential every two weeks
• Incremental every two days
  

It may depend on how much data changes on the servers that you back up, whether that makes sense. But on a server where only a few percent of the data changes every day, it may be a good solution.

Con: a little more space used because a differential backup would capture more than just the increments

Pros: more safety. If a differential or incremental backup becomes corrupt, you would lose 0-2 weeks of data as opposed to 0-6 months. And restores will be a bit faster because they don't have to apply the entire chain.

Cheers
 Christoph