file based backups and pruning

[Jon Schewe]

I'm doing backups to disk and have bareos setup to automatically label volumes as needed, I also have the ability to add disk space to the backup partition and I'm working to figure out the right size for my backups by watching how full the volume gets and then adding space when needed. However I'm finding that old volumes don't appear to be recycled despite having autoprune enabled and the purge action set to truncate.

I read through https://docs.bareos.org/TasksAndConcepts/VolumeManagement.html and found this note:

A few points from Alan Brown to keep in mind:

• If Maximum Volumes (Dir->Pool) is not set, Bareos will prefer to demand new volumes over forcibly purging older volumes.

• If volumes become free through pruning and the Volume retention period has expired, then they get marked as Purged and are immediately available for recycling - these will be used in preference to creating new volumes.

This suggests to me the for my particular setup where I have not limited the number of volumes nor do I have a limited amount of disk space bareos will never prune volumes. Am I understanding this correctly?

I believe that the solution for my situation is to execute the following command on a regular basis:

prune volume all  yes

From my reading this seems to be a safe operation as it will only mark volumes as purged once the appropriate retention periods have expired. 

My retention periods are set on the pools only.

Am I correct that this is a safe operation to run regularly and won't end up losing data that is within my retention periods?

[Andreas Rogge]

Am 12.12.24 um 20:23 schrieb Jon Schewe:

>
> Am I correct that this is a safe operation to run regularly and won't
> end up losing data that is within my retention periods?

Correct.

However, if there is a fixed amount of disk-space (which is usually the
case) my preferred method is as follows:

* Configure your disk pools as you need them (i.e. Full, Differential,
Incremental, Whatever)
* Disable auto-labeling
* Set maximum volume bytes for each of the pools to the same value
* Add a scratch pool
* Set "scratch pool" and "recycle pool" in your backup pools to point to
that scratch pool
* Pre-label N volumes in the scratch pool where N = total space /
maximum volume bytes

Now when your backups write to one of the pools, they'll fetch volumes
from the scratch pool as required. When a volume is recycled it will go
back to the scratch pool, so it is available to be reused in any other
backup pool.

Best Regards,
Andreas
--
Andreas Rogge andrea...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz

[Jon Schewe]

On Friday, December 13, 2024 at 3:12:34 AM UTC-6 Andreas Rogge wrote:

Am 12.12.24 um 20:23 schrieb Jon Schewe:
>
> Am I correct that this is a safe operation to run regularly and won't
> end up losing data that is within my retention periods?
Correct.

Thank you for that confirmation.

However, if there is a fixed amount of disk-space (which is usually the
case) my preferred method is as follows:

* Configure your disk pools as you need them (i.e. Full, Differential,
Incremental, Whatever)
* Disable auto-labeling
* Set maximum volume bytes for each of the pools to the same value
* Add a scratch pool
* Set "scratch pool" and "recycle pool" in your backup pools to point to
that scratch pool
* Pre-label N volumes in the scratch pool where N = total space /
maximum volume bytes

Now when your backups write to one of the pools, they'll fetch volumes
from the scratch pool as required. When a volume is recycled it will go
back to the scratch pool, so it is available to be reused in any other
backup pool.

I see how that makes sense in most cases. Where I'm at right now is that I have a set number of machines that need to be backed up and a requirement for how long I need to keep backups. Tthat is what is driving the amount of space that is needed for backups. Unfortunately I don't know how much space I expect the backups to take, so I'm starting with a small partition and growing it as needed and expect the usage to level off as long as we stop adding systems to the backup system and hit the max retention period.